Skip
To Content
|  | |||||||||||
 |
|
|||||||||||
|
A coordinated five-year sample design was developed for 1999 through 2003. The sample design for the 2000 main study, as a subsample of the five-year study, consisted of a deeply stratified, multi-stage, area probability design. Exhibit 2.1 presents details of the sample design.
The coordinated 1999-2003 design calls for 50 percent overlap in first stage units (area segments) between each successive year of the five-year study following completion of the 1999 survey.
The first stage of the sample selection procedures began by geographically partitioning each state into roughly equal-sized field interviewer (FI) regions. These regions were formed as a means of stratification so that each area would yield roughly the same expected number of interviews during each data collection period. This partitioning divided the United States into 900 FI regions made up of counties or groups/parts of counties.
These FI regions were subdivided into smaller geographic areas—called segments—that served as the primary sampling units. In general, segments consisted of adjacent Census blocks and were equivalent to area segments selected at the second stage of selection in previous NHSDAs. A total of 96 segments per FI region were selected (with probabilities proportional to size): 24 to field the five-year study and 72 to serve as backups in case of sample depletion or to field any supplemental studies SAMHSA may request. For the 2000 survey, a total of 7,200 segments within the 900 FI regions were selected. Of the total, 3,593 segments were overlap segments used during the 1999 survey, while 3,607 segments were new. (The segments are not evenly split because some of the 1999 segments had to be replaced due to insufficient remaining sample (see Section 2.5.3)).
After selecting these new areas, the process of counting and listing (C/L) the dwelling units (DUs) within each new segment ensued. Segments to be used in 2000 were listed between April and November of 1999. Once all DUs for a particular quarter were listed, the second-stage selection process identified sample dwelling units (SDUs) for inclusion in the study.
Similar to previous NHSDAs, at the final stages of selection, five age group strata were sampled at different rates. These five strata were defined by the following age group classifications: 12-17, 18-25, 26-34, 35-49, and 50 years old and over. Race/ethnicity groups were not purposely over-sampled for the 2000 main study. However, consistent with previous NHSDAs, the 2000 NHSDA was designed to over-sample younger age groups.
Preparations for C/L field activities began with the decision to use the existing NHSDA data collection management structure to supervise counting and listing. All current Field Supervisors (FSs) were asked to handle the administrative tasks for the listers hired for their area. These tasks included completion of the initial hiring process, segment assignment and weekly approval of time and expense reports. (Exceptions occurred in a few struggling states to allow those FSs and their field staff to concentrate solely on screening and interviewing work. In those states, traveling lister teams completed the C/L work.) For technical supervision such as how to handle a specific segment, all listers contacted the Regional Supervisor for Counting and Listing (C/L RS) for answers and advice.
Beginning in April, 1999, FSs recruited listing staff from their existing staff of field interviewers. Experienced listers not currently working as NHSDA interviewers were also available for hire. A total of 305 listers were hired and worked from April through November, 1999, to complete counting and listing operations for the 2000 NHSDA.
The training program varied by the listers' experience level and assignment:
| Traveling Listers: Classroom training was held in April to train a select group of 18 listers as the traveling listing team. Several RTI survey specialists were also trained in C/L procedures at this time. Training included detailed instruction in proper C/L protocol and the completion of actual segments selected for the state of North Carolina. These travelers reported directly to an NHSDA RS who provided administrative supervision in addition to managing their workload and assignments. | ||
| For all other training, staff received a homestudy training package containing a memorandum and materials including a newly revised C/L manual; C/L video tape; hire letter; Data Collection Agreement; 2000 NHSDA C/L Project Specification Sheet; Production, Time and Expense Reports; and general listing supplies. | ||
| – | RTI-Certified Listers: Staff previously certified as listers successfully completed the homestudy prior to receiving an assignment. | |
| – | Experienced but not RTI-Certified Listers: For staff with listing experience who had not been previously RTI-Certified, their training included the homestudy as well as path-of-travel exercises and a certification packet. | |
| – | New Listers: Staff with little or no listing experience received the same homestudy and certification package just described. However, they were given more time in which to complete these materials and received telephone training from RTI staff. This telephone training supplemented the homestudy before new listers completed their certification packages. In the event a new lister needed additional training, the C/L RS or FS arranged for in-person training or mentoring by an experienced lister. | |
Once the listers successfully completed the required materials/training process and returned signed Data Collection Agreements to RTI, they were authorized to begin their C/L assignments. All listers sent their completed assignments directly to the Sampling Department at RTI where they were carefully edited. Feedback was provided to any listers who had significant errors. Problem segments were either refielded (for correction of major errors) or were corrected by sampling staff through discussions with the lister. In some cases, the lister returned to the segment to review the items in question.
Prior to the start of actual C/L field work, segment kits were assembled at RTI. Each kit contained maps of the selected area, listing forms, and segment information sheets. A copy of the maps remained at RTI and another copy was given to the field supervisor for assisting with problems encountered in the field.
Beginning in April, segment kits were assigned and sent to those listers who had completed the certification process and were ready to begin listing. Once the remaining staff became certified, they received an assignment as well. Listers recorded the address or description of up to 400 dwelling units (DUs) in each segment.
To reduce the time required to count and list segments, several procedures were implemented to maximize efficiency. In many cases the "count" step was eliminated: the lister could immediately list the segment unless it was apparent the segment had experienced additional construction or the lister determined that the segment was large (i.e., 400+ DUs) during the initial trip around the boundaries of the segment. As had been done on prior rounds of the NHSDA, a rough count procedure was allowed for segments containing large geographic land areas, large DU counts (400+ DUs), or significant growth in residential DUs (typically, 1,000+ DUs). This procedure permitted listers to obtain an approximate count of residential DUs in these segments from secondary sources—such as the post office, fire department, or county or city planning office—without having to conduct an exact count.
If a lister came across a segment that needed subsegmenting, the lister called in the initial DU counts to RTI's Sampling Department, who could usually subsegment it over the telephone (any segment with more than 400 DUs generally required subsegmenting). In many cases, this allowed the lister to-in one trip—count and list a segment with 400 or more DUs, rather than experiencing a delay of one or two weeks and necessitating a second trip to the segment. For unusual or very difficult subsegmenting tasks, the segment materials were sent to RTI to be handled directly by sampling personnel.
The counting and listing of almost all of the segments was completed by the end of November 1999 (the exceptions involved a few access problems). Once the segments were listed and the completed segment kits were received at RTI, an editing process ensured that no DUs located outside segment boundaries were included, that listing sheets matched segment sketches/maps, and that proper listing order and related listing rules were observed. During this editing process, the sampling staff also checked all subsegmenting that occurred in the field to ensure it was done correctly.
Listed DUs were keyed into a computer control system. A selection algorithm then selected the specific sample dwelling units (SDUs) to be contacted for the study. Prior to the beginning of the appropriate quarter, FSs then assigned segments (or partial segments) to their interviewing staff. Interviewers received all assigned SDUs on their Newton handheld computer. Each selected unit and the next listed line (for use as a sample check to capture missed dwelling units during screening and interviewing) were also printed on Selected DU Lists. These lists, along with copies of the handwritten listing forms and maps, were placed in Segment Materials Envelopes and distributed to the assigned field staff before the start of each quarter.
During the screening process, Field Interviewers (FIs) were trained to identify any unlisted DUs that existed within the SDU or within the interval between the SDU and the next listed DU. If the missed DUs were housing units, they were automatically entered into the Newton (up to established limits) and selected for participation. At most, the FI could independently enter five added DUs per SDU and a maximum of ten missed DUs per segment. If the FI discovered more than these amounts or if the missed DUs were group quarters units, the FI called the FS. The FS then either called RTI's Sampling Department for further instructions or instructed the FI to call the Sampling Department directly, depending on the situation.
While no upper-limit was placed on the total number of DUs that could be added to a segment by RTI's Sampling Department, the FIs were instructed to notify RTI of any significant listing problems. A very small number of segments required re-listing during the screening and interviewing phase. Table 2.2 indicates the number of segments that experienced added DUs, as well as the total number of added DUs for the 2000 NHSDA.
In many of the major urban areas, field staff had some difficulties gaining access to locked buildings, and listers in particular had some trouble listing very large public housing complexes. Access in some suburban areas proved problematic as well; more and more planned communities have intercoms, guarded gatehouses or entryways outfitted with cameras and scrambled buzzer systems. Access to military bases, college dormitories, and large retirement communities also proved problematic at times. Based on experiences from 1999, these types of access problems were expected. Special mechanisms or protocols were in place to handle them promptly and in some cases avoid them entirely.
Access problems were typically resolved through effective follow-up efforts of supervisory staff, including situation-specific letters of request and in-person visits by the Field and/or Regional Supervisors. In particularly difficult situations, SAMHSA offered additional support via special refusal conversion letters or telephone follow-ups by the Project Officer.
In 2000, the often problematic access to military bases was handled with a formal and standardized approach. Through joint RTI/SAMHSA efforts, a contact person within the Pentagon for each branch of the service was identified. These individuals were advised in advance of base selections for the year. They then notified the base commanders regarding RTI's need to access these bases for both listing and screening/interviewing work. Additionally, standard letters and informational packages were sent by RTI staff to help obtain access to all selected bases. These efforts were effective: access to all but three selected bases was secured.
Access to colleges and universities is sometimes problematic. RTI used several standard approaches to accommodate the concerns of school administrators. Having standardized letters available that addressed reoccurring issues with a variety of attachment options was very effective.
Most schools requested or required only a letter stating the sponsor and the purpose of the study, and identifying the lister or data collection staff. However, some schools wanted more complete information and the right to approve the field data collection procedures and personnel working in and around their campuses. Most of these situations resulted in packages being sent that contained:
In the end, only one private educational institution denied the request for cooperation for the counting and listing phase of the 2000 NHSDA.
A small number of segments were identified during the counting and listing phase as difficult to access during months with unusual weather. Most involved roads made impassable by snow during the winter months. Others involved roads inaccessible due to rain, and one or two isolated locations involved water-only access that often froze during the winter months. If segments with weather or geographic access problems were selected for a quarter in which the access would be a problem (generally Quarters 1 or 4), the segment was switched with a segment in the same region for an appropriately paired time period. For example, inaccessible first quarter segments were switched with second quarter segments in the same region that would be more accessible during the first quarter; fourth quarter segments were switched with more easily accessed third quarter segments. Generally the "switched" segment was selected because it had more accessible road surfaces, was more urban, or had fewer inaccessible roads.
In a few locations, such as some areas in Alaska, there were no segments that were better for reassignment during the problematic time period. When that happened, staff made prompt assignments, emphasized early completion of the work, and tried to plan around good weather forecasts to accomplish the field work as early in the period as possible.
In the 1999 NHSDA, two samples were fielded simultaneously; the paper and pencil interviewing (PAPI) sample and the computer assisted interviewing (CAI) sample. Also, some special oversampling problems occurred at the beginning of the year. Thus, for some sample segments which were to be reused in 2000, the dwelling unit sample had been depleted. These segments were replaced with a segment from the same FI region drawn at random from the remaining sample. Thus, the 2000 segment sample consisted of 3,593 overlap segments used during the 1999 survey and 3,607 new segments.
First Stage of Selection for the Main Study: Segments
The 2000 design provided for estimates-by-state in all 50 states and the District of Columbia. States should therefore be viewed as the "first level" of stratification as well as a reporting variable. Eight states, labeled the "big" states in Table 2.1, had a sample designed to yield 3,600-4,630 respondents per state. The remaining 43 "small" states1 had a sample designed to yield 900-1,030 respondents per state.
The larger sample sizes obtained at the state level, along with small area estimation techniques refined under previous NHSDA contracts, enabled the development of estimates for all states, for several demographic subgroups within each state (i.e., age group and race/ethnicity group), and for some Metropolitan Statistical Areas and a few small areas in the "big" states.
The "second level" of stratification defined contiguous geographic areas within each state and also corresponded in size to the annual assignment for a single field interviewer (FI). These FI regions were of approximately equal population size in terms of allocated sample.
Additional implicit stratification was achieved by sorting the first-stage sampling units by an MSA/SES (Metropolitan Statistical Area/socioeconomic status) indicator2 and by percentage of non-Hispanic white. The first stage sample units for the 2000 NHSDA were selected from this well-ordered sample frame.
For the first stage of sampling for the 2000 NHSDA, each of the FI regions was partitioned into noncompact clusters of dwelling units by aggregating adjacent Census blocks. Consistent with the terminology used in previous NHSDA studies, these geographic clusters of blocks were referred to as segments. On average, segments were formed so that they contained at least 175 dwelling units and were constructed using 1990 Decennial Census data supplemented with revised population counts obtained from outside sources. A sample dwelling unit in the NHSDA refers to either a housing unit or a group quarters listing unit (such as a dormitory room or a shelter bed).
A sample of segments was selected within each FI region, with probabilities proportionate to a composite size measure and with minimum replacement. Segments were formed so that they contained sufficient numbers of dwelling units to support three annual NHSDA samples. This allowed half of the segments used in any given year's main sample to be used again in the following year as a means of improving the precision of measures of annual change. This also allows for any special supplemental sample or field test that SAMHSA may wish to conduct in any given NHSDA year within the same segments.
In order to coordinate the sample selection for 1999 through 2003, 96 segments were selected within each FI region. An equal probability subsample of eight segments was used for the 2000 NHSDA. These eight segments were randomly assigned to quarters and to two waves within each quarter. The waves used in the 2000 NHSDA were designated as Waves 2 and 3. Wave 2 segments were used for the 1999 and 2000 surveys. New dwelling units (i.e. those not previously selected for the 1999 study) were selected from the Wave 2 segments for 2000. Wave 3 segments were new for 2000 and will be used again for the 2001 survey.
Data from roughly one-fourth of the final sample of respondents was collected during each calendar quarter. This important design feature helped control any seasonal bias that might otherwise exist in drug use prevalence estimates and other important NHSDA outcome measures of interest.
Second Stage of Selection for the Main Study: Listed Lines
Before any sample selection within selected segments began, specially-trained staff listed all dwelling units and potential dwelling units within each newly selected area segment. A dwelling unit is either a housing unit for a single household or one of the eligible noninstitutional group quarters that are part of the defined target population. The listings were based primarily on observation of the area segment and could include vacant dwelling units and units that appeared to be dwelling units but were actually used for nonresidential purposes. The objective of the listing was to attain as complete a listing as possible of eligible residential addresses; any false positives for residences were eliminated during the household screening process after the sample was selected.
The sampling frame for the second stage of sample selection was the lines of listed dwelling units and potential dwelling units. After accounting for eligibility, nonresponse, and the third-stage sample selection procedures, it was determined that 280,273 lines were needed to obtain a sample of 70,000 responding persons distributed by state and age-group. During the study's implementation, however, a total of 215,860 lines were selected and yielded a final respondent sample of 71,764 (as shown in Table 2.1). These lines were selected among lines not used in the 1999 survey (overlap segments) and the complete list of dwelling units (new segments).
As in previous years, if an interviewer encountered any new dwelling unit in a segment or found a dwelling unit missed during the counting and listing activities, the new/missed dwellings were selected into the NHSDA using a half-open interval selection technique.3 That selection technique eliminated any frame bias that might have been introduced because of errors and/or omissions in counting and listing activities and also eliminated any bias that might have been associated with using "old" segment listings.
Third Stage of Selection for the Main Study: Persons
After dwelling units were selected within each segment, an interviewer visited each selected dwelling unit to obtain a roster of all persons aged 12 and over residing in the dwelling unit. This roster information was then used to select zero, one, or two persons for the survey. Sampling rates were pre-set by age group and state. Roster information was entered directly into the electronic screening instrument (the Newton) which automatically implemented this third stage of selection based on the state and age group sampling parameters.
Using an electronic screening instrument also provided the ability to impose a more complicated person-level selection algorithm at the third stage of selection. As a result of this unique design feature, any two survey-eligible people within a dwelling unit had some chance of being selected—i.e., all survey eligible pairs of people had some non-zero chance of being selected. This design feature is of interest to NHSDA researchers because it allows analysts to examine how the drug use propensity of one individual in a family relates to that of other family members residing in the same dwelling unit (e.g., the relationship of drug use between a parent and child).
As illustrated in Table 2.1, at the third stage of selection, 91,961 people were selected from 169,769 screened and eligible dwelling units. A total of 71,764 completed interviews were obtained from these 91,961 selected persons.
Expected Precision of NHSDA Estimates
The multi-stage, stratified NHSDA design has been optimally constructed to achieve specified precision for various person subpopulations of interest. These SAMHSA-specified, precision requirements call for the expected relative standard error on a prevalence of 10% not exceed the amounts listed below.
For the main study:
To achieve these precision requirements and meet state sample-size requirements, the optimal person-level sample distribution by strata was determined that minimized data collection costs while simultaneously meeting the above-specified precision requirements for several critical NHSDA outcome measures.
The precision constraints in the design optimization models were set up using local area predictions of drug use from a recent project involving small area estimation techniques used to generate local area estimates using 1991-1993 NHSDA data. Drug use estimates across strata were appropriately scaled to reflect the generic 10% prevalence.
| Statistic | Small States | Big States | Total |
|---|---|---|---|
| Total Sample | |||
| FI Regions | 516 | 384 | 900 |
| Segments | 4,128 | 3,072 | 7,200 |
| Selected Lines | 121,473 | 94,387 | 215,860 |
| Eligible dwelling units | 102,044 | 80,532 | 182,576 |
| Completed screening interviews | 95,376 | 74,393 | 169,769 |
| Selected persons | 51,753 | 40,208 | 91,961 |
| Completed Interviews | 40,744 | 31,020 | 71,764 |
| Average Per State | |||
| FI Regions | 12 | 48 | |
| Segments | 96 | 384 | |
| Selected Lines | 2,825 | 11,798 | |
| Completed Interviews | 948 | 3,878 | |
| Interviews Per Segment | 9.87 | 10.10 | |
| Average Per State And Quarter | |||
| Segments Per FI Region | 2 | 2 | |
| Interviews Per FI Region | 19.74 | 20.20 | |
| Interviews Per Segment | 9.87 | 10.10 | |
| Total States | 43 | 8 | 51 |
| Total Interviewers (approximate number that varied by quarter) | 516 | 384 | 900 |
| Note: "Small" states refers to states where the design yielded 948 respondents on average. "Big" states refers to states where the design yielded 3,878 respondents on average. |
|||
| Number of Added DUs per Segment (X) |
Number of Segments with X Added DUs |
Cumulative Number of Added DUs* |
|---|---|---|
| 1 | 449 | 449 |
| 2 | 174 | 797 | 3 | 81 | 1,040 |
| 4 | 43 | 1,212 |
| 5 | 35 | 1,387 |
| 6 | 12 | 1,459 |
| 7 | 13 | 1,550 |
| 8 | 7 | 1,606 |
| 9 | 10 | 1,696 |
| 10 | 8 | 1,776 |
| 11 | 4 | 1,820 |
| 13 | 2 | 1,846 |
| 15 | 2 | 1,876 |
| 16 | 3 | 1,924 |
| 17 | 1 | 1,941 |
| 18 | 1 | 1,959 |
| 22 | 1 | 1,981 |
| *Total number of added DUs = 1,981 | ||
1For reporting and stratification purposes, the District of Columbia is treated the same as a state and no distinction is made in the discussion.
2The four categories are defined as: (1) MSA/low SES, (2) MSA/high SES, (3) NonMSA/low SES, and (4) NonMSA/high SES.
3In summary, this technique states that if a dwelling unit is selected for the NHSDA and an interviewer observes any new or missed dwelling units between the selected dwelling unit and the dwelling unit appearing immediately after the selection on the counting and listing map page, then all new/missed dwellings between the selection and the next one listed will be selected. If a large number of new/missed dwelling units are encountered (generally greater than ten) then a sample of the missing dwelling units will be selected.
|
This page was last updated on December 29, 2008. |
|
SAMHSA, an agency in the Department of Health and Human Services, is the Federal Government's lead agency for improving the quality and availability of substance abuse prevention, addiction treatment, and mental health services in the United States. 
* PDF formatted files require that Adobe Acrobat Reader® program is installed on your computer. Click here to download this FREE software now from Adobe. |
