Various sources of measurement error have substantial implications for the accuracy of epidemiologic estimates. Exposure measurement error, for example, may arise when geographic information systems are trusted without recognizing the limitations of processes that rely on them. One such process is address matching, the automated pairing of coordinates (latitudes; longitudes) and statistical tabulation areas (e.g. census tracts) with street addresses, typically using TIGER/Line or other street data files 1. The process – which is also known as geocoding – has been described in detail 23. Geocoding usually involves matching addresses to specific street segments then positioning the addresses along the segments assuming an even distribution of street numbers within them. Although this form of geocoding involves linear interpolation and assumptions that can be inappropriate, its inaccuracy may be overlooked in large, population-based studies of associations between spatially interpolated environmental exposures, relevant health outcomes, and their contextual, socioeconomic effect modifiers. Nevertheless, geocoding accuracy is critical when such studies focus on exposure mechanisms that operate over short distances 4.

Although error in assignment of latitudes, longitudes, and census tracts has the potential to bias both estimation of location-specific exposures and socioeconomic contexts within which they occur 56, recent studies have reported mean positional errors in commercially geocoded address coordinates between fifty and 300 meters 7891011. This is a distance over which long-term average ambient air pollution concentrations, meteorological measures and their monitor-to-monitor temporal correlations are relatively constant 121314. However, concentrations of traffic-related emissions rapidly fall to ambient levels within comparable distances from street center-lines 15. Moreover, positional error may be relevant in an even wider range of studies if the previously reported range of distances (50 – 300 m) is an underestimate. Lack of adjustment for potentially important address characteristics suggests that this is a distinct possibility. Population density in the area surrounding an address, for example, is so strongly and inversely associated with positional error that reported distances may be biased by even small differences in the ratio of rural to urban and suburban address matches 1617. Positional error also varies markedly with match type, i.e. whether vendors match individual addresses to specific streets or to centers of statistical tabulation areas (centroids) 18, yet to date, most studies have not accounted for these factors.

Published studies of positional error have several additional features that are pertinent in this context. Many restricted their focus to a single geographic setting, address source or geocoding vendor, while those focusing on multiple vendors did not account for among-vendor heteroscedasticity or within-address correlation of positional error 1920. Others ignored potential for verification bias 21 and with a notable exception, none examined effects of positional error on exposure measures 7. Collectively, these observations suggest that the next generation of studies in this area should be designed with generalizability, validity and utility in mind.

To this end, we established three study objectives: (i) to compare multiple geocoding vendors using an identical sample of addresses with known coordinates selected from a broad range of data sources and geographic areas, (ii) to estimate geocoding accuracy and account for address characteristics that affect it using appropriate statistical procedures, and (iii) to estimate effects of observed inaccuracy on individual- and contextual-level exposure measures. We conducted this study to inform research emanating from two studies. The first, The Environmental Epidemiology of Arrhythmogenesis in WHI 22, is an ancillary study of electrocardiographic mechanisms linking air pollution and cardiovascular disease in 68,133 U.S. women aged 50–79 years at baseline in the Women's Health Initiative (WHI) clinical trial 23. The second, the Atherosclerosis Risk in Communities (ARIC) study, is a prospective study of cardiovascular disease in 15,792 U.S. men and women aged 45–64 years at baseline 24. This Institutional Review Board-approved ancillary study complied with all applicable regulations governing human subjects research (University of North Carolina Medical IRB# 03-EPID-12).

Door-to-door return times and geocoding costs were generally reasonable across vendors: range, 2–5 business days and $16–$25 per 1,000 addresses. However, analyses of the edited address database revealed large differences among vendors A-D in address match rate (98%; 82%; 81%; 30%), census tract concordance (85%; 88%; 87%; 98%) and mean ρ (1809; 748; 704; 228 m) (Table 3 and Figure 2). Address match rate and census tract concordance were relatively high and mean ρ, relatively low among WHI, complete, zip-coded, unedited, and urban or suburban addresses; addresses with NAD83 or WGS84 criterion standard coordinates; and street-type matches (Table 4).

In analyses restricted to vendors with minimally acceptable match rates (A-C), among-vendor differences in mean ρ were small for street-type matches (293; 287; 288 m). In contrast, differences between centroid-type matches were substantial in some vendor contrasts, but not others (6375; 4854; 5524 m), p for interaction < 10^-4. Adjustment for address characteristics, within-address correlation and heteroscedasticity of ρ reduced the mean and standard deviation of ρ (Table 5). The pattern of adjusted mean ρ among vendors reflected that of the adjusted odds of an address match: it was higher for vendor A versus C (OR = 66, 95% CI: 47, 93), but not B versus C (OR = 1.1, 95% CI: 0.9, 1.3). The adjusted odds of census tract concordance were, by comparison, no higher for vendor A versus C (OR = 1.0, 95% CI: 0.9, 1.2) or B versus C (OR = 1.1, 95% CI: 0.9, 1.3) (Table 6).

Further restricting analyses to records successfully geocoded by all vendors A-C attenuated mean ρ and its pattern of differences among them. Match rate and census tract concordance were much lower, and mean ρ, much higher in analyses of the unedited versus edited EPA addresses (data not shown).

The percent of street-type address matches < 100 meters away from the nearest highway was relatively constant across mean ρ (Table 7). This apparent absence of misclassification was related to counter-balancing effects of approximately equal false positive and false negative rates at values of mean ρ between 150 and 600 meters. Together, they accounted for a 14% increase in the total error rate over the same range. This increase was accompanied by a 20% decrease in census tract concordance.

In contrast, the percent of centroid-type address matches classified as < 100 meters away from the nearest highway was approximately two-fold higher at zero versus non-zero values of mean ρ (Table 7). This finding was related to the two- to three-fold excess of false negative versus false positive rates at values of mean ρ between 2,500 and 10,000 meters. The total error rate increased by 3% and census tract concordance decreased by 24% over the same range.

Persistent concerns about the potential effects of inaccurate geocoding on spatially interpolated environmental exposures, exposure-outcome associations, and their contextual effect modifiers have stimulated interest in the positional error of commercially geocoded address coordinates. However, studies of the topic have often reported average positional errors in the range of fifty to 300 meters 67891617181920. Although these reports have reduced such concerns, few studies have focused on multiple geographic areas, address sources and vendors; adjusted accuracy measures for important address and methodological characteristics; and estimated the influence of inaccuracy on individual- and contextual-level exposure measures. The generalizability, validity and utility of these estimates is therefore unclear.

We addressed this issue in a Women's Health Initiative ancillary study, the Environmental Epidemiology of Arrhythmogenesis in WHI, by submitting addresses selected from a broad range of data sources and geographic areas to four well-known vendors often contracted by epidemiologists for geocoding and related services or products (at the time of submission, they had been in business for a combined total of > 35 years, employed > 650 persons, and reported > $50 million of annual sales 38). We then examined differences between vendors in address match rate, census tract concordance and mean ρ.

We found that geocoding error depends on measures used to evaluate it and vendor. More specifically, vendors matching lower proportions of addresses geocoded them with higher spatial accuracy, i.e. higher census tract concordance and lower mean ρ. We also found that that geocoding error depends on address characteristics. Mean ρ, for example, was relatively high among EPA, incomplete, unzip-coded, edited and rural addresses; addresses with NAD27 criterion standard coordinates; and in particular, centroid-type address matches. After stratifying by match type, then adjusting for the remaining address characteristics and other methodological factors, mean ρ remained twenty times higher among vendor A's centroid- versus street-type address matches. The adjusted odds of an address match also remained more than sixty times higher for vendor A than either B or C. Lastly, by randomly displacing address coordinates over the range of mean ρ observed in this context, we found that traffic-related pollution exposure misclassification rates increased and census tract concordance decreased with corresponding increases in mean ρ.

Considered together, these findings suggest that vendor selection presents a trade-off between potential for missing data and error in estimating spatially defined attributes such as environmental exposure and socioeconomic context. They also indicate that the trade-off can be quite unbalanced. Vendor D, for example, matched an unacceptably low proportion of addresses, but geocoded them with a singularly high level of spatial accuracy. Moreover, the observed association between missing data and positional error across vendors suggests that while vendors may be targeting different points along the trade-off spectrum, they tend to retain observations that are likely to have positional errors. Deleting these observations would of course translate into reduced potential for bias due to individual- and contextual-level exposure measurement error, but it remains unclear whether vendors can increase data accuracy without compromising its availability.

Although these findings may have greater generalizability, validity and utility than those previously reported, our criterion standards may have been imperfect. Interpretation must therefore recognize potential for bias due to the elusiveness of a definitive criterion standard. Indeed, match rate and concordance may have been overestimated and mean ρ, underestimated because using imperfect criterion standards tends to artificially inflate accuracy 21.

Since errors in accuracy measures vary with errors in imperfect criterion standards, we therefore edited addresses when they failed to conform to U.S. postal standards or conflicted with field notes. Editing was intended to reduce misspelled, misspaced or inappropriately abbreviated state, street suffix or secondary unit designators like "apartment" 28. Though well-intentioned, editing may have introduced error instead of reducing it. Mindful of this possibility, we submitted both the unedited and edited versions of EPA addresses for geocoding. We found that, on average, match rate and census tract concordance were much higher and mean ρ, much lower in analyses of the edited versus unedited versions of the database. This finding confirmed that, on average, editing tended to correct addresses and thereby reduce error in accuracy measures, but as a precaution, we also adjusted measures of accuracy for edit type.

Even after editing addresses, our criterion standards may have contained erroneous coordinates of EPA monitors, NGS stations and WHI participants. Such errors have been identified, for example, within EPA databases of environmental hazards in South Carolina 39. Although theses errors vary across data sources, among states and over time, their potential existence in this context is no less a concern. The EPA implemented its Locational Data Policy in 1991 in response to concerns of this sort. It stipulated adoption of uniform methods, use of global positioning systems and collection of monitor coordinates according to a Federal Interagency Coordinating Committee on Digital Cartography accuracy standard of 25 meters 33. Five years later, the EPA also launched its Locational Data Improvement Project as a vehicle for further improvement in the accuracy of its databases 40. Moreover, the NGS adheres to a stricter, 1998 Federal Geographic Data Committee standard of less than ten meters 34 – a distance identical to that between parcel center points and true residential locations in urban settings and somewhat less than that in rural areas 16. We also adjusted measures of accuracy for differences among address sources despite these reassurances.

Interpretation of the findings reported here must also consider the challenges inherent in disentangling the general effect of vendor and the specific effect of a given geocoding method. Street offset – the perpendicular distance between vendor-assigned coordinates and the corresponding street centerline – serves as an illustrative example. Although researchers are often troubled by vendors' underlying assumption that this distance is equal for all addresses, a different study design would have been required to discriminate effects of vendor and offset because as a default, vendors A-D used distinct offsets between zero and fifty feet. However, a repeated-measures design – one in which the same addresses would have been geocoded repeatedly by the same vendors using different offsets – was not feasible: the option of changing defaults was not uniformly available among vendors A-D. Even if it had been, prior reports suggesting that the contribution of offset to geocoding accuracy is rather modest within the narrow range of defaults observed in this context are reassuring 1116.

With these caveats in mind, we conclude that informed selection of geocoding practices and approaches to data analysis involves estimating potential for, balancing the trade-off between, and when appropriate, adjusting for the effects of missing data and error in spatially defined attributes. We suggest beginning this process by submitting (masked) addresses associated with high quality criterion standard coordinates in a given study area to geocoding vendors, estimating the accuracy of vendor-assigned coordinates, and selecting vendors that balance the tradeoff between missing data and error in ways that best meet study needs. If edited and unedited forms of the same address are included in the geocoded data set, address cleaning procedures – which should (but may not) be standardized – can be simultaneously evaluated.

Comparing the limitations of methods commonly used to analyze incomplete data with those used to adjust for positional or exposure measurement error may help prioritize individual study needs in advance 41424344. Basic algebra, for instance, can be used to adjust associations for exposure measurement error 44. Consider the cell counts observed in a hypothetical case-control study of the association between distance to the nearest highway and coronary heart disease mortality (Table 8). The sensitivity (se) and specificity (sp) of the 100 m distance classification at mean ρ = 150 m can be calculated from the corresponding false negative (fn) and false positive (fp) rates in Table 7:

se = 1 - fn = 1 - 0.06 = 0.94

sp = 1 - fp = 1 - 0.08 = 0.92

Under non-differential misclassification, the corrected cell counts are

a = (a* - 0.08 × (a* + c*)) ÷ (0.94 + 0.92 - 1) = 81.40

b = (b* - 0.08 × (b* + d*)) ÷ (0.94 + 0.92 - 1) = 88.19

c = (a* + c*) - a = 143.61

d = (b* + d*) - b = 313.81

and in the absence of confounding, the corrected odds ratio is

OR = (a × d) ÷ (b × c) = (81.40 × 313.81) ÷ (88.19 × 143.61) = 2.0

This odds ratio is more extreme than its uncorrected counterpart, OR* (Table 8), which is biased toward the null. Its corrected probability distribution can be estimated using Monte Carlo simulation 45.

However, the magnitude of exposure measurement error in a continuous variable such as distance to the nearest highway may not vary directly with the magnitude of a given exposure-outcome association. When it is independent of disease status, the resulting misclassification of commonly used exposure categories (e.g. distance < or ≥ 100 meters) may be differential and vary in unanticipated ways. Seemingly appropriate adjustments may also be inaccurate even when this type of misclassification is non-differential 43. Such adjustments must therefore be applied with caution.

Nonetheless, uninformed selection of geocoding practices and data analysis appears to be a less desirable alternative, particularly in studies of exposure mechanisms operating within short distances. The positional errors reported here suggest that "short" should be defined as less than 280 meters for potentially geocodable addresses matched at the street level and less than 5.5 kilometers for those matched at the centroid level by well-known vendors with minimally acceptable match rates. Critical distances, though, may be substantially lower given the non-negligible misclassification rates we observed when mean ρ was approximately one-half as large as these values. More accurate geocoding methods that involve global positioning or parcel matching can be used to reduce potential for bias in studies requiring such high levels of spatial resolution 216. Use of the latter method is expected to grow over time as high quality, parcel-level databases become more uniformly available across larger study areas.

ARIC Atherosclerosis Risk in Communities

CASS Coding Accuracy Support System

EPA Environmental Protection Agency

FIPS Federal Information Processing Standards

NAD27 and NAD83 North American Datum of 1927 and 1983

NGS National Geodetic Survey

TIGER Topologically Integrated Geographic Encoding and Referencing

USPS United States Postal System

WHI Women's Health Initiative

WGS84 World Geodetic System of 1984

The author(s) declare that they have no competing interests.

EAW conceived of the study, designed it, and drafted the manuscript. PMQ assembled and analyzed the data, and helped draft the manuscript. RLS directed the statistical analysis and helped draft the manuscript. DJC helped direct the statistical analysis and draft the manuscript. DL helped design the study and draft the manuscript. ACH directed handling of geographic data and helped draft the manuscript. GH helped design the study and draft the manuscript.