Background and Limitations (4.1)
The estimation process draws on several data sources to derive estimates of the size of the A2H system:
- Information from the survey sample frame of providers, which was compiled from food bank records
- Information from the sampling and data collection operations on the observed numbers of clients served by providers, the providers’ days of operation, and similar factors
- Information from the client survey on respondents’ length and frequency of use of the emergency food system
- Information from A2H administrative files on the sizes of the food banks that participated in the study compared with those that did not participate
Given these rich data sources, several approaches can be taken in the estimation work. In much of the work below, we draw primarily on an approach, rooted in standard statistical estimation theory, where we (1) compute the probabilities of various providers and clients being in our survey sample, (2) compute analysis weights based on these probabilities, and (3) estimate the underlying population totals by summing the relevant analysis weights. In some instances, however, we use alternative approaches to develop certain estimates, compensate for limited information availability, add intuition to the estimation process, and test the robustness of our conclusions. We describe these approaches later in this chapter.
Our estimates unavoidably contain some uncertainty, which comes from several factors:
- Statistical Sampling Error. Sampling error results from the fact that many of the estimation parameters are based on statistical samples, rather than on surveys of all the relevant providers and clients.
- Reporting Error. Some of the interview questions on which our estimates are based were unavoidably complex. As a result, some error undoubtedly exists because respondents did not always understand the questions and did not always report accurately.
- Nonresponse Bias. As with any survey, it must be assumed that there is at least some error due to nonresponse. In this survey, it would be caused by the agencies and clients who did not respond to our surveys being different from those that did.
- Coverage Bias. Only about three-fourths of the A2H food banks participated in the study, which may lead to coverage bias. While we have adjusted for this, we cannot determine for sure exactly how accurate our adjustments are.
- Alternative Estimation Methods. As the subsequent discussion makes clear, several methods could be used in deriving the results presented below. Our discussion explains the reasons for the choices we make, but some judgment is involved in this and may influence the final results.
- Seasonality. Because of logistical requirements, most of the data were collected during winter and spring 2005. Therefore, it is not possible with this data set to fully examine and correct for fluctuations in the A2H system and clients over the entire year.
Despite these possible sources of error, the MPR research team for the study and A2H believe that the estimates derived below are based on the best survey methods and estimation procedures available, given the resources.
The next section provides an overview of our findings. After that we describe additional details of our calculations. We begin with pantries, since they are the largest component of the A2H Network.
