• About the Study
• Methodology
  • Instrument Development
  • Training
  • Agency Survey
  • Client Survey
  • Response Rates
  • Analysis Methods
  • Reporting Conventions
  • Degree of Comparability
    • Estimating Turnover Rates
    • Provider Screening
    • More Than One Visit
    • Longer Hours
    • Number of Participating Food Banks
    • Open Less Often
    • Small Providers
    • Summary
  • Number of A2H Clients Served
• Key Findings
• Who We Serve
• How We Serve
• Local Picture
• Snapshot Comparisons
• Media Center

Estimating Client Turnover Rates Within the A2H System (3.8.1)

An important goal of the periodic A2H surveys has been to develop annual estimates of the number of different clients served. However, this raises substantial methodological issues, as discussed below.

Importance of “Newcomer” Rates and Key Resulting Estimation Issues. An important factor that influences the annual number of clients participating in the A2H emergency food assistance system is the amount of client turnover during the year. It is much more straightforward to estimate the number of clients at a given point in time than to estimate the number over a year. This is because the annual number depends on turnover in the system. As an example, consider a pantry that serves 100 clients per month. If mostly the same clients go to the pantry month after month, then the annual number of clients for the pantry will be at least 100, or a small number more to account for a few clients leaving and others replacing them. If mostly different clients come each month, however, the pantry could serve 1,000 clients, or even more, in a year. Furthermore, even much more modest rates of turnover could increase the annual total from about 100 to perhaps 300 or 400. Thus, taking into account the amount of client turnover can have major implications for overall client estimates.

The study depends on information obtained during the client interviews to draw inferences about client usage of the system over a 12-month period. Survey recall problems pose formidable challenges to interpreting the data, however, because many clients may not accurately recall and report their past usage patterns for an entire year. Typically, clients are able to supply accurate information about their usage of the emergency food system during a recent period, such as a week (or even perhaps a month), but as the period gets longer, recall usually becomes less reliable. While long recall periods are a problem for many surveys, they may be particularly problematic for the A2H client population, because many of them are elderly, have disabilities, or are in low-income households where they are concentrating on how to meet day-to-day household needs with low resources, rather than thinking about the past year.

The 2001 Survey Questions Related to Client Turnover. In the 2001 and 2005 surveys, we tried to examine client turnover based on the self-reports of survey respondents about their patterns of using the A2H system. The research strategy focuses on what the 2001 report called the “newcomer rate,” defined as the percentage of clients at a given point in time who have started using A2H providers within the past month but had not used the A2H system in the previous 12 months. If we can estimate “newcomers” defined in this way for 12 months in a row, the sum yields a measure of all the people who entered the system during the past year. 1

The questions used to obtain this information have evolved between the 2001 and 2005 surveys. In 2001, we used the following two questions:

P66. For how many months in a row have you been using this food pantry or other pantries like this one?

|___|___| NUMBER OF . . .

1 o WEEKS
2 o MONTHS
3 o YEARS
N o TODAY WAS FIRST TIME
D o DON’T KNOW
R o REFUSED

P69. Not counting the last four weeks, when was the last time you or another member of your household received groceries from this or any other food pantry? 

|___|___| NUMBER OF . . .
1 o WEEKS AGO
2 o MONTHS AGO
3 o YEARS AGO
4 o NEVER
D o DON’T KNOW
R o REFUSED

(These were the questions for pantries; similar questions were asked of kitchen and shelter users, except that the second question started with “Not counting the past seven days.”)
In principle, answers to these questions are sufficient for estimating the number of clients who were “newcomers” as defined by the study. Many respondents did provide usable responses. However, we encountered two problems with these questions. First, during the survey fielding period, many interviewers and respondents said they found these two questions confusing. Second, in analyzing the results of that study, we found that many respondents had given inconsistent answers to these and related questions.

Because of these findings, in the 2001 analysis, we used a conservative definition of “newcomer” (in the sense of not overestimating the relevant population), with clients being treated as “newcomers” only if they gave consistent answers to both the questions reproduced above, indicating that they had just recently entered the A2H system for the first time in at least a year. 2 In addition, because of the uncertainty associated with the information available, the analysis of annual participation was undertaken for a range of possible estimates, rather than a single point estimate.

2005 Questions. At the outset of the 2005 study, to improve accuracy, we tried to develop alternative ways of obtaining these data on newcomer rates,. The result was that another question was added to the questionnaire:

P61b Now, thinking about the past year, did you or anyone in your household use a pantry…

1 o Every month, (12 MONTHS)
2 o Almost every month, (10-11 MONTHS)
3 o Most months, (6-9 MONTHS)
4 o Some months, (4-5 MONTHS)
5 o Just a few months, (2-3 MONTHS)
6 o Just this month?
D o DON’T KNOW
R o REFUSED

It was felt that this question was more straightforward and understandable and that it could perhaps elicit more accurate information. We kept the two questions used in the 2001 survey to provide backup if the new question did not work as hoped.

Findings. During fielding of the 2005 survey, examination of the answer patterns for the three 2005 questions and the two 2001 questions suggests that (1) the response patterns to the two previous questions are broadly consistent for 2001 and 2005, and (2) the responses to the new question were largely consistent with a “higher-estimate” interpretation of the old questions (compared to the more conservative, or lower-bound, interpretation used in the 2001 analysis).

2005 Analysis of Newcomers. Because of these findings, we have based the 2005 newcomer estimates on the responses to the new question. The result of this decision is that the newcomer estimates are significantly higher than in 2001. In particular, for pantries—by far the most important component in the total estimates—the point estimate of the percentage of clients that are newcomers in the previous month is 13%, compared with about 5% in the 2001 analysis.

This 2005 estimate was higher than expected, based on the 2001 analysis. It implies that, at the typical pantry on a given day, 13% of clients have started using the pantry that day or at some other time in the previous month and have not previously used the system for at least a year (or ever). However, several factors suggest that this estimate may be likely. First, as noted above, responses to both the two old questions and the new question seem to point to a newcomer rate higher than that used in the 2001 analysis. Second, many clients may use the A2H system just once or twice and then stop, either because their short-term food insecurity problem is resolved or other reasons.

Third, poverty rates and food insecurity rates have been rising in recent years,3,4 which has probably increased the need for emergency food. It is likely that as the low-income population and food insecure population has increased, more newcomers use safety net systems such as the emergency food system.

Fourth, evidence for Survey of Income and Program Participation (SIPP) data collected in the late 1990s suggests that there are quite high rates of transition into and out of poverty over time. For example, the available data suggest that 51.1% of poverty spells in the United States between 1996 and 1999 lasted only two to four months. This SIPP-based analysis concludes, “A little over half of the poverty spells, measured by using monthly income and poverty thresholds, were completed after 4 months. Another 19.3% of spells were over after 8 months.”5 The income volatility represented by these high rates of transition into and out of poverty provides further evidence that rates of transition into and out of the A2H system might be higher than were estimated in 2001.

Because of these considerations, the 2005 participation analysis has been based on responses to the new question. To exercise caution, however, we have kept the more conservative practice used in the 2001 study of conducting analysis for a range of assumptions around the 2005 point estimate.

1 Key to the approach outlined in the text is that a “newcomer” is defined as a person who starts using the A2H system and has not previously used it for at least a year. Of course, some people may enter and exit the system several times during the year; however, in making annual unduplicated estimates, we want to count these people only once a year.

2 The issues related to the 2001 analysis approach are discussed on page 38 of Hunger in America, 2001, the national report of the 2001 study (October 2001).

3 Between 2001 and 2004, the overall rate of persons in poverty in the United States rose from 11.7 to 12.7. See [www.census.gov/hhes/www/poverty/histpov/hstpov2.html], accessed on January 10, 2006.

4 See also Nord, Mark, Margaret Andrews, and Steven Carlson. Household Food Security in the United States, 2004. U.S. Department of Agriculture, Economic Research Service, Report Number 11, October 2005.

5 While now a little dated, the SIPP data are the most recent available to address the issue. See John Iceland, “Dynamics of Economic Well-Being: Poverty 1996-1999,” U.S. Census Bureau, Current Population Reports, #P70-91, July 2003. We are indebted to Kirk Johnson for making this point to us.