Portal de Periódicos da CAPES

Data Survey : The HILDA Survey: Progress and Future Developments

2010; Wiley; Volume: 43; Issue: 3 Linguagem: Inglês

10.1111/j.1467-8462.2010.00604.x

1467-8462

Nicole Watson, Mark Wooden,

Health disparities and outcomes

As described in previous data survey articles in this journal (Wooden, Freidin and Watson 2002; Watson and Wooden 2004), the Household, Income and Labour Dynamics in Australia (HILDA) Survey is Australia's first and only large-scale, nationally representative household panel survey. Like all longitudinal surveys, the main purpose of the HILDA Survey is to identify changes in the behaviour of the sample units being observed (in this case, residents of a representative sample of private households) and, where possible, to quantify the magnitude of those changes. This requires the repetitive collection over time, from the same sample members, of like, if not identical, measures of characteristics and behaviour. It might also be expected that the data collection procedures used should be replicated at each survey wave. Nevertheless, the process of survey administration and data collection is also affected by behaviours and changes in the real world. Individuals are not always cooperative and hence all panel surveys are confronted by sample attrition. At the other end of the spectrum, populations change because of births and immigration, raising questions about sample representativeness and thus the desirability of incorporating new sample members each survey wave to reflect these changes in the population. Very differently, changes in technology can have implications for the way surveys are administered. Finally, longitudinal surveys, especially those concerned with the measurement of human behaviour, need to be responsive to changes in both policy settings and the wider society, suggesting the need for survey content to gradually evolve over time. A good panel study can thus not remain static; it must evolve to reflect broader changes in society. With 9 years of data collection behind it, the HILDA Survey is now a very different creature from the one originally conceived and implemented in 2001. The aim of this article is to provide an update on progress with the survey, including the evolution of the sample, changes in the way the survey is being administered, and the changing nature of the survey content. All longitudinal surveys have to confront the problem that with each successive survey wave, some sample members are lost, either because of a failure to locate sample members that have moved, or because sample members withdraw their cooperation. This can be problematic, both because of the risk that sample sizes will eventually become unacceptably small, and because sample loss is often non-random, leading to concerns about the representativeness of the sample. Additionally, survey designers have to decide 'whether and how new entrants to the study population (births) should be included' (Lynn 2009, p. 11). In the case of the HILDA Survey, all members of responding households in wave 1 (the interviewing for which was conducted over the period August to December 2001), regardless of age, as well as any subsequent children (both biological and adopted), form the sample pursued each year. In addition, any person who at a later survey wave is co-residing with an original (or permanent) sample member is also added to the sample. These persons, however, remain in the sample only for as long as they are co-resident with an original sample member. There are, however, two exceptions to this rule. First, persons who have a child with a permanent sample member are converted to permanent status. Second, from wave 9, any sample members who are identified as having been born overseas and arrived in Australia after 2001 (the year wave 1 of the HILDA Survey was administered) are being converted to permanent status. Details about the evolution of the responding sample over the first nine waves are provided in Table 1. This table shows that of the 13 969 persons originally interviewed in wave 1, 9245 (or 66 per cent) were re-interviewed in wave 9. This total includes persons known to have died or moved out of scope (that is, persons who have moved overseas long term). These number 1037 and, if excluded, would give a nine-wave sample retention rate of 71.4 per cent. Perhaps the most important feature of the sample revealed by Table 1 is the relatively large proportion of the responding sample at later waves that did not participate in wave 1. Just over 30 per cent of wave 9 respondents were not interviewed in wave 1. These respondents are divided fairly equally between original sample members (either children turning 15 years and thus becoming eligible for interview, or adult members of partially cooperating wave 1 households who were not interviewed in wave 1) and new sample members (most of whom are temporary sample members). It can also be seen that, with this design, it is not inevitable that overall sample size will decline given attrition. Indeed, sample growth as a result of changing household composition has, since wave 4, more than offset the loss coming from attrition. Further information about response rates are provided in Table 2. This table provides the annual wave-on-wave response for waves 2 to 9. These response rates are intended to provide an indication of how successful the HILDA Survey has been at obtaining high rates of response, and thus exclude from the denominator persons who have died or moved out of scope (including temporary sample members who move out of the sample households). The table distinguishes between previous wave respondents (who account for the large majority of in-scope sample members in any year), previous wave non-respondents, children turning 15 years of age and new sample members. It also reports figures for both all persons and persons with an attachment to a household that responded in the previous wave. As can be seen, the wave-on-wave response rate for previous wave respondents has gradually improved over time, rising from 86.8 per cent in wave 2 to 96.2 per cent in wave 9. Furthermore, and as illustrated in Figure 1, these response rates are very similar to those reported over the first nine waves in other leading household panel studies, such as the British Household Panel Survey (BHPS) and the German Socio-Economic Panel (GSOEP). Wave-on-Wave Attrition Rates, HILDA, BHPS and GSOEP Compareda Notes: (a) HILDA, Household, Income and Labour Dynamics in Australia Survey; BHPS, British Household Panel Survey; GSOEP, German Socio-Economic Panel.(b) Excludes proxies and short telephone interviews. The response rates among other types of respondents are much lower. New sample members have less knowledge of, and attachment to, the survey than previous wave respondents, and hence we would expect both lower response rates and no discernible upward trend over time. The response rates are indeed lower, but do show a marked jump upwards in wave 5. We suspect this reflects the changed incentive arrangements that were introduced in this wave (see Subsection 3.4). Among children turning 15 years of age, and hence becoming eligible for interview for the first time, response rates vary between 70 and 80 per cent. The population here, however, includes many households where the parents have long since discontinued their involvement in the survey. Once we restrict the population to just children of households that participated in the previous wave, the response rate exhibits the same upward trend characteristic of wave 1 respondents, rising from 80.4 per cent in wave 2 to 92.6 per cent in wave 9. Finally, Table 2 reports very low response rates for persons that had not responded in the previous wave. This is hardly surprising, and indeed many longitudinal surveys make no effort to reach non-respondents in later waves. This is not the practice in the HILDA Survey, and as a result every year a sizeable number of individuals are recruited back into the responding sample despite a year or more missed. The following rules used in the HILDA Survey go some way to ensuring the sample remains representative of the Australian population in a cross-sectional sense; however, there is one group that is largely overlooked—immigrants arriving in Australia after the original sample was selected. Watson (2006) estimated that in the 10 years since the sample was selected, approximately 1.5 million immigrants arrived in Australia, representing 7 per cent of the Australian population. Some of these immigrants have joined HILDA Survey households, and, as noted earlier, we have recently started following them on a permanent basis. This group, however, is far from a random sample of recent immigrants. This suggests the need for a strategy for augmenting the sample targeted specifically at recent immigrants. A range of options for such a top-up sample were canvassed (see Watson 2006) but because of difficulties in targeting the sample, especially when the sampling frame is quite dated, it was decided that the top-up sample would not be limited to recent immigrants but would instead be a random sample of people living in non-remote parts of Australia. This will have the added benefit of helping to alleviate, to some degree, the biases from non-random attrition. The top-up will result in the addition of 2000 households to the ongoing sample, commencing with wave 11 (that is, 2011). Consistent with the wave 1 sample design, a random sample of 125 Census Collection Districts (CDs) will be selected across Australia, with a probability proportional to the number of occupied dwellings in each CD based on the 2006 Census. A minimum household response rate of 66 per cent is projected, yielding, on average, 16 dwellings from each CD. Administration of the HILDA Survey in wave 1 involved two main elements: (i) face-to-face interviews (using pencil and paper methods) with all household members aged 15 years and over; and (ii) the distribution of a self-completion questionnaire (SCQ). If maintaining longitudinal consistency in the data were the primary objective, then these same administration methods would be used in every succeeding wave. Cost pressures, together with the perceived advantages offered by new technology, however, have led to significant changes in the mode of survey delivery. In addition, other changes, and notably a change in the fieldwork provider, have resulted from factors outside of our control. For waves 1 to 8, all fieldwork functions as well as many data processing functions (data entry and coding) were subcontracted to the Nielsen Company. In early 2008, Nielsen informed us that it had no intention of re-tendering for the next fieldwork subcontract (apparently because it had made a business decision to move away from face-to-face interviewing), necessitating a change in fieldwork provider. The successful tenderer was Roy Morgan Research (RMR), which has the contract to perform the fieldwork functions for waves 9 to 12. Although such changes during the life of a longitudinal survey must be anticipated and planned for, they do have the potential to significantly disrupt data continuity. Most other long-running comparable household panel studies conducted elsewhere in the world, for example, have been able to avoid making such changes.1 A detailed transition plan was thus developed with both RMR and Nielsen that covered issues such as the strategy for communicating with sample members, knowledge and materials transfer, interviewer recruitment and the scope for RMR involvement in wave 8. It is difficult to distinguish the impact of the change in provider from other changes introduced at the same time, and notably the change in survey mode (see Subsection 3.2), but most available indicators suggest the change in provider has had minimal, if any, detrimental impacts on the survey (although it is perhaps too early to assess whether data quality has suffered). Most obviously, and as we have already seen, most response rates continued to improve in wave 9. We suspect that a major contributing factor to this good outcome is the relatively large number of HILDA experienced staff that moved with the project. One of our main concerns stemmed from the potential change in interviewer workforce. It has, for example, been consistently demonstrated, both in the HILDA Survey and in other longitudinal surveys, that interviewer continuity matters for response (Watson and Wooden 2009a). Actively pursuing the employment of experienced HILDA Survey interviewers was thus made a priority for RMR, something that was facilitated by both the casual nature of the interviewer workforce and the cooperation received from Nielsen. During the wave 8 interviewer training sessions, all interviewers were offered the opportunity to have their contact details passed over to RMR so that they could continue to work on the HILDA project. As a result, 113 face-to-face interviewers (92 per cent) agreed to have their contact details handed over, and of these 89 eventually worked on wave 9. Interviewer attrition was still high, with 34 per cent of all face-to-face interviewers being new interviewers. Nevertheless, this rate was similar to the rate experienced in wave 2. At the same time as we changed fieldwork provider, we also implemented a major change in the way the personal interviews were to be delivered, shifting from pencil and paper methods to computer-assisted personal interviewing (CAPI). The advantages that CAPI offers, both in terms of cost-effectiveness and data quality, are very attractive and thus the only issue for us was not whether to change, but when. Indeed, the shift to CAPI had been under consideration for a number of years, with a split sample trial of CAPI methods undertaken in conjunction with the wave 7 dress rehearsal. The benefits that CAPI offers are well documented and include allowing more complex routing, automated checking of responses for logical and internal consistency, delivery of more timely information for monitoring fieldwork progress, and the elimination of a separate data entry phase (see de Leeuw, Hox and Snijkers 1995). CAPI also better facilitates the feeding forward of information from one survey wave to the next. Again it is too early to assess what effects the introduction of CAPI may have had on the data, but the results of the wave 7 dress rehearsal trial were very encouraging. The evidence from that trial indicated that, relative to pencil and paper, CAPI was associated with unit response rates that were little different and lower rates of item non-response (Watson and Wilkins forthcoming). It was also reported that mode effects were limited to a few variables and were usually in the desirable direction (that is, subject to less social desirability bias). On the other hand, and unexpectedly, CAPI resulted in much longer interview times. This, however, seems to have been mainly a function of the use of laptops that placed a premium on keyboard and mouse skills. As a consequence, laptops were replaced in wave 9 by tablet computers that used a stylus rather than a mouse. It is also instructive that all of the longer running panel studies conducted overseas have had to go through a similar change, and none have reported evidence of any major data discontinuities (Laurie 2003; Schräpler, Schupp and Wagner 2006). There has been a gradual increase in the share of personal interviews administered by telephone. To reduce costs, the original HILDA Survey sample was clustered (see Wooden, Freidin and Watson 2002). Over time, the sample gradually unclusters, increasing the costs of reaching sample members. In some cases, where sample members move beyond the usual reach of our interviewer network, these costs become prohibitive and hence we are forced to resort to telephone. As shown in the last row of Table 3, the proportion of interviews undertaken by telephone has grown from just 0.5 per cent in wave 1 to more than 10 per cent in wave 8, before falling to 9.1 per cent in wave 9 (the result of a deliberate effort that year to restrict the proportion of interviews undertaken by telephone). Differences in survey mode, both over time and across sample units within waves, have the potential to affect consistency of responses. Telephone respondents, for example, do not have access to the same visual aids and cues that exist in the face-to-face interview. Very differently, telephone interviews might be less subject to reporting biases arising from the presence of others in the household (including the physical presence of the interviewer). We also know that telephone interviews are considerably shorter than face-to-face interviews—by 11 to 12 per cent (see Watson and Wooden 2009b)—suggestive of less considered answers, though this might be equally well explained by less time being spent on social interaction in telephone interviews. Whether such biases have any significant impact on the data is an issue currently under investigation, but preliminary findings are suggestive of little systematic variation in responses by mode (Watson and Wooden 2009b). Telephone interviewing is, however, associated with a much lower proportion of self-completion questionnaires returned. When personal interviews are conducted face-to-face the SCQs are handed directly to interviewees, with arrangements made for the interviewers to physically collect the completed questionnaire. Indeed, in many cases interviewers are able to take the completed SCQs away with them on the same day the interview is conducted (47 per cent of SCQs returned in wave 9 were completed on the same day as the personal interview). With telephone interviews, however, the SCQ has to be mailed to interviewees along with instructions to return the completed questionnaire in the post. As shown in Table 3, SCQ response rates have been falling more or less continuously since the HILDA Survey commenced. Table 3 also suggests that much of this decline is explained by the increase over time in the incidence of telephone interviewing. The SCQ response rate among telephone interview respondents averages only 63 per cent, whereas among face-to-face interview respondents it averages 92 per cent, although the trend in the latter is still distinctly downwards. Estimation of a pooled data probit model revealed, after controlling for interview wave as well as other interview and respondent characteristics, that telephone interview respondents are associated with an SCQ response rate that is 23 percentage points lower than otherwise comparable face-to-face interviewees (Watson and Wooden 2009b). The final administrative changes of note have been in the amount of financial incentive offered to sample members for participating and in the way those incentives are paid. In waves 1 to 4 all households were paid either $20 or $50 each year they participated, with the higher amount only being paid when interviews were completed with all in-scope household members. The payment was made by cheque delivered to households after the fieldwork for that household had been completed. In wave 5 the incentive was changed to $25 per completed personal interview, with a $25 bonus paid to households where all in-scope household members completed the personal interview. As noted earlier, we suspect this change, which substantially increases the incentive on offer in multi-person households as well as directly rewarding all household members rather than just the nominated household reference person, was responsible for the marked jump in response rates in wave 5, especially among new sample entrants and children turning 15. In wave 9 we made two further changes. First, the amount of the incentive payment was increased to $30 per head, with the household bonus also increasing to $30. Second, and more significantly, for interviews conducted in-person, incentives were no longer paid by cheque well after the interview date, but in the form of cash paid immediately following completion of the personal interview.2 Again, we suspect that these changes increased response rates above what they would otherwise have been. This is perhaps not so apparent in the response rate figures reported in Table 2, although we draw attention to the marked rise in the response rate among previous wave non-respondents. As already noted, although the main analytical advantages of longitudinal data derive from repeated measurement of the same variables, there a host of other reasons why content needs to be fluid (for example, shifting policy priorities, economic and social change). The HILDA Survey reflects this, with considerable new content being developed and included over its relatively short life. The demand for new content has been accommodated in part by more efficient questionnaire design and in part by placing added burden on respondents. Thus, new 'permanent' question sequences have been introduced into the individual interview questionnaire on work-related training (in wave 3, and extended in wave 7), on leave from work (in wave 5), and on caring activities (also in wave 5). More usual though is to accommodate new topics through rotating content across the different survey waves, and it is this that is the focus of discussion in this section. Those who want to know about the standard topic coverage should consult Wooden and Watson (2007). The HILDA Survey has always been designed with the idea of providing scope in each wave for asking questions on specific topics that will not be covered each year. The schedule for this rotating content is summarised in Table 4. As can be seen, the HILDA Survey now has four major topics that are converging on a 4-year cycle (coming into effect from wave 11). These topics cover household wealth (waves 2, 6 and 10); family formation and fertility (waves 5, 8 and 11, and thereafter moving on to a 4-year cycle); health (which was administered for the first time in wave 9); and education and skills (which is still in the planning stage and not yet confirmed, but nevertheless scheduled to be included for the first time in wave 12). In addition, there have been a number of 'minor' modules, either because they involve far fewer questions or because they are targeted at very specific subpopulations. The most significant example of the latter is the module on retirement, which is being conducted every 4 years starting in wave 3. Some of these minor modules are also being subsumed into the major modules. The modules on diet and health insurance were recently rolled into the wave 9 module on health, and the module on literacy and numeracy can expect to be next repeated in wave 12 as part of the proposed education and skills module. We now briefly turn to each of the four major modules. Of the four modules, the wealth module is the longest running and hence has already been the subject of much analysis. It seeks to provide estimates of total household wealth disaggregated by the type of asset or liability (see Headey, Marks and Wooden 2005). The scope of the wealth module was enhanced in wave 6 with the inclusion for the first time of questions on accounts payable. The level of disaggregation was also increased, with much more detail sought on different types of financial liabilities. We also sought to counter relatively high levels of item non-response by seeking banded estimates from respondents unable to provide a more precise estimate of the value of a specific type of asset or liability. The family formation and fertility module was implemented in part to provide Australian data that would provide comparability with the United Nations Generations and Gender Survey being implemented in many other countries. The HILDA Survey was seen as an ideal vehicle for this, given it was already collecting extensive information about families. This module thus sought complementary information about such things as recent pregnancies and whether they were intended; contraception methods; other factors influencing fertility decisions; and the recommencing of employment following childbirth. This module is also complemented by a series of questions about intimate non co-residential relationships, which in wave 8 were further extended through the inclusion of detailed questions about relationships with parents and siblings who were not co-resident. Given its central importance to quality of life, and the strong influence it has on economic opportunities and outcomes, there has always been significant content about individual health status in the HILDA Survey, especially in the SCQ. The health module, however, provides an opportunity to focus periodically on a wider array of health-related issues. Specifically, the module implemented in wave 9 covered the following topics: expectations about health; difficulties caused by health conditions/disabilities (previously included in wave 4); serious illness conditions; retrospective childhood health; private health insurance (previously included in wave 4); use of healthcare services; diet (previously included in wave 7); and the health status of, and use of health care services by children in the household. Like health, questions on education are included in the HILDA Survey every year. Indeed, there is a dedicated section devoted to recording education activity during the years preceding interview. Nevertheless, it is felt that that this is one area where the HILDA Survey collection could be enhanced. This was recognised in wave 7 when a series of questions about literacy and numeracy were included. As noted above, we would expect these to be repeated as part of this module. Other issues that are likely to be considered include the collection of more details about education histories (such as field of qualification and institution attended), the use of skills in the workplace, and the measurement of cognitive skills. One of the relatively novel features of the HILDA Survey is that, in addition to data collected by interview, all interviewees are also asked to complete a lengthy self-completion paper questionnaire. This questionnaire consists mainly of questions that are difficult to administer in a time-effective manner in a personal interview or that respondents may feel slightly uncomfortable answering in a face-to-face interview. Furthermore, the content of this instrument is much less stable, with many items rotating in and out of the instrument. The types of topics covered in most waves include health status; lifestyle behaviours, such as smoking, exercise and alcohol consumption; relationship quality and satisfaction; social interaction and support; time use; life events; financial stress; and work–family balance. In recent waves a considerable amount of new content has been included, facilitated in part by the expansion of the length of the SCQ in wave 5, from 16 pages to 20 pages. Included here are: height and weight, included for the first time in wave 6 and repeated every year since);3 the Kessler 10 measure of psychological distress, included for the first time in wave 7 and expected to be repeated every 2 years;4 personality, first included in wave 5 and repeated in wave 9;5 additional measures of alcohol consumption intended to quantify the frequency of 'excessive drinking'; measures of food consumption patterns and dieting behaviour, included for the first time in wave 7, but realigned to coincide in future with the inclusion of the health module in the interview component;6 a multi-item measure of community participation, included for the first time in wave 6 and to be repeated every 4 years (and discussed in more detail in Berry and Welsh 2010); and a household expenditure inventory, included for the time in wave 5 and extended in wave 6, which, together with expenditure items collected elsewhere in the HILDA Survey, such as housing, provides coverage of close to 80 per cent of total household expenditure.7 By necessity, the review provided here is very brief. Readers looking for more detail are thus advised to consult the online manual, available from the HILDA Survey website at . We also say nothing here about the uses being made of the data by researchers and policy makers, but note that a review of research (admittedly now already somewhat dated) can be found in Wooden and Watson (2007). Additionally, an extensive bibliography of publications that make use of the HILDA Survey can again be found on our website at .