Methodology
To be eligible for the sample, members need to have been a health plan member in the 4th quarter of the year prior to the survey, a current member at the time of the survey, be able to read in English, not be in a Skilled Nursing Facility or in hospice, and not have dementia, Alzheimer’s disease, or other types of cognitive disability.
Survey sample selection: 1993 through 2014/2015
Because the survey results were designed to be used to create profiles of KPNC’s 19 medical center service populations (MCSPs) as well as the region, separate age-gender stratified random samples were selected from among each MCSP. Members are administratively assigned to an MCSP based on where they receive or would be expected to receive most of their primary care. MCSPs are not the same as geographic catchment areas.
Most MCSP samples were comprised of 2,200 members, with set numbers of women and men selected from five age groups (20-44, 45-64, 65-74, and 75-79, and 80+). To ensure that adequate numbers of racial and ethnic groups were represented, we selected slightly larger samples (approximately 2,500 members) from our Oakland, Hayward, Richmond, San Francisco, and Vallejo medical center service populations which have our most ethnically diverse service populations. Members aged 65 and over were also oversampled for each MCSP so that we could learn as much as possible about the characteristics of this group that is a major consumer of health care.
Survey sample selection: 2017 and 2020
Because we needed to reduce our sample size, we changed our sampling design to obtain information from sufficient numbers of Black, Latino, and Asian members to study race/ethnic disparities in sociodemographic, health, and health care characteristics. Taking advantage of the greater number of adult members with race/ethnicity data in our electronic health records, we selected age-sex stratified random samples of members from our four largest race/ethnic groups: non-Hispanic White, Black, Latino, and Asian/Pacific Islander. We restricted the sample to those with spoken and written English language preference due to funding constraints. We get information about our Latino members who prefer to speak Spanish and Chinese members who prefer to speak a Chinese dialect through other targeted surveys and by analysis of electronic health data
Data collection methods
The majority of respondents participate using a self-administered print questionnaires mailed to their home. We usually begin to mail out questionnaires in mid-April, followed an additional mailing 8-10 weeks after the initial mailing to enhance the final response rate. Beginning in 2005, respondents were also able to complete the survey online using a URL printed in the survey materials and/or a link sent to them in an email close to the time of the print mailing. The survey was also administered by phone interview upon request.
For the survey cycles through 2015, separate questionnaires were used to collect data from women aged 20-64y, men aged 20-64y, and women and men aged 65y and over. The questionnaire for men aged 20-64y was virtually identical to that for similarly aged women but excluded questions or items specific to women and includes some items specific to men’s health. The questionnaire for men and women aged 65 and over was slightly longer than the women’s questionnaire, containing all of the core items plus additional items on functional health, use of health-related supportive services, and usual mode of transportation. For the 2017 and 2020 survey cycles, we used one questionnaire for both men and women aged 25-64y and one for men and women aged 65-90y.
To encourage participation, respondents were automatically entered into a gift card drawing held toward the end of the survey year. There were no individual financial incentives offered for participation.
Survey response rate
Information about overall and age group-gender specific survey response rates for each survey year is found in the introductions of the region and medical center service population profile reports for each survey year. The overall response rate for the most recent (2014/2015) survey was 38 percent. As with other similar surveys, the overall survey response rate has been declining over time (55 percent in 1993, 45 percent in 2005, 41 percent in 2008, 39% in 2011, 37.9% in 2014/2015, and 31.7% in 2020; because of an experimental sampling design, the response rate in 2017 was only 22.8%). Response rates have always been lower for the 20-44 and 45-64 age groups than for members aged 65 and over, who through 2011 had response rates hovering around 70 percent but in 2014/2015 dropped to approximately 65 percent and in 2020 to 53.4%.
Assignment of post-stratification weighting factors
For survey cycles through 2014/2015, we assign respondents a weighting factor based on the number of members of their sex and age group (5-year intervals) in their medical center service population. For example, data for 35-year-old and 50-year old male respondents drawn from the Oakland medical center service population might be weighted to represent 241 and 183 men, respectively, whereas data for male respondents of these ages drawn from the Kaiser Permanente-Sacramento service population might be weighted to represent 344 and 386 men, respectively. We use the weighted data, which reflect the actual age, gender, and geographic composition of the membership, to estimate the demographic characteristics and prevalence of different health conditions and health risks in the region and service populations. For the 2017 and 2020 survey cycles, survey respondents were assigned post-stratification weighting factors that would result in prevalence estimates based on a population that closely approximated the underlying age x sex x racial/ethnic (White non-Hispanic, Black, Latino, Asian/Pacific Islander, Other) composition of adults in the KPNC membership whose primary language was English in 2016 and 2019, respectively.
Linkage of survey data to other sources of information
Member Health Survey data are often linked at the individual level to Kaiser Permanente administrative and clinical information on health and health care use for research purposes. Survey data can also be linked to Census-derived and other geographic information data based on street address or zip code. The same health plan and geographic information can be obtained for non-respondents. All data linkage is done in a way to protect the confidentiality of people in the survey sample. We are pre-approved by our IRB to link certain data at the time of the survey as outlined in our study application. If we want to link additional data, we must request IRB approval to do so.
