«A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy (Survey Methodology) in The University of ...»
Vocal Characteristics, Speech, and Behavior of Telephone Interviewers
Jessica Susan Broome
A dissertation submitted in partial fulfillment
of the requirements for the degree of
Doctor of Philosophy
in The University of Michigan
Professor Frederick G. Conrad, Chair
Professor Susan Brennan, State University of New York at Stony Brook
Professor Norbert Schwarz
Research Scientist Steven Heeringa
Acknowledgments This work would not have been possible without funding from several sources. I was honored to receive the Harvey G. and Joyce H. Behner Graduate Fellowship for the 2011-2012 academic year. The Charles Cannell Fund in Survey Methodology provided generous support, as did the A. Regula Herzog Young Investigators Fund and the Margaret Dow Towsley Scholarship. The original project, which included transcription and coding of contacts used in my research, was funded by the National Science Foundation (grant # SES-0819734 and # SES-0819725), the University of Michigan Survey Research Center, and the Department of Communicative Sciences & Disorders, Michigan State University.
My chair, Fred Conrad, has been unwavering in his support of me, and in his belief in the importance of this work. I was grateful to have such a helpful and diverse committee. Norbert Schwarz inspired me with his enthusiasm and creative ideas. Susan Brennan offered a refreshing perspective from outside the Survey Methodology world. Steve Heeringa’s support as the lone statistician on my committee helped to make my work better.
Other faculty who had a hand in this project include Bob Groves, whose vision led to the creation of the unique dataset used in my dissertation; José Benkí, who offered crucial guidance on not only using Praat but also understanding concepts related to speech and language; Frauke Kreuter, who brought a unique perspective to the original “Voices” project and was consistently encouraging;
and Jim Lepkowski and Roger Tourangeau, who nurtured my research ideas when they were still in an embryonic stage.
The Program in Survey Methodology administrative staff make everything happen. My sincere thanks to Jill Esau, Patsy Gregory, Nancy Oeffner, Annmarie Thomas, Jodi Holbrook, Elisabeth Schneider, and Sumi Raj.
Thanks to all my colleagues at both Michigan and Maryland, past and present, for encouragement and advice, especially Brady West, Matt Jans, Rachel Levenstein, Jeff Gonzalez, and Chan Zhang.
This work would have been much harder without Dave Childers from CSCAR;
Jacki Spear and Steve Wisniewski from Lightspeed Research; Wil Dijkstra; my tireless coder Dylan Vollans; and Kristen Holt-Browning, whose editing eye proved invaluable. Huge thanks to Pete Batra, Joe Matuzak, and all 3,476 of my respondents!
ii My heart is full of gratitude for so many people who never stopped believing in me: Jennifer Scott, Heidi D’Agostino, Sarah Peterson, Lenore Robison, Manfred Kuechler, Pam Stone, Robert Gay, Dana Levin, Jay Pearson, Jeffrey Hudson, Lily Baldwin, Jenny Smith, Diane Wohland, Eleanor Stanford, Dan Imaizumi, Corin Hirsch, Damian Beil, Nicole Ray, Sam Wotring, Alisha Deen-Steindler, Jason Steindler, Guy Oliveri, Jared Pinnell, Susan Gottesfeld, Dean Hadin, Todd Tesen, Natalie Donnellon, Fred Bigliardi, Christine Young, Christian Breheney, Sheila Donnelly, Melina Shannon-diPietro, Chris Weiser, Kate Davidson, Christine Koch, Anna Daigle, Amanda Box, and most of all, Mom and Dad, Edythe and Johnny, Michael and Paul, thank you for everything you’ve done, always and in all ways.
iii Table of Contents Acknowledgements
List of Figures
List of Tables
List of Appendices
Chapter 1: A Proposed Conceptual Model for the Association Between Interviewers’ Speech and Vocal Characteristics and Success
1.1. Problem Statement, Significance, and Research Aims12
1.2. Overview of Studies
1.3. Literature Review
1.3.1 Vocal Characteristics and Survey Response
1.3.2 Formation of First Impressions
1.3.3 Voice and Personality Characteristics
1.3.4 Tailoring and Responsiveness
Chapter 2: Examining the Role of First Impressions
2.2. Data and Methods
2.2.1 Listeners’ Study: Questionnaire Development
2.2.2 Selection of Contacts
2.2.3 Listeners’ Survey: Editing of Contacts
2.2.4 Listeners’ Survey: Data Collection
2.2.5 Practitioners’ Survey: Questionnaire Development.............. 39 2.2.6 Practitioners’ Study: Sampling
2.2.7 Practitioners’ Study: Data Collection and Respondent Overview
2.3.1 Listeners’ Survey: Respondent Overview
2.3.2 Listeners’ Survey: Descriptives
2.3.3 Dimensions of Person Perception: Warmth and Competence
2.3.4 Judged Likelihood Ratings: Association with Rated Characteristics and Actual Outcome
2.3.5 Characteristic Ratings as Predictors of Actual Contact Outcome
2.3.6 Importance of First Impressions: Comparison Between Listeners’ and Practitioners’ Surveys
2.3.7 Relationship Between Vocal Characteristics, Ratings, and Contact Outcome
2.3.8 Vocal Characteristics: Contrast with Practitioners’ Study..... 58
2.5. Limitations and Suggestions for Future Research
Chapter 3: Interviewer Responsiveness
3.2. Data and Methods
3.3.1 Prevalence of Answerer Concerns
3.3.2 Classification of Answerer Concerns
3.3.3 Conversation Starters
3.3.4 Interviewer Responsiveness Scoring
3.3.5 Addressing Specific Concerns
3.3.6 Predictions of Contact Length and Outcome using Responsiveness Score and First Impressions
Chapter 4: Conclusion
4.1.1 Review of Methods
4.2. Summary of Findings
4.2.1 Findings on Distinction of Person Perception Dimensions.
4.2.2 Associations Between First Impressions, Vocal Characteristics, Predicted Outcomes and Actual Outcomes....... 110 4.2.3 Contrast Between Practitioners’ and Listeners’ Studies..... 111 4.2.4 Interviewer Responsiveness
4.2.5 Scriptedness and Responsiveness
4.3. Recommendations for Telephone Interviewing Practice Based on these and Other Studies
4.4. Recommendations for Future Research
4.4.1 Rewording to Clarify Contrast between Listeners and Practitioners
4.4.2 Exposure to Longer Clips of Interviewer Speech................ 120 4.4.3 Content Analyses of Interviewer Speech
4.4.4 Interviewer-level Analyses
4.4.5 Analyses of Variation in Ratings by Rater Characteristics.. 122 4.4.6 Analyses by Different Types of Scheduled Callbacks........ 123 4.4.7 Distinguishing True from False Concerns
4.4.8 Collecting Interviewer Ratings in a “Live” Situation............ 125 Appendices
Table 1.1: Summary of Studies on Interviewer Vocal Characteristics Table 2.
1: Rated Interviewer Characteristics Table 2.2: Audio File Groupings Table 2.3: Audio File Groupings by Study Table 2.4: Audio File Groupings by Outcome Table 2.5: Description of Ratings Table 2.6: Factor Loadings Table 2.7: Correlations Between Ratings of Positive Interviewer Characteristics Table 2.8: Factor Loadings When Likelihood is Included Table 2.9: Predicting Likelihood Ratings Table 2.10: Practitioners’ Ratings of Importance to Interviewer’s Success Table 2.11: Emphases in Interviewer Training Table 2.12: Predicting Ratings with Interviewer Speech Rate (controlling for exposure length and accounting for clustering by interviewer) Table 2.13: Predicting Ratings with Interviewer Fundamental Frequency (controlling for exposure length and accounting for clustering by
Table 2.15: Importance of Vocal Attributes in Hiring Decisions Table 3.
1: Inter-coder Reliability Measures Table 3.2: Proportion Contacts with One or More Concerns Expressed by
Table 3.3: Concern Rates by Outcome Table 3.
4: Concerns Expressed in Agree Contacts, by Call Number Table 3.5: Distribution of Call Number by Outcome Table 3.6: Prevalence of Concerns by Outcome in Contacts Containing any
Table 3.7: Proportion Contacts with Conversation Starters by Outcome Table 3.
8: Conversation Starter Rates by Outcome Table 3.9: Mean Responsiveness Score by Outcome Table 3.10: Decomposing Responsiveness Scores Table 3.11: Concerns Addressed by Outcome Table 3.12: Distribution of Contacts Containing Statements of Disinterest by
Table 3.13: Model Predicting Log Odds of Agree Table 3.
14: Model Predicting Contact Length Table 3.15: Agree Rates by Scriptedness/Responsiveness Quadrant
Appendix 1: Details on Preliminary Studies Appendix 2: Listeners’ Survey Appendix 3: Sampling Structure Appendix 4: Practitioners’ Survey Appendix 5: Email Request Sent to Practitioners Appendix 6: Details on Reliability of Initial Move Coding Appendix 7: Move Codes Appendix 8: Tailoring Study: Coding Scheme
Telephone survey interviewers vary widely in their success at persuading potential respondents to participate in phone surveys. This persuasive act can be viewed in two stages: first, the initial impression the interviewer makes on potential respondents, or telephone “answerers”; and, assuming that the contact continues past this stage, the interviewer’s ability to respond concerns expressed by answerers.
I report results from two studies looking at these stages in a corpus of audiorecorded telephone survey introductions, as well as a small study of research practitioners’ opinions on related issues. Initial impressions of telephone interviewers are assessed by asking Web survey respondents to listen to the initial seconds of an interviewer’s recorded introduction (typically “Hello, this is ___ and I’m calling from ___ about our study on ___”) and to rate the interviewer on twelve personal and vocal characteristics, including “professional,” “competent,” “friendly,” and “scripted.” The only characteristic that was predictive of contact-level success was scriptedness, which was negatively associated with success. This finding was in marked contrast to practitioners’ view that the first impression an interviewer gives to a sample member is important to the interviewer’s success, while his or her scriptedness matters little.
Interviewers’ responsiveness to concerns expressed by answerers is assessed through analysis of the entire introduction. These introductions have
specific concerns (for example, “I don’t have time”), and the interviewer’s response to the concern (for example, “We can call you back at a more convenient time”). Coding also captures “conversation starters” by answerers, including questions addressed to interviewers or any conversation peripheral to the task at hand, and interviewers’ responses to such utterances.
Findings support the hypothesis that interviewers who respond promptly and appropriately to answerers’ concerns and conversation starters have more success in persuading answerers to either participate immediately or defer participation (rather than refuse outright). Responsiveness over the course of the introduction can also make up for an initial perception of an interviewer as overly scripted.
1.1 Problem Statement, Significance, and Research Aims Nonresponse to telephone surveys has the potential to bias survey estimates (Groves, Presser, and Dipko 2004), which in turn can have policy and programmatic implications. It has been demonstrated (Oksenberg and Cannell
1988) that some telephone interviewers have higher response rates––that is, more success recruiting sample members to participate––than others. Identifying vocal characteristics and techniques of successful telephone interviewers promises to have a potentially broad impact on data quality, by allowing for more targeted screening and training of interviewers with the aim of reducing nonresponse, which can in turn benefit any disciplines in which telephone survey data is used.
Literature from both survey methodology (Oksenberg, Coleman, and Cannell