Lies, Damned Lies, and Surveys
Dr. Anthony R. Artino, Jr.
Date: Wednesday, January 16, 2019
Time: 12:00pm to 1:30pm (feel free to bring a bagged lunch)
Locations:
- DHCC 2267 (host venue)
- IRC 305
- RJH CA 120
- NHSC 9-374
Remote:
- Additional locations are available. Please email ches.communications@ubc.ca to request an additional site.
Abstract
Mark Twain famously expressed his disdain for statistics when he said, “There are three kinds of lies: lies, damned lies, and statistics.” Beyond referencing his own difficulties with math, Twain was making the point that statistics can have persuasive power, even when used inappropriately. In short, statistics often are used to bolster weak arguments.
The same can be said of surveys—results from a poorly designed and poorly executed survey can have considerable persuasive power. Flawed survey results can send educators off searching for fixes to non-existent problems, negatively influence policy decisions, and fill the scientific literature with unsubstantiated knowledge claims that take years to correct, if they are ever corrected.
In this talk, I discuss the challenge of designing and implementing high-quality, self-administered surveys. Using empirical data and a series of examples from health care, education, and politics, I describe how survey results can be used to shape science and policy, often inappropriately. In addition, I provide insight into the cognitive processes that guide how respondents make sense of survey questions, and I demonstrate what happens when these cognitive processes are ignored by educators and researchers. Next, I propose several design principles meant to improve surveys and introduce procedures that front-load the survey-design process by focusing heavily on item development and pretesting. Finally, I end by discussing the future of surveys as an evaluation and research methodology. Taken together, the goal of this talk is to demonstrate that Mark Twain’s lament, when applied to surveys, is often correct—but it certainly doesn’t have to be.
Biography
Anthony R. Artino, Jr., Ph.D., is Professor and Deputy Director for the Division of Health Professions Education at the Uniformed Services University of the Health Sciences in Bethesda, Maryland. He received his Ph.D. in educational psychology from the University of Connecticut and has accrued over 21 years of leadership experience as a Captain in the U.S. Navy. In his current role, he teaches graduate courses, conducts research, mentors students and junior faculty, and provides administrative leadership. As a researcher, he has been the Principal or Associate Investigator on more than $8 million in funded research. In this role, he collaborates with teams of scientists trained in medicine, psychology, epidemiology, and English. His research teams study various aspects of human motivation and learning in health professions education. Dr. Artino is Deputy Editor for the Journal of Graduate Medical Education and Assistant Editor for Academic Medicine. He has published 120 peer-reviewed journal articles and given lectures at international meetings around the globe.
Accreditation:
The University of British Columbia Division of Continuing Professional Development (UBC CPD) is fully accredited by the Committee on Accreditation of Continuing Medical Education (CACME) to provide study credits for continuing medical education for physicians. This event is an Accredited Group Learning Activity (Section 1) as defined by the Maintenance of Certification Program of the Royal College of Physicians and Surgeons of Canada, and approved by UBC CPD. You may claim a maximum of 1.5 (x 10 sessions) hours (credits are automatically calculated). This Group Learning program meets the certification criteria of the College of Family Physicians of Canada and has been certified by UBC CPD for up to 1.5 (x 10 sessions) Mainpro+ credits. Each physician should claim only those credits he/she actually spent in the activity.
