You can find the link to the survey here:
So let’s begin… how does this particular survey fail in methodology…
1. Bias is a systematic error that can prejudice the findings in some way
a. In the introduction letter the authors suggest that “The questions included in this survey have been put together to address some issues that the white paper either ignored or did not address.”
i. How does this influence the reader? Does this suggest that the NATA and those that served on this committee did not appropriately represent you? How would that make you feel? Angry? Disoriented? Unappreciated? Might this impact the findings?
2. Sampling bias occurs when a sample does not accurately reflect true representation within the target population
a. Also in the introduction, the authors tell us how they hope to distribute this survey… “We are distributing this survey to some members via email. But we are going to rely mainly on the power of the social media- Facebook, Twitter, professional blogs, etc., so please send the site of this survey or to as many of your athletic training colleagues or professional friends as possible including educators, clinical staff, and even undergraduate, graduate and doctoral students over e-mail and social media outlets.”
i. Does this seem systematic to you? What kind of respondents will you get if you focus solely on social media? Will you access some of the older, more seasoned athletic trainers in the profession? Will you only entice the youth… those of whom are 50% likely to leave the profession in the next 5 years? And how will that impact the findings?
3. Forced responses within a survey violate your rights as a respondent… you should have the right to choose which items you respond to or not. This falls under the protection for persons under federal guidelines and the Belmont Report of 1979.
4. The orientation of responses in this survey lists strongly agree from left to right suggesting that upon downloading the data a strongly agree response will be indicative of a “1” and a strongly disagree will be indicative of a “5”. Did the authors mean to provide the results backwards? That will certainly make data analysis difficult!
5. The orientation of the responses are also listed horizontally; however, methodologists in survey research, particularly Dillman and colleagues have identified that vertical responses are likely to yield the most accurate results. Apparently how we read things MATTERS!
6. Double barreling items occurs when a respondent is asked to evaluate two concepts within the same item.
a. “Advancing the entry-level degree for professional practice from the current Bachelors degree to a Masters degree will be more likely to improve patient outcomes and ensure longevity of the profession of athletic training.”
i. Here the authors ask that you rate your level of agreement (well actually, they don’t give you any directions at all, but they assume you will rate your level of agreement) with two concepts: 1) that an entry-level masters will improve patient care and 2) that it will ensure longevity in the profession. Are those two things the same? Or different? Can you have opposing views on each? Should you be able to?
7. Items that ask about factual things like the presence of evidence cannot be perceived… they are factual.
a. “There is clear evidence in the athletic training research journals that suggests improved patient outcomes are likely to result from transitioning to a Masters degree as our entry-level professional degree.”
i. Is this true or is this untrue? Furthermore… the use of the word “clear” to describe the evidence also incites confusion in the reader… is it clear to me? Or is it clear to you?
Need I go on? I understand people want their voices to be heard! But don’t do it by using my responses to a poor survey to manipulate my perceptions. This is a poorly constructed method of quantitative survey research… if you want to ask the question… ask the question… do you agree or disagree with the NATA Whitepaper finding… I AGREE and I will continue to support the progress of the profession and my national organization because I have faith in their due diligence. I believe in the NATA, the BOC, and the CAATE in that they have the best interest of the profession as the driving force…
Lindsey E. Eberman PhD, LAT, ATC