1/
Okay @IranahPris and #naccb2022, here’s a not so short 🧵

But let’s try not to open Pandora’s Box

***Note: @ConSocSci 🤖 retweets #HumanDimensions #ConSocSci #MarSocSci #EnvSocSci #EnvHum #EnvHis #EnvPsych #ConPsych #SocEcoSys. Original content and replies will be limited.
@IranahPris 2/
If you must do a survey, if it’s the appropriate method, a good starting point is to remember people are just people, not your guaranteed respondents; treat them like people and design your survey accordingly
@IranahPris 3/
And the “survey” includes all component of research design, sampling, etc., not just your instrument/questionnaire. DO NOT start with the qnr, start with your purpose, aims, questions, goals
@IranahPris 4/
But remember to ask, “Do I really need a survey?”

A survey is the default, go-to method but is more limited and complicated than expected, and we don’t talk about this enough

Will a survey get you what you need or will it just get you some data for data sake? Image
@IranahPris 5/
Survey deskilling is also a serious issue with so many DIY platforms Google Forms, Qualtrics, QuestionPro, SurveyMonkey, Alchemer, KoBo

These focus you on implementation & data collection rather than good research design, sampling, & qnr design Image
@IranahPris 6/
If you go no further on this thread, read and follow this:

aapor.org/Standards-Ethi…

And remember: "The quality of a survey is best judged not by its size, scope, or prominence, but by how much attention is given to dealing with the many important problems that can arise."
@IranahPris 7/
Important to remember that we must evolve & improve practices with the survey climate and how it changes, which has changed significantly over just the past 10yrs

What worked in 1971 will not work now. What works in US may not work elsewhere

*aapor.org/Education-Reso…
@IranahPris 8/
In the US, good survey case studies are the 1936 Literary Digest prez poll, the 1948 Gallup prez poll, and 2016 prez polls.

*researchgate.net/project/Sociol…

*pewresearch.org/fact-tank/2016…
@IranahPris 9/
Quick aside…

“Survey Climate” is a term defined by Loosveldt & Joye 2016, building on the work of Lyberg & Lyberg 1991; Groves & Couper 1998

researchgate.net/publication/30…
@IranahPris 10/
It is also important to recognize that people are more distracted than ever and receive almost constant survey requests for their “feedback”, "input", or "experience"

*doi.org/10.31235/osf.i…
@IranahPris 11/
Any product we purchase linked to email or phone# will request feedback

In app ratings, receipts, grocery checkouts, pollsters, kiosks in the public bathroom, and even meeting schedulers are feedback requests

surveystance.com/airport-feedba…
@IranahPris 12/
Then throw in research-focused surveys from universities and organizations and the survey commons gets crowded

researchgate.net/publication/26…
@IranahPris 13/
Assume at your own risk. Just don’t assume:

1. Your audience is as interested in a topic as you
2. Has anything to gain by taking your survey
3. Can distinguish your survey from marketing

…It is your job to make all this clear Image
@IranahPris 14/
Don’t borrow too much language or copy/paste from other surveys. Customize to your audience, context, & purpose

You must develop communicative language that will gain their attention, spark their interest, & legitimize your work and its benefits to them
@IranahPris 15/
Avoid pitfalls & make your survey design and communications engaging & salient to your audience.

Humanize it.

Use introspection, self-reflection, & internal assessment of your own survey research methodology and practices

This is essential.
@IranahPris 16/
Recognize you most likely aren’t a survey expert; most in conservation, natural resources, sustainability don’t have adequate training

Recognize & admit your limitations, ignorance, or arrogance. Ask for help, ask for a consult, use your network, build a team.
@IranahPris 17/
Another quick aside…

This goes for all aspects of social research design, theory, methodology, methods, analysis, and reporting, not just a survey

If you aren’t an expert, have limited experience, or don’t know, don’t be over-confident.

*researchgate.net/publication/33… Image
@IranahPris 18/
People often worry about response rate (RR), which is a major issue, but only one of many. Response is 1 of 4 cornerstones, the other 3 are sampling, coverage, measurement

*joophox.net/papers/SurveyH… Image
@IranahPris 19/
The survey invite, follow-up requests, how you design your survey flow and structure, among many things will effect RR

*Weisberg, H. F. (2009). The Total Survey Error Approach: A Guide to the New Science of Survey Research. University of Chicago Press. Image
@IranahPris 20/
7 influences on RR: (1) mode, (2) sponsor, (3) response task, (4) incentives, (5) response request structure, (6) communication content, and (7) attributes potential respondents
@IranahPris 21/
…8 is number of “survey” requests people get but hard to know

…9 is some people won’t respond regardless; see de Leeuw et al Survey Attitude Scale

*rdcu.be/cR0Ol
@IranahPris 22/
Reflect on your survey research practices…

Are you current with tactics that works for audience?

Does your qnr design that make sense for audience?

Are contact and response modes appropriate for audience and context?

Does mixed-mode, web-push, email augment make sense?
@IranahPris 23/
Also ask if your audience is over-surveyed? Is your survey design likely to cause burden and fatigue because of length and complexity. Could your survey be perceived as invasive, lengthy, or useless?

*Pro-tip: it’s probably too long
@IranahPris 24/
A pre-notification letter or formal survey invite is your first point of contact with your audience (sometimes it is us if we do onsite or drop-off/pick-up)

Take it seriously.
@IranahPris 25/
Know why YOU are doing the survey. Really. It sounds simple but is often overlooked, assumed, or forgotten as you distract yourself with sampling and question development
@IranahPris 26/
Know your audience. Who are they to you and your topic/issue? Why are you asking them and not someone else?
@IranahPris 27/
Pilot and pretest. Always.

Bonus: cognitive interviews and similar methods are helpful for both the questionnaire and invitation to assure your intended meaning is what a potential respondent interprets
@IranahPris 28/
Social exchange theory (SET) is a good starting point for invitation letters. Or use another communication theory and principles that will drive and customize how you present and frame the survey purpose and they language you use in your invitation.
@IranahPris 29/
SET is recommended by Dillman, Smyth, & Christian in “Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method” (2014)
@IranahPris 30/
If his book is #tldr, Don is a fan of “How to Conduct Your Own Survey” and the “Savvy Survey Series” from Glenn Israel @UF_IFAS

edis.ifas.ufl.edu/entity/topic/s…

wiley.com/en-us/How+to+C…
@IranahPris @UF_IFAS 31/
There are tons of established professional organizations and groups that can be resources

Groups like WERA-1010 have been discussing and working on these issues for over 25 years

nimss.org/projects/view/…
@IranahPris @UF_IFAS 32/
If this thread has miscommunicated anything, it's probably b/c it was written quickly

Some highly recommended reads and follow-ups are:
@IranahPris @UF_IFAS 33/
Guides from @AAPOR (or @ESRAsurvey). Seriously.

aapor.org/Education-Reso…

aapor.org/Standards-Ethi…

https://t.co/x3QLtIDVGO
@IranahPris @UF_IFAS @AAPOR @ESRAsurvey 34/
Pew Research Center @pewresearch has a good YouTube playlist and tons of other resources and examples:

youtube.com/playlist?list=…
@IranahPris @UF_IFAS @AAPOR @ESRAsurvey @pewresearch 35/
Research and books by Robert Grove, Don Dillman, Jon Krosnick, Jolene Smyth, Sharon Lohr, Mick Couper, Judith de Leeuw are good places to start
@IranahPris @UF_IFAS @AAPOR @ESRAsurvey @pewresearch 36/
And check @AASROorg if a nearby university has a survey research center or survey consulting

aasro.org

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Conservation Social Sciences

Conservation Social Sciences Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!

:(