,
14 tweets,
3 min read
Read on Twitter
As pre-registration becomes more popular, I've always wondered - if we can pre-register our hypotheses and analytical plan, then why don't we include our code? Today I decided to be as open as possible in my pre-registration and did just this. It's super easy, so here's how: 1/n
I started by drafting up the typical pre-reg (I personally like the aspredicted form on OSF). I filled it in as usual, including a detailed description of my analyses (Q5 on the AP form). I put a note indicating that an INITIAL draft of the code will be in the project folder. 2/n
Now, you might be wondering, how can you draft your code if you don't have a dataset to test it? If you use Qualtrics, it couldn't be easier. To streamline the data preparation phase, make sure to label the variables in your survey ahead of time... 3/n 
If you use a matrix-style question, you'll have to do a little bit of editing directly in the dataset later, but at least you won't have a completely unidentifiable dataset when you download the file (e.g., VAR001, VAR002...). 4/n
Once you've named all your variables, Qualtrics has a (new?) function that allows you to generate test responses (i.e., it basically simulates a bunch of fake data for you), which is what you will need to start writing and testing your code. 5/n
After you run the test responses, the fake cases will appear in the "Data & Analysis" tab, just like real responses. Now you're ready to download the data file and get going! No fancy gimmicks or coding...it's super simple. 6/n
As I mentioned before, you might need to do a little bit of editing to your variable names (as usual with Qualtrics), so now is the time to do that. I then saved this "finalized" simulated file...this way when I download my real data, I can just copy the variable names over 7/n
As usual with data prep/analysis, I wrote out all of my var computations, recodes, data exclusions, etc. in SPSS (I personally do my data prep there and import the fine file to Mplus for mlm). Because you have the simulated file, you can test your code, adjust it, etc. 8/n
Once the data prep phase was done, I then wrote out my primary analyses in mplus (which mapped directly on to what I wrote in my pre-reg). I uploaded everything to OSF (along with the materials, etc.) and that's it! 9/n
Normally I would do all of this once the data came in (sometimes weeks/months later), but since the prereg was super fresh in my mind, all of these steps were much easier (including messing with these new Qualtrics functions, I spent maybe an hour on all of this). 10/n
Note: In my pre-reg, I said I included an INITIAL draft of my code. This was a fail safe approach in case I need to make minor tweaks. After all, you are never strictly chained to your pre-reg, but I figured this might make some people more comfortable if they need to adjust 11/n
But essentially, once my data do come in, everything will be ready to go and the analysis phase will be so much faster/easier. This approach simply moves the back end of a project up front... 12/n
...but I would go as far as to argue that this actually SAVES time because you won't have to waste time trying to remember what you intended to do when you wrote your plan months before...you already prepared everything when it was fresh (i.e., during the design phase). 13/n
Now, I'm sure there are fancier ways to do all of this (e.g., simulating data in R or whatever), but I just wanted to show that you can use the same basic tools you already use to streamline open science & most importantly improve your workflow. /end
