Profile picture
Todd Wallack @TWallack
, 24 tweets, 7 min read Read on Twitter
Since it's #FOIA Friday, I wanted to say a few words about public records and persistence. And explain how it took three years to research an article I wrote with @GlobeKayLazar bostonglobe.com/metro/2018/07/…
A few years ago, I was working on some stories about police in Massachusetts. One was about the large number of crashes involving police cruisers. bostonglobe.com/metro/2014/10/…
Another was about police who were accused of driving drunk while off-duty. bostonglobe.com/metro/2014/12/…
Getting the records was difficult. We are still fighting @MassStatePolice & @bostonpolice in court to get records of police who were caught drunk driving. (A Superior Court judge ruled in our favor, but @MassAGO appealed)
bostonglobe.com/metro/2015/05/…
Anyway, during the course of reporting the stories, I was curious how State Police spent their time. So, I asked the agency for five years' of data from their public logs.
But State Police demanded $20,625 to produce the data, mostly to manually review the data just to make sure it didn't accidentally include any confidential data in the public logs. $20,625 estimate
A separate state law says public police logs are supposed to be available for free, but the state Supervisor of Records ruled M.G.L. ch. 41 §98F. only applies to local and college police.
And the Supervisor of Records found that the $20,625 fee estimate was "reasonable." documentcloud.org/documents/2643…
Did I mention the @MassStatePolice won a national award for secrecy? And the former Supervisor of Records was a finalist? ire.org/blog/ire-news/…
But I eventually figured out the State Police would give me a single day's log for free if I filed a public records request.
So, I started asking for the daily State Police log. Every morning. Since January 1, 2016.
That adds up. Over the course of the past 2.5 years, I have filed more than 900 public records requests. Just for the daily police log. excerpt from state police FOIA log
I actually wrote a computer program in Python to automatically file the request for the daily police log each morning. (It currently emails the request at 9:10 am each day.)
Just so you know, I tried to figure out if there was an easier way first. I first asked the State Police if there was some easier way to share the logs. I got no response.
The logs are not in a database or spreadsheet. They're ugly PDFs. That look like this.
So, I wrote another program to go through all 900+ PDFs, grab key pieces of data, and turn them into spreadsheets. So I could analyze all that data. #ThanksPython
The data showed some stations are less busy than others, especially the stations on islands - Nantucket and Martha's Vineyard.
Nantucket listed on average of just over 1 incident a day on its public police log.
Nantucket was so sleepy that the phone number on its web site was missing a digit for at least nine months and no one seemed to notice (until I tweeted about it).
Meanwhile, I also asked for copies of any reports on barracks closings. State Police fought my request, but eventually gave me recommendations from a 2011 study.
The study recommended a number of stations be closed or consolidated, including Nantucket and Martha's Vineyard. (The state hasn't closed any stations since getting the report.)
So it took three years. It took a little programming. But we eventually got the data. And we got a good story. bostonglobe.com/metro/2018/07/…
But I am still baffled why @MassStatePolice would prefer to process 900+ public records requests for daily police logs, instead of posting them online or sharing them with reporters automatically.
Or simply running a database query and giving me all the data I originally asked for years ago. (The end.)
Missing some Tweet in this thread?
You can try to force a refresh.

Like this thread? Get email updates or save it to PDF!

Subscribe to Todd Wallack
Profile picture

Get real-time email alerts when new unrolls are available from this author!

This content may be removed anytime!

Twitter may remove this content at anytime, convert it as a PDF, save and print for later use!

Try unrolling a thread yourself!

how to unroll video

1) Follow Thread Reader App on Twitter so you can easily mention us!

2) Go to a Twitter thread (series of Tweets by the same owner) and mention us with a keyword "unroll" @threadreaderapp unroll

You can practice here first or read more on our help page!

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just three indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member and get exclusive features!

Premium member ($3.00/month or $30.00/year)

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!