,
14 tweets,
3 min read
Read on Twitter
Never been a fan of framing decisions as being "data-driven." "Data-informed" is more apt, IMO. We also need to be clear on the limits of enterprise level "big data." /thread I suppose. insidehighered.com/quicktakes/201…
That "big data" can tell you "what" is happening, e.g., melt is higher, alumni are donating less, students who get a C in X course are at risk for a D in Y course, but it does not and cannot tell you "why" that thing is happening.
There is no magic point where "more" data suddenly reveals an answer to a question the data isn't designed to gather information about. Big data will also never be predictive down to the individual level. It's probabilities, aggregates.
Trying to apply aggregated data to the choices of individuals without understanding the underlying "why" of the phenomenon risks some very bad consequences.
Aggregated data can predict how many bags of chips will be purchased over Labor Day weekend. It will not tell us whether or not a particular shopper is going to buy a bag of chips. If we're using big data to steer students according to aggregates, we're limiting their freedom.
The organizations issuing the report at the top frame data use as a necessity for institutional survival, but higher ed is not a grocery store and we cannot lose sight of the mission to serve students, not in aggregate, but as individuals.
That enterprise level data is useful, for sure, but it is not sufficient. I say this as an analyst and strategist for a research firm who works with data every single day. I wrote this post on our company blog about the limits of big data. willowresearch.com/big-data-risks/
The company I work for does projects for higher ed institutions who have reached the limits of enterprise level data. Often those dashboards show a problem, but after some flailing, they realize that there's missing information, the "why." You have to talk to students themselves.
This isn't a pitch to hire us. (Though we're awesome.) But a plea for people in positions to act on this data to recognize the limits of that data and seek out additional insight when necessary. That data is a tool, not a cure all.
Data by itself cannot tell you what to do. It can inform a choice, but that choice is ultimately dependent on your underlying values. If institutions don't have that foundation of values data will have you lurching recklessly this way and that.
I'm really worried about how this stuff is being sold, can you tell?
The exact same data can suggest wildly different paths depending on your goals and values. Data does not speak independent of context. It doesn't speak at all. I Algorithms aren't going to obviate the need for human judgement.
Colleges should make value-driven, data-informed decisions. There, only took me like a dozen tweets to get to the pithy counter to the message in the report at the top of the thread.
Useful comment from someone inside a university analytics group pointing out additional problems with simplified “mor data!” approaches. insidehighered.com/quicktakes/201…
