I don’t know if I would characterize myself as a “data scientist.” That said, I have done a lot of analytics work in my career and have designed a few statistical algorithms which has made me, at least at times, a fellow-traveler to those who call themselves data scientists.
As a result, I was quite interested in this post by Cathy O’Neil about what she does in her role as a data scientist. She lists four points. The first two come down to finding ways to make data make sense to businesses and forecasting.
The next two points are these:
3. I measure. This is where the old-school statistics comes in, in deciding whether things are statistically significant and what is our confidence interval. It’s related to reporting as well, but it’s a separate task.
4. I help decide whether business ideas are quantitatively reasonable. Will there be enough data to answer this question? How long will we need to collect data to have a statistically significant answer to that? This is kind of like being a McKinsey consultant on data steroids.
O’Neil goes on to note, correctly I think, that most data scientists don’t view 3 and 4 as part of their job. Why?
It is far less sexy to try to honestly find the confidence interval of a prediction than it is to model behavior. Data scientists are considered magical when they forecast behavior that was hitherto unknown, and they are considered total downers when they tell their CEO, hey there’s just not enough data to start that business you want to start, or hey this data is actually really fat-tailed and our confidence intervals suck.
In closing she notes:
How do you select for a good data scientist? Look for one that speaks clearly, directly, and emphasizes skepticism. Look for one that is ready to vent about how people trust models too much, and also someone who’s pushy enough to speak up at a meeting and be that annoying person who holds people back from drinking too much kool-aid.
I think O’Neil is right. But its been my observation that not all that many organizations really want someone who will do just that. Standing up at a meeting and saying the business model is based on poorly reasoned out conclusions and will probably fail often gets you put on the chopping block during the next round of layoffs. Especially if you were right.