As a result, I was quite interested in this post by Cathy O’Neil about what she does in her role as a data scientist. She lists four points. The first two come down to finding ways to make data make sense to businesses and forecasting.

The next two points are these:

3. I measure. This is where the old-school statistics comes in, in deciding whether things are statistically significant and what is our confidence interval. It’s related to reporting as well, but it’s a separate task.

4. I help decide whether business ideas are quantitatively reasonable. Will there be enough data to answer this question? How long will we need to collect data to have a statistically significant answer to that? This is kind of like being a McKinsey consultant on data steroids.

O’Neil goes on to note, correctly I think, that most data scientists don’t view 3 and 4 as part of their job. Why?

It is far less sexy to try to honestly find the confidence interval of a prediction than it is to model behavior. Data scientists are considered magical when they forecast behavior that was hitherto unknown, and they are considered total downers when they tell their CEO, hey there’s just not enough data to start that business you want to start, or hey this data is actually really fat-tailed and our confidence intervals suck.

In closing she notes:

How do you select for a good data scientist? Look for one that speaks clearly, directly, and emphasizes skepticism. Look for one that is ready to vent about how people trust models too much, and also someone who’s pushy enough to speak up at a meeting and be that annoying person who holds people back from drinking too much kool-aid.

I think O’Neil is right. But its been my observation that not all that many organizations really want someone who will do just that. Standing up at a meeting and saying the business model is based on poorly reasoned out conclusions and will probably fail often gets you put on the chopping block during the next round of layoffs. Especially if you were right.

This relationship has been spotted in all sorts of data, including measures of population, mass of heavenly bodies, and death rates to name a few. Note that this only applies if the numbers are not dimensionless. That is, larger numbers have to relate to more people, more massive objects, greater death rates, or whatever it is that is being measured. Note also that Benford’s law doesn’t depend on scale – when measuring distances, it shows up whether the distances are measured in inches, yards, miles, or metric units.

Why is this interesting? Well, deviations from Benford’s law usually point to artificial constraints being placed on outcomes. For example, consumer psychology, in the form of a weakness for prices that end in “99″ can throw the distribution of observed prices off Benford’s law. But an equally likely artificial constraint is fraud. Benford’s Law, or rather, deviations from Benford’s Law have been used as pointers for election or accounting fraud. In one recent study I liked a lot, researchers applied Benford’s Law to national level data reported by European countries. They found that, of data submitted all European states, Greek data deviated from Benford’s Law more than data from any other country. No wonder Greece is in trouble.

But I was also interested in another application I recently saw on a blog by Jialan Wang, a Finance Professor at Washington University in St. Louis. To quote Professor Wang,

In fact, Benford’s law has been used in legal cases to detect corporate fraud, because deviations from the law can indicate that a company’s books have been manipulated. Naturally, I was keen to see whether it applies to the large public firms that we commonly study in finance.

I downloaded quarterly accounting data for all firms in Compustat, the most widely-used dataset in corporate finance that contains data on over 20,000 firms from SEC filings. I used a standard set of 43 variables that comprise the basic components of corporate balance sheets and income statements (revenues, expenses, assets, liabilities, etc.).

And here’s what she found:

Deviations from Benford’s law have increased substantially over time, such that today the empirical distribution of each digit is about 3 percentage points off from what Benford’s law would predict. The deviation increased sharply between 1982-1986 before leveling off, then zoomed up again from 1998 to 2002. Notably, the deviation from Benford dropped off very slightly in 2003-2004 after the enactment of Sarbanes-Oxley accounting reform act in 2002, but this was very tiny and the deviation resumed its increase up to an all-time peak in 2009.