While in London recently, I attended some of the Big Data Week events. One big draw was a Community Meetup featuring a panel of Big Data celebs. One question that came up: “How important is business knowledge?” To make a long story short, the panel did not band together and rush to the defense of business knowledge. There’s the difference between the philosophy of data mining, which was created to empower the business person, and the emerging culture of data science.
As nearly as I can recall her words, bit.ly’s Hilary Mason said that smart people could solve any problem. She did use the words “smart people,” and that was a real thorn in my side. Hilary is a force in the analytics community and she is much to be admired, yet my experience leaves me at odds with her on this one. It’s not that I doubt that people can adapt to new business situations or unfamiliar issues. On that point, I’m a believer. But it shouldn’t be done in a vacuum.
What’s wrong with letting an analyst dive into a problem without business knowledge? For one thing, it’s inefficient. The patterns that the analyst finds may not be meaningful – like predicting an event based on factors that only happen afterward. Or making assumptions that aren’t reasonable for the situation. And then, there is reinventing the wheel. Just recently I heard about a fabulous new analytic technology, complete with a group of fans, and I was dying to see it in action. After a huge buildup, I finally got to see the stuff in action, and my heart fell. It was nearly identical to something I worked with in the nineties that was a big flop with clients. The developers clearly hadn’t researched the history of their market.
It’s not realistic to expect that every project must be tackled by an analyst who is an expert in the business behind it. Maybe that wouldn’t even be desirable. But if the analyst doesn’t know the business, then working closely with someone who does is a must.
