Big data is the future, but no one seems to understand it and we’re not teaching it enough. Can this end well?

Let’s follow the bouncing ball here:

1. The Society for Industrial and Organizational Psychology (SIOP) released the top 10 office trends of 2014 recently; admittedly it reads a little bit like a press release from someone in their 50s who’s nervous about the future, but there are interesting components. Unsurprisingly, “big data” is the No. 1 trend for 2014 (and presumably beyond).

2. There have been numerous studies showing that while top executives at different-sized companies value big data, no one completely understands what exactly to do with it.

3. The base assumption is that there isn’t “good talent” out there who can work well with data — track it, analyze it, and then present it in a brief and contextual fashion. 

4. If the talent isn’t there, is the problem the schools? Some schools, like GW in DC, seem to understand that developing business analytics people would be a good thing:

“You have to acknowledge that the kinds of data that we’re dealing with, the sources they’re coming from, the volume of it … these are all clearly things that we haven’t seen before or we haven’t had access to,” said Srinivas Y. Prasad, an associate professor of decision sciences and faculty director of the business analytics degree at GWU.

And look here: IBM is now on board.

Jim Spohrer, director of IBM’s Global University Programs, said about 40 percent of the institutions that participate in its big data program are business schools. Another 40 percent are engineering schools, with the balance comprised of other academic fields, such as social science or health.

“It’s not like we could solve the problem if we had more computer scientists in data analytics,” Spohrer said. “The fact of the matter is we need marketing people who know big data analytics. We need health care people who know big data analytics.”

5. This is actually a fairly big deal, contextually. Check out this June 2011 report from McKinsey:

There will be a shortage of talent necessary for organizations to take advantage of big data. By 2018, the United States alone could face a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts with the know-how to use the analysis of big data to make effective decisions.

6. Alright, so … now we get into tricky waters. There is a base stereotype that Asians and Indians are better at math/data science than your good ol’ fashioned American. I’m not sure that’s conclusively true — although it can often seem true — and right now, about 40 percent of the Fortune 500 (and as high as 70 percent of Silicon Valley) are companies founded by immigrants. I’m not saying every successful company was founded by an Asian who understands “big data” — far from it — but this is a trend line to watch for the next 10 years. Companies want big data analysts, and those may be coming from … the Far East. That could shift the immigration discussion, or cause more companies to relocate major offices abroad, or … any number of things could happen, but it’s interesting to consider.

7. There are ideas around teaching big data in high school. I’m a big fan of this. I went to a top-three private high school (I mean top three nationally, as in, that’s where it’s often ranked) and despite being in HS from 1995 to 1999, when I’m pretty sure Excel was in full use, I never once touched Excel in high school. That seems like a bit of a failure, even though overall it’s a really strong high school. Excel is the most basic way most people understand “data” — rows and cells — and what we’re talking about above isn’t even that (it’s bigger than that), but shouldn’t even fifth graders be learning how to observe, record, analyze and report back on data and findings? If we really believe the knowledge economy is replacing the former behemoth that is the industrial economy, then the most important skill we can provide (outside of respect, broader lessons) is the ability to capture, synthesize, and engage others in data, right? I mean, that seems logical — but I could be misguided.

I’ve been in a ton of MBA classes over the past two years — top-25 program too — and I’ve only had one, or maybe 1.5, focused on data analysis and reporting. I had an entire strategy class where we did SWOTs for three hours. SWOTs. I don’t feel like the business world of 2020 will be interested in a SWOT so much as a three-minute PowerPoint saying “Our 51 million customers are here, here and here and like this, this, and this, so going forward we should do A, B, and C.” Sometimes I think the biggest problem with the job side of everything is that businesses change rapidly (because they must), whereas the premier universities that ultimately feed the businesses can be equivalent to dinosaurs (because they’re allowed that privilege). Big data should be a focus all the way down to second grade science experiments. Let’s do this thing, America.

6 comments on “Big data is the future, but no one seems to understand it and we’re not teaching it enough. Can this end well?

  1. I’ve been in digital marketing for a while now, but the reality of having to learn new techniques to accommodate the massive size my analysis files has been tough. I wish there was a start to finish course that I could take. For now, I’m getting involved in Python and refreshing my memory on college level stats – I really hope I can jump ahead of the curve (whatever this curve looks like). I wrote something similar here, would appreciate your feedback (

  2. The problem of the Jargon play is re-naming things that has been existing in the past. Everyone has databases that were filled with data – these are the big data.

    There’s nothing new about big data that everyone must embrace it like the way ‘iPod fever’ influenced our generations.

    • dsottimano says:

      Fair enough, I see your point. “Big data” however, is important trend as a search marketer. Maybe the negative side is the jargon as you say, but it’s only been recently that we’re getting the software to be able to gather and manipulate large datasets, correctly. Up until 2 years ago, I was happy enough not having to interact with database applications (only Excel), simply because it wasn’t needed.

      I do feel the need to embrace the big data trend – because I need to figure out how to analyse larger, more complex datasets. I think I got slightly off topic here, regardless, I liked the article.

  3. […] expensive, it creates too much shackled debt for kids coming out, and the pay-off isn’t even that tremendous in the grand scheme of things (it’s great to have a college degree, yes, but it by no means guarantees a job or even living […]

  4. […] Essentially, the idea is this: search engines (i.e. Google) are great for helping us find recipes if they already exist — i.e. “chicken pasta” — but what if you have a bunch of random crap in your fridge/cabinets and want to figure out what could be made (aside from just putting it all on a pizza) but no recipe yet exists for what you’re about to create? Cognitive computing can help with this, which is also known as a “Big Data Problem” — essentially, there’s so much data (inputs, ingredients) and so many options (things can be prepared tons of ways) that it can be overwhelming. (This is all why Big Data needs to be taught better in schools.) […]

  5. […] but boys are still outperforming them. This is especially relevant contextually right now, because many consider “big data jobs” to be the future of work, and those tend to involve math. (It should be noted that across OECD nations, girls outperform […]

Leave a Reply