Security Blogs

Defining an Approach to Industrial Big Data

A few years back, in 2008, I was working at a digital advertising company. In measuring the audiences that together make up the internet population, and building a quantitative understanding of the people that make up those audiences, I learned the meaning of “big data”. An easy rule of thumb was: when you have enough data to fill a box of hard drives that you can’t pick up, you’ve got big data.

What we were building back then was really big data 1.0, developed for the Consumer Internet to support web search and ad targeting. The key challenges were to get the technology to work at all, and to scale cheaply enough to let us run our business. Now, internet-scale data processing is a commodity. Big data is getting ready for the Industrial Internet and with that, comes a whole new set of challenges.

Read More