Chasing That Killer Application Of Big Data

I often get asked, "what is the killer application of Big Data?" Unfortunately, the answer is not that simple.

In the early days of enterprise software, it was the automation that fueled the growth of enterprise applications. The vendors that eventually managed to stay in business and got bigger were/are the ones that expanded their footprint to automate more business processes in more industries. The idea behind the killerness of some of these applications was merely the existence and some what maturity of business processes in alternate forms. The organizations did have financials and supply chain but those processes were paper-based or part-realized in a set of tools that didn't scale. The objective was to replace these homegrown non-scalable processes and tools and provide standardized package software that would automate the processes after customizing it to the needs of an organization. Some vendors did work hard to understand what problems they were set out to solve, but most didn't; they poured concrete into existing processes.

Traditional Business Intelligence (BI) market grew the same way; the customers were looking for a specific set of reporting problems to be solved to run their business. The enterprise applications that automated the business processes were not powerful enough to deliver the kind of reporting that organizations expected to gain insights into their operations and make decisions. These applications were designed to automate the processes and not to provide insights. The BI vendors created packaged tools and technology solutions to address this market. Once again, the vendors didn't have to think about what application problems the organizations were trying to solve.

Now with the rise of Big Data, the same vendors, and some new vendors, are asking that same question: what's the killer application? If Big Data turns out to be as big of a wave as the Internet or cloud we are certainly in a very early stage. This wave is very different than the previous ones in a few ways; it is technology-led innovation which is opening up new ways of running business. We are at an inflection point of cheap commodity hardware and MPP software that is designed from ground up to treat data as the first class citizen. This is not about automation or filling a known gap. I live this life working with IT and business leaders of small and large organizations worldwide where they are struggling to figure out how best they can leverage Big Data. These organizations know there's something in for them in this trend but they can't quite put a finger on it.

As a vendor, the best way to look at your strategy is to help customers with their Big Data efforts without chasing a killer application. The killer applications will be emergent when you pay attention and observe patterns across your customers. Make Big Data tangible for your customers and design tools that would take your customers away from complexity of a technology layer. The organizations continue to have massive challenges with semantics as well as the location and format of their data sources. This is not an exciting domain for many vendors but help these organizations bring their data together. And, most importantly, try hard to become a trusted advisor and a go-to vendor for Big Data regardless of your portfolio of products and solutions. Waiting for a killer application to get started or marketing your product as THE killer application of Big Data are perhaps not the smartest things to do right now.

Big Data is a nascent category; an explosive, promising, but a nascent category. The organizations are still trying to get a handle on what it means to them. The maturity of business processes and well-defined unsolved problems in this domain are not that clear. While this category plays out on its own don't chase those killer applications or place your bets on one or two killer applications. Just get started and help your customers. I promise you shall stumble upon that killer application during your journey.

About the picture: I took this picture inside a historic fort in Jaisalmer, India that has rich history. History has taught me a lot about all things enterprise software as well as non-enterprise-software.