When To Use Big Data and When Not – Is Big Data For All?

Big data has been one of the biggest hypes over the past years. According to Gartner hype cycle, big data is however falling into the trough of disillusionment.

Recently discussion has surfaced about backlash for big data. For example, Google flu trend mixed correlation with causation. Yet sources like GigaOm have a different opinion about the backlash.

What is the situation now, should you still proceed with big data? Are there successful use cases for big data?

What’s behind big data?

The following properties are generally considered characteristics of big data:

• Volume – Data comes in very large volumes, consisting of terabytes and petabytes of data.
• Variety – Data appears in different formats and originates not only from internal but from external sources outside the organization. In addition to structured data, the contents can be unstructured data like text, images, locations to name a few.
• Velocity – Data is arriving at increased speed, every millisecond 24/7.
• Veracity – Data may be uncertain due to unverified sources. Data from anonymous Twitter account is not as reliable as data coming from a strongly authenticated and identified user.

What are typical use cases for big data?

There are obvious use cases like

• Amazon has a large-scale recommendation engine for books and other products. Amazon recently applied for a patent for anticipatory shipping.
• Google uses PageRank search engine to predict search queries based on just two or three letters. Google now provides cloud-based fast analytics with its BigQuery.
• Netflix is one of the most notable big data users and it has applied big data and analytics even to design House of Cards TV series (TV series may no longer be the correct term as the series is available on the net only).

New angles to the traditional business models have emerged by utilizing big data.

One example is Trulia which provides information for apartment and house buyers, sellers and renters.

Recently Trulia started a service called Trulia Insight. This service allows agents to access detailed analytics about property markets and prospective customers.

When not to use big data – at least for now

Big data may not be advisable in all the cases, at least not for now in these cases:

• When the current operation is not very data-intensive
• When the data is not properly understood, managed or governed
• When there are privacy concerns due to utilizing personal data from different sources

The current situation may change as big data and especially advanced analytics will become mainstream, part of the daily operations for any business.

When big data can be a massive opportunity

Advanced analytics used with big data can also be applied to ‘small data’. Descriptive and predictive analytics with the data visualization can reveal new insights about the way the business is now operated.

New businesses without legacy systems should definitely investigate big data approaches and possibilities available through advanced analytics.

As Amazon and Nextflix have shown, the use of analytics has spread to unexpected and surprising areas. Similar surprise element can eventually occur in every organization should the step towards big data and analytics first take place.