It’s tempting in the technology world to think that big data is all about the processing engine…as if putting a quarter in Hadoop will automatically spit out big data. There are also some who think that Big Data itself is a technology!
Big data is not a singular topic
Gartner analyst Nick Heudecker even suggests that “big data is no longer a topic unto itself,” it has evolved into separate topics, including: 1
- Advanced analytics and data science
- Business intelligence and analytics
- Enterprise information management
- In-memory computing technology
- Information infrastructure
What he’s suggesting, in part, is that big data has become a de facto part of business, and there is no reason to treat it otherwise.
If that’s true, then why do so many organizations struggle to adopt and align a big data mindset with their day-to-day business?
In truth, Big Data is first, a strategy. A strategy that requires asking the right questions and analyzing the right data…with the support of the right executives and the right infrastructure. Without that foundation, getting mired in the “big data or not” and “Hadoop vs Spark” debate is pointless.
A Big Data foundation starts with a solid understanding of what you’re trying to achieve. A marketing director at a pharmaceutical company wants to cut the time and expense of getting the next “PharmaStar” to market. A loan officer at a mortgage company wants to minimize risky loans to ensure profitability. Retail buyers want to gauge trends so that they can deliver the right product at the right time—and through the right channel—to their customers.
Understanding what questions to ask keeps the process from going down the rabbit hole.
A big data strategy also requires not only the right data, but good data. This is often referred to as the “single source of truth”—the one source of data that everyone in the company agrees is the real, trusted number. In other words, bad data cannot be trusted to deliver good results.
Key to a big data strategy is executive buy-in. While this means showing the executive team the value of a big data solution, that’s often the easy part. The more difficult part is getting them to commit the resources necessary to establish a single source of truth—integrating all the data and siloes—and ensuring the infrastructure has the capacity and performance needed to analyze all of that data.
You’ve got the right questions, a single source of truth, and the resources and support you need. But herein lies perhaps your biggest obstacle: The Infrastructure. This is especially true for rigid legacy infrastructures that lack the capacity and performance requirements needed to quickly scale and ingest massive amounts of data.
“Enterprises want a platform that graciously allows them to move from one scale to the next and the next. You just can’t get this if you drop a huge chunk of change on a data center that is frozen in time,” explains AWS Data Science Chief Matt Wood.2
That’s why many companies are turning to cloud services to run their big data workloads. Even though cloud computing and big data are still evolving, they are completely conjoined. They provide a cost effective and scalable infrastructure that agilely supports big data and business analytics.
Colocation is another option to consider for big data and analytics, particularly with data housed in multiple locations. Colocation providers, like carrier-neutral IO, offer multiple connectivity options so that you can move different types of data from different sources into a single repository for analysis.
Like cloud computing, colocation also offers ample scalability so you can store massive amounts of data and adjust capacity as needed. Colocation has the added advantage of allowing you to build a high-performance computing (HPC) environment for more research-intensive needs—something that’s nearly impossible in most corporate data centers.
Finally, both cloud computing and colocation offer considerable business value for big data projects, making it an excellent return on investment.
Pilot projects are an excellent way to determine if big data strategies can benefit your company. They can also be very persuasive with getting your executive team on board. And a cloud environment is the perfect way to run a pilot project.
Whatever happens, it’s clear that most successful big data strategies will incorporate a range of big data technologies running in the cloud—or in a colocation environment.
1 Source: “Why your big data strategy is a bust,” InfoWorld.com, 09/04/2015.
2 Source: “Why your big data strategy is a bust,” InfoWorld.com, 09/04/2015.