A real-time data architecture that can support streaming analytics applications is an enticing, yet overwhelming, prospect for organizations looking to act quickly on information.
When I worked at a fast-food eatery in high school, a co-worker friend and I decided its motto should be “Speed, Not Perfection.” We silkscreened t-shirts for the two of us with that phrase embedded in the corporate logo — two smart-aleck teenagers gently sticking it to the man.
These days, data management and data analytics teams increasingly find themselves being asked to fulfill the speed part to enable real-time data analysis in their organizations.
But they don’t have the luxury of being able to get by with the same sort of occasional sloppiness that my friend and I did in slapping burgers together.
Also, that puts them under a great deal of weight, on the grounds that making a constant design and utilizing it to run streaming data analytics applications is a convoluted endeavor.
For starters, streaming analytics systems don’t come in a crate not even a large one. Setting them up is an artisanal process that requires prospective users to piece together various data processing technologies and analytics tools to meet their particular application needs.
In addition, the technology options have increased significantly over the past few years, thanks largely to the emergence of multiple big data analytics platforms that provide stream processing capabilities in different ways.
A Plethora Of Streaming Platforms
Spark Streaming, Flink, Storm, Samza, Pulsar, Druid, Kylin they’re all open source processing engines vying for a piece of the data streaming and real-time data analytics action. Even Kafka, originally a messaging technology for feeding data from one system to another, now also functions as a stream processing platform in its own right.
In addition to the open source tools, various IT vendors offer more traditional complex event processing systems that began emerging in the late 1990s. Specific databases – in-memory ones, for instance – are also built to handle streaming data analytics.
On the data analytics software side, broader use of machine learning data algorithms is making it more feasible to build predictive analytics models that can churn through large amounts of streaming data on things like financial transactions, equipment performance and internet clickstreams.
But again, there are a multitude of technology choices to consider: tools from mainstream data analytics vendors and machine learning specialists, cloud-based services, open source platforms.
As with building a big data architecture in general, the surfeit of software available to underpin a real-time data analytics architecture can be a boon for users or mire them in a veritable boondoggle of a deployment.
Finding the right technologies and combining them into an effective data analytics framework is a perilous process; missteps can send a project careening off the intended path.
Streaming Forward On Real-Time Projects
That isn’t stopping companies, particularly large ones with lots of data and ample IT resources, from giving it a go. In an ongoing survey being conducted by Search Business Analytics publisher TechTarget Inc., 28.1% of the 7,000-plus IT, data analytics and business Intelligence analyst who had responded as of mid-January said their organizations were looking to invest in real-time data analytics technology over the ensuing 12 months. Furthermore, 13.4% said they planned to buy stream processing software.
The ability to pull useful information out of data streams in real time lets business intelligence operations act fast, and that clearly can be to their advantage.
Predictive analytics applications run against streaming data on the web activity of consumers can drive website personalization programs and targeted online advertising and marketing campaigns. Fraud detection, predictive analytics maintenance and satellite imaging are other applications that can benefit from streaming data analytics.
More of the time, real time might be the only time to take advantage of what’s in the data being collected. Streaming analytics tools point to “perishable insights” that need to be acted on quickly before the opportunity is lost, Forrester Research analyst Mike Gualtieri and then-colleague Rowan Curran wrote in a 2016 Forrester Wave report.
And you can’t get those kinds of insights simply by throwing data into a Hadoop cluster, as Darryl Smith, chief data platform architect at Dell EMC, said during a presentation on the data storage vendor’s real-time streaming efforts at Strata + Hadoop World 2016 in New York.
Speed is indeed a wonderful thing. Just be sure your team has a well-thought-out plan before turning up the heat on a streaming data analytics initiative.
Otherwise, it might end up getting flame-grilled by disappointed business executives.
No comments:
Post a Comment