The Next ‘BIG DATA’ ?
————-by Asis Mohanty
As Big Data projects grow, the complexities multiply exponentially. Even as the cost of storage has fallen, the cost of managing Big Data infrastructure grows exponentially as Big Data scales up. That includes the cost of hiring rare and expensive talent like the modern data engineer.
The potential benefits of putting Big Data in the Cloud have been known for years. The simplicity and flexibility that comes with having infrastructure as software, appeared to be an attractive solution to the complex management Big Data projects required. And yet, enterprises were reticent to move their Big Data projects to the cloud. Five years ago, the enterprises that were experimenting with Big Data did so primarily in the form of oneoff applications. The infrastructure demands for a Big Data project, while complex, were manageable. On top of that, concerns about security, privacy and cost largely outweighed the benefits that would come with the flexibility and resource efficiency of the cloud.
But now, enterprises have moved beyond Big Data projects and are striving to operationalize and leverage Big Data across their entire organization. Big Data can improve every part of a business from providing the insights for new analytic applications to augmenting traditional onpremise systems. And as Big Data aspirations grow, managing complex Big Data infrastructure h a s gone from being a painful annoyance to a fullblown IT infrastructure bottleneck that threatens to stifle the value of data.
The Cloud has emerged as an increasingly popular means of scaling Big Data usage. Further signaling this sea of change, enterprise software giants Oracle and IBM both announced Big Data analytics cloud platforms late this year, joining a push already underway by AWS, Google and Microsoft. ❏