Alan Eldridge, Asia Pacific Director of Sales Engineering at Snowflake, describes the evolution of the cloud from infrastructure into a service.
During the past 15 years, cloud computing has evolved from being a technical novelty to something that underpins many aspects of daily business activity.
Organizations that were wary of entrusting their core applications and data to a third party have become comfortable with the concept and understand the significant benefits it can deliver. Some have reached the stage where they are now 100% cloud based.
This evolution has been supported by the rise of a number of large public cloud service providers. Lead by companies such as Amazon Web Services, Microsoft Azure and Google, these providers offer access to virtually unlimited storage and compute resources.
Yet, while these services deliver clear and significant business benefits, they have also created some challenges when it comes to data management. Many organizations have, probably inadvertently, reached the point where they have created a number of data silos with different sets stored within different cloud providers.
For example, the finance department might be using Azure while the sales department keeps its data within salesforce.com. At the same time, marketing might have become reliant on AWS while the research and development team makes extensive use of Google’s cloud.
This can make it difficult to access and use data in a holistic way. To access and combine different data sets can be come a complex and time-consuming technical challenge. As a result, many organizations can find they are missing out on opportunities to extract the maximum value from the data they have at their disposal.
The rise of the data cloud
To overcome this challenge, there is a new development within the public cloud environment. The data cloud evolves cloud computing from being an infrastructure into being a service.
In this new environment, organizations can make use of the cloud without needing to worry about the specifics of the platform underpinning it. Just like you can make use of Netflix or Spotify without needing to know the cloud resources they use, so an organization can take the same approach with its data. The cloud shifts from being a resource to being more like an application that can be put to work within an organization.
Thanks to the data cloud, an organization can access and use its data as though it is being stored in a single location. IT teams no longer have to worry about having separate connections to multiple stores of data.
Time and effort can instead be focused on making better use of all available data. It can be accessed, analyzed and reported on as easily as it if was all stored in a single database or data warehouse.
Introducing a data exchange
For organizations with large numbers of data sets, it can be challenging for everyone to know about everything that is available for use. In these circumstances, a data exchange can provide an effective solution.
A data exchange is essentially a centralized catalogue that contains details of all data sets help by the organization. These could include budget projections and plans from the finance department, records of last quarter’s revenues from the sales team, and output and inventory data from the manufacturing facility.
When a staff member wants to run a new query, they can consult the data exchange and determine exactly what is available to them. This can significantly increase the value an organization can derive from its data without adding additional workloads on to the IT team.
In some circumstances, it might make sense to open this exchange up to trusted third parties. This could include business partners, suppliers and business consultants. Each will have a clear view of all available data across the organization.
Creating a data marketplace
The concept of a data exchange can be taken even further with the creation of a data marketplace. Here, an organization’s data sets can be made publicly visible and available to interested third parties.
A data marketplace can be thought of in the same way as an app store. Software developers submit their apps to the store which is then accessed by interested prospective users. These users can then access the apps, either for free or by paying an agreed fee.
Data marketplace users are then able to combine data sourced from third parties with their own internal data sets to derive insights in new and powerful ways. Such activity would simply not have been possible without the centralized and public marketplace.
One recent example can be seen in the sharing and use of data around infection rates associated with COVID-19. Data from multiple sources are being combined in real time and used to provide a complete and accurate picture of exactly what is going on around the world.
The marketplace also provides a unique opportunity for organizations to monetize their data in new ways. Rather than just regarding it as a resource to be consumed internally, it can be positioned as one that drives an additional source of revenue.
In addition, the data cloud allows users to also share functions, allowing them to develop additional layers of value on top of the raw data. For example, a company that conducts credit risk scoring could provide a service (function) that their customers could invoke in a query, passing in the name/ID of a company and returning the credit risk score for that company.
The result is a data set enriched by the data provider without either party having to exchange their potentially confidential data. This dramatically expands the scope of what is possible with the data cloud versus traditional data sharing techniques.
A data-filled future
The impact of the data cloud is only just starting to be understood by many organizations. They are coming to terms with the significant business benefits that can stem from having a platform that provides ready access to all data regardless of where that data is being stored.
The shift also makes a big difference for IT teams, rather than needing to juggle multiple different links to a myriad data stores and platforms, they can readily access required data quickly and easily. Their time and efforts can therefore be more focused on activities that will add increased value to the organization.
The need for efficient access to data will continue to increase in the months and years ahead. The organizations that succeed and flourish will be the ones that use their data resources in the most effective way to derive new insights.
Armed with the power of the data cloud, they will be best placed to capitalize on new market opportunities as they emerge.
