Traditionally, transferring data between cloud providers is fraught with red tape, interoperability issues and high egress fees, locking organisations into lengthy deals. AI is now exacerbating the issue. Paul Mackay, Regional Vice President Cloud – EMEA & APAC at Cloudera, explains the current landscape of cloud.
In October 2022, the practices of the cloud hyperscalers – Google, Amazon and Microsoft – were brought into sharp focus. Ofcom, the UK’s communications regulator, initiated an investigation into whether these three cloud giants were limiting competition in the cloud market.
Clearly, there was no smoke without fire.
The case has since been referred to the Competition and Markets Authority (CMA), and the age-old issue of vendor lock-in has reared its head once more.
After almost two years of investigation, we now know the date for the CMA’s conclusion. A provisional report is expected in Autumn 2024, followed by the full report in April 2025.
At this stage, it’s impossible to tell what the regulator will say, but it has recently published a set of qualitative interviews with hyperscalers’ customers that outlined their biggest concerns. Unsurprisingly, interoperability – the technical challenge of moving data between cloud environments – was the main issue.
It was more surprising to see that the egress fees for moving data out of a provider’s cloud were described by some customers as ‘negligible’. However, that changed when the customers were asked about the impact of AI on egress fees.
The future is now with AI
AI – Generative AI in particular – has been at the forefront of the tech industry for the last two years or so. A multitude of business processes stand to benefit from AI, and it has now matured, organisations are moving out of the experimental stage of deployment to releasing their AI into the wild with real-life use cases.
As some of the largest tech companies in the world, with wallets to match, the hyperscalers have naturally invested heavily in developing AI. Microsoft fired the starting gun by investing US$13 billion into OpenAI, and all three hyperscalers have continued the trend since then.
It’s now a key battleground for these cloud giants, each with its own arsenal of exclusive products.
But ultimately, it’ll be the hyperscalers’ customers using these AI products. So, the CMA rightly asked questions about AI as part of its initial investigation into the cloud market.
When queried on egress fees in this context, the responses were quite different. Currently, organisations only tend to move small amounts of data between cloud providers, hence the ‘negligible’ egress fees. But that will likely change as AI usage becomes more widespread.
For example, over time organisations may find their AI goals might be better suited to one hyperscaler’s AI offerings over another’s. In this scenario, where it’d be necessary for customers to move all their data, the egress fees would be enormous.
A customer may also want to switch between hyperscalers to use AI models for a particular task. This requires the ability to move data sets from one environment to another easily and quickly. However, organisations are effectively experiencing a new form of lock-in. They either stick with their current cloud provider and only use the AI tools it offers, or spend large amounts of money and time to ensure their data is optimised for migrating between clouds.
This was outlined perfectly during one customer interview in the CMA report:
“One of the things that is a concern currently is lock-in. So for our analysis work, we’ve used AWS, its tooling, its modelling and the lock-in, in terms of AI feels a lot stronger than it does in other services. For example, the optimisation of certain models on certain clouds would make it very difficult from my understanding to move elsewhere. But it’s definitely something that we’re looking more into. I don’t think we understand what the answer is currently. But it is a concern of ours, and the lock-in is a big concern because I think it takes us down a certain way of using AI with certain models.”
Overcoming cloud constraints
The interim report clearly indicates that the interoperability and egress fees hyperscaler customers experience today will be exacerbated once they start deploying AI. However, a more flexible cloud market won’t necessarily spark a cloud-switching frenzy and regular mass data migrations. Even if the CMA does mandate an easing of interoperability barriers, the task of moving all an organisation’s data would still be costly and time-consuming.
Instead, as organisations look to take advantage of the various AI tools offered by hyperscalers, we’re more likely to see customers moving subsets of data between clouds in a highly adaptable multi-cloud model. The regular migration of these smaller subsets means that the hyperscalers will have as much data entering their clouds as leaving.
However, applying this flexible cloud model to drive effective AI is only possible if organisations have a modern data architecture in place. Whether customers are moving a small subset of data to use a specific AI tool or undertaking a full-scale migration, a unified data platform offers a layer of abstraction – enabling them to switch data between environments easily and securely.
A return to the original cloud innovation ethos?
With AI now shaping the future of business strategy, innovation will rely on having flexibility and choice over which AI products to use. Progress for the wider business landscape shouldn’t be gatekept by the few companies that own AI models.
In its infancy, cloud was heralded as a tool to bring people together and enable greater flexibility, but that ethos seems to have been lost now that cloud and AI are intrinsically linked, and choice is being restricted. Hopefully, these barriers will be removed. Then the onus will be on customers to ensure they can freely move data between clouds.
