Zahi Yaari, VP of EMEA at SnapLogic, discusses the importance and benefits of data integration and why the possibility of fragmented data requires CIOs to ensure that all departments are aligned and the data integration process is followed carefully.
One of the ironies of Big Data is that it has made is more difficult to see the big picture. The sheer volume of enterprise data generated per day has become overwhelming: the average enterprise has over 100 applications in use at any given time. At these volumes, data becomes a double-edged sword. Many organisations are at the point where teams and departments form splinter groups in an attempt to manage the flow of information.
It’s now the case that departments within an enterprise will buy in an IT tool or platform to address a highly specific data problem, without utilising the full functionality of what they have purchased. It’s akin to buying a Porsche and only driving it at 30mph. In some cases, companies will be buying multiple Porsches – duplicating costs.
CIOs need to regain control when the data situation gets messy and the risk of a shadow IT system rears its head. The way to do this is to prioritise data integration, which in the process will dismantle data siloes and ensure joined-up thinking – and purchasing.
What is data integration?
Data integration is the process of bringing data together, so different types of data form a unified source to make the data more useful and valuable for the organisation.
Data integration falls into three broad categories (operational, analytical and hybrid) and some businesses will need to use all three, depending on how and where their data is stored.
With operational data integration, the goal is to synchronise and replicate operational data and make it available across applications, systems and databases.
Analytical data integration involves the integration of data for all your systems of record, transforming and summarising data so it can be pushed to your analytics stack, which is usually comprised of a data warehouse and a business intelligence endpoint.
Hybrid data integration is – as the name suggests – a combination of operation and analytical data integration, where insights from BI tools are fed into operational systems to improve customer experience and drive operational efficiency.
Why is data integration important?
As it goes, data integration has a number of benefits: aside from reducing costs, it can also reduce data complexity, make data more readily available, provide a seamless and easy data collaboration process and ensure smarter business decisions are made overall.
Some reading this might say that they have ‘done’ data integration within their business already. However, data integration is an ongoing process, not a ‘once and done’ project. And when struggling with data overload and the feeling that different departments have started to build their own mini-siloes, a back-to-basics focus on data integration recalibrates the entire business. Alongside this, organisations can guarantee that their data will be cleansed and validated in order to ensure that all data is robust, free of errors, duplication and inconsistencies.
The risks of fragmented data
Research from Freeform Dynamics shows that 82% of IT managers feel decision-making is hampered by data fragmentation, particularly data availability and inconsistency issues.
This is evidence that CIOs need to ensure that all departments are aligned and the data integration process is followed carefully to avoid any mishaps. It will benefit the organisation, while largely avoiding the risk of data being managed by the dreaded shadow IT, which only causes further data fragmentation, with the added possibility of data loss.
Also, when data is fragmented it is likely that it will be stored in multiple places, leaving no room for other information. This will also increase storage costs and cause ongoing overhead management, based on the complexity of handling large amounts of data.
Getting to grips with data integration
Data integration is all about selecting the right tools for the job. Every business has different needs and budgets, so ask yourself some of the following questions before you make a start – the answers will define what you want. You may also realise in this process that you already own the tools you need, idling their metaphorical engines as they are not being used at full capacity.
Some questions to ask when selecting a data integration platform:
- Is it likely to have legacy components you’ll need to worry about? Is it self-upgrading?
- How scalable is it? Will it flex to your needs if they change?
- How much skill is needed to use it? Is it so complex that no one except the IT department can understand it?
- Is it AI- and ML-enabled to help speed up the process and support compliance with GDPR and other regulations?
Think about those answers before making any decisions. Data integration is for life, not just for Christmas – you don’t want to be paying again and again for something that doesn’t work for your business.
Next steps
As with any situation that feels overwhelming, the best thing to do is break it down into manageable steps. For CIOs struggling to control the flow and volume of data within their business, data integration offers a clean slate that unifies siloes, puts clear rules and boundaries in place and starts to deliver results almost instantly. The process also reveals instances of duplication or under-utilisation in the system – platforms or tools brought in by different teams and departments that all do essentially the same job, or running at only a fraction of their full potential. This is a great opportunity to make cost savings and fully use the platforms you’re already paying for. If you discover you have a Porsche already sitting in the garage – now is the time to drive it.
