Big data is a combination of structured, semi-structured and unstructured data collected by organisations that can be mined for information and used in Machine Learning projects, predictive modelling and other advanced analytics applications. Industry experts look at how enterprise IT is using Big Data to glean information that can be applied to improve business performance and productivity in an organisation.
Systems that process and store Big Data have become a common component of data management architectures in organisations, combined with tools that support Big Data analytics uses across the Middle East region. Big Data is a combination of structured, semi-structured and unstructured data collected by organisations that can be mined for information and used in Machine Learning projects, predictive modelling and other advanced analytics applications.
According to the latest insights from International Data Corporation (IDC), retailers across the Middle East and Africa (MEA) spent US$1.95 billion on IT in 2019. The global technology research, consulting and events firm forecasts this figure will reach US$2.04 billion this year and continue rising at a compound annual growth rate of 5.7% over the coming years to reach US$2.43 billion in 2023.
“The Middle East retail industry is going through a transition where the digital marketplace is gaining prominence,” said Ranjit Rajan, Associate Vice President, Research, Middle East, Turkey and Africa (META), IDC. “While the bigger players are directing their money and efforts towards building digital platforms, the market’s smaller players are exploring new business models in partnership with digital pure plays.
“Regardless of the size or the type of marketplace, the focus is on providing a personalised customer experience that is enabled by data-driven processes and a cloud-based foundation. Technologies such as robotics, Artificial Intelligence and Big Data analytics are becoming an integral part of the retail value chain –from personalised marketing and seamless digital or physical stores to warehouse and delivery automation.”
Big Data application
With many consumers wary that organisations are collecting too much data about them online, IDC believes that 60% of major retailers worldwide will implement a consent-based approach over the next two years, with the aim of maximising the value of contextualised customer journey personalisation and automated conversations. Those retailers that fail to do so may find that they can only leverage anonymous and aggregated data, severely limiting their ability to deliver personalised customer experiences.
Mohammed Alkhotani, Area Vice President – Middle East and Africa, Sitecore, agreed with Rajan on the need to offer personalised customer experiences and pointed out that Middle East organisations of all sizes and industry verticals are prioritising using Big Data to transform customer experiences, which will become critical for driving their business.
Alkhotani said in the Middle East and North Africa, 88% of IT decision-makers have changed their customer experiences due to the pandemic, with 84% of brands creating a deeper connection with customers, according to a recent survey by YouGov. “These results shows that organisations, their vendors and partners should prioritise using Big Data solutions to achieve business goals, In the meantime the digital skills are required for not only IT people but also marketers,” he said.
Talal Shaikh, Associate Professor and Director, Undergraduate Studies, School of Mathematical and Computer Sciences, Heriot-Watt University Dubai, said a great deal of a company’s decisions is now based on information collected from Big Data. Shaikh said large corporations have a problem in that they handle a lot of data and it is a complex situation with data entering the system at variable speeds from various sources, requiring some decisions to be made on the fly. “It is possible to use Big Data techniques in a variety of different contexts. Some of them are healthcare providers, especially those with multiple branches and larger hospitals, can use diagnostic data to pinpoint or predict illness and outbreak trends,” he said. “Regular check-up data can also be used to understand public health trends, for example, herd immunity can be tracked or predicted based on diagnoses and check-ups from local health centres. This would also allow time to arrange supplies in time to cope with the shift in healthcare requirements.”
According to Shaikh, traditionally, businesses have used Big Data to analyse market trends and make strategic decisions in order to grow. “During the on-going pandemic, for example, marketers are able to present customer-specific content when and where it was most effective to improve online and in-store brand recognition. Another great benefit of using Big Data for marketing purposes is to improve customer acquisition. Cloud computing permits the gathering and analysis of consistent and personalised data from multiple sources, such as web, mobile applications, email, live chat, and even in-store interactions,” he added.
Shaikh said with so many free tools available, it is not surprising that enterprises often get confused while selecting the best tool for data mining. “A good place to start are open-source tools, since these are constantly being updated towards greater flexibility and efficiency by a large developers’ community,” he noted. “Most open-source data mining tools are similar to each other with some key distinctions. When selecting the best data mining tools, it is important to consider several factors such as type or format of data, size of data, objectives of data mining and the programmer’s preferred language (Java vs R vs Python).”
Importance of Big Data
Mazhar Hussain, Digital Lighthouse Leader, KPMG Saudi Arabia, said Big Data is incredibly important for powering the ambitious Vision 2030 goals in the Kingdom of Saudi Arabia. Hussain said data is the fuel of the digital economy and accelerating data maturity is a key enabler for transformation. “Big Data can be used to unlock efficiencies, automate existing processes and help both the public and private sector to drive better decisions,” he said. “Big Data can improve operations in two main ways: helping your organisation to run its core business more efficiently and powering innovation through data-driven insights to create competitive advantage.”
Abed Hamandi, Senior Director – Professional Services, South and East Europe, Central Asia, Middle East and Africa, SAS, said Big Data is a term that describes the large volume of data – both structured and unstructured – that inundates a business on a day-to-day basis. “But it’s not the amount of data that’s important. Its what organisations do with the data that matters. Big Data can be analysed for insights that lead to better decisions and strategic business moves,” he said. “Well managed, trusted data leads to trusted analytics and trusted decisions. To stay competitive, businesses need to seize the full value of Big Data and operate in a data-driven way – making decisions based on the evidence presented by Big Data rather than gut instinct. The benefits of being data driven are clear. Data-driven organisations perform better, are operationally more predictable and are more profitable.”
Gaurav Mohan, VP Sales, SAARC and Middle East, NETSCOUT, said the primary goal of these data sets is to create closed loops that automatically establish and provide services and optimise the network resources required to supply these services. “This is achieved by anticipating faults, bottlenecks and other potential networking issues. This is especially true in a hybrid networking environment, where physical and virtual network operations must communicate. The following level of analytics is more predictive,” he said.
According to Hussain, Big Data is pervasive in every aspect of peoples’ lives. “From consumer experiences such as being recommended what movie to watch based on your viewing preferences or using a smart-watch to monitor your health – to an oil and gas company using Digital Twins to run real-time simulation. The full spectrum of our citizen experience is impacted by Big Data,” he said.
Business benefits
While its undeniable that Big Data offers huge business benefits, pundits warn that CIOs and IT teams need to harness and glean information from Big Data systems in order for it to have any business value.
Volha Smirnova, Internet of Things and Analytics BDM, EMEA Centre of Excellence, Software AG, said Big Data has been rapidly elevated business results across organisations and sectors.
Smirnova explained that according to the BCG Global Innovators Survey: committed innovators are winning (which are only 40%), with 60% of them, as reported, are generating a rising proportion of sales from products and services launched is the past three years. “This means that Big Data is becoming a source of creating new revenue streams and launching new data-enabled services, rather than activating new production cycles and modernising on the product or manufacturing cycles,” she said. “Other key benefits would revolve around improved customer insight, data governance, smarter customer targeting and data-driven innovation.”
Alkhotani noted that Middle East organisations that integrate Big Data on to one single platform, such as the Sitecore Experience Database, can find numerous business benefits. “These include optimised costs, new levels of citizen, customer, and employee engagement and happiness, and enhanced digital revenue and business competitiveness,” he said.
Bill Scudder, SVP, AIoT General Manager, Aspen Technology, said the biggest challenges that organisations face is operating under the assumption that there is a one-size-fits-all solution. Scudder pointed out that this is not true – organisations must continuously re-evaluate their workflows and processes for collecting, storing, optimising and presenting data to ensure they’re reaping the greatest business value from it. “This shows up in practice when thinking about auto discovery. Many IT leaders believe that there are tools that will auto-discover relevant information across all of your data, whereas there’s an age limit on what these tools can work with – data that is past a certain time frame is usually undetectable, for example,” he explained.
Another challenge according to Scudder, comes from a generational, operational expertise gap. “Many organisations are having difficulty finding the right people who have the means and knowledge of where data is stored and what format it’s in. This all circles back to making sure you have the correct data integration strategy in place – it makes it infinitely easier on the employees when a set plan has been made and executed on,” he said.
CIO challenges
With all the momentum around Big Data across the region, CIOs and IT leaders are being urged to have robust systems and tools to store and process Big Data.
Heriot-Watt University’s Shaikh said for storage and processing Big Data, two features are preferred: scalability and high-speed access to massive volumes of information. “Relational database management systems (DBMS), whether centralised or distributed, are based on the use of SQL language and transactional properties that ensure atomicity, consistency, isolation, resiliency and durability (ACID) compliance,” he said. “Massive parallel analytic databases (MPADs) make up a significant part of relational database technology that is useful for Big Data. As opposed to traditional data warehouse, these DBMSs may implement large amounts of structured data with little to no data modelling and may scale up to multiple terabytes and petabytes.”
Hamandi added that the CIO has to make the right technology choices that enable business users to become more data driven. “Today, the CIO’s and IT head roles have evolved much more and become more centred around business transformation. The CIO is almost like the change agent who ensures that all IT functions that support the business work smoothly to effect transformational change,” he said. “To implement transformational change, you need information and intelligence –something that you can get only with data and analytics. As with any area of IT, there are strategic considerations involved in selecting a Big Data and analytics strategy for an organisation.”
He said the CIO is also the main “godfather” of business-led analytics. “While IT data infrastructure remains IT’s responsibility, the capacity to use data and the ideas to best use data comes from the frontlines and the business users. For this, it is important to democratise data and use advanced analytics platforms that allow business users to build and manage data flows and gain access to powerful data visualisations, create predictive models, and streamline and automate forecasting processes,” he added.
According to Hamandi, the objective of Big Data and analytics is to enable collaboration to drive transformation and lead to successful business outcomes. When designing Big Data and analytics strategies, CIOs have one main mission to promote collaboration between technology teams and the business while remaining compliant with all regulatory requirements, he said.
Sitecore’s Alkhotani said one of the biggest challenges is that Big Data solutions should not be deployed in isolation, but rather as part of a wider Digital Transformation strategy for enterprises. “Already, the region is a leader in adopting the technology infrastructure needed to leverage Big Data,” he said. According to a recent YouGov survey, 83% of Middle East and North Africa (MENA) brands have advanced their marketing stack more in 2020 than in 2019 and 84% of brands experimented with new marketing approaches during the pandemic.
Moving forward, the Middle East is ideally positioned to optimise Big Data for personalised marketing. One of the biggest opportunities for enterprises is to simplify the connected commerce experience beyond web and mobile, such as with voice, kiosk Augmented Reality and Virtual Reality.
With Big Data enabling organisations across the Middle East to gain the quality of predictive personalisation for e-commerce and TV shows by accurately understanding current and future customers’ needs, data utilisation will take on increased importance, for enterprise to improve ROI and achieve KPIs.
