Malwarebytes is an industry leading anti-malware and Internet security software provider. The company’s innovative real-time security tools detect and prevent malware infections for customers across the globe. The Malwarebytes Data and AI team is able to provide interactive dashboards that trace the trajectories and velocities of detected threats as they spread around the world.
Malwarebytes’ use of Redis Enterprise for fast data ingestion, session management, centralised stateful storage, time series analysis and geospatial analysis allows the company to aggregate, correlate and visualise data in a manner and speed it believes would not be possible without Redis Enterprise.
Before Redis Enterprise, Malwarebytes was struggling to harness the sheer enormity of data its systems was capturing. The company had access to a wealth of malware data, but leveraging that data with the speed and efficiency necessary to drive intelligence into global and local attack vectors was a daunting task.
One of the challenges at hand was to create stateful storage for several of Malwarebytes’ lifeblood data streams. “We get billions and billions of records of malware detection information,” said Darren Chinen, Senior Director of Data and AI at Malwarebytes.
As malware is detected, threat details are streamed to a centralised data platform. Stateful environment information is also streamed and collected separately in stateful storage for streaming data joins. “Understanding environment state as malware detections are found in realtime is game-changing,” said Chinen. “This technology has provided deep insights into malware proliferation, velocities and attack vectors that were previously impossible.”
Malwarebytes’ advanced visualisations posed another big storage challenge. “I call them our zombie apocalypse maps,” said Chinen. “The visualisations provide an analysis of outbreak geography, velocities and even have provided insight into gestational periods of early malware formation. The problem is that they’re built on vast amounts of data and require tremendous amounts of compute resources to generate; we needed to find a database that could provide centralised stateful storage and perform real-time streaming joins, both at massive scale.”
Since its implementation at Malwarebytes, NoSQL Redis Enterprise has an essential part of the backbone of the company’s real-time streaming layer. Malwarebytes relies on several standout Redis features uniquely suited to its challenging use cases.
Redis has been benchmarked to handle over one million read/write operations per second. The blazing fast performance of in-memory Redis is critical in addressing Malwarebytes’ incredibly high throughput, requirement for real-time streaming joins and need to access massive amounts of data at caching speeds.
The company tried other solutions but found there were problems with speed and difficulty ensuring consistency of data.
Redis’ built-in data structures were another big draw for Malwarebytes. The database’s Set, Hash and Geo Set data structures optimise the complex time series and geospatial analyses that power Malwarebytes’ dashboards. “Redis has excellent key-value flexibility,” said Chinen. “The values are not just strings, but actual data structures that are quite sophisticated. Once you understand the data structures that Redis provides, its easy to see why it outperforms other key-value databases.”
Malwarebytes needed a centralised stateful storage solution that all workers could consume from. “Redis provided the much-needed real-time indexing and retrieval capability for us to create joins on streaming data,” said Chinen. “Using Redis solved our problem by providing a blazing fast, centralised stateful storage.”
Malwarebytes had originally been running Amazon ElastiCache for Redis. But the company decided it needed true high availability, scalability and reliability, as well as an expert support team to call if something went wrong. “Redis Enterprise provided the high availability and scaling we needed to move code from a conceptual prototype to a high-performing production system,” said Chinen.
Redis Labs and its enterprise-grade Redis brought all of those things to the table and something else: an engineering mindset. “One thing we really love about Redis Labs is that the team are willing to sit with us and help us architect the best solution,” said Chinen. “Unlike a lot of other software who cater to IT organisations, Redis Labs was willing to help us architect and engineer the best solution before we got to production. The process was a fluid conversation among our collective engineering teams. The team’s commitment to the engineering mindset ‘do it right so it doesn’t break in production’ and not just ‘break-fix support’ was a breath of fresh air and a key deciding factor for us.”
