C-Suite On Deck
Keep me plugged in with the best
Join thousands of your peers and receive our weekly newsletter with the latest news, industry events, customer insights, and market intelligence.
I agree to the
terms of service
PLEASE CORRECT THE FOLLOWING:
Please Enter Some Keywords
Not just for Processing: How Kafka Streams as a Distributed Database Boosted our Reliability and Reduced Maintenance
The Apache Kafka Streams library is used by enterprises around the world to perform distributed stream processing on top of Apache Kafka. One aspect of this framework that is less talked about is its ability to store local state, derived from stream processing. In this blog post we describe how we took advantage of this ability in Imperva’s Cloud Application Security product. Using Kafka Streams, we built shared state microservices that serve as fault-tolerant, highly-available Single Sources of Truth about the state of objects in the system, which is a step up both in terms of reliability and ease of maintenance. If you’re looking for an alternative approach to relying on a single central database to maintain the formal state of your objects, read on. Why we felt it was time for a change for our shared state scenarios. At Imperva, we need to maintain the state of various objects based on the reports of agents (for example: is a site under attack?). Prior to introducing Kafka Streams, we relied in many cases on a single central database (+ Service API) for state management. This approach comes with its downsides: in data-intensive scenarios, maintaining consistency and synchronization becomes a challenge, and the database can become a bottleneck or be prone to race conditions and unpredictability.
I'm for real
Enter your email once to access all our information and resources.
(Your email address is required so we know you're a real person)
By downloading this content, you give permission for your contact information to be shared with the content provider who may contact you in regards to the content.