Conference Program

JUNE 17-18, 2020 | MESSAGING & STREAMING EVERYWHERE

Testing Event-Driven Applications over Pulsar

11:30 AM - 12:10 PM PDT, June 18, 2020 | Track 3

We helped one of our clients upgrade a synchronous monolith application into an asynchronous event-driven microservices application over Pulsar. The resulting application had more than a dozen microservices all communicating over Pulsar. In this context, we were forced to rethink how we would automate the testing of this application without slowing down the development velocity. Our primary constraints were as follows.
a) We must enable independent development and deployment of each microservice. Each team developing a microservice must be able to develop new functionality, ensure that these changes are not breaking existing functionality, and release at their own cadence.
b) Adapting the existing test automation from the monolith application required a lot of effort from our engineers to recreate and update thousands of test cases and scripts in the context of the new Pulsar-based architecture. We had to update test cases, modify test data, and (re-)script these test cases. Unfortunately, we did not have the engineering resources, the budget, and the time needed to hire and train a team of test automation engineers. This was, by far, the most challenging constraint.
c) We needed to enable QE and development teams to easily create and debug end-to-end functional tests. A critical need here is to obtain visibility into the changes and differences of any request as it traverses through the pipeline.
In this talk, we discuss how we addressed the above challenge within the imposed constraints. We will also share our experience in deploying this solution to enable the SDLC workflows. One of the most important and encouraging outcomes was that the client’s team derived a lot of confidence from this framework and with very little engineering effort. In fact, their confidence with each release was much higher than what they had with their automation for the monolith.

Operations

Venky Ganti

Venky is currently busy learning how to enable engineering teams across all enterprises spend most of their time in creating new business value rather than on various types of testing during the software development and deployment lifecycle. He is a co-founder and the CEO of Mesh Dynamics, Inc. Prior to Mesh Dynamics, he was a co-founder and CTO of Alation where he helped define and evangelize the new product category of Data Catalogs. While at Google, he worked on a data product called Dynamic Search Ads where he created data pipelines (for machine learning and data enrichment as well as transformation) at web scale. While at Microsoft Research, he initiated and productized several projects and holds more than 35 granted patents. Venky graduated with a PhD in Machine Learning and Databases from the University of Wisconsin, Madison.