The Cloud Messaging team at Verizon Media has been running a large installation of Pulsar in data-centers across the globe, since the inception of Pulsar. The system is run as a hosted platform to provide pub-sub messaging as a service for hundreds of use cases within Verizon Media. It hosts millions of topics in full mesh global replication, with automated on-boarding and resource management. In this talk, we share the operating experience in scaling Pulsar, its evolution over the last 5 years on multiple hardware configurations, tuning and capacity modeling practices. The talk will also cover the operating practices like rolling upgrades, automation and monitoring of the systems. Some of the important issues experienced in production and learnings will be discussed. We will also discuss some areas for operational improvements in Pulsar
Joe Francis Joe Francis heads the team that provides Key-Value Storage and Messaging systems for all of Verizon Media. Joe has led the development and operation of multiple large scale distributed systems, including Pulsar, while at Yahoo. Prior to joining Yahoo, Joe was the architect for database kernel storage and recovery, at Sybase. He is a member of the PMC of Apache Pulsar. Joe is well-versed in the development of Pulsar from its very inception, its growing pains to a large-scale system, and operating at scale in production.
Ludwig Pummer is a Principal Production Engineer at Verizon Media working on messaging and distributed key-value storage systems. He is the lead production engineer for Pulsar in Verizon Media and has supported it since prior to the inception of the current architecture. He has worked on many large scale platforms at Verizon Media, including Login and Account Registration, and is experienced in automated deployment and monitoring of large scale globally distributed systems.