How Apache Kafka got faster by switching ext4 to XFS

The Backend Engineering Show with Hussein Nasser - A podcast by Hussein Nasser - Fridays

Try Bookbeat 60! days for free, click here

Try Bookbeat 60! days for free, click here

Enjoy a whole world of audiobooks and e-books, everything from new releases to the classics

 Sponsored

Categories:

Technology

https://oscourse.win


Allegro improved their Kafka produce tail latency by over 80% when they switched from ext4 to xfs. What I enjoyed most about this article is the detailed analysis and tweaking the team made to ext4 before considering switching to xfs. This is a classic case of how a good tech blog looks like in my opinion. 


0:00 Intro

0:30 Summary

2:35 How Kafka Works?

5:00 Producers Writes are Slow

7:10 Tracing Kafka Protocol 

12:00 Tracing Kernel System Calls

16:00 Journaled File Systems

21:00 Improving ext4

26:00 Switching to XFS



Blog

https://blog.allegro.tech/2024/03/kafka-performance-analysis.html

Visit the podcast's native language site