Benchmark Software Testing With Real Production Traffic

Benchmarking software testing is all about knowing (not guessing) whether your app is performing faster/slower or is becoming more/less fragile after you have made a change. The information in this post will be shared with DevOps engineers as well as engineers in general, creating an understanding of what benchmark testing really is from both DevOps and general engineering perspectives. My approach to this topic comes from actual production failures, late-night responses from developers to alerts sent via logs, and expensive lessons I learned along the way. You will learn how to create an appropriate performance baseline, which metrics are meaningful to you, and why the majority of teams perform benchmarking incorrectly; this is because they use synthetic data and averages for their determination. Additionally, by capturing real production traffic, you can improve benchmark accuracy and identify edge cases before your end users do. If you have ever shipped “working” code that has reduced performance without initially being addressed, this guide will assist you in creating and maintaining an accurate, repeatable, practicable benchmarking process that is built on the foundation of being scalable to your environments.