Action Deduplication: Faster and Cheaper Remote Builds Without Lifting a Finger - Christian Scott, Canva
In this session I will discuss remote action deduplication, a feature of some Bazel RBE backends. I will explain the problem it solves, how it works, and some details about how well it has worked for us. I'll also quickly touch on merge queues, and mention some RBE backends that support the feature. --- If you run builds on every master commit, and you have commits landing more quickly than your slowest action can finish, you'll end up rebuilding unnecessarily. We estimated that our slowest webpack action was being re-run unnecessarily 66% of the time! This action is an input to a large number of expensive integration tests so it's very likely that we're also wasting a lot of compute on those tests as well. The web page builds and tests account for the vast majority of the compute we use in CI. The number of wasted builds would be even higher if we were using a merge queue, so we have been forced to stick to a conventional merge strategy. By running these builds remotely and using an RBE backend that supports action deduplication, we've massively (25%, more expected by BazelCon) reduced the amount of compute needed to build and test web pages.