TruffleRuby in Shopify CI
Shopify is deeply committed to the future of Ruby. One of our key investments is in TruffleRuby — a high-performance implementation of the Ruby language based on cutting edge compiler research. While there are now several JIT compilers available for CRuby, they’re somewhat constrained by CRuby’s architecture. The same approach that allows CRuby to start up quickly and run fast during warm-up also makes it challenging to perform Ruby-level optimizations such as function inlining. In contrast, TruffleRuby is authored predominately in Ruby, allowing its JIT compiler to optimize both application code and the core libraries provided by the language. One big trade-off of this approach is TruffleRuby has needed to implement all of CRuby’s extensive core library in Ruby. It has taken several years for TruffleRuby to reach the desired level of compatibility, but it’s now a robust implementation that can run many complex Ruby workloads, including Rails applications and CRuby native extensions.
When discussing TruffleRuby publicly, I’m often asked a simple and straightforward question: “Does Shopify use TruffleRuby in production?” The answer is “not yet”, but perhaps not for the reasons you suspect. As a major Ruby shop, Shopify has historically relied on CRuby. Some of our deployment infrastructure, whether by design or incidentally, was built to only work on CRuby. We’re actively working to add support for TruffleRuby, but it’s a situation where we don’t know what needs updating until we try to use it. To help guide our work, we’re preparing our first production deployment: a large internal Rails application named Vault.
I’m happy to say our internal CI system recently added support for TruffleRuby in both release and snapshot builds. We utilize ruby-build to handle the installation, enabling us to use TruffleRuby’s published snapshot builds instead of waiting for the interpreter to be compiled from source. Having snapshot support is important because it allows us to quickly address any bugs that we discover and fix upstream without having to wait for TruffleRuby’s release cycle.
Building CI support required revisiting tools that were inadvertently overfit to CRuby, such as the part of our CI process responsible for building a Debian package.
To comply with Debian’s package naming conventions, a version number must appear in the package name and it must adhere to a specific pattern.
Our tooling was built with the assumption that ruby-build’s version strings would be valid in dpkg
as well.
That assumption holds for CRuby because the dev builds have names like “3.1.0-dev” and “3.2.0-dev”, which incidentally match Debian’s version pattern.
However, the ruby-build recipe name for TruffleRuby snapshots is “truffleruby-dev” and this string does not pass Debian’s version validation.
Consequently, our tooling needed to be updated to get the Ruby version directly from the installation instead of relying on ruby-build’s recipe names.
It was ultimately a small fix, but one that blocked running TruffleRuby in CI for any Shopify project.
With TruffleRuby now in CI, we’ve begun the process of adding it to the build of Vault’s dependencies and will ultimately add it directly to Vault’s CI. It may take a while to update other devops tooling, but in the meanwhile we can be sure TruffleRuby continues to work in common dependencies. Moreover, running CI with dev builds allows us to quickly check any upstream bug fixes and helps build confidence that code will continue to work in the next TruffleRuby release. There’s still more work to go, but we’re well on the way to our first TruffleRuby production deployment at Shopify!