Look! A trace of slow events in a benchmark! Hover over the image to see it get bigger.

Now read on to see what the slow events are and how we got this pretty picture.

The rules

The first rule of just-in-time compilers is: you stay in JIT code. The second rule of JIT is: you STAY in JIT code!

When control leaves the compiled code to run in the interpreter—what the ZJIT team calls either a “side-exit” or a “deopt”, depending on who you talk to—things slow down. In a well-tuned system, this should happen pretty rarely. Right now, because we’re still bringing up the compiler and runtime system, it happens more than we would like.

We’re reducing the number of exits over time.

Lies, damned lies, and statistics

We can track our side-exit reduction progress with --zjit-stats, which, on process exit, prints out a tidy summary of the counters for all of the bad stuff we track. It’s got side-exits. It’s got calls to C code. It’s got calls to slow-path runtime helpers. It’s got everything.

Here is a chopped-up sample of stats output for the Lobsters benchmark, which is a large Rails app:

$ WARMUP_ITRS=0 MIN_BENCH_ITRS=20 MIN_BENCH_TIME=0 ruby --zjit-stats benchmarks/lobsters/benchmark.rb
...
***ZJIT: Printing ZJIT statistics on exit***
...
Top-20 side exit reasons (100.0% of total 12,549,876):
                   guard_type_failure: 6,020,734 (48.0%)
                  guard_shape_failure: 5,556,147 (44.3%)
  block_param_proxy_not_iseq_or_ifunc:   445,358 ( 3.5%)
                   unhandled_hir_insn:   215,168 ( 1.7%)
                        compile_error:   181,474 ( 1.4%)
...
compiled_iseq_count:                               5,581
failed_iseq_count:                                     2
compile_time:                                    1,443ms
...
guard_type_count:                            133,425,094
guard_type_exit_ratio:                              4.5%
guard_shape_count:                            49,386,694
guard_shape_exit_ratio:                            11.3%
...
code_region_bytes:                            31,571,968
side_exit_size_ratio:                              33.1%
zjit_alloc_bytes:                             19,329,659
total_mem_bytes:                              50,901,627
...
ratio_in_zjit:                                     82.8%
$

(I’ve cut out significant chunks of the stats output and replaced them with ... because it’s overwhelming the first time you see it.)

The first thing you might note is that the thing I just described as terrible for performance is happening over twelve million times. The second thing you might notice is that despite this, we’re staying in JIT code seemingly a high percentage of the time. Or are we? Is 80% high? Is a 4.5% class guard miss ratio high? What about 11% for shapes? It’s hard to say.

The counters are great because they’re quick and they’re reasonably stable proxies for performance. There’s no substitute for painstaking measurements on a quiet machine but if the counter for Bad Slow Thing goes down (and others do not go up), we’re probably doing a good job.

But they’re not great for building intuition. For intuition, we want more tangible feeling numbers. We want to see things.

Building intuition

The third thing is that you might ask yourself “where are these exits coming from?” Unfortunately, counters cannot tell you that. For that, we want stack traces. This lets us know where in the guest (Ruby) code triggers an exit.

Ideally also we would want some notion of time: we would want to know not just where these events happen but also when. Are the exits happening early, at application boot? At warmup? Even during what should be steady state application time? Hard to say.

So we need more tools. Thankfully, Perfetto exists. Perfetto is a system for visualizing and analyzing traces and profiles that your application generates. It has both a web UI and a command-line UI.

We can emit traces for Perfetto and visualize them there.

A look at Perfetto

Take a look at this sample ZJIT Perfetto trace generated by running Ruby with --zjit-trace-exits¹. What do you see?

I see a couple arrows on the left. Arrows indicate “instant” point-in-time events. Then I see a mess of purple to the right of that until the end of the trace.

Hover over an arrow. Find out that each arrow is a side-exit. Scream silently.

But it’s a friendly arrow. It tells you what the side-exit reason is. If you click it, it even tells you the stack trace in the pop-up panel on the bottom. If we click a couple of them, maybe we can learn more.

We can also zoom by mousing over the track, holding Ctrl, and scrolling. That will get us look closer. But there are so many…

Fortunately, Perfetto also provides a SQL interface to the traces. We can write a query to aggregate all of the side exit events from the slice table and line them up with the topmost method from the backtrace arguments in the args table:

SELECT
  s.name AS reason,
  a.display_value AS method,
  COUNT(*) AS count
FROM slice s
JOIN args a ON a.arg_set_id = s.arg_set_id AND a.key = '0'
GROUP BY s.name, a.display_value
ORDER BY count DESC

This pulls up a query box at the bottom showing us that there are a couple big hotspots:

It even has a helpful option to export the results Markdown table so I can paste (an edited version) into this blog post:

Looks like we should figure out why we’re having shape misses so much and that will clear up a lot of exits. (Hint: it’s because once we make our first guess about what we think the object shape will be, we don’t re-assess… yet.)

This has been a taste of Perfetto. There’s probably a lot more to explore. Please join the ZJIT Zulip and let us know if you have any cool tracing or exploring tricks.

Now I’ll explain how you too can use Perfetto from your system. Adding support to ZJIT was pretty straightforward.

Implementation

The first thing is that you’ll need some way to get trace data out of your system. We write to a file with a well-known location (/tmp/perfetto-PID.fxt), but you could do any number of things. Perhaps you can stream events over a socket to another process, or to a server that aggregates them, or store them internally and expose a webserver that serves them over the internet, or… anything, really.

Once you have that, you need a couple lines of code to emit the data. Perfetto accepts a number of formats. For example, in his excellent blog post, Tristan Hume opens with such a simple snippet of code for logging Chromium Trace JSON-formatted events (lightly modified by me):

event_name = ...
timestamp = ...
duration = ...
f = open('trace.json','a')
f.write("[\n")

# ... emit some events here ...

# Log a single event
f.write('{"name": "%s", "ts": %d, "dur": %d, "cat": "hi", "ph": "X", "pid": 1, "tid": 1, "args": {}},\n' %
  (event_name, timestamp, duration))

# ... emit some events here ...

# ... at process exit, close the file ...
f.write("]") # this closing ] isn't actually required
f.close()

This snippet is great. It shows, end-to-end, writing a stream of one event. It is a complete (X) event, as opposed to either:

• two discrete timestamped begin (B) and end (E) events that book-end something, or
• an instant (i) event that has no duration, or
• a couple other event types in the Chromium Trace Event Format doc

It was enough to get me started. Since it’s JSON, and we have a lot of side exits, the trace quickly ballooned to 8GB large for a several second benchmark. Not great. Now, part of this is our fault—we should side exit less—and part of it is just the verbosity of JSON.

Thankfully, Perfetto ingests more compact binary formats, such as the Fuchsia trace format. In addition to being more compact, FXT even supports string interning. After modifying the tracer to emit FXT, we ended with closer to 100MB for the same benchmark.

We can reduce further by sampling—not writing every exit to the trace, but instead every K exits (for some (probably prime) K). This is why we provide the --zjit-trace-exits-sample-rate=K option.

Check out the trace writer implementation from the point this article was written.

Tracing more things

We could trace:

• When methods get compiled
• How big the generated code is
• How long each compile phase takes
• When (and where) invalidation events happen
• When (and where) allocations happen from JITed code
• Garbage collection events
• and more!

Conclusion

Visualizations are awesome. Get your data in the right format so you can ask the right questions easily. Thanks for Perfetto!

Also, looks like visualizations are now available in Perfetto canary. Time to go make some fun histograms…

The Rails Infrastructure team has been working on making Bundler faster and our work has paid off. A cold bundle install is 3x faster on a Gemfile with 452 gems compared to Bundler 2.7. But “faster” only means something if everyone agrees on what’s being measured. If two people are running benchmarks with different definitions of “cold install” or different cache states, the results aren’t comparable. We needed a shared tool that would give us confidence, both internally and externally, that we’re tackling the right problems and actually making Bundler faster. And along the way we learned when Claude can be helpful and when to not outsource our own expertise and thinking.

What affects bundle install time

Before building anything, we had to understand what variables go into bundle install performance. There are more than you’d expect:

• Number of gems: A Gemfile with 35 gems takes less time to install than a Gemfile with 500 gems.
• Depth of dependencies: A flat Gemfile with no transitive dependencies resolves faster than a deep dependency tree.
• Native extensions: Gems like bigdecimal take orders of magnitude longer to install than pure Ruby gems (bigdecimal alone takes 3 seconds to install). The ratio of native extension gems to pure Ruby gems in your Gemfile changes the install profile significantly. It’s rare to have a Gemfile without at least a few dependencies on native extensions.
• How Ruby is compiled: Optimization flags, compiler version, and platform all affect gem compilation time.
• Network time: Downloading from rubygems.org introduces latency and rate limiting that can skew results.
• Number of cores: Bundler parallelizes installs across worker threads, so core count matters.
• Endpoint security software: On company issued, managed machines, security software that scans file writes adds measurable time to every gem install. Running the same benchmark on a personal non-managed device vs a managed device produced very different numbers with no code change. If your benchmarks aren’t reproducible across machines, this is worth checking.

We needed a way to remove as many of these variables as possible so when we made changes, we could trust our benchmarks were correct. The goal was to remove guesswork, ensure everyone is testing from the same starting point, and provide a straightforward way to run benchmarks when making changes.

What we built

It took a few weeks to get reliable results and as part of building the benchmark we also implemented a full toolkit that includes scripts for installing, benchmarking and profiling Ruby package managers.

Getting a reliable benchmark took a lot of iteration. The first version of the benchmark was basic, and every time we ran it we’d find something that wasn’t quite right. We worked with Claude to make the original benchmark and tweak it as we found issues with the runs. In some cases we had to do the tedious work of debugging the benchmark ourselves.

Back in 2018 when I was working on improving Rails integration test performance, I kept an entire repo with all my benchmarks and profile scripts so I could track changes over time, but also so I could share it with the community. When your benchmark is open source, you’re not working in a vacuum and everyone else can check your assumptions. That lesson stuck with me, and it’s why this toolkit follows a similar pattern.

The toolkit includes everything you need to benchmark and profile Bundler.

• Setup scripts: Scripts to install for both macOS and Linux that lets you choose which package managers you want to benchmark.
• Benchmarking tool: The benchmark tool uses hyperfine for statistical timing with standard deviation, min/max, and outlier detection.
  • It supports running against multiple branches and package managers, switching the Ruby version, changing the number of iterations, and provides multiple Gemfile scenarios.
  • It automatically runs both warm and cold scenarios and outputs how much faster or slower each is than the baseline.
  • It includes a fake gemserver. Thanks to Claude I was able to quickly build a fake gemserver that served real gems based on Aaron’s slow-gemserver and use that to eliminate deviations caused by network round trips to Rubygemsorg and/or rate limiting.
• Profiler tool for Bundler: The profiler tool can currently only profile Bundler but also includes everything you need using either Samply or Vernier
  • It supports switching the Ruby version, running with cold or warm cache mode, choosing the Gemfile scenario, and changing the output path of the profile.
  • It also can optionally use the fake gemserver to avoid profiling network time.

How we defined what to measure

As part of building this benchmark we also needed to define what we wanted to measure in order to have the same understanding of the scenarios we are trying to improve.

Cold is defined as a first-ever install. Before each iteration, hyperfine’s --prepare hook nukes all caches: download cache, compact index cache, installed gems, bundle home, and removes the lockfile. The install has to resolve dependencies, download every gem, and install from scratch. This is the case where nothing is compiled or installed.

Warm is defined as reinstalling gems that have been downloaded previously. The benchmark setup first runs one full cold install to populate the download cache. Then for each timed iteration, the --prepare hook removes only installed gems and the .bundle directory, keeping the download cache and lockfile intact. The install runs with BUNDLE_FROZEN=1 so it skips resolution and only extracts and installs.

Getting “warm” right was harder than it sounds. Bundler, gel, rv, and scint package managers all have different cache structures. Early on Bundler’s warm results were barely faster than cold, and we spent time debugging before realizing it was a cache isolation issue in the benchmark itself, not a Bundler problem. The benchmark was wrong, not the code. Interestingly, this was a case where AI wasn’t that helpful. Claude kept missing this specific environment variable, so we had to debug the hard way (this is also why the script has a BENCH_DEBUG mode). But it paid off because we gained a better understanding of which environment variables affect the caches.

In the future we may want to define other cache scenarios to measure. There are scenarios between our pre-defined cold and warm scenarios like bundle update or having the gems installed but no lockfile so resolution needs to occur again. The beauty of this toolkit being open source is that if there’s a scenario you want to test, we can easily add that to the benchmark script.

Using the benchmark script

In order to support multiple tools we implemented a --run argument that is specified as a LABEL:TOOL[:PATH] triple. The label isolates caches, the tool selects the package manager, and the path optionally points to a local checkout or git worktree. Multiple runs can be compared in a single invocation, with the first treated as baseline.

ruby run_benchmark.rb \
  --run master:bundler:~/rubygems \
  --run patched:bundler:~/rubygems-patched \
  --scenario rails \
  --iterations 5 \
  --source http://localhost:9292

Caches are fully isolated per label under .caches/<label>/ using environment variables so comparing two Bundler versions in the same invocation won’t contaminate results. The comparison output shows relative speed so you can quickly see if your change is how many times faster or slower it is than the baseline.

Each scenario is just a directory containing a Gemfile. The rails scenario represents a typical Rails application with 35 gems. The large scenario is a stress test with 452 gems. You can add your own by creating a directory with a Gemfile and passing --scenario yourdir to the script.

$ ruby run_benchmark.rb --run bundler27:bundler:~/bundler27 --run master:bundler:~/rubygems/ --scenario large --iterations 3 --source http://localhost:9292 --ruby /usr/local/bin/ruby
Benchmark matrix
Ruby: ruby 4.0.1 (2026-01-13 revision e04267a14b) +PRISM [x86_64-linux]
Source: http://localhost:9292
Iterations: 3
Runs:
  bundler27: bundler (/home/ubuntu/bundler27)
  master: bundler (/home/ubuntu/rubygems)

=== Scenario: large (452 gems, bundler27) ===
  Running cold benchmark (3 runs)...
Benchmark 1: bundler27 (cold)
  Time (mean ± σ):     51.368 s ±  0.058 s    [User: 106.055 s, System: 22.181 s]
  Range (min … max):   51.332 s … 51.435 s    3 runs

  Running warm benchmark (3 runs)...
Benchmark 1: bundler27 (warm)
  Time (mean ± σ):      8.895 s ±  0.150 s    [User: 7.743 s, System: 4.658 s]
  Range (min … max):    8.742 s …  9.042 s    3 runs

  Cold median: 51.34s  Warm median: 8.9s

=== Scenario: large (452 gems, master) ===
  Running cold benchmark (3 runs)...
Benchmark 1: master (cold)
  Time (mean ± σ):     16.012 s ±  0.023 s    [User: 107.344 s, System: 21.568 s]
  Range (min … max):   15.987 s … 16.033 s    3 runs

  Running warm benchmark (3 runs)...
Benchmark 1: master (warm)
  Time (mean ± σ):      7.202 s ±  0.027 s    [User: 4.908 s, System: 3.061 s]
  Range (min … max):    7.173 s …  7.228 s    3 runs

  Cold median: 16.02s  Warm median: 7.2s

Results written to /home/ubuntu/bundler-bench/results/bundler27_20260318_161305.json
Results written to /home/ubuntu/bundler-bench/results/master_20260318_161305.json

=== Comparison Summary ===

Scenario: large (452 gems)
                             Cold     +/-                        Warm     +/-
  ------------------------------------------------------------------------------
  bundler27                51.34s   0.06s  baseline             8.90s   0.15s  baseline
  master                   16.02s   0.02s  3.21x faster         7.20s   0.03s  1.24x faster

Note these numbers will vary across macOS and Linux, as well as machines with endpoint security software. While we aimed to reduce many variables, you still may not see the same numbers, however times faster should be between 2-3.5x for cold and 1-1.5x for warm for bundle install. This script was run on an AWS Sandbox and therefore has no other traffic or endpoint security altering the numbers. It is also using the fake gemserver, so network round trips aren’t involved.

Using the profiling script

Benchmarks tell you whether something got faster. Profiles tell you why it’s slow. The profiling tool runs a single bundle install under Vernier or samply to produce flamegraphs.

ruby profile_bundler.rb \
  --run master:bundler:~/rubygems \
  --scenario rails \
  --mode warm \
  --profiler vernier

It supports both cold and warm modes so you can profile the specific phase you’re investigating. Profiles are written to profiles/ with filenames that include the label, scenario, mode, platform, and timestamp so you can compare across runs and machines.

Here’s an example of the Vernier output for the master branch on the AWS linux sandbox for the cold cache mode.

See all the yellow bars? Those are native extensions compiling and blocking the threads from doing other work.

Setup scripts

Reproducing someone else’s benchmark results is only possible if you’re starting from the same place. The repository includes setup scripts for both macos and linux which will install Ruby, hyperfine, profiling tools, and clone the repos you need:

./setup-benchmark-mac.sh --tools bundler,bundler27,gel,scint

The --tools flag lets you pick which package managers or versions to install. It defaults to bundler,bundler27 so you can compare the current master against the last stable release without extra setup. We wanted anyone on the team (or in the community) to be able to spin up a fresh machine and get comparable results without hunting down the right Ruby version, compiler flags, or repo branches.

What we learned

A shared benchmark is a source of truth. When someone says “my change is faster” and someone else disagrees, you need a neutral tool that everyone agreed on beforehand. Without that, performance discussions turn into competing anecdotes. The toolkit gives us a way to settle those disagreements with data instead of intuition.

Reproducibility matters just as much. If you can’t reproduce similar results on someone else’s machine, you can’t verify the claims. “It’s faster on my machine” isn’t useful if it’s not faster for everyone, or worse faster on Linux but slower on macOS. When results differ across machines, we can start asking why instead of arguing about whether.

Back in 2018 I gave a talk called How to Performance which was on the surface about how I sped up integration tests in Rails, but really it was a talk about how to write benchmarks you could trust so you know when you actually made something faster. Many of the lessons I learned back then came up again during this project.

Profiles and guesswork are only one part of the equation. A profile can show you a hot spot, and you can write a fix that looks faster, but without a proper benchmark you don’t actually know. You don’t know if the gain on macOS is a regression on Linux. You don’t know if “cold” got faster but “warm” got slower. You don’t know if the improvement holds across different Gemfile sizes. The benchmark is what turns a hypothesis into evidence.

This matters even more now than it did in 2018. Engineering rigor is more important in the AI world than it was before. It’s easy to generate output that looks correct or looks faster. It’s easy to make a benchmark that looks reasonable but cheats on the warm caches. AI is good at producing plausible code and plausible explanations. Humans are good at critical thinking and using our gut to know when something doesn’t look right. We have taste, discernment, and scrutiny, AI has data.

That’s not a dig on AI. I used Claude extensively throughout this project. It was great at writing the setup scripts, which are uninteresting and error prone, and it wrote the original benchmark tool. But it also got things wrong. The warm cache bug I mentioned earlier? Claude missed setting BUNDLE_USER_HOME in the environment, which meant Bundler was writing to the system bundle home instead of the isolated one. Warm caches on the master branch looked broken because they were being shared across runs. I spent time debugging Bundler before I realized the benchmark itself was wrong. Claude didn’t catch it because it doesn’t have the deep institutional knowledge of how Bundler’s cache layers interact. I caught it because I knew what the numbers should look like and they didn’t add up.

That’s not a reason to stop using AI. It’s a reminder to not outsource our thinking and to always test our assumptions. Applying engineering rigor is how we can be sure the work we’re doing, whether it’s us or AI doing it, is valid and achieves our goals.

The toolkit is available at bundler-perf-toolkit. If you’re working on Bundler performance or just curious about how your Gemfile affects install times, give it a try. We welcome PRs with new scenarios, corrections to cache handling if you spot something we got wrong, and support for other tools to test against.

Intro

Since the post at the end of last year, ZJIT has grown and changed in some exciting ways. This is the story of how a new, self-contained optimization pass causes ZJIT performance to surpass YJIT on an interesting microbenchmark. It has been 10 months since ZJIT was merged into Ruby, and we’re now beginning to see the design differences between YJIT and ZJIT manifest themselves in performance divergences. In this post, we will explore the details of one new optimization in ZJIT called load-store optimization. This implementation is part of ZJIT’s optimizer in HIR. Recall that the structure of ZJIT looks roughly like the following.

flowchart LR
        A(["Ruby"])
        A --> B(["YARV"])
        B --> C(["HIR"])
        C --> D(["LIR"])
        D --> E(["Assembly"])

This post will focus on optimization passes in HIR, or “High-level” Intermediate Representation. At the HIR level, we have two capabilities that are distinct from other compilation stages. Our optimizations in HIR typically utilize the benefits of our SSA representation in addition to the HIR instruction effect system.

These are the current analysis passes in ZJIT without load-store optimization, as well as the order in which the passes are executed.

run_pass!(type_specialize);
run_pass!(inline);
run_pass!(optimize_getivar);
run_pass!(optimize_c_calls);
run_pass!(fold_constants);
run_pass!(clean_cfg);
run_pass!(remove_redundant_patch_points);
run_pass!(eliminate_dead_code);

Here’s where load-store optimization gets added.

  run_pass!(type_specialize);
  run_pass!(inline);
  run_pass!(optimize_getivar);
  run_pass!(optimize_c_calls);
+ run_pass!(optimize_load_store);
  run_pass!(fold_constants);
  run_pass!(clean_cfg);
  run_pass!(remove_redundant_patch_points);
  run_pass!(eliminate_dead_code);

Overview

Ruby is an object-oriented programming language, so CRuby needs to have some notion of object loads, modifications, and stores. In fact, this is a topic already covered by another Rails at Scale blog post. The shape system provides performance improvements in CRuby (both interpreter and JIT), but there is still plenty of opportunity to improve JIT performance. Sometimes optimizing interpreter opcodes one at a time leaves repeated loads or stores that can be cleaned up with a program analysis optimization pass. Before getting into the weeds about this pass, let’s talk performance.

Results

The setivar benchmark for ZJIT changes dramatically on 2026-03-06. This is when load-store optimization landed in ZJIT. At the time of this writing, ZJIT takes an average of 2ms per iteration on this benchmark, while YJIT takes an average of 5ms.

This is the second time that ZJIT has clearly surpassed YJIT. The first example is here.

At a high level, this means that ZJIT is over twice as fast as YJIT for repeated instance variable assignment, and more than 25 times faster than the interpreter!

A Troubling Development

However, there’s an important question we have to address - why should an optimization pass for object loads and stores have anything to do with instance variable assignment? It turns out that ZJIT’s High Intermediate Representation (HIR) uses LoadField and StoreField instructions both for both object instance variables, and for object shapes. We’re going to have to dig deeper into CRuby shapes and ZJIT HIR internals in order to make sense of this.

Background

So far, we’ve learned that HIR has LoadField and StoreField instructions. We’ve claimed that they are multi-purpose and that the performance wins come from optimizing object shapes, but that they can also apply to object instance variables. Because the algorithm works just as well for both situations, the rest of this post will focus on object instance variables. This allows us to demonstrate concepts in pure Ruby to make things more approachable.

Example

Let’s start with a simple example we can all agree on. Clearly this code snippet has a double store, and we can safely remove one of the @a = value calls.

class C
  def initialize
    value = 1
    @a = value
    @a = value
  end
end

Here’s the same code snippet with an example of the call we remove. Here, we have elided a redundant StoreField instruction.

  class C
    def initialize
      value = 1
      @a = value
-     @a = value
    end
  end

When should we remove LoadField and StoreField instructions? The HIR code snippets will come later. For now, we only need to know the mapping between Ruby and HIR for instance variable loads and stores.

Note: In a class’s initialize method, instance variable operations are likely to cause LoadField and StoreField instructions due to shape transitions. Outside of an initialize method, the loads and stores are more likely to be related to the instance variables themselves. We decided that more complicated Ruby code snippets would clarify the kind of LoadField or StoreField but overly clutter the code snippets in this post.

Cases

Let’s consider every edge case for our algorithm through short Ruby snippets to illustrate scenarios where we can and cannot elide LoadField or StoreField HIR instructions.

Note: The following examples could replace the value variable with the constant 1, but in ZJIT this could cause other optimizations such as constant folding to interfere with our load-store demonstrations. We will use these more complex code snippets in case the reader wants to follow along with a compiler explorer.

Redundant Store

class C
  def initialize
    value = 1
    @a = value
    # This store is redundant and should be elided in HIR
    @a = value
  end
end

Redundant Load

class C
  def initialize
    value = 1
    @a = value
    # We already know that this load is `value` and should be replaced
    @a
  end
end

Redundant Store with Aliasing

class C
  attr_accessor :a

  def initialize(value)
    @a = value
  end
end

class D
  attr_accessor :a

  def initialize(value)
    @a = value
  end
end

def multi_object_test
  x = C.new(1)
  y = D.new(1)
  new_x_val = 2
  new_y_val = 3
  x.a = new_x_val
  y.a = new_y_val
  # We would like to elide this (but currently do not)
  x.a = new_x_val
end

With variables pointing to distinct objects, we could elide the second store to object x. This is not currently implemented, but is a possible improvement with a technique called type-based alias analysis.

Required Store with Aliasing

class C
  attr_accessor :a

  def initialize(value)
    @a = value
  end
end

def multi_object_test
  x = C.new(1)
  y = x
  new_x_val = 2
  new_y_val = 3
  x.a = new_x_val
  y.a = new_y_val
  # We should not elide the second `x.a` assignment because the `y.a` assignment modifies `x`
  # The `x.a` store after this comment is no longer redundant
  x.a = new_x_val
end

With multiple multiple variables aliasing to the same object, we cannot elide the second store to x. While technically we could elide y.a = new_y_val and the initial y = x assignment, these improvements are out of scope for this post. The key point here is that aliasing needs to be considered. If we assume that y and x reference different objects and elide the second x.a = new_x_val call, we alter program behavior.

Required Store with Effects

def scary_method(obj)
  obj.a = "We have modified the object. The second store is no longer redundant"
end

class C
  attr_accessor :a

  def initialize(value)
    @a = value
  end
end

def effectful_operations_between_stores_test
  x = C.new(1)
  x.a = 5
  scary_method(x)
  # We want to elide this but `scary_method` can modify `x`
  x.a = 5
end

In this case, the second store looks redundant, but it might not be. An arbitrary Ruby method (or C call, or some HIR instructions) could modify the x object and breaks the assumptions we can make about the state of the x object. In such cases, we cannot perform load-store optimization.

The Algorithm

Key Idea

With these cases, we have covered everything needed to implement our load-store optimization algorithm. The algorithm is a lightweight abstract interpretation over objects. This approach allows us to minimize the computation required to perform our optimization pass while ensuring soundness. In layperson’s terms, this means that every load we replace and every store we eliminate will not change program behavior, but that we will potentially miss some loads or stores that could be eliminated.

Tricky Details

Basic Blocks

Our load-store optimization pass scans through basic blocks, searches for redundant loads and stores, and updates the HIR instructions accordingly. Unnecessary StoreField operations are elided, and unnecessary LoadField operations are replaced with the instruction already holding the value. While one key benefit of ZJIT is that it can optimize entire methods, load-store optimization is (for now) block-local only.

LoadField and StoreField Distinctions

So far, we’ve talked about elision and instruction removal. We can get away with deleting StoreField instructions because no other instructions point to StoreField instructions. Conversely, LoadField instructions do have dependencies and are referenced by other instructions. These references need to be fixed up. Each reference to LoadField gets replaced with the cached value that was the target of a load.

The WriteBarrier Instruction

ZJIT has WriteBarrier instructions to support garbage collection. These also can modify objects and act similarly to stores. We need to handle this case in our algorithm.

Pointer Intricacies

The pseudo code we are about to introduce uses the term “offset” to denote the number of bytes from the object’s base address in memory. We use this to detect redundant loads and stores, as well as clear the cache from effectful instructions and write barriers. However, it is not immediately obvious that simply checking offsets would be enough. How can we be sure that the memory regions we are tracking remain untouched by some other instruction? Fortunately, HIR instructions always point to the base of an object and use offsets that are in bounds of the object. If we have two offsets that are not equal, they cannot reference the same region of memory. If the offsets are equal, then object aliasing must be considered.

Algorithm Sketch

Here’s the pseudo-code for a given basic block.

For each HIR instruction in the basic block
    initialize an empty cache as a hashmap
    
    if instruction is `LoadField`
        check if the object, offset, and value triple is in the cache
        if so, delete instruction and replace references to it with the loaded value
        else, cache the loaded value with the object, offset pair as a key
        
    if instruction is `StoreField`
        check if the object, offset, and value triple is in the cache
        if so, delete the instruction
        else, remove each cache entry with the same offset (the flags field) to avoid aliasing issues
        
    if instruction is `WriteBarrier`
        # This instruction is needed for the garbage collector and is complex
        # It works similarly to `StoreField` in practice
        # This instruction is never removed but the cache cleaning is still needed
        remove each cache entry with the same offset to avoid aliasing issues
        
    if instruction can modify objects
        flush the cache
        
    else
        continue
          
return the pruned HIR instructions

Source Code

The source at the time of this writing can be found here.

HIR Improvements

After the optimization, here are examples of how the HIR changes.

This the new HIR for our first redundant load example.

  fn initialize@../scripts/double_load.rb:3:
  bb1():
    EntryPoint interpreter
    v1:BasicObject = LoadSelf
    v2:NilClass = Const Value(nil)
    Jump bb3(v1, v2)
  bb2():
    EntryPoint JIT(0)
    v5:BasicObject = LoadArg :self@0
    v6:NilClass = Const Value(nil)
    Jump bb3(v5, v6)
  bb3(v8:BasicObject, v9:NilClass):
    v13:Fixnum[1] = Const Value(1)
    PatchPoint SingleRactorMode
    v30:HeapBasicObject = GuardType v8, HeapBasicObject
    v31:CShape = LoadField v30, :_shape_id@0x4
    v32:CShape[0x80000] = GuardBitEquals v31, CShape(0x80000)
    StoreField v30, :@a@0x10, v13
    WriteBarrier v30, v13
    v35:CShape[0x80008] = Const CShape(0x80008)
    StoreField v30, :_shape_id@0x4, v35
-   v20:HeapBasicObject = RefineType v8, HeapBasicObject
    PatchPoint SingleRactorMode
-   v38:CShape = LoadField v20, :_shape_id@0x4
-   v39:CShape[0x80008] = GuardBitEquals v38, CShape(0x80008)
-   v40:BasicObject = LoadField v20, :@a@0x10
    CheckInterrupts
-   Return v40
+   Return v13

This the new HIR for our first redundant store example.

bb1():
  EntryPoint interpreter
  v1:BasicObject = LoadSelf
  v2:NilClass = Const Value(nil)
  Jump bb3(v1, v2)
bb2():
  EntryPoint JIT(0)
  v5:BasicObject = LoadArg :self@0
  v6:NilClass = Const Value(nil)
  Jump bb3(v5, v6)
bb3(v8:BasicObject, v9:NilClass):
  v13:Fixnum[1] = Const Value(1)
  PatchPoint SingleRactorMode
  v35:HeapBasicObject = GuardType v8, HeapBasicObject
  v36:CShape = LoadField v35, :_shape_id@0x4
  v37:CShape[0x80000] = GuardBitEquals v36, CShape(0x80000)
  StoreField v35, :@a@0x10, v13
  WriteBarrier v35, v13
  v40:CShape[0x80008] = Const CShape(0x80008)
  StoreField v35, :_shape_id@0x4, v40
  v20:HeapBasicObject = RefineType v8, HeapBasicObject
  PatchPoint NoEPEscape(initialize)
  PatchPoint SingleRactorMode
- v43:CShape = LoadField v20, :_shape_id@0x4
- v44:CShape[0x80008] = GuardBitEquals v43, CShape(0x80008)
- StoreField v20, :@a@0x10, v13
  WriteBarrier v20, v13
  CheckInterrupts
  Return v13

And that’s load-store optimization!

Design Discussion

You may notice that our optimization is pruning the graph of loads and stores on an object. We are solving a very similar problem to the SSA form baked into the HIR. While it would be great to have “more SSA” at the object level, this comes at a cost. Computing SSA at this level could necessitate structural changes to HIR and make things less ergonomic or more confusing in regions of the codebase outside of load-store optimization. In fact, this question of “more SSA” is a complex design decision and contentious topic with a rich history in compilers such as V8 or Jikes RVM. So far, we’ve decided to use a lightweight SSA representation in ZJIT that causes us to work a bit harder for certain optimization passes, yielding subtle design simplifications across the rest of HIR.

Future Work

There’s still a lot of exciting work to be done and there are improvements to be made before we hit diminishing returns. Dead store elimination utilizes many of the same ideas and could help improve object initialization performance. We could implement type based alias analysis, though this requires care, as type confusion bugs are quite dangerous in JIT compilers. See section 4.1 in the phrack article for further details.

Conclusion

Thanks for reading the first post about ZJIT’s optimizer. We have lots more to come, so stay tuned.

At Shopify, we want our development environments to be fast. Installing dependencies is slow, especially in an application as large as Shopify. bun and uv have dramatically improved install times for TypeScript and Python dependencies. What if we could do the same for Bundler and the Ruby community?

Our team at Shopify has been working on a series of improvements to Bundler and RubyGems. Bundler downloads gems up to 200% faster. Cloning git gems is now 3x faster in our monolith.

We were also able to decrease the overall bundle install time by 3.5x in one of our applications by precompiling gems thanks to cibuildgem, a new precompilation toolchain we’d love you to try!

Here’s an overview of the improvements we’ve made in the last few months:

Faster gem downloads

One impactful change was deceptively simple. Bundler’s HTTP fetcher had a connection pool size of 1. This meant that during parallel gems installation, every thread was fighting over a single HTTP connection.

Pink spikes are threads waiting for the connection to be available.

By increasing the pool of HTTP connections, Bundler can download more gems in parallel. The speed gain with this change is even more dramatic during peak hours when RubyGems.org is under heavy load, or when you are geographically far from the CDN, where latency amplifies the cost of waiting on a single connection.

To benchmark this change, we opted to only measure download and extraction time (no compilation of native extensions) and built a local gem server where we can control latency at our will.

This is the result in a freshly generated Rails application when all gems are served with a 100ms latency.

Scenario: rails (164 gems)
                             Cold     +/-                        Warm     +/-
  ------------------------------------------------------------------------------
  5 HTTP connections       5.86s   0.10s  baseline             4.17s   0.02s  baseline
  1 HTTP connection       19.80s   0.02s  237.6% slower        4.16s   0.02s  0.3% faster

Hotspots and optimizations

We regularly profile Bundler with different Gemfiles and identify hotspots we might optimize. While no single optimization is dramatic, their collective impact has been significant.

One such optimization involves gem installation. A .gem file is a compressed tarball — and gzip has a built-in integrity check: if decompression succeeds, the content is guaranteed to be intact. Despite this, RubyGems was walking every entry in the tarball and reading all bytes upfront as an explicit corruption check, before proceeding with installation. This redundant verification step was thrown away entirely, since a successful decompression already provides the same guarantee.

9-17% of the time installing a gem is spent verifying the tarball’s content.

Another hotspot during installation is the check RubyGems performs to determine whether a gem includes a RubyGems plugin and, if so, whether its plugin file needs to be regenerated. The vast majority of gems don’t include a RubyGems plugin, yet every gem pays the cost of a Dir.glob with an expensive pattern just to handle the small minority that do.

It turns out that unconditionally regenerating the plugin file is faster than performing this upfront check.

Bundler checking whether regenerating a gem plugin is required

Parallel git clones

Many Rails applications depend on gems sourced directly from git repositories. This is particularly useful if a gem has upstream changes that aren’t yet released. Previously, Bundler would fetch each git repository sequentially, even though there’s no technical limitation on fetching them all at once.

Shopify’s Core Rails monolith includes 33 git gems. After introducing this change to parallelize git clone, we saw a 3x performance improvement for fetching git gems.

Native extensions

By far the biggest bottleneck when running bundle install is the compilation of native extensions. Many gems in the Ruby ecosystem include C code that must be compiled on each developer’s machine when installed. Common examples are json, date, and bigdecimal. Even if your Gemfile doesn’t directly depend on native extensions, it’s likely they will be included in your Gemfile.lock as transitive dependencies.

An installer thread spending 92% of the time compiling the gem.

To illustrate how slow compilation is, we can run bundle install on a freshly generated Rails application.

Installing the 18 native extension gems accounts for 85% of the time spent running bundle install.

Precompiled gems

Remember when Nokogiri used to take forever to install? Those days are behind us thanks to the amazing work of its maintainer, Mike Dalessio. Mike updated the gem’s publishing pipeline to precompile its native extensions into platform-specific binaries and releases separate gems for each supported platform (macOS, Windows, Linux). Now Nokogiri installs as fast as pure Ruby gems.

Imagine if we extended this to the rest of the Ruby ecosystem. If the community works together to ship precompiled binaries for our most popular native-extension gems, everyone will benefit from a lightning-fast bundle install.

One way to build binary gems is with the popular Rake-compiler-dock toolchain, which provides a cross-compilation environment and allows compilation to run inside Docker containers. However, cross-compiling can be brittle and presents hard-to-debug issues. Compiling on the target platform is ultimately far more reliable.

Many CI providers now offer free access to cloud machines. GitHub Actions, for example, is widely popular, and the Ruby community has built many easy-to-use actions around it (e.g., ruby/setup-ruby). Could we apply the same approach and leverage those machines to natively compile binary gems?

Introducing cibuildgem

At Shopify, we wanted to build an easy-to-use tool to help developers release gems with precompiled binaries using a native compilation approach via GitHub Workflows.

cibuildgem lets you generate a standard GitHub Actions workflow. Once triggered, multiple jobs run to:

1. Compile the binaries and package the gems
2. Run a matrix of test suites
3. Verify the .gem files are not corrupted and installable
4. Release the gems to RubyGems.org

Releasing a binary gem with cibuildgem

We aimed to make cibuildgem easy and fast to set up. Since many gems with native extensions are already configured with Rake Compiler for development compilation, we chose to piggyback on that so cibuildgem can run without any extra configuration for most gems.

The workflow generated by cibuildgem is intentionally standard.

• Want to compile your gem on Linux AArch64? Add it to the matrix.
• Want to trigger the workflow automatically when pushing a new git tag? No problem — tweak it to your liking.

We also wanted to ensure that the binaries compiled by cibuildgem would work in a macOS development environment and a Linux production environment on a real Rails application.

As an experiment, we used cibuildgem to compile dozens of open-source gems and publish them under a “namespace” on RubyGems.org (e.g., sassc -> precompiled-sassc).

The goal was to see how much performance improvement we could get with precompiled binaries. To test this, we created a Bundler plugin that hijacked the Bundler resolver to download the gems with precompiled binaries we had just published. For example, it would force-install precompiled-json if the json gem was requested anywhere in the dependency tree.

We tested this and deployed on an internal application which included 235 gems. By precompiling 17 of them, we saw a 3.5x performance improvement.

This experiment demonstrates how much faster bundle install could be when gems are precompiled. It has also given us confidence that cibuildgem builds compatible binaries for macOS and Linux.

In fact, a few gems at Shopify are now released with precompiled binaries (stack_frames, heap_profiler, rubydex) thanks to cibuildgem.

If you maintain a gem with a native extension, we’d love for you to give it a try and share your feedback ❤️!

ZJIT is a new just-in-time (JIT) Ruby compiler built into the reference Ruby implementation, YARV, by the same compiler group that brought you YJIT. We (Aaron Patterson, Aiden Fox Ivey, Alan Wu, Jacob Denbeaux, Kevin Menard, Max Bernstein, Maxime Chevalier-Boisvert, Randy Stauner, Stan Lo, and Takashi Kokubun) have been working on ZJIT since the beginning of this year.

In case you missed the last post, we’re building a new compiler for Ruby because we want to both raise the performance ceiling (bigger compilation unit size and SSA IR) and encourage more outside contribution (by becoming a more traditional method compiler).

It’s been a long time since we gave an official update on ZJIT. Things are going well. We’re excited to share our progress with you. We’ve done a lot since May.

In brief

ZJIT is compiled by default—but not enabled by default—in Ruby 4.0. Enable it by passing the --zjit flag or the RUBY_ZJIT_ENABLE environment variable or calling RubyVM::ZJIT.enable after starting your application.

It’s faster than the interpreter, but not yet as fast as YJIT. Yet. But we have a plan, and we have some more specific numbers below. The TL;DR is we have a great new foundation and now need to pull out all the Ruby-specific stops to match YJIT.

We encourage you to experiment with ZJIT, but maybe hold off on deploying it in production for now. This is a very new compiler. You should expect crashes and wild performance degradations (or, perhaps, improvements). Please test locally, try to run CI, etc, and let us know what you run into on the Ruby issue tracker (or, if you don’t want to make a Ruby Bugs account, we would also take reports on GitHub).

State of the compiler

To underscore how much has happened since the announcement of being merged into CRuby, we present to you a series of comparisons:

Side-exits

Back in May, we could not side-exit from JIT code into the interpreter. This meant that the code we were running had to continue to have the same preconditions (expected types, no method redefinitions, etc) or the JIT would safely abort. Now, we can side-exit and use this feature liberally.

For example, we gracefully handle the phase transition from integer to string; a guard instruction fails and transfers control to the interpreter.

def add x, y
  x + y
end

add 3, 4
add 3, 4
add 3, 4
add "three", "four"

This enables running a lot more code!

More code

Back in May, we could only run a handful of small benchmarks. Now, we can run all sorts of code, including passing the full Ruby test suite, the test suite and shadow traffic of a large application at Shopify, and the test suite of GitHub.com! Also a bank, apparently.

Back in May, we did not optimize much; we only really optimized operations on fixnums (small integers) and method sends to the main object. Now, we optimize a lot more: all sorts of method sends, instance variable reads and writes, attribute accessor/reader/writer use, struct reads and writes, object allocations, certain string operations, optional parameters, and more.

For example, we can constant-fold numeric operations. Because we also have a (small, limited) inliner borrowed from YJIT, we can constant-fold the entirety of add down to 3—and still handle redefinitions of one, two, Integer#+, …

def one
  1
end

def two
  2
end

def add
  one + two
end

Register spilling

Back in May, we could not compile many large functions due to limitations of our backend that we borrowed from YJIT. Now, we can compile absolutely enormous functions just fine. And quickly, too. Though we have not been focusing specifically on compiler performance, we compile even large methods in under a millisecond.

C methods

Back in May, we could not even optimize calls to built-in C methods. Now, we have a feature similar to JavaScriptCore’s DOMJIT, which allows us to emit inline HIR versions of certain well-known C methods. This allows the optimizer to reason about these methods and their effects (more on this in a future post) much more… er, effectively.

For example, Integer#succ, which is defined as adding 1 to an integer, is a C method. It’s used in Integer#times to drive the while loop. Instead of emitting a call to it, our C method “inliner” can emit our existing FixnumAdd instruction and take advantage of the rest of the type inference and constant-folding.

fn inline_integer_succ(fun: &mut hir::Function,
                       block: hir::BlockId,
                       recv: hir::InsnId,
                       args: &[hir::InsnId],
                       state: hir::InsnId) -> Option<hir::InsnId> {
    if !args.is_empty() { return None; }
    if fun.likely_a(recv, types::Fixnum, state) {
        let left = fun.coerce_to(block, recv, types::Fixnum, state);
        let right = fun.push_insn(block, hir::Insn::Const { val: hir::Const::Value(VALUE::fixnum_from_usize(1)) });
        let result = fun.push_insn(block, hir::Insn::FixnumAdd { left, right, state });
        return Some(result);
    }
    None
}

Fewer C calls

Back in May, the machine code ZJIT generated called a lot of C functions from the CRuby runtime to implement our HIR instructions in LIR. We have pared this down significantly and now “open code” the implementations in LIR.

For example, GuardNotFrozen used to call out to rb_obj_frozen_p. Now, it requires that its input is a heap-allocated object and can instead do a load, a test, and a conditional jump.

fn gen_guard_not_frozen(jit: &JITState,
                        asm: &mut Assembler,
                        recv: Opnd,
                        state: &FrameState) -> Opnd {
    let recv = asm.load(recv);
    // It's a heap object, so check the frozen flag
    let flags = asm.load(Opnd::mem(64, recv, RUBY_OFFSET_RBASIC_FLAGS));
    asm.test(flags, (RUBY_FL_FREEZE as u64).into());
    // Side-exit if frozen
    asm.jnz(side_exit(jit, state, GuardNotFrozen));
    recv
}

More teammates

Back in May, we had four people working full-time on the compiler. Now, we have more internally at Shopify—and also more from the community! We have had several interested people reach out, learn about ZJIT, and successfully land complex changes. For this reason, we have opened up a chat room to discuss and improve ZJIT.

A cool graph visualization tool

You have to check out our intern Aiden’s integration of Iongraph into ZJIT. Now we have clickable, zoomable, scrollable graphs of all our functions and all our optimization passes. It’s great!

Try zooming (Ctrl-scroll), clicking the different optimization passes on the left, clicking the instruction IDs in each basic block (definitions and uses), and seeing how the IR for the below Ruby code changes over time.

class Point
  attr_accessor :x, :y
  def initialize x, y
    @x = x
    @y = y
  end
end

P = Point.new(3, 4).freeze

def test = P.x + P.y

More

…and so, so many garbage collection fixes.

There’s still a lot to do, though.

To do

We’re going to optimize invokeblock (yield) and invokesuper (super) instructions, each of which behaves similarly, but not identically, to a normal send instruction. These are pretty common.

We’re going to optimize setinstancevariable in the case where we have to transition the object’s shape. This will help normal @a = b situations. It will also help @a ||= b, but I think we can even do better with the latter using some kind of value numbering.

We only optimize monomorphic calls right now—cases where a method send only sees one class of receiver while being profiled. We’re going to optimize polymorphic sends, too. Right now we’re laying the groundwork (a new register allocator; see below) to make this much easier. It’s not as much of an immediate focus, though, because most (high 80s, low 90s percent) of sends are monomorphic.

We’re in the middle of re-writing the register allocator after reading the entire history of linear scan papers and several implementations. That will unlock performance improvements and also allow us to make the IRs easier to use.

We don’t handle phase changes particularly well yet; if your method call patterns change significantly after your code has been compiled, we will frequently side-exit into the interpreter. Instead, we would like to use these side-exits as additional profile information and re-compile the function.

Right now we have a lot of traffic to the VM frame. JIT frame pushes are reasonably fast, but with every effectful operation, we have to flush our local variable state and stack state to the VM frame. The instances in which code might want to read this reified frame state are rare: frame unwinding due to exceptions, Binding#local_variable_get, etc. In the future, we will instead defer writing this state until it needs to be read.

We only have a limited inliner that inlines constants, self, and parameters. In the fullness of time, we will add a general-purpose method inlining facility. This will allow us to reduce the amount of polymorphic sends, do some branch folding, and reduce the amount of method sends.

We only support optimizing positional parameters, required keyword parameters, and optional parameters right now but we will work on optimizing optional keyword arguments as well. Most of this work is in marshaling the complex Ruby calling convention into one coherent form that the JIT can understand.

Performance

We have public performance numbers for a selection of macro- and micro-benchmarks on rubybench. Here is a screenshot of what those per-benchmark graphs look like. The Y axis is speedup multiplier vs the interpreter and the X axis is time. Higher is better:

You can see that we are improving performance on nearly all benchmarks over time. Some of this comes from from optimizing in a similar way as YJIT does today (e.g. specializing ivar reads and writes), and some of it is optimizing in a way that takes advantage of ZJIT’s high-level IR (e.g. constant folding, branch folding, more precise type inference).

We are using both raw time numbers and also our internal performance counters (e.g. number of calls to C functions from generated code) to drive optimization.

Try it out

While Ruby now ships with ZJIT compiled into the binary by default, it is not enabled by default at run-time. Due to performance and stability, YJIT is still the default compiler choice in Ruby 4.0.

If you want to run your test suite with ZJIT to see what happens, you absolutely can. Enable it by passing the --zjit flag or the RUBY_ZJIT_ENABLE environment variable or calling RubyVM::ZJIT.enable after starting your application.

On YJIT

We devoted a lot of our resources this year to developing ZJIT. While we did not spend much time on YJIT (outside of a great allocation speed up), YJIT isn’t going anywhere soon.

Thank you

This compiler was made possible by contributions to your PBS station open source project from programmers like you. Thank you!

• Aaron Patterson
• Abrar Habib
• Aiden Fox Ivey
• Alan Wu
• Alex Rocha
• André Luiz Tiago Soares
• Benoit Daloze
• Charlotte Wen
• Daniel Colson
• Donghee Na
• Eileen Uchitelle
• Étienne Barrié
• Godfrey Chan
• Goshanraj Govindaraj
• Hiroshi SHIBATA
• Hoa Nguyen
• Jacob Denbeaux
• Jean Boussier
• Jeremy Evans
• John Hawthorn
• Ken Jin
• Kevin Menard
• Max Bernstein
• Max Leopold
• Maxime Chevalier-Boisvert
• Nobuyoshi Nakada
• Peter Zhu
• Randy Stauner
• Satoshi Tagomori
• Shannon Skipper
• Stan Lo
• Takashi Kokubun
• Tavian Barnes
• Tobias Lütke

(via a lightly touched up git log --pretty="%an" zjit | sort -u)

Ruby has always been a joy to write. But for a long time, reading Ruby documentation on docs.ruby-lang.org hasn’t really matched that experience.

Last year, I brought a new look to the Darkfish theme by updating its visuals and improving mobile support. It was a visible improvement, but it wasn’t enough.

So this year, I built something new from the ground up. Starting with RDoc 7.0.0, Aliki is now the default theme for RDoc.

This release also coincides with Ruby’s 30th anniversary and the redesign of ruby-lang.org—a great moment to give Ruby’s documentation a fresh look as we head into the next chapter with Ruby 4.0.

Why a New Theme?

Even after last year’s improvements, I still didn’t enjoy using docs.ruby-lang.org as much as I wanted. Every time I needed to look something up, the experience felt dated.

And it was difficult to further improve Darkfish because:

• It lacks documentation, especially around the original design decisions
• Some of the patterns it uses were outdated
• Some third-party themes build on Darkfish, so updating it too much risks breaking them

RDoc itself added more constraints: It can’t depend on any gem that doesn’t ship with Ruby itself, and it can’t run a modern JavaScript build pipeline.

RDoc was created in the pre-Node.js era and hasn’t evolved with frontend tooling. Adopting modern toolchains would raise the dependency requirements for everyone—Ruby’s documentation generation pipeline, gems like IRB and Reline, and so on.

So all JavaScript, CSS, and HTML had to be written directly—no frameworks, no build tools, no npm packages.

And honestly, given all the constraints, this project wouldn’t have been possible without AI coding agents. Last year, just getting Darkfish’s code block styling right took me hours. It was a struggle for me to implement the look I wanted, and then to make it work with surrounding elements. At that pace, building an entire new theme wasn’t realistic.

This year, however, I discovered that I could try three different UI styles in an hour with AI agents. So I decided to take on the impossible task.

My goal was simple: make docs.ruby-lang.org look modern and actually enjoyable to use.

I collected all the features I wished a documentation site would have, gathered feedback from Rubyists around me, cherry-picked the improvements the community added to Darkfish last year (SEO, search enhancements, etc.), and put them together into a new theme.

Ok, enough of the back stories. Let’s see what Aliki brings:

Search

The old search wasn’t intuitive. It supported fuzzy matching, but getting the sorting right was difficult—searching Arr never actually got you the Array class as the first result.

After a few patch-ups, it was still buggy, so I rewrote it with a new UI:

Some notable new features/improvements:

• Case-aware ranking: If you search parse (lowercase), methods show up first. If you search Parser (capitalized), classes and modules come first.
• Fuzzy matching: This existed before, but fuzzy results used to pollute the top of the list. Now we have a smarter ranking system to make sure exact/substring matches show up before fuzzy results.
• Constants included: You can now search for constants, along with classes, modules, and methods.
• Type labels: Each result shows whether it’s a class, module, method, or constant.
• Keyboard support: Did you know you can press / to focus the search bar? This existed in Darkfish too, but I thought it was worth mentioning.

Dark Mode

Aliki has a light/dark toggle. It saves your preference and respects your OS dark mode setting by default.

Layout

The layout has three columns:

• Left sidebar: Navigation for pages, ancestors, methods, and class/module index
• Center: The documentation content
• Right sidebar: A table of contents generated from headings, with the current section highlighted as you scroll

Sidebar sections can collapse. For example, when you’re on a class or module page, the pages section is automatically collapsed so you can focus on the relevant navigation, with page documents still accessible if you need them.

Speaking of pages, I also reorganized the pages list this year. It used to be a long, rather flat list—now pages are grouped and much easier to navigate. In the coming year, we’ll continue improving page documentation so it feels more like a coherent guide rather than a collection of loosely related pages.

On mobile, the layout is a single column with a hamburger menu and a full-screen search modal—same as before.

Code Features

Code blocks now have copy buttons:

C code is now highlighted too:

For Gem Documentation

Aliki works for any gem, not just Ruby core. If you generate documentation with RDoc 7.0+, your users will see this theme automatically.

You can also customize the footer links now. For example, in your .rdoc_options:

footer_content:
  DOCUMENTATION:
    Home: index.html
  RESOURCES:
    GitHub Repository: https://github.com/your/repo
    Issue Tracker: https://github.com/your/repo/issues

This is useful for linking to your gem’s repository, issue tracker, or other resources.

To keep using Darkfish in your project, you can switch back with:

generator_name: darkfish

Acknowledgments

Thanks to @tompng and @earlopain for reviewing the code and helping polish things up.

Try It Out

You can see Aliki at docs.ruby-lang.org/en/master/ or ruby.github.io/rdoc/.

If you find issues or have suggestions, open an issue on GitHub.

What’s Next

Now that reading docs is enjoyable again, the next step for RDoc is to make writing docs enjoyable too.

About the Name

Aliki is my cat. I’m not good at naming things, so I just named the new theme after her.

This post explores Rails’s swappable migration backend, a little-known feature that lets applications customize how migrations run. At Shopify, we relied on monkey patches and a brittle SQL parser to make Rails migrations work with our Schema Migrations Service. We developed the swappable backend feature to more simply adapt Rails’s migration runner to our needs. We’ll cover why and how we built this, and how Shopify uses it to power database migrations at scale.

At Shopify, we run hundreds of database migrations across many Rails applications every week. Each migration needs to be vetted for safety and executed in a way that doesn’t cause downtime for our merchants. For years, we relied on bespoke tooling and LHMs to perform online schema changes at scale. In 2021, Shopify’s database team began designing a new, centralized system for running schema migrations, the Schema Migrations Service. One of their goals was to enable developers to use vanilla Rails migrations to perform schema changes safely and with zero downtime.

Our database team built the schema migrations gem to solve this problem, but the implementation wasn’t simple. The gem relied on monkey patches and a complicated RACC parser to handle safety checking migrations and submitting them to the schema migrations service. Shopify’s Rails Infrastructure team took the opportunity to build something into the framework that would help us address our schema migration needs more elegantly. We built the swappable migration backend (available as of Rails 7.0) to give applications flexibility over how their migrations execute. Let’s dive into how Shopify uses this feature to power safe database migrations at scale.

Why Production Migrations Require a Different Approach

When you run bin/rails db:migrate in development, Rails executes your migration methods directly against the database. Each call to create_table, add_column, or add_index immediately translates to SQL that modifies your schema. This works great for local development, but at Shopify’s scale, we can’t afford to run schema changes this way in production.

LHMs are a tool for performing online schema migrations. This means that migrations can be performed without locking tables, enabling the system to stay up while the migration is running. We used LHMs for many years to perform schema changes without downtime, but this also meant that we couldn’t use Rails’s native migration API.

Shopify’s database team decided to build a Schema Migrations Service to allow developers to return to using vanilla Rails migrations, while ensuring that schema changes were still performed online behind the scenes. The idea was also to improve the developer experience around migrations by:

1. Requiring migrations to pass safety checks before execution (e.g. blocking column-change operations, ensuring a migration only operated on a single table, etc.).
2. Submitting migrations to a centralized manager to more easily orchestrate schema changes across multiple database shards, with better testing and retries behaviour.
3. Providing developers with more insight into which migrations were running, their progress, etc. from a comprehensive UI.

The schema migrations gem built by our DB team handled safety checking migrations and submitting them to the centralized manager. The initial implementation, however, relied heavily on monkey patches to existing migration codepaths in Rails. Rather than executing migration SQL, the gem patched Rails to capture any SQL statements. It relied on a RACC parser to extract schema change operations from the SQL, safety check them, and then transform them into a JSON DDL (Data Definition Language) to be sent to the manager.

The Rails Infrastructure team realized that this was a great opportunity to make Rails’ migration execution more flexible, so that we could meet Shopify’s schema migration needs without needing to monkey patch a bunch of code or maintain a complicated RACC parser.

Building Rails’s Swappable Migration Strategy

When we started this project in early 2022, we explored several approaches that would allow us to move away from monkey patching Rails in the gem. One idea was to use static analysis, and try to parse migration files without running them. Another was to propose schema definition objects for every migration operation, where Rails would expose Ruby representations of schema changes (like AddColumnDefinition, CreateTableDefinition, etc.) that could be translated into any format: SQL, JSON, or otherwise.

The Rails Core team had concerns about the complexity that schema definitions would introduce to Active Record, and we soon pivoted to a simpler approach: the strategy pattern. Instead of fundamentally changing how migrations represent schema changes, we’d introduce an intermediary object between migrations and the connection adapter that could customize execution behavior. This was a cleaner abstraction that solved our problem without requiring massive changes to Active Record’s internals.

In June 2022, we opened a pull request to Rails proposing this “execution strategy” pattern for migrations. The PR introduced a strategy object between the Migration class and the connection adapter. Instead of migrations directly delegating schema statement commands to the connection via method_missing, they would delegate to a strategy object that could be swapped out.

For example, suppose you call a method like create_table in a migration. Rails routes that call through a migration strategy object, which by default, is ActiveRecord::Migration::DefaultStrategy:

module ActiveRecord
  class Migration
    class DefaultStrategy < ExecutionStrategy
      private
        def method_missing(method, ...)
          connection.send(method, ...)
        end

        def respond_to_missing?(method, include_private = false)
          connection.respond_to?(method, include_private) || super
        end

        def connection
          migration.connection
        end
    end
  end
end

The default strategy sends migration methods to the connection, which executes SQL against your database. This is how migrations worked before, so most Rails developers are unaware that there’s now a strategy object working behind the scenes! However, the migration strategy class can be configured to customize how migrations are executed. As of Rails 7.0, you can set config.active_record.migration_strategy in your environment configuration (for example, in config/environments/production.rb). Pass it either a class object or a string with the class name:

# lib/custom_migration_strategy.rb

class CustomMigrationStrategy < ActiveRecord::Migration::DefaultStrategy
  def drop_table(*)
    raise "Dropping tables is not supported!"
  end
end

# config/environments/production.rb

Rails.application.configure do
  config.active_record.migration_strategy = CustomMigrationStrategy
end

Now, when you run bin/rails db:migrate, Rails will delegate all migration methods to your custom strategy, giving you complete control over how migrations are executed.

Note: Outside of production, you will likely want to stick with the default strategy for local development. This setup lets you safely use advanced migration tooling in production while keeping things fast and simple for local development. We do this at Shopify.

Serializing Production Migrations to JSON

Once Rails supported swappable migration backends, we implemented a custom strategy that serialized migrations as JSON, making them easy to submit to a remote manager. To accomplish this, our gem introduced a JsonSerializationStrategy class. This class implemented each schema change method available in migrations, using Rails’s schema definition APIs to build the necessary schema objects. We then converted these objects into JSON payloads that described each schema operation. Here’s an example of how we capture create_table operations:

class JsonSerializationStrategy < ActiveRecord::Migration::DefaultStrategy
  attr_accessor :connection, :operations

  def initialize(connection)
    @connection = connection
    @operations = []
  end

  def create_table(...)
    td = connection.build_create_table_definition(...)
    ddl = connection.schema_creation.accept(td)
    definition = extract_table_definition(td.name, ddl)

    operations << {
      type: :sql,
      op: :create_table,
      params: {
        name: td.name,
        definition: definition,
      },
    }
  end

  private

  def extract_table_definition(table_name, ddl)
    table_name_pattern = /^CREATE TABLE #{connection.quote_table_name(table_name.to_s)} /
    ddl.sub(table_name_pattern, "")
  end
end

Here’s a simplified look at how migrations are run in production, using the swappable strategy:

class ExternalMigrationsRunner
  def upload_migration(migration)
    # Run the migration, but since we're using the JsonSerializationStrategy,
    # we won't execute SQL; instead, the strategy captures all operations as JSON
    runnable_migration = migration.migration_class.new
    if runnable_migration.respond_to?(:change)
      runnable_migration.change
    elsif runnable_migration.respond_to?(:up)
      runnable_migration.up
    end

    # Extract the serialized operations from the strategy
    operations = runnable_migration.execution_strategy.operations

    # Upload to the migrations service via API
    ApiClient.upload_migration(
      name: migration.name,
      database: database_name,
      identifier: migration.version,
      operations: operations,  # JSON representation of schema changes
      table_name: migration.table,
      author: migration.author
    )
  end
end

Configuring the Migration Strategy Automatically

Rather than requiring each application to configure the migration strategy in their config file for production, the schema migrations gem leveraged an initializer to set this automatically:

# lib/schema_migrations/railtie.rb
require "rails/railtie"

class Railtie < Rails::Railtie
  ...

  initializer "schema_migrations.migration_strategy_config" do |app|
    next unless Rails.env.production?

    app.config.active_record.migration_strategy = JsonSerializationStrategy
  end
end

This initializer ensures that any application that includes the schema migrations gem has its migrations intercepted and serialized in production environments.

Reimagining Safety Checks: From SQL Parsing to Runtime Analysis

While working on the upstream strategy feature, our team was simultaneously tackling another critical problem: safety checks. Before any migration runs in production at Shopify, the gem performs safety checks to catch common mistakes that could cause downtime, such as:

• Adding a NOT NULL column without a default value (check out this blog post if you’re interested in learning more)
• Renaming a column (breaks downstream consumers)
• Changing a column type in an incompatible way

These checks run in development too, giving developers immediate feedback before they deploy.

The old implementation of the gem’s safety checker relied on a RACC parser to analyze SQL strings, which was brittle: every time SQL syntax changed or we encountered a new edge case, the parser had to be updated. We wanted a standalone workflow for being able to safety check migrations, separate from the migrations actually being executed and submitted to the manager. Consequently, we couldn’t rely on the migration strategy to do this. Instead, we settled on a new approach that would allow us to move away from the RACC parser and reduce a lot of the complexity. We developed a MigrationOperationRecorder that “runs” a migration and records all method calls performed:

class MigrationOperationRecorder
  def initialize(migration_class)
    @migration = migration_class.new
  end

  def record
    singleton_class = @migration.singleton_class
    singleton_class.include(RecordMigrationOperations)

    if @migration.respond_to?(:change)
      @migration.change
    elsif @migration.respond_to?(:up)
      @migration.up
    end

    @migration.method_calls
  end
end

The RecordMigrationOperations module works by leveraging the same method_missing mechanism that Rails uses for migrations. Since ActiveRecord::Migration uses method_missing to route commands to the execution strategy, we define RecordMigrationOperations#method_missing to store the method call instead:

module RecordMigrationOperations
  def method_missing(method, *args, **options, &block)
    @method_calls << MigrationOperation.new(
      method: method,
      args: args,
      options: options
    )
  end

  def method_calls
    @method_calls ||= []
  end
end

Once operations are recorded, individual safety checks can inspect the migration data. Here’s an example of the SingleTableCheck:

class SingleTableCheck < BaseSafetyCheck
  def initialize(migration)
    @inspected_migration = migration
  end

  def check
    # @inspected_migration is a specialized object containing info
    # about all of the operations the migration performs, as returned
    # from MigrationOperationRecorder#record
    tables = @inspected_migration.tables

    return if tables.one?

    raise SafetyCheckError,
      "You must work with exactly one table per migration. " \
      "Split tables #{tables.to_sentence} into #{tables.length} migrations."
  end
end

This check accesses @inspected_migration.tables, which is extracted during the analysis phase, and validates that exactly one table is involved. If the check fails, it raises a SafetyCheckError with a clear message telling developers how to fix the issue.

Why Not Use a Migration Strategy for Safety Checking?

You might wonder why we used method_missing for the MigrationOperationRecorder instead of creating another strategy pattern. Couldn’t we use our newly built feature for safety checking? The answer comes down to separation of concerns and simplicity. Safety checking and migration execution serve different purposes:

• Migration execution needs to be swappable because different environments (development vs. production) require different behaviours. In development, we execute SQL directly. In production, we serialize to JSON and submit to a remote service.

• Safety checking needs to happen the same way everywhere. We’re analyzing which operations the migration is performing, not executing schema changes. The same safety checks run in development, CI, and production.

Using method_missing for safety checks gives us a simpler implementation that automatically captures all migration DSL methods without needing to explicitly enumerate them all. A strategy pattern would have required us to implement every migration method explicitly. Given that we only wanted to record the migration methods being called and their arguments, opting for a simpler method_missing approach made more sense.

Per-Adapter Migration Strategies

One challenge with using a global migration strategy is that it’s insufficient for applications using multiple database systems. Since its inception, Shopify has primarily used MySQL, but more recently we’ve been exploring running non-MySQL databases. Different databases have different requirements for how migrations should be serialized, which means that the migration strategy needs to be tailored to database the migrations are running against.

We could make this work by having our gem’s migration strategy inspect the database adapter at runtime and dispatch to the appropriate serialization logic. This is not ideal, though; we’re reimplementing adapter dispatch logic that Rails can handle natively. It felt like this was a missing piece in our upstream solution, so last month, we opened a PR to add per-adapter migration strategies to Rails. This feature will be available in Rails 8.2.

Instead of setting one global strategy:

config.active_record.migration_strategy = JsonSerializationStrategy

You can now register strategies directly on adapter classes:

ActiveSupport.on_load(:active_record_trilogyadapter) do
  ActiveRecord::ConnectionAdapters::TrilogyAdapter.migration_strategy =
    MysqlStrategy
end

ActiveSupport.on_load(:active_record_postgresqladapter) do
  ActiveRecord::ConnectionAdapters::PostgreSQLAdapter.migration_strategy =
    PostgreSQLStrategy
end

Rails automatically selects the correct strategy based on the database adapter in use for each migration. For example, if you’re running migrations against a MySQL database configured with the Trilogy adapter, Rails chooses MysqlStrategy. If your migrations are running against a PostgreSQL database, Rails selects PostgreSQLStrategy. If the current adapter does not have a strategy configured, Rails will fall back to using the global strategy.

Making Rails Work for You

One of Rails’s design philosophies is convention over configuration. The majority of Rails apps don’t need to think about how their Rails migrations are performed, so we keep things simple with a default migration strategy. At the point where an application needs to customize how their migrations run, the framework provides a clear extension point. Applications can opt-into configurable behaviour as their requirements evolve.

This is also a story about how working in the open benefits everyone. We could have kept our monkey patches internal to Shopify, continuing to patch Rails as needed. Instead, we built a more maintainable solution for ourselves while also providing the Rails community with a new tool for customizing migration behaviour. If you’re running into limitations with Rails for your specific use case, consider whether there’s an opportunity for an upstream contribution that could solve your problem while benefitting the rest of the community.

ZJIT adds support for Iongraph, which offers a web-based, pass-by-pass viewer with a stable layout, better navigation, and quality-of-life features like labeled backedges and clickable operands.

Prelude

I’m an intern on the ZJIT team for the fall term. I also have a rather bad habit of being chronically on lobste.rs.

While idly browsing, I spotted an article by Ben Visness titled Who needs Graphviz when you can build it yourself?, which covers his work on creating a novel graph viewer called Iongraph.

Immediately, I was intrigued. I like looking at new technology and I wondered what it might be like to integrate the work done in Iongraph with ZJIT, getting all sorts of novel and interesting features for free. I suspected that it could also help other engineers to reason about how their optimizations might affect the control flow graph of a given function.

Also, it just looks really cool. It’s got nice colours, good built-in CSS, and is built in a fairly extensible way. The underlying code isn’t hard to read if you need to make changes to it.

Investigating further

Iongraph is compelling for a few reasons.

It supports stable layouts, which means that removing or adding nodes (something that can happen when you run an optimization pass) doesn’t shift the location of other nodes to an extreme degree. Iongraph also gives all sorts of interactive options, like clickable operands, scrollable graphs, or arrow keys to navigate between different nodes.

An especially useful feature is the ability to switch between different compiled methods with a small selector. In our codebase, ZJIT compiles each method on its own, so using a tool like this allows us to inspect method level optimizations all in one pane of a web browser. Of course, there are other great features, like loop header highlighting or being able to click on optimization passes to see what the control flow graph looks like after they’re applied.

Proposal

Roughly an hour after I read through said article, I noticed that my mentor, Max, had also posted it in an internal team chat, mentioning that it would be cool to support it.

Of course, I was tempted by this project. As is a common trait for interns, I was tempted to take on a new, shiny project despite not knowing what it might imply to actually develop it. After talking to Max further, he clarified that this would require significant infrastructure work — or at the very least, more than initially apparent.

Building

A JSON library inside ZJIT?

Looking into the Iongraph format, I figured that I would have to use some sort of JSON crate. Since ZJIT as a project doesn’t rely strictly on using Rust tooling like cargo, directly adding serde_json as a dependency was out of the question. Another compelling option was vendoring it (or a smaller JSON library), but that was likely to include features that we did not want or introduce licensing issues.

After a quick discussion, I settled on implementing the functionality myself. I read a bit of the JSON specification, and got a sense of the ideal way to design the library’s API. Ultimately, I chose to opt for readability and usability over raw performance. This decision I think is reasonable given that the serialization code is not in the critical path of the compiler. It’s also accurate to say that the interface is clean enough to replace the internals in the future with minimal issue should there be more performance needed.

In designing the serializer, I chose to target RFC 8259, which provides more freedom than previous specifications. As noted in said RFC, historical specifications constrained the top level value to be an array or an object, but this spec (and my implementation) don’t require that constraint. I also opted to avoid comments, encode strictly in UTF-8, and escape control characters. Notably, RFC 8259 does not impose a limit on precision of numbers, just that infinity, negative infinity, or NaN are restricted.

Computing control flow graph properties

With JSON serialization handled, the more challenging work was computing the graph metadata that Iongraph requires. The format expects explicit successor and predecessor relationships, loop headers, and back edge sources — information that ZJIT doesn’t normally compute since it’s not needed for compilation at this stage of compiler development.

One constraint I had to contend with was that the Iongraph format needs the user to manually provide the successor and predecessor nodes for a given node in a control flow graph. In ZJIT, we compile individual methods at a time as Functions (our internal representation) that hold a graph of Blocks. Each Block is a basic block that you would find in a compiler textbook. (One caveat to understand is that we use extended basic blocks, meaning that blocks can have jump instructions at any point in their contained instructions — not just at the end.)

The process of computing successors and predecessors is fairly simple. As you iterate through the list of blocks, all blocks referenced as the target of a jump-like instruction (whether conditional or unconditional) are added to the successor set. Then for each successor, update their predecessor set to include the block currently being operated on.

The next task I had to solve was computing the loop headers and back edge sources.

Required in the process of computing both of these are computing the dominators for blocks in a control flow graph. We can state that a block i dominates a block j if all paths in the control flow graph that reach j must go through i. Several algorithms exist for computing dominators. There exist both simple iterative options and more complicated versions. Initially, I heard of a fixed point iteration option that was very straightforward to implement but perhaps not the most efficient. That one (which I will discuss shortly) runs in quadratic time to the number of blocks available. In A Simple, Fast Dominance Algorithm by Cooper, Harvey, and Kennedy, both this iterative solution as well as one that is optimized to use less space are mentioned. A third option is the Lengauer-Tarjan algorithm, which has better worst case bounds compared to both the iterative and tuned implementations.

Based on the goals of the project, I opted to use the iterative algorithm, since it performs well and doesn’t incur serious memory use penalties for a small number of blocks in a control flow graph. It can be described as such:

dom = {}
nodes.each do |node|
  if entry_nodes.include?(node)
    dom[node] = Set[node]
  else
    dom[node] = nodes.to_set
  end
end

changed = true
while changed
  changed = false
  nodes.reverse_post_order.each do |node|
    preds = predecessors(node)
    pred_doms = preds.map { |p| dom[p] }

    # Intersection of all predecessor dominators
    intersection = if pred_doms.empty?
                     Set.new
                   else
                     pred_doms.reduce(:&)
                   end

    # Union with {node}
    new_set = intersection | Set[node]

    if new_set != dom[node]
      dom[node] = new_set
      changed = true
    end
  end
end

Implementing this is fairly simple, and it runs quickly enough for the limited number of nodes that it is totally acceptable.

To compute successors we use the following snippet:

let successors: BTreeSet<BlockId> = block
    .insns
    .iter()
    .map(|&insn_id| uf.find_const(insn_id))
    .filter_map(|insn_id| {
        Self::extract_jump_target(&function.insns[insn_id.0])
    })
    .collect();

Here we go through all the instructions in a given block. We use a union find data structure to map instructions to their canonical representatives (since some optimizations may have merged or aliased instructions). We then filter by extract_jump_target, which returns an Option that contains a BlockId for jump-like instructions.

After finding successors, we can set the predecessors by iterating through the nodes in the successor set and adding the current node to their predecessor sets.

The last important thing we need to consider is finding the loop depth.

For finding this, we need to consider first how we can find a natural loop in the first place.

We identify natural loops by detecting back edges. A back edge occurs when a block has a predecessor that is dominated by that block (all paths to the predecessor pass through this block). When we find such an edge, the target block is a loop header and the predecessor is the source of a back edge. The natural loop consists of all blocks on paths from the back edge source to the loop header (excluding the header itself). Each block within this natural loop then has its loop depth incremented.

These additional computations are used within the Iongraph layout engine to determine the height at which a given block should be vertically, or where lines should be routed within the graph. Loop headers and back edge sources are also marked!

The final result

You can click around this demo graph showing a simple example from ZJIT to get a sense of how Iongraph works! Operands are clickable to get to their definition. You can click on the phases of optimization on the left side - note that only the non-grayed out passes will have made changes. The graph is also zoomable and scrollable!

Hopefully this post was educational! I learned a lot implementing this feature and enjoyed doing so.

If you would like to do some work on ZJIT (and learn a lot in the process), you are welcome to make pull requests to github.com/ruby/ruby/ with the commit prefix ZJIT:. You can find issues here.

Also, feel free to join our Zulip!

This blog post was adapted from our paper and talk at the International Symposium on Memory Management 2025.

We would first like to acknowledge the late Chris Seaton, who initiated our collaboration with the Australian National University on this project. We are thankful for his contribution, vision, and leadership. Without him, none of this would have been possible.

Background

The Australian National University (ANU) and Shopify are collaborating on integrating the Memory Management Toolkit (MMTk) with Ruby. We are supporting the project and working alongside ANU researchers to explore how to build a next-generation garbage collector for Ruby.

If you’re not familiar with MMTk, it offers a highly modular, VM-neutral framework for rapidly building high-performance garbage collectors. Once a language plugs into MMTk, it can leverage a wide range of built-in garbage collection algorithms, ranging from canonical collectors such as NoGC, Mark and Sweep, and Immix to more performant collectors such as Generational Immix and Sticky Immix. Many of these algorithms are considerably more sophisticated than the Mark and Sweep algorithm used in Ruby and have the potential to deliver significant performance gains.

There are currently two implementations of MMTk in Ruby: one is maintained by the MMTk team and is a fork of Ruby (in the mmtk/ruby and mmtk/mmtk-ruby repositories), the other lives inside Ruby using the modular GC framework (in the ruby/mmtk repository). You might be wondering, why are there two implementations? The MMTk team’s implementation is much more advanced, with around 5 years of development. They continue to use it to experiment and develop new techniques to further leverage MMTk’s powers and improve performance. The implementation upstreamed to Ruby uses the modular GC framework and is designed to be part of an ecosystem of garbage collectors for Ruby. However, it is a reimplementation that uses techniques and knowledge from the MMTk team’s implementation, but is still quite behind.

In this blog post, we will follow the paper and mostly focusing on MMTk team’s implementation. However, if you want to learn more about the modular GC framework, you can watch this talk at RubyKaigi 2025 or read this blog post.

Challenges

In the paper, we discuss some of the challenges we faced and solutions we used while integrating MMTk with Ruby. In this blog post, we highlight some of these challenges, but please read the paper if you want the entire picture.

Copying Garbage Collector

When Ruby 2.7 introduced a moving garbage collector, it marked the first time that the memory location of objects could be moved. To facilitate this, there needed to be additional code in each of the data types in Ruby to update the address of the object after it has been moved. To ensure backwards compatibility, each data type needed to opt-in to using a new API that supports object movement, and all the existing types would pin the objects they refer to. A pinned object cannot move.

This pinning system works for Ruby’s default (built-in) garbage collector, because it has a marking phase to determine objects that are live and objects that are pinned followed by a compaction phase to move non-pinned objects. However, many of MMTk’s algorithms combine the marking and moving phases, meaning that an object is moved the moment it is marked. For algorithms like Immix, objects can be pinned, but they must be specified ahead of time. One solution would be to scan the heap twice: first to determine which objects get pinned, and again to mark all live objects and move the unpinned objects. However, this is inefficient because it essentially involves scanning the whole Ruby heap twice.

Fortunately, it’s been more than 5 years since a moving garbage collector was introduced to Ruby, so almost all the types in Ruby and many native gems support it. We introduced a new concept called Potentially Pinning Parents, or PPP for short. An object is a PPP if it could potentially contain references that cannot be moved. Earlier this year, we made an effort to reduce PPP objects. In fact, as of the time of writing, there are no user-facing Ruby objects that are PPPs except for ones defined in native gems (which we do not have any control over). There are still a few internal Ruby objects that are PPPs, but we are working on eliminating those as well.

Since we now know whether an object is a PPP at allocation time, MMTk keeps a list of PPP objects that are alive. Using that list, during a garbage collection cycle, it inspects every PPP object to determine the child objects that should be pinned before moving onto the phase to mark and move objects. Since the set of PPP objects is now small, this phase can be completed very quickly.

Finalization

Before Ruby 3.2, all Ruby objects were allocated out of the garbage collector in fixed 40-byte slots. This meant that any additional data for the object needed to be allocated externally, usually through the system using malloc. In Ruby 3.2, we introduced Variable Width Allocation which allows us to allocate dynamic slot sizes through the garbage collector. However, because of legacy reasons and technical limitations of Variable Width Allocation, there are still many cases where we need to allocate memory out of the system through malloc.

One of the superpowers of MMTk is that it supports parallelism in the garbage collector. Unlike Ruby’s default garbage collector, MMTk can split the work that needs to be done during a GC cycle (marking, sweeping, moving, etc.) into small chunks (MMTk calls these “work packets”) and process these work packets in parallel across multiple CPU cores.

It’s important to note however that while MMTk can perform its GC work in parallel, it does not run concurrently with the VM. In that sense, MMTk is a parallelized GC implementation, but it is not concurrent, meaning that Ruby code cannot run while the garbage collector is running, so it still requires the Ruby VM to be stopped.

There were many challenges that we had to overcome to move from a serial garbage collector to a parallel one, including removing dependence on thread-local variables and race conditions. However, while those issues were apparent as crashes and unexpected behavior, we ran into a tricky problem: our garbage collection cycles were slower the more threads we used!

This was counterintuitive, because if each CPU core does less work, then shouldn’t it run faster? We looked at performance profiles more closely, and saw that it was the finalization phase that was slower. The finalization phase iterates over all dead objects to run code to do things like reclaim memory or close file descriptors. Specifically, we found that the culprit was free, the function that frees memory allocated through malloc. In the following graph, we freed 100 million 32-byte pieces of memory using free. We measure the time taken (in milliseconds) with the work split across a varying number of threads and using various implementations of malloc. We see that for glibc, jemalloc, and tcmalloc, they all scale negatively with the number of threads. The only allocator that offers any scalability is mimalloc, but we see little to no gain past a factor of 4. This is likely due to mimalloc’s design for a fast free that maximizes concurrency.

Another difference between MMTk and the default GC is that if an object does not require finalization (i.e. it does not have any resources that need to be reclaimed), then we don’t need to visit it at all, further improving performance. MMTk can use a bump pointer allocator, which increments a pointer for every allocation until it reaches the end of the allocation space. Meanwhile, the default GC in Ruby uses a freelist allocator, which uses a linked list of free slots to allocate objects into. Since building the freelist requires visiting all dead objects anyway, the default GC won’t be able to take advantage of this improvement.

The solution to this challenge was to avoid using malloc. Instead, MMTk allocates the buffer for common types (Array, String, and MatchData objects) using hidden Ruby objects instead. Since these buffer objects are now Ruby objects, they are also allocated through MMTk. As a result, these buffers now have automatic memory management, rather than manual memory management like malloc. This means that Array, String, and MatchData need to mark their buffer objects to keep those buffers alive in the marking phase, but, in return, they don’t need to do anything anymore during the finalization phase.

Future Work & Conclusion

In this blog post, we looked at a few of the challenges we encountered in integrating MMTk with Ruby and the solutions we used. We hope that sharing our experiences can provide insights for Ruby developers, garbage collector researchers, and language designers.

Work continues in MMTk’s fork of Ruby to experiment with more optimized memory layouts, new techniques for object movement, and integrations between JIT compilers and the garbage collector. We are also using the lessons we learned with MMTk to make improvements into Ruby upstream.

Ever since YJIT’s introduction, I’ve felt simultaneously close to and distant from Ruby’s JIT compiler. I know how to enable it in my Ruby programs. I know it makes my Ruby programs run faster by compiling some of them into machine code. But my understanding around YJIT, or JIT compilers in Ruby in general, seems to end here.

A few months ago, my colleague Max Bernstein wrote ZJIT has been merged into Ruby to explain how ZJIT compiles Ruby’s bytecode to HIR, LIR, and then to native code. It sheds some light on how JIT compilers can compile our program, which is why I started to contribute to ZJIT in July. But I still had many questions unanswered before digging into the source code and asking the JIT experts around me (Max, Kokubun, and Alan).

So I want to use this post to answer some questions/mental gaps you might also have about JIT compilers for Ruby:

1. Where does JIT-compiled code actually live?
2. How does Ruby actually execute JIT code?
3. How does Ruby decide what to compile?
4. Why does JIT-compiled code fall back to the interpreter?

While we use ZJIT (Ruby’s experimental next-generation JIT) as our reference, these concepts apply equally to YJIT as well.

Where JIT-Compiled Code Actually Lives

Ruby ISEQs and YARV Bytecode

When Ruby loads your code, it compiles each method into an Instruction Sequence (ISEQ) - a data structure containing YARV (CRuby virtual machine) bytecode instructions.

(If you’re not familiar with YARV instructions or want to learn more, Kevin Newton wrote a great blog series to introduce them)

Let’s start with a simple example:

def foo
  bar
end

def bar
  42
end

Running ruby --dump=insn example.rb shows us the bytecode:

== disasm: #<ISeq:foo@example.rb:1 (1,0)-(3,3)>
0000 putself                                                          (   2)[LiCa]
0001 opt_send_without_block                 <calldata!mid:bar, argc:0, FCALL|VCALL|ARGS_SIMPLE>
0003 leave                                  [Re]

== disasm: #<ISeq:bar@example.rb:5 (5,0)-(7,3)>
0000 putobject                              42                        (   6)[LiCa]
0002 leave                                  [Re]

JIT-Compiled Code Lives on ISEQ Too

I assumed JIT-compiled code would replace bytecode—after all, native code is faster. But Ruby keeps both, for good reason.

Here’s what an ISEQ looks like initially:

ISEQ (foo method)
├── body
│   ├── bytecode: [putself, opt_send_without_block, leave]
│   ├── jit_entry: NULL  // No JIT code yet
│   ├── jit_entry_calls: 0  // Call counter

After the method is called repeatedly and gets JIT-compiled:

ISEQ (foo method)
├── body
│   ├── bytecode: [putself, opt_send_without_block, leave]  // Still here!
│   ├── jit_entry: 0x7f8b2c001000  // Pointer to native machine code
│   ├── jit_entry_calls: 35  // Reached compilation threshold

The jit_entry field is the gateway to native code. When it’s NULL, Ruby interprets bytecode. When it points to compiled code, Ruby can jump directly to machine instructions. But the bytecode never goes away - Ruby needs it for de-optimization, which we will explore a bit later.

The Execution Switch: From Bytecode to Native Code

This is easier than I expected. Since each ISEQ points to its JIT compiled code when it’s available, Ruby simply checks the jit_entry field on every ISEQ it’s going to execute:

When there’s no JIT code (jit_entry is NULL), it continues interpreting. Otherwise, it runs the compiled native code.

How Ruby Decides What to Compile

Ruby doesn’t compile methods randomly or all at once. Instead, methods earn compilation through repeated use. In ZJIT, this happens in two phases:

if (body->jit_entry == NULL && rb_zjit_enabled_p) {
    body->jit_entry_calls++;

    // Phase 1: Profile the method
    if (body->jit_entry_calls == rb_zjit_profile_threshold) {
        rb_zjit_profile_enable(iseq);
    }

    // Phase 2: Compile to native code
    if (body->jit_entry_calls == rb_zjit_call_threshold) {
        rb_zjit_compile_iseq(iseq, false);
        // After this, jit_entry points to machine code
    }
}

As of now, ZJIT’s default profile threshold is 25 and compile threshold is 30 (both may change in the future). So a method’s lifecycle may look like this:

Calls:     0 ─────────── 25 ────────── 30 ─────────────────►
           │              │             │
Mode:      └─ Interpret ──┴── Profile ──┴─ Native Code (JIT compiled)

This is why we need to “warm up” the program before we get the peak performance with JIT.

When JIT Code Gives Up: Understanding De-optimization

JIT code makes assumptions to run fast. When those assumptions break, Ruby must “de-optimize” - return control to the interpreter. It’s a safety mechanism that ensures your code always produces correct results.

Consider this method:

def add(a, b)
  a + b
end

which would generate these instructions:

== disasm: #<ISeq:add@test.rb:1 (1,0)-(3,3)>
0000 getlocal_WC_0                          a@0                       (   2)[LiCa]
0002 getlocal_WC_0                          b@1
0004 opt_plus                               <calldata!mid:+, argc:1, ARGS_SIMPLE>[CcCr]
0006 leave                                                            (   3)[Re]

Because Ruby doesn’t know what opt_plus would be called with beforehand, the underlying C function vm_opt_plus needs to handle various classes (like String, Array, Float, Integer, etc.) that can respond to +.

But, if profiling shows add is always called with integers (Fixnums), JIT compilers can generate optimized code that only handles integer addition. But it includes “guards” to check this assumption:

When the assumption is broken, like when add(1.5, 2) is called:

1. The guard check fails
2. JIT code jumps to a “side exit”
3. The side exit restores interpreter state (stack, instruction pointer..etc.)
4. Control returns to the interpreter
5. The interpreter executes opt_plus and calls the vm_opt_plus function

Other triggers for falling back include:

• TracePoint activation - TracePoint needs bytecode execution for properly emitting events (more details below)
• Redefined core methods - Someone changed what + means on Integer
• Ractor usage - Multi-ractor changes some YARV instruction’s behaviour. So the compiled code could perform differently than the interpreter in that situation

These assumption checks, or patch points as we call them in ZJIT, make sure your program performs correctly when any of the assumptions change.

Answering Some Additional Questions

Why does enabling TracePoint slow everything down?

(TracePoint is a Ruby class that can be used to register callbacks on specific Ruby execution events. It’s commonly used in debugging/development tools.)

Most of TracePoint’s events are triggered by corresponding YARV bytecode. When TracePoint is activated, instructions in ISEQs will be replaced with their trace_* counterpart. Like opt_plus will be replaced with trace_opt_plus.

If Ruby only executes the compiled machine code, then those events wouldn’t be triggered correctly. Therefore, when ZJIT and YJIT compilers detect TracePoint’s activation, they immediately throw away the optimized code to force Ruby to interpret YARV instructions instead.

Why doesn’t Ruby just compile everything?

Many methods are called rarely. Compiling them would waste memory and compilation time for no performance benefit. Also, compiling methods without profiling would mean that JIT compilers either make wrong assumptions that get invalidated pretty quickly, or don’t make specific enough assumptions that miss further optimization opportunities.

Final Notes

I hope this post helped you understand JIT compilers, a now essential part of Ruby, a little bit more.

If you want to learn more about Ruby’s new JIT compiler: ZJIT, I highly recommend giving ZJIT has been merged into Ruby a read. And if you want to learn more about Ruby’s YARV instructions, Kevin Newton’s Advent of YARV series is the best resource.