If you’ve ever written a test for your code, you’re probably familiar with typical test framework methods: test/it to define test cases, and assert/expect to make assertions about the behavior of your code.

However, I want to highlight a less commonly used method: in other languages or frameworks it goes by other names, but in Ruby’s minitest it’s called skip. In this post, I’ll cover what skip does, when it may be useful, and, most importantly, when you should probably use something else.

Just skip to the good stuff

Okay, so what does skip do? Put simply, it allows you to not run a test.

More concretely: in minitest, none of the test code after skip is run, an S will be printed instead of the usual ./F/E, and you’ll see it included in the number of skipped tests in the summary:

# test.rb
require "minitest/autorun"

class SkipTest < Minitest::Test
  def test_skip
    skip "This test is skipped."
    assert_equal 1, 2 # Notice that this assertion _would_ fail
  end

  def test_normal
    assert_equal 1, 1
  end
end

$ ruby test.rb
Run options: --seed 9367

# Running:

.S

Finished in 0.000576s, 3472.2225 runs/s, 1736.1112 assertions/s.

2 runs, 1 assertions, 0 failures, 0 errors, 1 skips

You have skipped tests. Run with --verbose for details.

So it’s not quite as simple as “just not running a test”. skip also includes some signals to make sure you know “hey, by the way, this test didn’t actually run”.

skip, don’t run

The most common use of skip is to temporarily disable a test. Let’s say you have a newly failing test, and maybe it’s caused by a dependency upgrade, or maybe you’re just in the middle of a really big refactor. In either case, you know you need to fix the test eventually, but you don’t want to deal with it right now. This is a good use case for skip!

Instead of skip, you could comment out the test and leave a TODO. However, this approach is worse because it’s much easier to forget that the test exists at all. With skip, you get a reminder every time you run your test suite that “you should probably fix these”.

In the rails/rails test suite, we also use skip to indicate something is missing from a developer’s environment. For example, the Active Support test suite contains tests for ActiveSupport::Cache that depend on redis and memcached. If those services aren’t running locally, the tests depending on them are skipped¹ and a message is printed telling the developer why.

This is another good use of skip! It allows developers who aren’t actively working on ActiveSupport::Cache to run the Active Support test suite without requiring them to set up more complex dependencies. But it also signals to those developers that there are more tests to run, they just aren’t currently running.

Don’t skip this next part

We’ve looked at a few good examples of using skip, but I also see it used in places where it shouldn’t be.

Here’s a (not real) example using ActiveSupport::Cache:

module SharedCacheTests
  def test_some_redis_specific_thing
    skip unless cache_store.is_a?(ActiveSupport::Cache::RedisCacheStore)

    # ... test code that only works for RedisCacheStore
  end
end

Don’t use skip for this!

This is bad because it completely ruins the value of the skip signal. If any tests are always skipped, then the test output will always have Ss, and the final skips count will always be nonzero. Both of these signals are useful because of their rarity; if a developer sees them, they know there’s something for them to do. If skip is used for tests where there isn’t anything for a developer to do, then the useful signals gets drowned out by the noise.

Another issue with using skip like this is the runtime cost: minitest’s skip happens at test runtime. That means all the code before the skip call still runs: any setup/teardown hooks in the test’s own class as well as any setup/teardown hooks in the test class’ ancestors. Maybe you’re lucky and your test suite is fast enough that this doesn’t matter, but in a larger codebase this could add up to a significant amount of wasted time.

So, if you shouldn’t use skip in these scenarios, what should you use instead?

There are (at least) three good alternatives.

In the ActiveSupport::CacheStore example above, the skipped test is specific to a particular cache store (redis). So really, it doesn’t belong in the shared tests for all cache stores. Put it where it belongs!

module SharedCacheTests
  # ... shared tests for all cache stores
end

class RedisCacheStoreTest < Minitest::Test
  include SharedCacheTests

  def test_some_redis_specific_thing
    # ... test code that only works for RedisCacheStore
  end
end

Maybe you run your entire test suite with different configurations: instead of a test class per backend you run tests against each backend in a separate process². Since each backend will have different capabilities, some tests may not apply to every backend. Instead of conditionally skipping those tests, you can lift the conditional out of the test so that the test isn’t even defined if it shouldn’t run.

class CacheTest < Minitest::Test
  if cache_store.supports_multi_get?
    def test_multi_get
      # ...
    end
  end
end

Or, if you aren’t using minitest, your test framework may have a way to annotate tests so that they only run in certain scenarios.

class CacheTest < Megatest::Test
  test "only works with redis", store: :redis do
    # ...
  end
end

# Skip tests that only work with redis
$ megatest ! :@store=redis

In all of these cases, the skip signals in the test output remain actionable and the test suite remains fast, while still ensuring only the relevant tests are run in each scenario.

skip to the end…

skip is a powerful tool for signaling to developers that some tests aren’t running. However, it must be used conservatively to ensure the signal retains its value.

Luckily, there are many alternatives to skip for those cases where action isn’t required. Don’t skip out on using them!

We recently landed a new register allocator in ZJIT and I thought I’d write a post about it!

What is a register allocator?

Whenever a compiler generates machine code it needs to decide where to put values. Those values usually take the shape of a variable in your function, though the compiler can also compute intermediate values as well. When we need to perform a calculation on some value, the CPU needs to know how to find the value.

The CPU can typically only compute output based on inputs that are in registers, though some architectures (like x86) allow computations on values stored in memory. That said, reading and writing to registers is much faster than memory, so it behooves the compiler to keep values in registers as long as possible.

Any particular function in your program could have tons of variables, but the number of available registers is finite, and architecture dependent. This is where a register allocator comes in. The register allocator looks at all of the variables, then figures out which registers they should go in, and if there aren’t enough registers figures out how to “spill” those variables to memory.

How does it work?

There are several well-known approaches to register allocation, each with different trade-offs between compile time and code quality. For ZJIT, we chose to implement a linear scan register allocator based on a reduced version of Christian Wimmer’s paper titled “Linear Scan Register Allocation on SSA Form”. The paper isn’t very long so I highly recommend giving it a read when you have time! Alternatively, Max Bernstein wrote a blog post breaking down the paper as well.

ZJIT uses Static single-assignment form, or SSA form in its back end. “SSA form” is a representation of code that only allows a variable to be assigned once.

For example, this Ruby code:

a = 123
a += 1
a

Would be represented in our backend Low-level Intermediate Representation (or “LIR”) kind of like this:

v1 = Const(123)
v2 = Const(1)
v3 = Add v1, v2
CRet v3

In the above pseudocode (it’s not exactly the same as the IR we use in the backend, but quite close), v1, v2, and v3 are all different variables. No variables are allowed to be re-assigned like in the corresponding Ruby code. The generated intermediate representation has exactly the same semantics as the Ruby code, but adds the restriction that a variable can only be written to once. When ZJIT generates an intermediate representation of the Ruby code, it translates all variables to these numeric SSA variables and also uses SSA variables for temporary values. In the example translation above we can think of v1 as being equivalent to a, v2 as a temporary variable with the value 1, and v3 as a new version of a that has been added with v2.

Variables can be written to once, but you can read from the variable as many times as you want. We call the place where the variable was written the “definition” and the place where the variable is read a “use”.

Lifetimes

The first thing a register allocator needs to understand is when each value is alive and for how long. We call this duration a “lifetime” or “live range”. A value’s lifetime (or live range) starts at the point where it’s defined (the definition) and ends at the place it is last used (its “last use”). If two values have overlapping lifetimes they can’t share the same register. If there are more overlapping lifetimes than registers, then we know we need to spill a value to memory.

Since these ranges refer to the “lifetime” of the variable, it’s common to say that the variable “came to life” at its definition, and then “died” at its last use.

Let’s look at an example. Consider this Ruby method that is already in SSA form:

def add_twice(a, b, c)
  d = a + b
  e = d + c
  e
end

Here are the lifetimes for each value:

#  Instruction     |  a  |  b  |  c  |  d  |  e
-------------------+-----+-----+-----+-----+----
1  d = a + b       |  x  |  x  |  .  |  .  |
2  e = d + c       |     |     |  x  |  x  |  .
3  return e        |     |     |     |     |  x

A . character means that the variable is alive at that instruction. An x character means the variable dies, but is still used at that instruction.

We can also express these lifetimes as ranges. The live range for each value is [definition, last use]:

a: [0, 1]
b: [0, 1]
c: [0, 2]
d: [1, 2]
e: [2, 3]

Where instruction 0 represents the function entry (parameter definitions).

• The parameters a, b, and c are all alive at instruction 1 because they were defined before the method body.
• a and b are last used at instruction 1, so they die after that.
• c stays alive through instruction 2 because that’s where it’s last used.
• d is defined at instruction 1 and last used at instruction 2, so it’s alive for both.
• e is defined at instruction 2 and last used at instruction 3.

At instruction 1, a and b die and d comes to life. Since we know that a and b are never used again, we’re free to reuse one of their registers for d. So we only need three registers at instruction 1: one each for a/d, b, and c. Similarly at instruction 2, c and d die as e is born, so we can reuse a register again.

Computing lifetimes involves a backward dataflow analysis over the control flow graph. We walk the instructions in reverse order, tracking which values are currently live. When we see the definition or use of a value, we know it is live at that instruction.

ZJIT has a debugging option to dump live range graphs like this:

$ ruby --zjit-call-threshold=2 --zjit-dump-lir=live_intervals ../test.rb

Here is an example of the output graph from one basic block as an excerpt from many blocks in a larger function:

          v0  v1  v2  v3  v4  v5  v6  v7  v8  v9  v10 v11
          --- --- --- --- --- --- --- --- --- --- --- ---
i0     :   .   .   .   .   .   .   .   .   .   .   .   .  FrameSetup
i2     :   v   .   .   .   .   .   .   .   .   .   .   .  v0 = Load [x21 - 0x28]
i4     :   █   v   .   .   .   .   .   .   .   .   .   .  v1 = Load [x21 - 0x20]
i6     :   █   █   .   .   .   .   .   .   .   .   .   .  Jmp bb3_l2([x19 + 0x18], v0, v1)

As mentioned earlier, in ZJIT, SSA variables names are just numbers. In the block above we have 12 variables numbered 0 through 11. These variables are listed in the first row along the top of the graph, numbered v0 through v11.

The first column lists the instruction numbers. In this case we have 4 instructions (i0 through i6), and for reasons outside the scope of this post, they’ve been numbered with even numbers.

The last column of the graph lists the actual LIR instruction. I’m not going to dive in to exactly what each of these instructions means, but you can see that some of them define variables (like i2 and i4 define v0 and v1), and some of them use variables (like i6 uses v0 and v1).

In the center of the chart, the v character indicates when a variable came to life, the solid block is when a variable is “alive”, and a ^ character indicates last use (though there are no “last uses” in this example).

Interference graphs

Once we know the lifetimes of all values, we need to figure out how many ranges overlap and where. One way to do this is to build an “interference graph”. An interference graph is a straightforward graph data structure where each node in the graph represents a live range, and each edge in the graph represents an “overlap”, or “interference”, with another live range.

Once you have an interference graph, you can treat allocation as a graph coloring problem, where each color represents a physical register available on that CPU. If we have k physical registers, we need a valid k-coloring of the interference graph. This is NP-complete in the general case, but heuristics can help simplify the problem.

While graph coloring produces excellent results, building and manipulating the interference graph can be expensive in terms of both time and memory, especially for large functions. JIT compilers should be fast, so we opted for a different algorithm: Linear Scan.

Linear Scan

As I mentioned earlier, we’re using a linear scan register allocator based on Christian Wimmer’s paper.

The algorithm is fairly straightforward. Once you’ve computed live ranges, iterate over those live ranges in order. When you get to an instruction where a live range starts, “pull” a free register from a pool, and assign the register to the live range. When you get to an instruction where a live range ends, put the register back in the pool.

If, at some point, you run out of registers in the pool, spill either the new live range, or an existing one.

Local vs global allocation

The techniques we’ve discussed so far like lifetimes, interference graphs, and linear scan can all be applied at different scopes. In the register allocator world there are two main types you’ll read about, and the difference is to do with how they process the program’s control flow graph.

First is the “local” register allocator. Every function is broken down in to multiple basic blocks, and program control flows through these blocks. A local register allocator will only ever “see” one single block at a time.

The second allocator is a “global” register allocator. A global register allocator will process the function’s entire graph at once. Confusingly, this allocator is not “global” in the sense of “global variables” in a program, but “global” in the sense that it analyzes an entire function at once rather than one basic block.

ZJIT’s previous register allocator was a local allocator inherited from YJIT. It only tracked lifetimes within a single basic block. This means that at every block boundary, all live values had to be moved to well-known locations (like the stack or fixed registers) so the next block could pick them up.

For a very lazy block-at-a-time compiler like YJIT, this is a reasonable trade-off. YJIT compiles one basic block at a time, so it never has the full picture of the function anyway. Using YJIT’s allocator helped us to bootstrap ZJIT quickly.

But ZJIT compiles entire methods, so we have all the information we need to do better.

A global allocator can keep a variable in the same register across block boundaries. If a value is defined in one block and used in a later block, the allocator can let its live range span both blocks and assign it a single register for the whole duration. This avoids unnecessary stores and loads at block boundaries, which can make a real difference in tight loops.

A global allocator also unlocks features that are difficult or impossible with a local one. For example, splitting and adding new basic blocks in various optimization passes was very tricky due to keeping track of what variables went where. Now basic blocks can be easily manipulated without worrying about which blocks use what variable. It’s also a prerequisite for method inlining, where the inlined callee’s code becomes part of the caller’s control flow graph and its values need to participate in the same allocation.

Where we are now

The new register allocator has landed and is working well. We’re now building on top of it! Method inlining is actively in progress and relies heavily on the global allocation we now have.

There are still improvements to make to the allocator itself. One big one is lifetime holes. Right now, a value’s live range is a single contiguous interval from its definition to its last use. But in practice, a value can be “dead” in the middle of its range.

For example, if a value is used in one branch of an if but not the other, the register allocator could accidentally keep the value “live” for one branch. This can be a problem because the value will end up using valuable resources (physical registers) that should be used for values that are actually alive at that time. The below Ruby program kind of visualizes this issue:

def example(cond, a, b)
  x = a + b

  if cond
    y = a * 2
    use(y)
  else
    use(x)
  end
end

Depending on how the code is linearized, the value x could be considered “alive” inside the true side of the if statement even though we can clearly see it’s not used. In other words, the register assigned to the value x will be reserved inside the true side even though it’s not actually used.

Representing these holes would let us reuse registers more aggressively in those gaps, reducing spills.

We’re excited about the foundation this gives us and looking forward to building on it!

One engine, many tools

A few years ago, the new Ruby parser Prism was released. One of its primary goals was to unify the community since we had multiple implementations of Ruby parsers, each with their own bugs, differences in implementation and portability. By having a single parser, community investments in performance and correctness benefit every single tool built on top of it (including Ruby itself!).

However, the story of repeated implementations of highly complex foundational blocks doesn’t end at the parser level. Move one level up the stack and the pattern repeats. Today, we have multiple tools that implement code indexing and related static analysis algorithms. Consider just a few examples:

• Language servers: tools like Ruby LSP and Solargraph need code indexing to provide go to definition, hover, signature help, completion and so on
• Type checkers: tools like Sorbet and Steep need code indexing for all of the previous reasons plus having the ability to type check code
• Documentation generators: tools like RDoc and YARD need code indexing to aggregate all declarations and their respective documentation for navigating and generating the static website
• Dead code detectors: tools like Spoom and debride need code indexing to match declarations and references, so that they can identify what declarations are dead (i.e.: unused)
• Linters: tools like RuboCop and Standard don’t currently use code indexing, but could provide much more sophisticated linting capabilities given a global knowledge of the codebase

The story we have here is the same. Multiple implementations of code indexing with varying performance, implementation differences and correctness discrepancies. On top of that, none of them are packaged and portable as an API that any other project can use.

It’s another case of our community’s efforts being diluted when we could instead have compounding benefits of working together in a foundational building block. We ought to do something about it.

Introducing Rubydex

Rubydex is a new portable static analysis engine intended to provide features such as code indexing and type analysis through a convenient API.

An important thing to note is that Rubydex is a framework/engine. It is not a tool by itself, but rather the core building block to create other tools. Despite being early in its development, Rubydex can already:

• Collect all definitions in a codebase and its dependencies, including classes, modules, constants, singleton classes, instance variables, class variables, global variables and methods
• Index RBS documents (including the bundled core and stdlib files and any RBS files in the workspace)
• Resolve constant references
• Track constant and instance variable references completely and method references with limitations¹
• Create declarations from the discovered definitions and constant references²
• Linearize ancestor chains
• Track descendants
• Query the resulting graph in many ways

Built for portability

One of our goals with Rubydex is portability. If someone wants to write sophisticated tooling for Ruby using a different language or maybe target the browser through WASM, then they should be able to!

For this reason, Rubydex is built with Rust, C and Ruby to ship 3 distinct components:

• The main Rust crate: this is where the entire logic is implemented. Rust allows for high performance and easy parallelism, which are incredibly valuable when implementing static analysis tooling where the performance constraints are intense. Other Rust projects can use this directly, like creating a Zed extension that can understand Ruby code or writing a linter in Rust.
• A Rust FFI crate: this crate provides C compatible bindings to use the main crate’s logic, allowing other languages to integrate with Rubydex. Developers can use this to write tooling in other languages, like a VS Code extension that can analyze Ruby codebases with no Ruby runtime dependency.
• A Ruby gem: a native extension that provides the Ruby API, which interacts with the underlying Rust implementation through the FFI crate. The gem ships with pre-compiled binaries for macOS (Intel and M series), Linux (x64 and ARM64) and Windows. For any other platforms, rubydex has a dependency on cargo (the Rust package manager) in order to compile correctly when installed.

Impact on existing tools

As of the time of this writing, we have either completed or started migrating our existing tools to use Rubydex. The impact story for all of them is essentially the same: better performance, higher accuracy and a lot less code to maintain.

Tapioca

You may know Tapioca for all of its runtime analysis, which is what allows the tool to output static RBI information for more accurate Sorbet type checking. However, Tapioca also consumes static information. There are two main use cases for static analysis in Tapioca:

• Fetching documentation for a given declaration so that it can be included in RBIs. This is important so that Sorbet can show documentation on hover
• Bootstrapping the initial analysis for generating gem RBIs. We discover the initial set of constants defined in a gem statically and then proceed to uncover the rest of the information by using the runtime

Tapioca adopted Rubydex to handle documentation, replacing the original approach which used YARD. After the change, we saw a massive performance improvement, dropping the total execution time for generating gem RBIs in our Core monolith from ~6 minutes to ~20 seconds, while using much less memory. We also replaced the gem RBI bootstrapping.

In addition to the performance gains, the correctness of Rubydex’s constant resolution algorithm and declaration handling allows comments to be attached in a more predictable way. This ensures that the comments are associated with the declaration they were meant to document, improving the quality of hover results for the Sorbet language server.

You can enjoy the Rubydex improvements starting in Tapioca v0.19.0.

Packwerk

Packwerk relies heavily on constant resolution to identify package violations. The problem is that constant resolution in Ruby is hard. The algorithm is complex and the amount of edge cases for static analysis increases the difficulty in getting it right. Investing the time to get it right in Rubydex means we don’t have to re-invent the wheel multiple times.

In our initial exploration of replacing Packwerk’s logic with Rubydex as the engine, we see the power of being able to reuse battle tested algorithms:

• Considerable reduction in codebase size (about -3000 lines)
• Significantly more violations found thanks to correct constant resolution (about 3x more constant references resolved in our monolith)
• Varying levels of performance improvements depending on the conditions (single-threaded vs parallel mode)

Spoom

We started prototyping the migration of Spoom’s dead code candidate detection to Rubydex. The story repeats once again. Faster analysis and higher accuracy thanks to being able to account for constant resolution and ancestors correctly. However, in the middle of the work, a question struck the team. Why not bundle this into Rubydex? It feels like dead code detection can be a part of the core engine, even if it’s only detecting candidates.

For that reason, we paused the Spoom migration to explore moving it into Rubydex instead. We hope to be able to ship the ability to find dead code candidates directly through our API.

Ruby LSP

For the Ruby LSP, Rubydex replaces the entire old code indexer with a significantly more performant and correct implementation. Several cases that we could not handle correctly are now understood by the language server and performance should be better across the board. The features directly impacted by Rubydex are:

• Go to definition, hover and signature help: better handling of nesting resolution will provide more accurate results
• Completion: the new completion engine in Rubydex provides significantly better results than the previous implementation
• Workspace symbol: new fuzzy search is much faster than the previous implementation
• Type hierarchy: ancestor information is more complete and correct. Additionally, descendant tracker allows us to implement the sub-types feature
• Test discovery for the explorer: faster analysis for test discovery
• Find references and rename: fast reference finding and renaming since Rubydex automatically tracks those
• Foundation: the previous indexer was not fast enough, so we ran indexing asynchronously in a thread. This improved responsiveness, but created several possible race conditions where taking an action in the editor before indexing was complete could result in discrepancies between the features and the actual state of the codebase. Rubydex is fast enough that we can simply index and manage the analysis synchronously, allowing us to eliminate the chances for race conditions and greatly simplifying the code in the Ruby LSP

In addition to the features already migrated to Rubydex, there are other improvements we are now able to deliver that weren’t possible before. For example, in semantic highlighting we treat all constant references with the token namespace. The reasoning behind this was performance. Resolving every constant reference in the old indexer was slow and semantic highlighting runs on every keypress, which would cause the editor to grind to a halt. With Rubydex, we can now solve this issue and correctly classify constant references as class, namespace or variable.constant.

Another possibility to explore is better test discovery. The current implementation of the test explorer relies on glob patterns to find test files in a performant way. With Rubydex, identifying all classes that inherit from Minitest::Test as a test group and all public methods prefixed with test_ is a breeze. Maybe we can finally ditch the glob pattern and achieve higher accuracy in discovery?

Safe to say, there are lots of opportunities for more sophisticated and correct functionality in the Ruby LSP, without compromising performance in any way.

Add-ons

The switch to Rubydex means breaking changes for most add-ons. Please refer to the Ruby LSP’s documentation and release notes for migration instructions.

MCP server

As part of Rubydex, we are also including an experimental MCP server. The purpose is to provide powerful tools for LLMs to explore the codebase more efficiently. Instead of reading files in their entirety and grepping based on text, the LLM can directly request information about declarations, definitions, references, ancestors, descendants (basically anything that you could query through the regular API).

We are still working on improvements and ironing out the best way to distribute the MCP server so that it can find dependencies correctly, integrating with version managers and Bundler (which can mutate where dependencies are installed in the system, making discovering them not so straightforward). For now, the MCP server must be built from source, but we hope to ship it as part of the Ruby gem in a future release.

In early testing, we’re seeing 15 to 80% reduction in token usage, cost and duration for certain LLM tasks thanks to the ability of quickly navigating the codebase. One example where this is easy to understand is asking the LLM to perform a refactor. Instead of scanning the entire codebase for usages of a class/module, it can directly ask Rubydex to get the information.

Current API

The Rubydex API is quite extensive to support all use cases in the Ruby LSP, Tapioca, Packwerk and other tools. Here are a few simple examples of how to use it, but please refer to the repo for more complete API documentation.

For this example, consider the following files present in the workspace:

# foo.rb
class Foo
  def found_me; end
end

# bar.rb
class Bar < Foo
  def found_me; end
end

# qux.rb
module Zip
  class Bar::Qux
    Foo
  end
end

We can use Rubydex to ask several questions about the state of the code:

require "rubydex"

# The graph is Rubydex's representation of the codebase. It's where everything starts and where all data is stored. To
# populate the graph, we first index the workspace (all .rb, .rbs files + dependencies) and then run the resolution step
# to transform definition information into semantic declarations.
#
# Find all of the other methods for managing the state of the graph in the repository's documentation.
graph = Rubydex::Graph.new
graph.index_workspace
graph.resolve

# Once the graph is ready, we can query it for any information we would like. This is what powers all language server's
# features in the new v0.27 and forward for the Ruby LSP.
#
# Find all of the querying features in the repository's documentation.
declaration = graph["Foo"]
puts declaration.ancestors.map(&:name) # => ["Foo", "Object", "Kernel", "BasicObject"]
puts declaration.descendants.map(&:name) # => ["Bar"]

puts graph.search("#found_me()") # => [#<Declaration name=Foo#found_me()>, #<Declaration name=Bar#found_me()>]

# Resolve a constant reference to `Foo` inside the `Zip, Baz::Qux` nesting
puts graph.resolve_constant("Foo", ["Zip", "Baz::Qux"]) # => [#<Declaration name=Foo>]

Future plans

This announcement marks the first iteration on Rubydex. There are a lot of interesting experiments that we want to do and lots to build. Here are some of the future ideas we have in our backlog.

Shared database

One of the ideas we partially experimented with and are eager to build is allowing Rubydex to save the analysis into a database. This benefits tools in a few ways:

• Reducing boot time: since the analysis is saved in the database, Rubydex wouldn’t have to start from scratch every time it is booted. This is regardless of how it gets initiated (language server, linter, MCP server)
• Loading partial data: developers rarely interact with 100% of their codebase all at the same time. Usually, you’re really only working in a small subset of the codebase at a time. This is true of LLMs too, they don’t need to read every single file in the codebase if you’re making a targeted change. Maintaining the entire analysis data in-memory at all times is wasteful. A lot of the information is simply not relevant for the changes being executed. If Rubydex can trace all of the dependencies of a document (and we believe it’s possible), then we can load a subset of the graph into memory and leave the rest in the database. This approach reduces the amount of memory being used by offloading to disk and it makes it so that memory usage does not scale linearly with the number of tools based on Rubydex (since they can share the same database)

If we can get this right, it would mean that you can have a language server, MCP server, linter, type checker all operating at the same time, reusing the same database to avoid multiple rounds of analysis and consuming little memory since most of the data is offloaded to disk.

Semantic linting

Another hope we have for Rubydex is enabling semantic linting. Rubydex can already collect diagnostics such as undefined constants. The idea is to offer a rich set of APIs so that linting rules can be created with the multitude of data available across the different phases of analysis.

The main benefits of semantic linting are reducing false positives and writing more complex rules that simple AST inspection cannot achieve. Here are some example rules that are possible (and even easy to write) with Rubydex, but not possible without a complete code indexing engine:

• Prohibiting inheriting from a certain ancestor: Maybe you’re deprecating the use of a module or parent class and you want to detect all existing uses (direct or transitive)
• Detecting potential dead code: Dead code is basically any declaration for which no references have been found. Of course, with meta-programming and untyped code, we can’t always be 100% sure it’s dead. However, our team has seen great benefit in flagging dead code candidates that then get reviewed by a developer to remove
• Detecting invalid mocks in tests: For example, stubbing a method to return a boolean when in fact it returns a string
• Detecting methods that are always shadowed: For example, a method in a parent class with real behaviour, but all of the subclasses override the method without invoking super
• Detecting incorrectly decomposed modules: For example, a module that depends on two instance variables, but none of the classes that include it define both instance variables, leading to module methods that can only be invoked when included into specific classes

And many, many others.

Type aware analysis

Currently, Rubydex has no concept of types and performs no inference. This limits the framework’s ability to understand method calls. For example:

var = Foo.something
var.other_thing

To know what defines other_thing, we need to know the type of var, and to know the type of var, we need to know the type returned by Foo.something. Only by propagating types through the analysis can we know for sure which methods are being used (or if they exist at all). Without this, Rubydex cannot associate method references to their declarations like it can for constants and instance variables.

The natural next step is to consume type annotations to improve the accuracy of the analysis. For that to work, we need to design the internal type system, consume annotations already used by the community (e.g.: Sorbet sigs and RBS), and experiment with different algorithms. The experimentation lies in the approaches we can take and understanding how well they can fit the Ruby language (e.g.: constraint solving vs set theoretic types).

Conclusion

Prism showed what happens when the Ruby community consolidates around a shared parser: every tool got better and every improvement compounded. We believe the same is possible one layer up. Analyzing Ruby statically is inherently hard, but we don’t have to solve the same problems separately.

Rubydex is our bet on a shared foundation. If you maintain a linter, a language server, a documentation generator, a type checker, a code-mod tool, or something we haven’t imagined yet, we’d love to build this with you.

Try it through the Ruby LSP, Tapioca or Packwerk, or directly use Rubydex’s API. File bugs, open issues, send PRs, tell us what’s missing in the repo. Every improvement made to Rubydex benefits all tools—even the ones that haven’t been built yet.

The story starts with Rust 1.85.0 giving me a surprise SIGTRAP in some test code. For reasons not important here, I had to bump opt-level from 0 to 1 for tests. That made a bunch of tests fail with SIGTRAP, whose default signal handler terminates the process.

The SIGTRAP happened in Rust code code that call C.

Passing a Rust closure through C code

The code that broke passes a Rust closure through a C function and invokes the closure in a callback.

The C API takes a user data argument in addition to the callback function, for passing through context:

// Calls (*proc)(data). VALUE is uintptr_t.
VALUE rb_protect(VALUE (* proc) (VALUE), VALUE data, int *pstate);

Each closure has a distinct anonymous type as they can vary in size depending on what’s captured. There is no way to separate out the captures and the function pointer to fit the C API. But, we can use a trait object to talk about a category of closure types, and the trait object has one uniform size:

fn closure_info(closure: impl FnMut()) {
    use std::mem::size_of_val;
    let trait_object: &dyn FnMut() = &closure;
    println!(
        "closure_size={} trait_object_size={}",
        size_of_val(&closure),
        size_of_val(&trait_object)
    );
}

fn main() {
    let mut int = 0;
    let one_capture = || int = 42;
    let no_capture = || {};
    closure_info(one_capture); // closure_size=8 trait_object_size=16
    closure_info(no_capture);  // closure_size=0 trait_object_size=16
    // Varying closure size, same trait_object_size.
}

The trait object is too big to fit the pointer-sized data argument, but we can solve that by taking a reference of it. That gives &mut &mut dyn FnMut(), a double reference to the trait object. &mut is pointer-sized, unlike &mut dyn. Great, we can now pass the closure through. Now we need to write the callback that invokes the closure.

The transmute

To call the closure, we need to first turn the VALUE back into the double reference to trait object. I used std::mem::transmute for this, and it blew up. Rust 1.85.0 compiles the following function to a single UD2 instruction which raises SIGTRAP.

#[repr(transparent)]
struct VALUE(usize);

extern "C" fn c_callback(obj: VALUE) {
    let closure: &mut &mut dyn FnMut() = unsafe { std::mem::transmute(obj) };
    closure();
}

Surprise SIGTRAPs like these from LLVM makes me think of trap mode in UndefinedBehaviorSanitizer. What rules did I break?

Probably pointer provenance

Integer to pointer transmute is documented to have unspecified behavior, but our struct VALUE(usize) definition is not an integer. Like many parts of Unsafe Rust, it’s hard to say how the transmute should have behaved. Rust 1.78.0 includes a change that seems to draw a clear line. “Lower transmutes from int to pointer type as gep on null” (“transmute patch” from here on) changes how transmute picks the provenance of the target pointer. Before, the transmute acted like an integer-to-pointer as cast, picking a previously exposed provenance. Now, it’s based on the null pointer. The null pointer is invalid for access, and so is the derived pointer. The SIGTRAP is probably trying to say, in an obtuse way, that I’m calling an invalid function pointer.

I had a hunch this is related to pointer provenance changes. I also remember from discussions about provenance something about writing a signature on the Rust side that uses a pointer type to accept an integer argument from the C side. I’m not sure if it’s a good idea to misrepresent types like this, but it’s useful to do it as an experiment to see if the SIGTRAP goes away:

// Experiment: the C side calls this with an integer (VALUE), but we claim it's a pointer
extern "C" fn c_callback(obj: *mut ()) {
    let closure: &mut &mut dyn FnMut() = unsafe { std::mem::transmute(obj) };
    closure();
}

And this version doesn’t SIGTRAP! So this is probably related to pointer provenance, rules that stipulate that in addition to having a good address, a pointer is valid for dereference only when obtained a certain way. The pointer in both the working version and the SIGTRAP version have the same address and in-memory representation, but only one is valid for dereference. The ptr module has documentation about provenance rules.

Searching in rust-lang/rust found me the transmute patch, and experiments on Compiler Explorer with various Rust versions showed 1.78.0 to be the first version that compiles the function to a single UD2 instruction. It’s the first release that includes the transmute patch.

But why does it SIGTRAP only on older Rusts? What changed?

Revert due to LLVM provenance bug

In the rust_has_provenance RFC, the lack of proper treatment of provenance in LLVM is stated as a drawback. Turns out, the transmute patch triggers such bugs and so was reverted in version 1.91.0. In some situations, the existence of a pointer with invalid provenance in the system can have LLVM confused and wrongly decide that an unrelated pointer is invalid for access. To be clear, the LLVM bugs are not the reason my code raises SIGTRAP; the transmute patch is designed to break such code. The unintended breakages from the transmute patch were due to the LLVM bugs giving bad output for code completely absent of Unsafe Rust.

There are many reports of code breakages due to the transmute patch, but the Rust team did not consider it to be a breaking change. I think it deserved a place in the release note as a compatibility issue. It also would have been nice if it panicked with a clear message rather than a nondescript SIGTRAP, but maybe the UD2 comes from a place too deep in LLVM’s pipeline to realistically replace.

With the revert, the SIGTRAP is gone. Absence of problematic runtime behavior does not imply absence of Undefined Behavior, though, and the revert did not change rules of the language. The current documentation for std::mem::transmute is clear about this subject:

Transmuting integers to pointers is a largely unspecified operation. It is likely not equivalent to an as cast. Doing non-zero-sized memory accesses with a pointer constructed this way is currently considered undefined behavior.

Let’s try to follow the rules.

The fix

Using an as cast instead of transmute avoids the SIGTRAP in all Rust versions:

extern "C" fn c_callback(obj: VALUE) {
    let closure = obj.0 as *const *mut dyn FnMut();
    unsafe { (**closure)() };
}

I understand why as cast works here better than transmute as follows: transmute works solely with the bits of the input value, but provenance is not represented in the input integer (integers have no provenance), so the output pointer has no provenance and is invalid for dereference. On the other hand, as casts can add things to the output not in the input. For casting from a smaller integer size to a larger one, it adds bits. Here, it adds provenance.

Ready for round two

I wrote code that triggers Undefined Behavior. We fix the code and move on, having learned a bit about pointer provenance language rules. If and when the transmute patch is reintroduced after the LLVM bugs are fixed or mitigated, our code will work.

Look! A trace of slow events in a benchmark! Hover over the image to see it get bigger.

Now read on to see what the slow events are and how we got this pretty picture.

The rules

The first rule of just-in-time compilers is: you stay in JIT code. The second rule of JIT is: you STAY in JIT code!

When control leaves the compiled code to run in the interpreter—what the ZJIT team calls either a “side-exit” or a “deopt”, depending on who you talk to—things slow down. In a well-tuned system, this should happen pretty rarely. Right now, because we’re still bringing up the compiler and runtime system, it happens more than we would like.

We’re reducing the number of exits over time.

Lies, damned lies, and statistics

We can track our side-exit reduction progress with --zjit-stats, which, on process exit, prints out a tidy summary of the counters for all of the bad stuff we track. It’s got side-exits. It’s got calls to C code. It’s got calls to slow-path runtime helpers. It’s got everything.

Here is a chopped-up sample of stats output for the Lobsters benchmark, which is a large Rails app:

$ WARMUP_ITRS=0 MIN_BENCH_ITRS=20 MIN_BENCH_TIME=0 ruby --zjit-stats benchmarks/lobsters/benchmark.rb
...
***ZJIT: Printing ZJIT statistics on exit***
...
Top-20 side exit reasons (100.0% of total 12,549,876):
                   guard_type_failure: 6,020,734 (48.0%)
                  guard_shape_failure: 5,556,147 (44.3%)
  block_param_proxy_not_iseq_or_ifunc:   445,358 ( 3.5%)
                   unhandled_hir_insn:   215,168 ( 1.7%)
                        compile_error:   181,474 ( 1.4%)
...
compiled_iseq_count:                               5,581
failed_iseq_count:                                     2
compile_time:                                    1,443ms
...
guard_type_count:                            133,425,094
guard_type_exit_ratio:                              4.5%
guard_shape_count:                            49,386,694
guard_shape_exit_ratio:                            11.3%
...
code_region_bytes:                            31,571,968
side_exit_size_ratio:                              33.1%
zjit_alloc_bytes:                             19,329,659
total_mem_bytes:                              50,901,627
...
ratio_in_zjit:                                     82.8%
$

(I’ve cut out significant chunks of the stats output and replaced them with ... because it’s overwhelming the first time you see it.)

The first thing you might note is that the thing I just described as terrible for performance is happening over twelve million times. The second thing you might notice is that despite this, we’re staying in JIT code seemingly a high percentage of the time. Or are we? Is 80% high? Is a 4.5% class guard miss ratio high? What about 11% for shapes? It’s hard to say.

The counters are great because they’re quick and they’re reasonably stable proxies for performance. There’s no substitute for painstaking measurements on a quiet machine but if the counter for Bad Slow Thing goes down (and others do not go up), we’re probably doing a good job.

But they’re not great for building intuition. For intuition, we want more tangible feeling numbers. We want to see things.

Building intuition

The third thing is that you might ask yourself “where are these exits coming from?” Unfortunately, counters cannot tell you that. For that, we want stack traces. This lets us know where in the guest (Ruby) code triggers an exit.

Ideally also we would want some notion of time: we would want to know not just where these events happen but also when. Are the exits happening early, at application boot? At warmup? Even during what should be steady state application time? Hard to say.

So we need more tools. Thankfully, Perfetto exists. Perfetto is a system for visualizing and analyzing traces and profiles that your application generates. It has both a web UI and a command-line UI.

We can emit traces for Perfetto and visualize them there.

A look at Perfetto

Take a look at this sample ZJIT Perfetto trace generated by running Ruby with --zjit-trace-exits¹. What do you see?

I see a couple arrows on the left. Arrows indicate “instant” point-in-time events. Then I see a mess of purple to the right of that until the end of the trace.

Hover over an arrow. Find out that each arrow is a side-exit. Scream silently.

But it’s a friendly arrow. It tells you what the side-exit reason is. If you click it, it even tells you the stack trace in the pop-up panel on the bottom. If we click a couple of them, maybe we can learn more.

We can also zoom by mousing over the track, holding Ctrl, and scrolling. That will get us look closer. But there are so many…

Fortunately, Perfetto also provides a SQL interface to the traces. We can write a query to aggregate all of the side exit events from the slice table and line them up with the topmost method from the backtrace arguments in the args table:

SELECT
  s.name AS reason,
  a.display_value AS method,
  COUNT(*) AS count
FROM slice s
JOIN args a ON a.arg_set_id = s.arg_set_id AND a.key = '0'
GROUP BY s.name, a.display_value
ORDER BY count DESC

This pulls up a query box at the bottom showing us that there are a couple big hotspots:

It even has a helpful option to export the results Markdown table so I can paste (an edited version) into this blog post:

Looks like we should figure out why we’re having shape misses so much and that will clear up a lot of exits. (Hint: it’s because once we make our first guess about what we think the object shape will be, we don’t re-assess… yet.)

This has been a taste of Perfetto. There’s probably a lot more to explore. Please join the ZJIT Zulip and let us know if you have any cool tracing or exploring tricks.

Now I’ll explain how you too can use Perfetto from your system. Adding support to ZJIT was pretty straightforward.

Implementation

The first thing is that you’ll need some way to get trace data out of your system. We write to a file with a well-known location (/tmp/perfetto-PID.fxt), but you could do any number of things. Perhaps you can stream events over a socket to another process, or to a server that aggregates them, or store them internally and expose a webserver that serves them over the internet, or… anything, really.

Once you have that, you need a couple lines of code to emit the data. Perfetto accepts a number of formats. For example, in his excellent blog post, Tristan Hume opens with such a simple snippet of code for logging Chromium Trace JSON-formatted events (lightly modified by me):

event_name = ...
timestamp = ...
duration = ...
f = open('trace.json','a')
f.write("[\n")

# ... emit some events here ...

# Log a single event
f.write('{"name": "%s", "ts": %d, "dur": %d, "cat": "hi", "ph": "X", "pid": 1, "tid": 1, "args": {}},\n' %
  (event_name, timestamp, duration))

# ... emit some events here ...

# ... at process exit, close the file ...
f.write("]") # this closing ] isn't actually required
f.close()

This snippet is great. It shows, end-to-end, writing a stream of one event. It is a complete (X) event, as opposed to either:

• two discrete timestamped begin (B) and end (E) events that book-end something, or
• an instant (i) event that has no duration, or
• a couple other event types in the Chromium Trace Event Format doc

It was enough to get me started. Since it’s JSON, and we have a lot of side exits, the trace quickly ballooned to 8GB large for a several second benchmark. Not great. Now, part of this is our fault—we should side exit less—and part of it is just the verbosity of JSON.

Thankfully, Perfetto ingests more compact binary formats, such as the Fuchsia trace format. In addition to being more compact, FXT even supports string interning. After modifying the tracer to emit FXT, we ended with closer to 100MB for the same benchmark.

We can reduce further by sampling—not writing every exit to the trace, but instead every K exits (for some (probably prime) K). This is why we provide the --zjit-trace-exits-sample-rate=K option.

Check out the trace writer implementation from the point this article was written.

Tracing more things

We could trace:

• When methods get compiled
• How big the generated code is
• How long each compile phase takes
• When (and where) invalidation events happen
• When (and where) allocations happen from JITed code
• Garbage collection events
• and more!

Conclusion

Visualizations are awesome. Get your data in the right format so you can ask the right questions easily. Thanks for Perfetto!

Also, looks like visualizations are now available in Perfetto canary. Time to go make some fun histograms…

The Rails Infrastructure team has been working on making Bundler faster and our work has paid off. A cold bundle install is 3x faster on a Gemfile with 452 gems compared to Bundler 2.7. But “faster” only means something if everyone agrees on what’s being measured. If two people are running benchmarks with different definitions of “cold install” or different cache states, the results aren’t comparable. We needed a shared tool that would give us confidence, both internally and externally, that we’re tackling the right problems and actually making Bundler faster. And along the way we learned when Claude can be helpful and when to not outsource our own expertise and thinking.

What affects bundle install time

Before building anything, we had to understand what variables go into bundle install performance. There are more than you’d expect:

• Number of gems: A Gemfile with 35 gems takes less time to install than a Gemfile with 500 gems.
• Depth of dependencies: A flat Gemfile with no transitive dependencies resolves faster than a deep dependency tree.
• Native extensions: Gems like bigdecimal take orders of magnitude longer to install than pure Ruby gems (bigdecimal alone takes 3 seconds to install). The ratio of native extension gems to pure Ruby gems in your Gemfile changes the install profile significantly. It’s rare to have a Gemfile without at least a few dependencies on native extensions.
• How Ruby is compiled: Optimization flags, compiler version, and platform all affect gem compilation time.
• Network time: Downloading from rubygems.org introduces latency and rate limiting that can skew results.
• Number of cores: Bundler parallelizes installs across worker threads, so core count matters.
• Endpoint security software: On company issued, managed machines, security software that scans file writes adds measurable time to every gem install. Running the same benchmark on a personal non-managed device vs a managed device produced very different numbers with no code change. If your benchmarks aren’t reproducible across machines, this is worth checking.

We needed a way to remove as many of these variables as possible so when we made changes, we could trust our benchmarks were correct. The goal was to remove guesswork, ensure everyone is testing from the same starting point, and provide a straightforward way to run benchmarks when making changes.

What we built

It took a few weeks to get reliable results and as part of building the benchmark we also implemented a full toolkit that includes scripts for installing, benchmarking and profiling Ruby package managers.

Getting a reliable benchmark took a lot of iteration. The first version of the benchmark was basic, and every time we ran it we’d find something that wasn’t quite right. We worked with Claude to make the original benchmark and tweak it as we found issues with the runs. In some cases we had to do the tedious work of debugging the benchmark ourselves.

Back in 2018 when I was working on improving Rails integration test performance, I kept an entire repo with all my benchmarks and profile scripts so I could track changes over time, but also so I could share it with the community. When your benchmark is open source, you’re not working in a vacuum and everyone else can check your assumptions. That lesson stuck with me, and it’s why this toolkit follows a similar pattern.

The toolkit includes everything you need to benchmark and profile Bundler.

• Setup scripts: Scripts to install for both macOS and Linux that lets you choose which package managers you want to benchmark.
• Benchmarking tool: The benchmark tool uses hyperfine for statistical timing with standard deviation, min/max, and outlier detection.
  • It supports running against multiple branches and package managers, switching the Ruby version, changing the number of iterations, and provides multiple Gemfile scenarios.
  • It automatically runs both warm and cold scenarios and outputs how much faster or slower each is than the baseline.
  • It includes a fake gemserver. Thanks to Claude I was able to quickly build a fake gemserver that served real gems based on Aaron’s slow-gemserver and use that to eliminate deviations caused by network round trips to Rubygemsorg and/or rate limiting.
• Profiler tool for Bundler: The profiler tool can currently only profile Bundler but also includes everything you need using either Samply or Vernier
  • It supports switching the Ruby version, running with cold or warm cache mode, choosing the Gemfile scenario, and changing the output path of the profile.
  • It also can optionally use the fake gemserver to avoid profiling network time.

How we defined what to measure

As part of building this benchmark we also needed to define what we wanted to measure in order to have the same understanding of the scenarios we are trying to improve.

Cold is defined as a first-ever install. Before each iteration, hyperfine’s --prepare hook nukes all caches: download cache, compact index cache, installed gems, bundle home, and removes the lockfile. The install has to resolve dependencies, download every gem, and install from scratch. This is the case where nothing is compiled or installed.

Warm is defined as reinstalling gems that have been downloaded previously. The benchmark setup first runs one full cold install to populate the download cache. Then for each timed iteration, the --prepare hook removes only installed gems and the .bundle directory, keeping the download cache and lockfile intact. The install runs with BUNDLE_FROZEN=1 so it skips resolution and only extracts and installs.

Getting “warm” right was harder than it sounds. Bundler, gel, rv, and scint package managers all have different cache structures. Early on Bundler’s warm results were barely faster than cold, and we spent time debugging before realizing it was a cache isolation issue in the benchmark itself, not a Bundler problem. The benchmark was wrong, not the code. Interestingly, this was a case where AI wasn’t that helpful. Claude kept missing this specific environment variable, so we had to debug the hard way (this is also why the script has a BENCH_DEBUG mode). But it paid off because we gained a better understanding of which environment variables affect the caches.

In the future we may want to define other cache scenarios to measure. There are scenarios between our pre-defined cold and warm scenarios like bundle update or having the gems installed but no lockfile so resolution needs to occur again. The beauty of this toolkit being open source is that if there’s a scenario you want to test, we can easily add that to the benchmark script.

Using the benchmark script

In order to support multiple tools we implemented a --run argument that is specified as a LABEL:TOOL[:PATH] triple. The label isolates caches, the tool selects the package manager, and the path optionally points to a local checkout or git worktree. Multiple runs can be compared in a single invocation, with the first treated as baseline.

ruby run_benchmark.rb \
  --run master:bundler:~/rubygems \
  --run patched:bundler:~/rubygems-patched \
  --scenario rails \
  --iterations 5 \
  --source http://localhost:9292

Caches are fully isolated per label under .caches/<label>/ using environment variables so comparing two Bundler versions in the same invocation won’t contaminate results. The comparison output shows relative speed so you can quickly see if your change is how many times faster or slower it is than the baseline.

Each scenario is just a directory containing a Gemfile. The rails scenario represents a typical Rails application with 35 gems. The large scenario is a stress test with 452 gems. You can add your own by creating a directory with a Gemfile and passing --scenario yourdir to the script.

$ ruby run_benchmark.rb --run bundler27:bundler:~/bundler27 --run master:bundler:~/rubygems/ --scenario large --iterations 3 --source http://localhost:9292 --ruby /usr/local/bin/ruby
Benchmark matrix
Ruby: ruby 4.0.1 (2026-01-13 revision e04267a14b) +PRISM [x86_64-linux]
Source: http://localhost:9292
Iterations: 3
Runs:
  bundler27: bundler (/home/ubuntu/bundler27)
  master: bundler (/home/ubuntu/rubygems)

=== Scenario: large (452 gems, bundler27) ===
  Running cold benchmark (3 runs)...
Benchmark 1: bundler27 (cold)
  Time (mean ± σ):     51.368 s ±  0.058 s    [User: 106.055 s, System: 22.181 s]
  Range (min … max):   51.332 s … 51.435 s    3 runs

  Running warm benchmark (3 runs)...
Benchmark 1: bundler27 (warm)
  Time (mean ± σ):      8.895 s ±  0.150 s    [User: 7.743 s, System: 4.658 s]
  Range (min … max):    8.742 s …  9.042 s    3 runs

  Cold median: 51.34s  Warm median: 8.9s

=== Scenario: large (452 gems, master) ===
  Running cold benchmark (3 runs)...
Benchmark 1: master (cold)
  Time (mean ± σ):     16.012 s ±  0.023 s    [User: 107.344 s, System: 21.568 s]
  Range (min … max):   15.987 s … 16.033 s    3 runs

  Running warm benchmark (3 runs)...
Benchmark 1: master (warm)
  Time (mean ± σ):      7.202 s ±  0.027 s    [User: 4.908 s, System: 3.061 s]
  Range (min … max):    7.173 s …  7.228 s    3 runs

  Cold median: 16.02s  Warm median: 7.2s

Results written to /home/ubuntu/bundler-bench/results/bundler27_20260318_161305.json
Results written to /home/ubuntu/bundler-bench/results/master_20260318_161305.json

=== Comparison Summary ===

Scenario: large (452 gems)
                             Cold     +/-                        Warm     +/-
  ------------------------------------------------------------------------------
  bundler27                51.34s   0.06s  baseline             8.90s   0.15s  baseline
  master                   16.02s   0.02s  3.21x faster         7.20s   0.03s  1.24x faster

Note these numbers will vary across macOS and Linux, as well as machines with endpoint security software. While we aimed to reduce many variables, you still may not see the same numbers, however times faster should be between 2-3.5x for cold and 1-1.5x for warm for bundle install. This script was run on an AWS Sandbox and therefore has no other traffic or endpoint security altering the numbers. It is also using the fake gemserver, so network round trips aren’t involved.

Using the profiling script

Benchmarks tell you whether something got faster. Profiles tell you why it’s slow. The profiling tool runs a single bundle install under Vernier or samply to produce flamegraphs.

ruby profile_bundler.rb \
  --run master:bundler:~/rubygems \
  --scenario rails \
  --mode warm \
  --profiler vernier

It supports both cold and warm modes so you can profile the specific phase you’re investigating. Profiles are written to profiles/ with filenames that include the label, scenario, mode, platform, and timestamp so you can compare across runs and machines.

Here’s an example of the Vernier output for the master branch on the AWS linux sandbox for the cold cache mode.

See all the yellow bars? Those are native extensions compiling and blocking the threads from doing other work.

Setup scripts

Reproducing someone else’s benchmark results is only possible if you’re starting from the same place. The repository includes setup scripts for both macos and linux which will install Ruby, hyperfine, profiling tools, and clone the repos you need:

./setup-benchmark-mac.sh --tools bundler,bundler27,gel,scint

The --tools flag lets you pick which package managers or versions to install. It defaults to bundler,bundler27 so you can compare the current master against the last stable release without extra setup. We wanted anyone on the team (or in the community) to be able to spin up a fresh machine and get comparable results without hunting down the right Ruby version, compiler flags, or repo branches.

What we learned

A shared benchmark is a source of truth. When someone says “my change is faster” and someone else disagrees, you need a neutral tool that everyone agreed on beforehand. Without that, performance discussions turn into competing anecdotes. The toolkit gives us a way to settle those disagreements with data instead of intuition.

Reproducibility matters just as much. If you can’t reproduce similar results on someone else’s machine, you can’t verify the claims. “It’s faster on my machine” isn’t useful if it’s not faster for everyone, or worse faster on Linux but slower on macOS. When results differ across machines, we can start asking why instead of arguing about whether.

Back in 2018 I gave a talk called How to Performance which was on the surface about how I sped up integration tests in Rails, but really it was a talk about how to write benchmarks you could trust so you know when you actually made something faster. Many of the lessons I learned back then came up again during this project.

Profiles and guesswork are only one part of the equation. A profile can show you a hot spot, and you can write a fix that looks faster, but without a proper benchmark you don’t actually know. You don’t know if the gain on macOS is a regression on Linux. You don’t know if “cold” got faster but “warm” got slower. You don’t know if the improvement holds across different Gemfile sizes. The benchmark is what turns a hypothesis into evidence.

This matters even more now than it did in 2018. Engineering rigor is more important in the AI world than it was before. It’s easy to generate output that looks correct or looks faster. It’s easy to make a benchmark that looks reasonable but cheats on the warm caches. AI is good at producing plausible code and plausible explanations. Humans are good at critical thinking and using our gut to know when something doesn’t look right. We have taste, discernment, and scrutiny, AI has data.

That’s not a dig on AI. I used Claude extensively throughout this project. It was great at writing the setup scripts, which are uninteresting and error prone, and it wrote the original benchmark tool. But it also got things wrong. The warm cache bug I mentioned earlier? Claude missed setting BUNDLE_USER_HOME in the environment, which meant Bundler was writing to the system bundle home instead of the isolated one. Warm caches on the master branch looked broken because they were being shared across runs. I spent time debugging Bundler before I realized the benchmark itself was wrong. Claude didn’t catch it because it doesn’t have the deep institutional knowledge of how Bundler’s cache layers interact. I caught it because I knew what the numbers should look like and they didn’t add up.

That’s not a reason to stop using AI. It’s a reminder to not outsource our thinking and to always test our assumptions. Applying engineering rigor is how we can be sure the work we’re doing, whether it’s us or AI doing it, is valid and achieves our goals.

The toolkit is available at bundler-perf-toolkit. If you’re working on Bundler performance or just curious about how your Gemfile affects install times, give it a try. We welcome PRs with new scenarios, corrections to cache handling if you spot something we got wrong, and support for other tools to test against.

Intro

Since the post at the end of last year, ZJIT has grown and changed in some exciting ways. This is the story of how a new, self-contained optimization pass causes ZJIT performance to surpass YJIT on an interesting microbenchmark. It has been 10 months since ZJIT was merged into Ruby, and we’re now beginning to see the design differences between YJIT and ZJIT manifest themselves in performance divergences. In this post, we will explore the details of one new optimization in ZJIT called load-store optimization. This implementation is part of ZJIT’s optimizer in HIR. Recall that the structure of ZJIT looks roughly like the following.

flowchart LR
        A(["Ruby"])
        A --> B(["YARV"])
        B --> C(["HIR"])
        C --> D(["LIR"])
        D --> E(["Assembly"])

This post will focus on optimization passes in HIR, or “High-level” Intermediate Representation. At the HIR level, we have two capabilities that are distinct from other compilation stages. Our optimizations in HIR typically utilize the benefits of our SSA representation in addition to the HIR instruction effect system.

These are the current analysis passes in ZJIT without load-store optimization, as well as the order in which the passes are executed.

run_pass!(type_specialize);
run_pass!(inline);
run_pass!(optimize_getivar);
run_pass!(optimize_c_calls);
run_pass!(fold_constants);
run_pass!(clean_cfg);
run_pass!(remove_redundant_patch_points);
run_pass!(eliminate_dead_code);

Here’s where load-store optimization gets added.

  run_pass!(type_specialize);
  run_pass!(inline);
  run_pass!(optimize_getivar);
  run_pass!(optimize_c_calls);
+ run_pass!(optimize_load_store);
  run_pass!(fold_constants);
  run_pass!(clean_cfg);
  run_pass!(remove_redundant_patch_points);
  run_pass!(eliminate_dead_code);

Overview

Ruby is an object-oriented programming language, so CRuby needs to have some notion of object loads, modifications, and stores. In fact, this is a topic already covered by another Rails at Scale blog post. The shape system provides performance improvements in CRuby (both interpreter and JIT), but there is still plenty of opportunity to improve JIT performance. Sometimes optimizing interpreter opcodes one at a time leaves repeated loads or stores that can be cleaned up with a program analysis optimization pass. Before getting into the weeds about this pass, let’s talk performance.

Results

The setivar benchmark for ZJIT changes dramatically on 2026-03-06. This is when load-store optimization landed in ZJIT. At the time of this writing, ZJIT takes an average of 2ms per iteration on this benchmark, while YJIT takes an average of 5ms.

This is the second time that ZJIT has clearly surpassed YJIT. The first example is here.

At a high level, this means that ZJIT is over twice as fast as YJIT for repeated instance variable assignment, and more than 25 times faster than the interpreter!

A Troubling Development

However, there’s an important question we have to address - why should an optimization pass for object loads and stores have anything to do with instance variable assignment? It turns out that ZJIT’s High Intermediate Representation (HIR) uses LoadField and StoreField instructions both for both object instance variables, and for object shapes. We’re going to have to dig deeper into CRuby shapes and ZJIT HIR internals in order to make sense of this.

Background

So far, we’ve learned that HIR has LoadField and StoreField instructions. We’ve claimed that they are multi-purpose and that the performance wins come from optimizing object shapes, but that they can also apply to object instance variables. Because the algorithm works just as well for both situations, the rest of this post will focus on object instance variables. This allows us to demonstrate concepts in pure Ruby to make things more approachable.

Example

Let’s start with a simple example we can all agree on. Clearly this code snippet has a double store, and we can safely remove one of the @a = value calls.

class C
  def initialize
    value = 1
    @a = value
    @a = value
  end
end

Here’s the same code snippet with an example of the call we remove. Here, we have elided a redundant StoreField instruction.

  class C
    def initialize
      value = 1
      @a = value
-     @a = value
    end
  end

When should we remove LoadField and StoreField instructions? The HIR code snippets will come later. For now, we only need to know the mapping between Ruby and HIR for instance variable loads and stores.

Note: In a class’s initialize method, instance variable operations are likely to cause LoadField and StoreField instructions due to shape transitions. Outside of an initialize method, the loads and stores are more likely to be related to the instance variables themselves. We decided that more complicated Ruby code snippets would clarify the kind of LoadField or StoreField but overly clutter the code snippets in this post.

Cases

Let’s consider every edge case for our algorithm through short Ruby snippets to illustrate scenarios where we can and cannot elide LoadField or StoreField HIR instructions.

Note: The following examples could replace the value variable with the constant 1, but in ZJIT this could cause other optimizations such as constant folding to interfere with our load-store demonstrations. We will use these more complex code snippets in case the reader wants to follow along with a compiler explorer.

Redundant Store

class C
  def initialize
    value = 1
    @a = value
    # This store is redundant and should be elided in HIR
    @a = value
  end
end

Redundant Load

class C
  def initialize
    value = 1
    @a = value
    # We already know that this load is `value` and should be replaced
    @a
  end
end

Redundant Store with Aliasing

class C
  attr_accessor :a

  def initialize(value)
    @a = value
  end
end

class D
  attr_accessor :a

  def initialize(value)
    @a = value
  end
end

def multi_object_test
  x = C.new(1)
  y = D.new(1)
  new_x_val = 2
  new_y_val = 3
  x.a = new_x_val
  y.a = new_y_val
  # We would like to elide this (but currently do not)
  x.a = new_x_val
end

With variables pointing to distinct objects, we could elide the second store to object x. This is not currently implemented, but is a possible improvement with a technique called type-based alias analysis.

Required Store with Aliasing

class C
  attr_accessor :a

  def initialize(value)
    @a = value
  end
end

def multi_object_test
  x = C.new(1)
  y = x
  new_x_val = 2
  new_y_val = 3
  x.a = new_x_val
  y.a = new_y_val
  # We should not elide the second `x.a` assignment because the `y.a` assignment modifies `x`
  # The `x.a` store after this comment is no longer redundant
  x.a = new_x_val
end

With multiple multiple variables aliasing to the same object, we cannot elide the second store to x. While technically we could elide y.a = new_y_val and the initial y = x assignment, these improvements are out of scope for this post. The key point here is that aliasing needs to be considered. If we assume that y and x reference different objects and elide the second x.a = new_x_val call, we alter program behavior.

Required Store with Effects

def scary_method(obj)
  obj.a = "We have modified the object. The second store is no longer redundant"
end

class C
  attr_accessor :a

  def initialize(value)
    @a = value
  end
end

def effectful_operations_between_stores_test
  x = C.new(1)
  x.a = 5
  scary_method(x)
  # We want to elide this but `scary_method` can modify `x`
  x.a = 5
end

In this case, the second store looks redundant, but it might not be. An arbitrary Ruby method (or C call, or some HIR instructions) could modify the x object and breaks the assumptions we can make about the state of the x object. In such cases, we cannot perform load-store optimization.

The Algorithm

Key Idea

With these cases, we have covered everything needed to implement our load-store optimization algorithm. The algorithm is a lightweight abstract interpretation over objects. This approach allows us to minimize the computation required to perform our optimization pass while ensuring soundness. In layperson’s terms, this means that every load we replace and every store we eliminate will not change program behavior, but that we will potentially miss some loads or stores that could be eliminated.

Tricky Details

Basic Blocks

Our load-store optimization pass scans through basic blocks, searches for redundant loads and stores, and updates the HIR instructions accordingly. Unnecessary StoreField operations are elided, and unnecessary LoadField operations are replaced with the instruction already holding the value. While one key benefit of ZJIT is that it can optimize entire methods, load-store optimization is (for now) block-local only.

LoadField and StoreField Distinctions

So far, we’ve talked about elision and instruction removal. We can get away with deleting StoreField instructions because no other instructions point to StoreField instructions. Conversely, LoadField instructions do have dependencies and are referenced by other instructions. These references need to be fixed up. Each reference to LoadField gets replaced with the cached value that was the target of a load.

The WriteBarrier Instruction

ZJIT has WriteBarrier instructions to support garbage collection. These also can modify objects and act similarly to stores. We need to handle this case in our algorithm.

Pointer Intricacies

The pseudo code we are about to introduce uses the term “offset” to denote the number of bytes from the object’s base address in memory. We use this to detect redundant loads and stores, as well as clear the cache from effectful instructions and write barriers. However, it is not immediately obvious that simply checking offsets would be enough. How can we be sure that the memory regions we are tracking remain untouched by some other instruction? Fortunately, HIR instructions always point to the base of an object and use offsets that are in bounds of the object. If we have two offsets that are not equal, they cannot reference the same region of memory. If the offsets are equal, then object aliasing must be considered.

Algorithm Sketch

Here’s the pseudo-code for a given basic block.

For each HIR instruction in the basic block
    initialize an empty cache as a hashmap
    
    if instruction is `LoadField`
        check if the object, offset, and value triple is in the cache
        if so, delete instruction and replace references to it with the loaded value
        else, cache the loaded value with the object, offset pair as a key
        
    if instruction is `StoreField`
        check if the object, offset, and value triple is in the cache
        if so, delete the instruction
        else, remove each cache entry with the same offset (the flags field) to avoid aliasing issues
        
    if instruction is `WriteBarrier`
        # This instruction is needed for the garbage collector and is complex
        # It works similarly to `StoreField` in practice
        # This instruction is never removed but the cache cleaning is still needed
        remove each cache entry with the same offset to avoid aliasing issues
        
    if instruction can modify objects
        flush the cache
        
    else
        continue
          
return the pruned HIR instructions

Source Code

The source at the time of this writing can be found here.

HIR Improvements

After the optimization, here are examples of how the HIR changes.

This the new HIR for our first redundant load example.

  fn initialize@../scripts/double_load.rb:3:
  bb1():
    EntryPoint interpreter
    v1:BasicObject = LoadSelf
    v2:NilClass = Const Value(nil)
    Jump bb3(v1, v2)
  bb2():
    EntryPoint JIT(0)
    v5:BasicObject = LoadArg :self@0
    v6:NilClass = Const Value(nil)
    Jump bb3(v5, v6)
  bb3(v8:BasicObject, v9:NilClass):
    v13:Fixnum[1] = Const Value(1)
    PatchPoint SingleRactorMode
    v30:HeapBasicObject = GuardType v8, HeapBasicObject
    v31:CShape = LoadField v30, :_shape_id@0x4
    v32:CShape[0x80000] = GuardBitEquals v31, CShape(0x80000)
    StoreField v30, :@a@0x10, v13
    WriteBarrier v30, v13
    v35:CShape[0x80008] = Const CShape(0x80008)
    StoreField v30, :_shape_id@0x4, v35
-   v20:HeapBasicObject = RefineType v8, HeapBasicObject
    PatchPoint SingleRactorMode
-   v38:CShape = LoadField v20, :_shape_id@0x4
-   v39:CShape[0x80008] = GuardBitEquals v38, CShape(0x80008)
-   v40:BasicObject = LoadField v20, :@a@0x10
    CheckInterrupts
-   Return v40
+   Return v13

This the new HIR for our first redundant store example.

bb1():
  EntryPoint interpreter
  v1:BasicObject = LoadSelf
  v2:NilClass = Const Value(nil)
  Jump bb3(v1, v2)
bb2():
  EntryPoint JIT(0)
  v5:BasicObject = LoadArg :self@0
  v6:NilClass = Const Value(nil)
  Jump bb3(v5, v6)
bb3(v8:BasicObject, v9:NilClass):
  v13:Fixnum[1] = Const Value(1)
  PatchPoint SingleRactorMode
  v35:HeapBasicObject = GuardType v8, HeapBasicObject
  v36:CShape = LoadField v35, :_shape_id@0x4
  v37:CShape[0x80000] = GuardBitEquals v36, CShape(0x80000)
  StoreField v35, :@a@0x10, v13
  WriteBarrier v35, v13
  v40:CShape[0x80008] = Const CShape(0x80008)
  StoreField v35, :_shape_id@0x4, v40
  v20:HeapBasicObject = RefineType v8, HeapBasicObject
  PatchPoint NoEPEscape(initialize)
  PatchPoint SingleRactorMode
- v43:CShape = LoadField v20, :_shape_id@0x4
- v44:CShape[0x80008] = GuardBitEquals v43, CShape(0x80008)
- StoreField v20, :@a@0x10, v13
  WriteBarrier v20, v13
  CheckInterrupts
  Return v13

And that’s load-store optimization!

Design Discussion

You may notice that our optimization is pruning the graph of loads and stores on an object. We are solving a very similar problem to the SSA form baked into the HIR. While it would be great to have “more SSA” at the object level, this comes at a cost. Computing SSA at this level could necessitate structural changes to HIR and make things less ergonomic or more confusing in regions of the codebase outside of load-store optimization. In fact, this question of “more SSA” is a complex design decision and contentious topic with a rich history in compilers such as V8 or Jikes RVM. So far, we’ve decided to use a lightweight SSA representation in ZJIT that causes us to work a bit harder for certain optimization passes, yielding subtle design simplifications across the rest of HIR.

Future Work

There’s still a lot of exciting work to be done and there are improvements to be made before we hit diminishing returns. Dead store elimination utilizes many of the same ideas and could help improve object initialization performance. We could implement type based alias analysis, though this requires care, as type confusion bugs are quite dangerous in JIT compilers. See section 4.1 in the phrack article for further details.

Conclusion

Thanks for reading the first post about ZJIT’s optimizer. We have lots more to come, so stay tuned.

At Shopify, we want our development environments to be fast. Installing dependencies is slow, especially in an application as large as Shopify. bun and uv have dramatically improved install times for TypeScript and Python dependencies. What if we could do the same for Bundler and the Ruby community?

Our team at Shopify has been working on a series of improvements to Bundler and RubyGems. Bundler downloads gems up to 200% faster. Cloning git gems is now 3x faster in our monolith.

We were also able to decrease the overall bundle install time by 3.5x in one of our applications by precompiling gems thanks to cibuildgem, a new precompilation toolchain we’d love you to try!

Here’s an overview of the improvements we’ve made in the last few months:

Faster gem downloads

One impactful change was deceptively simple. Bundler’s HTTP fetcher had a connection pool size of 1. This meant that during parallel gems installation, every thread was fighting over a single HTTP connection.

Pink spikes are threads waiting for the connection to be available.

By increasing the pool of HTTP connections, Bundler can download more gems in parallel. The speed gain with this change is even more dramatic during peak hours when RubyGems.org is under heavy load, or when you are geographically far from the CDN, where latency amplifies the cost of waiting on a single connection.

To benchmark this change, we opted to only measure download and extraction time (no compilation of native extensions) and built a local gem server where we can control latency at our will.

This is the result in a freshly generated Rails application when all gems are served with a 100ms latency.

Scenario: rails (164 gems)
                             Cold     +/-                        Warm     +/-
  ------------------------------------------------------------------------------
  5 HTTP connections       5.86s   0.10s  baseline             4.17s   0.02s  baseline
  1 HTTP connection       19.80s   0.02s  237.6% slower        4.16s   0.02s  0.3% faster

Hotspots and optimizations

We regularly profile Bundler with different Gemfiles and identify hotspots we might optimize. While no single optimization is dramatic, their collective impact has been significant.

One such optimization involves gem installation. A .gem file is a compressed tarball — and gzip has a built-in integrity check: if decompression succeeds, the content is guaranteed to be intact. Despite this, RubyGems was walking every entry in the tarball and reading all bytes upfront as an explicit corruption check, before proceeding with installation. This redundant verification step was thrown away entirely, since a successful decompression already provides the same guarantee.

9-17% of the time installing a gem is spent verifying the tarball’s content.

Another hotspot during installation is the check RubyGems performs to determine whether a gem includes a RubyGems plugin and, if so, whether its plugin file needs to be regenerated. The vast majority of gems don’t include a RubyGems plugin, yet every gem pays the cost of a Dir.glob with an expensive pattern just to handle the small minority that do.

It turns out that unconditionally regenerating the plugin file is faster than performing this upfront check.

Bundler checking whether regenerating a gem plugin is required

Parallel git clones

Many Rails applications depend on gems sourced directly from git repositories. This is particularly useful if a gem has upstream changes that aren’t yet released. Previously, Bundler would fetch each git repository sequentially, even though there’s no technical limitation on fetching them all at once.

Shopify’s Core Rails monolith includes 33 git gems. After introducing this change to parallelize git clone, we saw a 3x performance improvement for fetching git gems.

Native extensions

By far the biggest bottleneck when running bundle install is the compilation of native extensions. Many gems in the Ruby ecosystem include C code that must be compiled on each developer’s machine when installed. Common examples are json, date, and bigdecimal. Even if your Gemfile doesn’t directly depend on native extensions, it’s likely they will be included in your Gemfile.lock as transitive dependencies.

An installer thread spending 92% of the time compiling the gem.

To illustrate how slow compilation is, we can run bundle install on a freshly generated Rails application.

Installing the 18 native extension gems accounts for 85% of the time spent running bundle install.

Precompiled gems

Remember when Nokogiri used to take forever to install? Those days are behind us thanks to the amazing work of its maintainer, Mike Dalessio. Mike updated the gem’s publishing pipeline to precompile its native extensions into platform-specific binaries and releases separate gems for each supported platform (macOS, Windows, Linux). Now Nokogiri installs as fast as pure Ruby gems.

Imagine if we extended this to the rest of the Ruby ecosystem. If the community works together to ship precompiled binaries for our most popular native-extension gems, everyone will benefit from a lightning-fast bundle install.

One way to build binary gems is with the popular Rake-compiler-dock toolchain, which provides a cross-compilation environment and allows compilation to run inside Docker containers. However, cross-compiling can be brittle and presents hard-to-debug issues. Compiling on the target platform is ultimately far more reliable.

Many CI providers now offer free access to cloud machines. GitHub Actions, for example, is widely popular, and the Ruby community has built many easy-to-use actions around it (e.g., ruby/setup-ruby). Could we apply the same approach and leverage those machines to natively compile binary gems?

Introducing cibuildgem

At Shopify, we wanted to build an easy-to-use tool to help developers release gems with precompiled binaries using a native compilation approach via GitHub Workflows.

cibuildgem lets you generate a standard GitHub Actions workflow. Once triggered, multiple jobs run to:

1. Compile the binaries and package the gems
2. Run a matrix of test suites
3. Verify the .gem files are not corrupted and installable
4. Release the gems to RubyGems.org

Releasing a binary gem with cibuildgem

We aimed to make cibuildgem easy and fast to set up. Since many gems with native extensions are already configured with Rake Compiler for development compilation, we chose to piggyback on that so cibuildgem can run without any extra configuration for most gems.

The workflow generated by cibuildgem is intentionally standard.

• Want to compile your gem on Linux AArch64? Add it to the matrix.
• Want to trigger the workflow automatically when pushing a new git tag? No problem — tweak it to your liking.

We also wanted to ensure that the binaries compiled by cibuildgem would work in a macOS development environment and a Linux production environment on a real Rails application.

As an experiment, we used cibuildgem to compile dozens of open-source gems and publish them under a “namespace” on RubyGems.org (e.g., sassc -> precompiled-sassc).

The goal was to see how much performance improvement we could get with precompiled binaries. To test this, we created a Bundler plugin that hijacked the Bundler resolver to download the gems with precompiled binaries we had just published. For example, it would force-install precompiled-json if the json gem was requested anywhere in the dependency tree.

We tested this and deployed on an internal application which included 235 gems. By precompiling 17 of them, we saw a 3.5x performance improvement.

This experiment demonstrates how much faster bundle install could be when gems are precompiled. It has also given us confidence that cibuildgem builds compatible binaries for macOS and Linux.

In fact, a few gems at Shopify are now released with precompiled binaries (stack_frames, heap_profiler, rubydex) thanks to cibuildgem.

If you maintain a gem with a native extension, we’d love for you to give it a try and share your feedback ❤️!

ZJIT is a new just-in-time (JIT) Ruby compiler built into the reference Ruby implementation, YARV, by the same compiler group that brought you YJIT. We (Aaron Patterson, Aiden Fox Ivey, Alan Wu, Jacob Denbeaux, Kevin Menard, Max Bernstein, Maxime Chevalier-Boisvert, Randy Stauner, Stan Lo, and Takashi Kokubun) have been working on ZJIT since the beginning of this year.

In case you missed the last post, we’re building a new compiler for Ruby because we want to both raise the performance ceiling (bigger compilation unit size and SSA IR) and encourage more outside contribution (by becoming a more traditional method compiler).

It’s been a long time since we gave an official update on ZJIT. Things are going well. We’re excited to share our progress with you. We’ve done a lot since May.

In brief

ZJIT is compiled by default—but not enabled by default—in Ruby 4.0. Enable it by passing the --zjit flag or the RUBY_ZJIT_ENABLE environment variable or calling RubyVM::ZJIT.enable after starting your application.

It’s faster than the interpreter, but not yet as fast as YJIT. Yet. But we have a plan, and we have some more specific numbers below. The TL;DR is we have a great new foundation and now need to pull out all the Ruby-specific stops to match YJIT.

We encourage you to experiment with ZJIT, but maybe hold off on deploying it in production for now. This is a very new compiler. You should expect crashes and wild performance degradations (or, perhaps, improvements). Please test locally, try to run CI, etc, and let us know what you run into on the Ruby issue tracker (or, if you don’t want to make a Ruby Bugs account, we would also take reports on GitHub).

State of the compiler

To underscore how much has happened since the announcement of being merged into CRuby, we present to you a series of comparisons:

Side-exits

Back in May, we could not side-exit from JIT code into the interpreter. This meant that the code we were running had to continue to have the same preconditions (expected types, no method redefinitions, etc) or the JIT would safely abort. Now, we can side-exit and use this feature liberally.

For example, we gracefully handle the phase transition from integer to string; a guard instruction fails and transfers control to the interpreter.

def add x, y
  x + y
end

add 3, 4
add 3, 4
add 3, 4
add "three", "four"

This enables running a lot more code!

More code

Back in May, we could only run a handful of small benchmarks. Now, we can run all sorts of code, including passing the full Ruby test suite, the test suite and shadow traffic of a large application at Shopify, and the test suite of GitHub.com! Also a bank, apparently.

Back in May, we did not optimize much; we only really optimized operations on fixnums (small integers) and method sends to the main object. Now, we optimize a lot more: all sorts of method sends, instance variable reads and writes, attribute accessor/reader/writer use, struct reads and writes, object allocations, certain string operations, optional parameters, and more.

For example, we can constant-fold numeric operations. Because we also have a (small, limited) inliner borrowed from YJIT, we can constant-fold the entirety of add down to 3—and still handle redefinitions of one, two, Integer#+, …

def one
  1
end

def two
  2
end

def add
  one + two
end

Register spilling

Back in May, we could not compile many large functions due to limitations of our backend that we borrowed from YJIT. Now, we can compile absolutely enormous functions just fine. And quickly, too. Though we have not been focusing specifically on compiler performance, we compile even large methods in under a millisecond.

C methods

Back in May, we could not even optimize calls to built-in C methods. Now, we have a feature similar to JavaScriptCore’s DOMJIT, which allows us to emit inline HIR versions of certain well-known C methods. This allows the optimizer to reason about these methods and their effects (more on this in a future post) much more… er, effectively.

For example, Integer#succ, which is defined as adding 1 to an integer, is a C method. It’s used in Integer#times to drive the while loop. Instead of emitting a call to it, our C method “inliner” can emit our existing FixnumAdd instruction and take advantage of the rest of the type inference and constant-folding.

fn inline_integer_succ(fun: &mut hir::Function,
                       block: hir::BlockId,
                       recv: hir::InsnId,
                       args: &[hir::InsnId],
                       state: hir::InsnId) -> Option<hir::InsnId> {
    if !args.is_empty() { return None; }
    if fun.likely_a(recv, types::Fixnum, state) {
        let left = fun.coerce_to(block, recv, types::Fixnum, state);
        let right = fun.push_insn(block, hir::Insn::Const { val: hir::Const::Value(VALUE::fixnum_from_usize(1)) });
        let result = fun.push_insn(block, hir::Insn::FixnumAdd { left, right, state });
        return Some(result);
    }
    None
}

Fewer C calls

Back in May, the machine code ZJIT generated called a lot of C functions from the CRuby runtime to implement our HIR instructions in LIR. We have pared this down significantly and now “open code” the implementations in LIR.

For example, GuardNotFrozen used to call out to rb_obj_frozen_p. Now, it requires that its input is a heap-allocated object and can instead do a load, a test, and a conditional jump.

fn gen_guard_not_frozen(jit: &JITState,
                        asm: &mut Assembler,
                        recv: Opnd,
                        state: &FrameState) -> Opnd {
    let recv = asm.load(recv);
    // It's a heap object, so check the frozen flag
    let flags = asm.load(Opnd::mem(64, recv, RUBY_OFFSET_RBASIC_FLAGS));
    asm.test(flags, (RUBY_FL_FREEZE as u64).into());
    // Side-exit if frozen
    asm.jnz(side_exit(jit, state, GuardNotFrozen));
    recv
}

More teammates

Back in May, we had four people working full-time on the compiler. Now, we have more internally at Shopify—and also more from the community! We have had several interested people reach out, learn about ZJIT, and successfully land complex changes. For this reason, we have opened up a chat room to discuss and improve ZJIT.

A cool graph visualization tool

You have to check out our intern Aiden’s integration of Iongraph into ZJIT. Now we have clickable, zoomable, scrollable graphs of all our functions and all our optimization passes. It’s great!

Try zooming (Ctrl-scroll), clicking the different optimization passes on the left, clicking the instruction IDs in each basic block (definitions and uses), and seeing how the IR for the below Ruby code changes over time.

class Point
  attr_accessor :x, :y
  def initialize x, y
    @x = x
    @y = y
  end
end

P = Point.new(3, 4).freeze

def test = P.x + P.y

More

…and so, so many garbage collection fixes.

There’s still a lot to do, though.

To do

We’re going to optimize invokeblock (yield) and invokesuper (super) instructions, each of which behaves similarly, but not identically, to a normal send instruction. These are pretty common.

We’re going to optimize setinstancevariable in the case where we have to transition the object’s shape. This will help normal @a = b situations. It will also help @a ||= b, but I think we can even do better with the latter using some kind of value numbering.

We only optimize monomorphic calls right now—cases where a method send only sees one class of receiver while being profiled. We’re going to optimize polymorphic sends, too. Right now we’re laying the groundwork (a new register allocator; see below) to make this much easier. It’s not as much of an immediate focus, though, because most (high 80s, low 90s percent) of sends are monomorphic.

We’re in the middle of re-writing the register allocator after reading the entire history of linear scan papers and several implementations. That will unlock performance improvements and also allow us to make the IRs easier to use.

We don’t handle phase changes particularly well yet; if your method call patterns change significantly after your code has been compiled, we will frequently side-exit into the interpreter. Instead, we would like to use these side-exits as additional profile information and re-compile the function.

Right now we have a lot of traffic to the VM frame. JIT frame pushes are reasonably fast, but with every effectful operation, we have to flush our local variable state and stack state to the VM frame. The instances in which code might want to read this reified frame state are rare: frame unwinding due to exceptions, Binding#local_variable_get, etc. In the future, we will instead defer writing this state until it needs to be read.

We only have a limited inliner that inlines constants, self, and parameters. In the fullness of time, we will add a general-purpose method inlining facility. This will allow us to reduce the amount of polymorphic sends, do some branch folding, and reduce the amount of method sends.

We only support optimizing positional parameters, required keyword parameters, and optional parameters right now but we will work on optimizing optional keyword arguments as well. Most of this work is in marshaling the complex Ruby calling convention into one coherent form that the JIT can understand.

Performance

We have public performance numbers for a selection of macro- and micro-benchmarks on rubybench. Here is a screenshot of what those per-benchmark graphs look like. The Y axis is speedup multiplier vs the interpreter and the X axis is time. Higher is better:

You can see that we are improving performance on nearly all benchmarks over time. Some of this comes from from optimizing in a similar way as YJIT does today (e.g. specializing ivar reads and writes), and some of it is optimizing in a way that takes advantage of ZJIT’s high-level IR (e.g. constant folding, branch folding, more precise type inference).

We are using both raw time numbers and also our internal performance counters (e.g. number of calls to C functions from generated code) to drive optimization.

Try it out

While Ruby now ships with ZJIT compiled into the binary by default, it is not enabled by default at run-time. Due to performance and stability, YJIT is still the default compiler choice in Ruby 4.0.

If you want to run your test suite with ZJIT to see what happens, you absolutely can. Enable it by passing the --zjit flag or the RUBY_ZJIT_ENABLE environment variable or calling RubyVM::ZJIT.enable after starting your application.

On YJIT

We devoted a lot of our resources this year to developing ZJIT. While we did not spend much time on YJIT (outside of a great allocation speed up), YJIT isn’t going anywhere soon.

Thank you

This compiler was made possible by contributions to your PBS station open source project from programmers like you. Thank you!

• Aaron Patterson
• Abrar Habib
• Aiden Fox Ivey
• Alan Wu
• Alex Rocha
• André Luiz Tiago Soares
• Benoit Daloze
• Charlotte Wen
• Daniel Colson
• Donghee Na
• Eileen Uchitelle
• Étienne Barrié
• Godfrey Chan
• Goshanraj Govindaraj
• Hiroshi SHIBATA
• Hoa Nguyen
• Jacob Denbeaux
• Jean Boussier
• Jeremy Evans
• John Hawthorn
• Ken Jin
• Kevin Menard
• Max Bernstein
• Max Leopold
• Maxime Chevalier-Boisvert
• Nobuyoshi Nakada
• Peter Zhu
• Randy Stauner
• Satoshi Tagomori
• Shannon Skipper
• Stan Lo
• Takashi Kokubun
• Tavian Barnes
• Tobias Lütke

(via a lightly touched up git log --pretty="%an" zjit | sort -u)

Ruby has always been a joy to write. But for a long time, reading Ruby documentation on docs.ruby-lang.org hasn’t really matched that experience.

Last year, I brought a new look to the Darkfish theme by updating its visuals and improving mobile support. It was a visible improvement, but it wasn’t enough.

So this year, I built something new from the ground up. Starting with RDoc 7.0.0, Aliki is now the default theme for RDoc.

This release also coincides with Ruby’s 30th anniversary and the redesign of ruby-lang.org—a great moment to give Ruby’s documentation a fresh look as we head into the next chapter with Ruby 4.0.

Why a New Theme?

Even after last year’s improvements, I still didn’t enjoy using docs.ruby-lang.org as much as I wanted. Every time I needed to look something up, the experience felt dated.

And it was difficult to further improve Darkfish because:

• It lacks documentation, especially around the original design decisions
• Some of the patterns it uses were outdated
• Some third-party themes build on Darkfish, so updating it too much risks breaking them

RDoc itself added more constraints: It can’t depend on any gem that doesn’t ship with Ruby itself, and it can’t run a modern JavaScript build pipeline.

RDoc was created in the pre-Node.js era and hasn’t evolved with frontend tooling. Adopting modern toolchains would raise the dependency requirements for everyone—Ruby’s documentation generation pipeline, gems like IRB and Reline, and so on.

So all JavaScript, CSS, and HTML had to be written directly—no frameworks, no build tools, no npm packages.

And honestly, given all the constraints, this project wouldn’t have been possible without AI coding agents. Last year, just getting Darkfish’s code block styling right took me hours. It was a struggle for me to implement the look I wanted, and then to make it work with surrounding elements. At that pace, building an entire new theme wasn’t realistic.

This year, however, I discovered that I could try three different UI styles in an hour with AI agents. So I decided to take on the impossible task.

My goal was simple: make docs.ruby-lang.org look modern and actually enjoyable to use.

I collected all the features I wished a documentation site would have, gathered feedback from Rubyists around me, cherry-picked the improvements the community added to Darkfish last year (SEO, search enhancements, etc.), and put them together into a new theme.

Ok, enough of the back stories. Let’s see what Aliki brings:

Search

The old search wasn’t intuitive. It supported fuzzy matching, but getting the sorting right was difficult—searching Arr never actually got you the Array class as the first result.

After a few patch-ups, it was still buggy, so I rewrote it with a new UI:

Some notable new features/improvements:

• Case-aware ranking: If you search parse (lowercase), methods show up first. If you search Parser (capitalized), classes and modules come first.
• Fuzzy matching: This existed before, but fuzzy results used to pollute the top of the list. Now we have a smarter ranking system to make sure exact/substring matches show up before fuzzy results.
• Constants included: You can now search for constants, along with classes, modules, and methods.
• Type labels: Each result shows whether it’s a class, module, method, or constant.
• Keyboard support: Did you know you can press / to focus the search bar? This existed in Darkfish too, but I thought it was worth mentioning.

Dark Mode

Aliki has a light/dark toggle. It saves your preference and respects your OS dark mode setting by default.

Layout

The layout has three columns:

• Left sidebar: Navigation for pages, ancestors, methods, and class/module index
• Center: The documentation content
• Right sidebar: A table of contents generated from headings, with the current section highlighted as you scroll

Sidebar sections can collapse. For example, when you’re on a class or module page, the pages section is automatically collapsed so you can focus on the relevant navigation, with page documents still accessible if you need them.

Speaking of pages, I also reorganized the pages list this year. It used to be a long, rather flat list—now pages are grouped and much easier to navigate. In the coming year, we’ll continue improving page documentation so it feels more like a coherent guide rather than a collection of loosely related pages.

On mobile, the layout is a single column with a hamburger menu and a full-screen search modal—same as before.

Code Features

Code blocks now have copy buttons:

C code is now highlighted too:

For Gem Documentation

Aliki works for any gem, not just Ruby core. If you generate documentation with RDoc 7.0+, your users will see this theme automatically.

You can also customize the footer links now. For example, in your .rdoc_options:

footer_content:
  DOCUMENTATION:
    Home: index.html
  RESOURCES:
    GitHub Repository: https://github.com/your/repo
    Issue Tracker: https://github.com/your/repo/issues

This is useful for linking to your gem’s repository, issue tracker, or other resources.

To keep using Darkfish in your project, you can switch back with:

generator_name: darkfish

Acknowledgments

Thanks to @tompng and @earlopain for reviewing the code and helping polish things up.

Try It Out

You can see Aliki at docs.ruby-lang.org/en/master/ or ruby.github.io/rdoc/.

If you find issues or have suggestions, open an issue on GitHub.

What’s Next

Now that reading docs is enjoyable again, the next step for RDoc is to make writing docs enjoyable too.

About the Name

Aliki is my cat. I’m not good at naming things, so I just named the new theme after her.