In our last post on Ractors, Edouard demonstrated the potential benefits of using Ractors in a Rails application. He also explained how our team has been working towards making Rails and its dependencies Ractor-friendly. While we’ve been able to get smaller applications running on Ractors, we still have a long way to go to unlock larger applications, which use more features of Rails and many more gems. However, we have reached a good point where we can start to talk about the process itself.

In this post, I want to discuss some of the patterns we’ve used to refactor code to work with Ractors. But first, let’s talk about why any of this is necessary by looking at the constraints that Ractors impose, and why those constraints even exist.

Fearless Concurrency Comes at a Cost

Let’s start by talking about “what’s wrong” with one of today’s common concurrency primitives, Threads. Threads provide an illusion of parallelism by interleaving their execution, but every thread shares CRuby’s Global VM Lock (GVL), so only one Thread can ever execute Ruby code at a time. If the GVL could be removed, then every Thread could execute in parallel, but that would also expose anything shared between Threads to data races. What we’d really like is something that removes the GVL, but also prevents those data races.

That’s exactly what Ractors do.

Each Ractor has its own GVL, which allows them to execute Ruby code in parallel. They also prevent data races by disallowing mutable state from being shared between Ractors.

Of course, it’s that second bit that necessitates making changes to existing code. Let’s look at some examples to see what that means:

module Rails
  VERSION = [8, 2, 0].join(".")
end

Here’s a simplified implementation of Rails::VERSION. Can we use it in a Ractor?

Ractor.new { Rails::VERSION }.value
# can not access non-shareable objects in constant Rails::VERSION of a class/module created by another Ractor. (Ractor::IsolationError)

Nope!¹ To use Strings created in other Ractors, they have to be shareable:

module Rails
  VERSION = [8, 2, 0].join(".").freeze
end

Ractor.new { Rails::VERSION }.value
# => "8.2.0"

And this “shareable” requirement applies to almost everything you could want to use across Ractors. For simple objects like String, Array, and Hash, “shareable” just means the objects need to be deeply frozen. You can either freeze values individually, or you can use Ractor.make_shareable to deep freeze an entire object graph.

# frozen_string_literal: true

A = { msgs: ["This won't work"] }.freeze

Ractor.new { A[:msgs].first }.value
# can not access non-shareable objects in constant Object::A of a class/module created by another Ractor. (Ractor::IsolationError)

B = { msgs: ["But this will!"].freeze }.freeze

Ractor.new { B[:msgs].first }.value
# => "But this will!"

C = Ractor.make_shareable({ msgs: ["This will also work!"] })

Ractor.new { C[:msgs].first }.value
# => "This will also work!"

💡 Tip

You can use RuboCop’s Style/MutableConstant cop to help enforce this.² We also added a new Recursive mode which ensures constants aren’t just frozen shallowly.

Okay, so just deep freeze everything and we should be good, right? Unfortunately, it’s not quite that simple.

Stop Being so Lazy!

We’ve previously written about how the memoization pattern can affect performance by increasing the number of Object Shapes a class will have. The memoization pattern can also be a problem for Ractors, since Ractors need shared objects to be immutable and instance variables can’t be set on frozen objects.

We’ve solved these kinds of issues a few different ways.

Remove the Memoization

The first is to re-evaluate the use of memoization. Does the value’s computation really require caching? Perhaps it was justified when originally written, but maybe it’s fast enough today to run on demand and the memoization can be removed.

For example, Active Record used to memoize whether a model had overridden default_scope. We removed this memoization and now perform the check each time instead:

# Before
if default_scope_override.nil?
  self.default_scope_override = !Base.is_a?(method(:default_scope).owner)
end

if default_scope_override
  # ...
end

# After
if !Base.is_a?(method(:default_scope).owner)
  # ...
end

Compute on freeze

If the computation really needs to only happen once, another option is to perform it on freeze. By defining a freeze method, we can hook into the exact moment where the object will be frozen and perform the computation at the last possible moment.

For example, ActiveSupport::TaggedLogging::Formatter lazily computes a key the first time its tag stack is accessed:

def tag_stack
  @thread_key ||= "activesupport_tagged_logging_tags:#{object_id}"
  IsolatedExecutionState[@thread_key] ||= TagStack.new
end

If the formatter were frozen before tag_stack was called, setting @thread_key would raise a FrozenError. We added a freeze hook which forces the key to be computed first:

def freeze
  tag_stack
  super
end

Compute Eagerly

The last option I’ll mention is performing the computation eagerly. This is similar to hooking into freeze, but we’ll “hook” into initialize instead. This option tends to be best for immutable values that won’t change over the object’s lifetime and, as mentioned, can result in fewer object shapes as well.

For example, ActiveSupport::Cache::Strategy::LocalCache used to lazily memoize its local_cache_key. Since the key never changes for the lifetime of a Cache, we moved the computation to initialize:

# Before
def local_cache_key
  @local_cache_key ||=
    "#{self.class.name.underscore}_local_cache_#{object_id}".gsub(/[\/-]/, "_").to_sym
end

# After
def initialize(...)
  super
  @local_cache_key =
    "#{self.class.name.underscore}_local_cache_#{object_id}".gsub(/[\/-]/, "_").to_sym
end

attr_reader :local_cache_key

Immutable, but also Configurable?

For values which never change, deep freezing is quick and easy. However, Rails allows many things to be configured during boot, so we can’t just freeze everything.

One of the patterns we used for making configurable values Ractor-compatible is Read-Copy-Update (RCU). For example, let’s look at how we changed attr_readonly:

# Before
def attr_readonly(*attributes)
  self._attr_readonly |= attributes.map(&:to_s)
end

# After
def attr_readonly(*attributes)
  self._attr_readonly = Ractor.make_shareable(self._attr_readonly | attributes.map(&:to_s))
end

Before, _attr_readonly was a mutable array, and new attributes are just added to it. To make it Ractor compatible, it must be frozen, so now it:

• reads the old array
• creates a copy that adds the new attributes
• deeply freezes it with make_shareable
• and finally updates the variable to point to the new value

This pattern works well because _attr_readonly is an internal/private API, so we can make it frozen without worrying about breaking applications. For public APIs, we have to be more careful.

Both Rails and some of its dependencies have values which are documented as being mutable. We can’t freeze these values by default, or even use Read-Copy-Update internally, as that would break the compatibility guarantee.

For these, we typically have to deprecate the existing API in favor of a new one. But this also comes with another challenge: how can we make progress towards Ractor compatibility while still supporting the existing API?

For these cases, we have to get creative.

One example of this creativity is how we changed Rack::MethodOverride. This is the Rack middleware that enables forms to submit PUT requests by including a hidden _method field. It has two constants, HTTP_METHODS and ALLOWED_METHODS, which are both currently public and unfrozen.

To make the middleware compatible with Ractors, we introduced a new API: initialize will now accept http_methods and allowed_methods keywords which default to the current constants, and store the values as instance variables. This solves both the immediate Ractor compatibility issue as well as the backwards compatibility issue:

• if an application is currently mutating the constants, it can continue to do so (but will get a deprecation warning)
• if an application wants to be Ractor compatible, it can use Ractor.make_shareable to deep freeze the middleware and the constants will be frozen since they are stored as instance variables!³

Just Try to be Shareable

So far, we’ve mostly looked at values that Rails controls. However, many Rails APIs accept objects from applications, and Rails frequently stores those objects inside its own object graph. Callbacks are a particularly common example:

# frozen_string_literal: true

class Post < ActiveRecord::Base
  prefix = +"Draft: "
  before_create { title.prepend(prefix) }
end

The user provided object here is the block passed to before_create. Since Rails can’t guarantee that blocks like this can be made shareable with Ractor.shareable_proc, we introduced an internal helper which can be configured to warn or raise if the block can’t be made shareable:

# frozen_string_literal: true

class Post < ActiveRecord::Base
  prefix = +"Draft: "
  before_create { title.prepend(prefix) }
  # Logs:
  #
  # DEPRECATION WARNING: Rails attempted to make a Proc from your application Ractor shareable but a Ractor
  # Isolation error was raised. The proc being returned is not Ractor safe and a runtime
  # error may occur anytime during the request lifecycle.
  #
  # #<Proc:0x0000000128ff3ad8 app/models/post.rb:3>
  #  (called from <class:Post> at app/models/post.rb:3)
end

This can be configured by setting ActiveSupport::Ractors.unshareable_proc_action:

• nil doesn’t attempt to make application objects shareable (same as today)
• :warn attempts to make them shareable, but emits a deprecation warning if it can’t
• :raise attempts to make them shareable and raises the Ractor::IsolationError if it can’t

Note that the issue with Post’s callback above is that the block references prefix, which is a mutable String. Freezing the string or inlining the value would allow the block to successfully be made shareable.

Meanwhile, Back on the Main Ractor

Finally, there’s some stuff which is just… harder to fix. For example, Active Record wants to cache the list of columns which INSERT ... RETURNING statements will return. However, for some adapters this list depends on queried database metadata which is only available after database connections are established. Since booting a Rails application shouldn’t depend on a database connection being available, this work must be performed lazily. So, how do we do it?

Well, we can’t set class instance variables on non-main Ractors, but we can set them on the main Ractor.

For these cases, we added a wrapper around John Hawthorn’s ractor-dispatch gem. As the name implies, it provides a simple API for non-main Ractors to dispatch work to the main Ractor.

# Before
module ModelSchema
  def _returning_columns_for_insert(connection)
    @_returning_columns_for_insert ||= do_computation
  end
end

# After
module ModelSchema
  def _returning_columns_for_insert(connection)
    @_returning_columns_for_insert || ActiveSupport::Ractors.on_main(self) do
      @_returning_columns_for_insert ||= do_computation
    end
  end
end

The downside of dispatching work to the main Ractor is that we can lose the parallelism benefits that Ractors provide. If every Ractor is waiting on the main Ractor to do its work, then we’ve effectively re-introduced a single GVL. However, as long as it’s used sparingly, it can be a useful tool for making code Ractor-safe without having to do significant refactors. In our case, we only used it for a few memoized values, so the performance impact is negligible.

So, are we Ractor Yet?

As Edouard concluded in his post, this is only the beginning of the journey towards Ractor-age Rails. We’ve been able to run small applications on Ractors to demonstrate the performance benefits, and we’ve established a series of patterns for making code Ractor-safe that has worked well for us so far. We also know there’s some patterns which we still need to figure out, like using some sort of Ractor safe equivalent to concurrent-ruby, which currently only provides Thread safety.

However, we still have a long way to go. Big applications have more lines of code, more dependencies, and use more features of Rails which all need to be Ractor-safe. Luckily, we have a few more tricks up our sleeve for making progress… you should come learn about them at Rails World!

What is Rubydex linter

Rubydex is a static analysis toolkit for Ruby. My teammate Vini introduced it in more detail in One engine, many tools — Introducing Rubydex. Rubydex linter is Rubydex’s built-in linter for writing project-specific structural rules that use Rubydex’s understanding of a codebase.

Why structural linting

Many checks can be decided by reading one file. But in large codebases, we sometimes need to require or prohibit patterns that span several files.

Consider this example: a project has two modules, LegacyFooHelper and FooHelper, that provide different implementations of the same foo method.

module LegacyFooHelper
  def foo
    behaviour_for_legacy_models
  end
end

module FooHelper
  def foo
    new_behaviour
  end
end

We never want the same model class to include both helpers. But how can we check that?

A linter that reads one file at a time can prevent this direct case:

class User
  include LegacyFooHelper
  include FooHelper
end

But add a bit of inheritance, and the check becomes more difficult:

# user.rb
class User
  include LegacyFooHelper
end

# admin_user.rb
class AdminUser < User
  include FooHelper
end

The same conflict can also happen through a mixin:

# bar_helper.rb
module BarHelper
  include LegacyFooHelper
  # and does something else
end

# user.rb
class User
  include FooHelper
  include BarHelper # oops, now foo is overridden
end

To reliably detect every violation, we need the complete ancestor chain for every model class. We can then check whether both helpers appear in each chain.

A tool that only reads one file at a time does not have this information. Before Rubydex, we could only get it at runtime: load the entire codebase, then inspect every model class.

def test_no_duplicate_foo_helper_inclusion
  eager_load_the_app! # Slow, but necessary to load every model class.
  violated_models = model_classes.select do |model|
    include_both_foo_helpers?(model.ancestors)
  end
  assert_empty violated_models
end

Rubydex statically builds ancestor chains for the classes and modules it indexes, including dependencies. We can now prohibit this pattern without adding a slow runtime test.

How we use it in Shopify monolith

In Shopify’s Rails monolith, we found many tests that did the same work as test_no_duplicate_foo_helper_inclusion: they eager-loaded the whole application and then inspected class hierarchies, method definitions, constant definition locations, allowlists, etc.

For example, we use rules to check that controllers inherit from the correct shared controller class, serializers do not override prohibited methods, and allowlists still refer to real classes.

Those checks are important, but runtime tests are an expensive way to express them. Running each of those tests took minutes because they eager-loaded the whole app.

After we built the linter, we moved the checks that Rubydex can represent into more than 20 linter rules. They now run during local checks and CI without starting the application. The whole process, from indexing the monolith to running every rule, now takes about 30 to 45 seconds.

Write a rule for our example

Unlike most linters, Rubydex linter does not ship with a default set of structural rules for your application. This is because these structural rules are highly specific to individual projects. So the best way to try it is to write a new rule for your project!

First, add Rubydex to your project:

# Gemfile
gem "rubydex"

Then run bundle install.

Rubydex loads rules only from rubydex_linter/rules/ or lib/rubydex_linter/rules/, so create rubydex_linter/rules/no_conflicting_foo_helpers.rb:

# Prevents a class from including both FooHelper and LegacyFooHelper.
#
# This description appears in `rdx lint explain NoConflictingFooHelpers`.
class Rubydex::Linter::Rules::NoConflictingFooHelpers < Rubydex::Linter::CustomRule
  def self.default_severity = Rubydex::Severity::Error

  def lint
    # Find every class with FooHelper in its ancestor chain, including indirect ancestors.
    child_classes("FooHelper").each do |model|
      # Check whether LegacyFooHelper is also in the ancestor chain.
      next unless model.has_ancestor?("LegacyFooHelper")

      # Report the class definition when both helpers are present.
      definition = model.definitions.first

      add_diagnostic(
        "`#{model.name}` must not include both `FooHelper` and `LegacyFooHelper`.",
        diagnostic_location(definition),
      )
    end
  end
end

Once it’s there, run the rule with:

bundle exec rdx lint

Rules can use Rubydex::Linter::RuleTestCase for focused tests. It builds a small Ruby workspace for each example and lets you mark the expected diagnostic directly below the source code.

Here are a few directions you can explore for your codebase:

1. Require certain ancestor relationships: if a class includes A, it must also include either B or C.
2. Connect file names, class names, and inheritance: every file ending in _test.rb must define a class ending in Test that inherits from MyProjectTestCase.
3. Prohibit certain method overrides: descendants of BaseSerializer must not define serializable_hash.
4. Enforce dependency boundaries: files under app/domain/**/*.rb must not reference constants under Web::*.
5. Restrict where classes can be reopened: User must not be defined outside app/models/user.rb.

Configure and share rules

By default, Rubydex recursively indexes every .rb, .rake, .rbs, and .ru file it finds in the workspace. Use the [graph] section of rubydex.toml to prevent generated or irrelevant files from being indexed:

[graph]
exclude = [
  "vendor",
  "**/*.generated.rb",
]

Graph exclusions are relative to the workspace root and support glob patterns. They prevent matching files and directories from being indexed.

Rubydex already skips these directories at the workspace root: .bundle, .claude, .git, .github, .ruby-lsp, .vscode, log, node_modules, and tmp. Entries under [graph].exclude are added to these defaults.

Rules can be configured separately:

[linter.rules.NoConflictingFooHelpers]
severity = "warning"
exclude = ["test/**"]

• Loaded rules are enabled by default. Set enabled = false in a rule’s section to disable it.
• severity overrides the rule’s default severity. Available values are hint, information, warning, and error. The process exits with code 1 only when the linter reports at least one diagnostic with error severity.
• A rule-level exclude only suppresses diagnostics in matching files; it does not prevent those files from being indexed.

Running bundle exec rdx lint explain NoConflictingFooHelpers displays the documentation written above the rule. Rules can also be distributed in gems by placing them under lib/rubydex_linter/rules/.

Editor integration

At the time of writing, editor integration requires the Ruby LSP beta server, which you can enable in VS Code via:

{
  // VS Code settings. Enable the beta version of the server.
  "rubyLsp.featureFlags": {
    "betaServer": true
  }
}

For other editors, pass the betaServer flag through initialization options.

To learn more about writing rules for your project, see our linter guide.

How does this compare with RuboCop’s Rubydex integration?

RuboCop added optional Rubydex-backed project indexing in 1.87 and expanded it in 1.89. RuboCop users can write custom cops for the same structural checks shown above. The rule APIs are different, so a Rubydex linter rule cannot run unchanged as a RuboCop cop. As the team behind Rubydex, we have our own vision for it and its built-in tools. We have always envisioned Rubydex with a built-in linter, and we will continue building toward that vision.

The two tools can coexist. Both support structural linting, and users can choose the tool that best suits the needs of their project.

Current scope and limitations

Rubydex does not yet understand every kind of Ruby code. It does not expose local variables or the context around conditions and rescue clauses, for example. It also does not understand what DSL calls such as belongs_to mean.

For some rules, we used Prism to parse the targeted files and collect the missing information. We aim to reduce these cases as we continue to expand the context Rubydex captures.

Conclusion

Rubydex linter is the first Ruby linter designed specifically for checking project-defined structural patterns across the whole codebase. We look forward to discovering and learning from the community about all the things we can use it for.

As One engine, many tools — Introducing Rubydex states: “one engine, many tools.” Rubydex linter is one of the tools we’re experimenting with on top of Rubydex, and we will share more in future posts.

It’s been 8 years since the first just-in-time compiler was introduced in Ruby. However, throughout this time, the garbage collector has been a black box to the JIT compilers. This meant that JIT compilers were not able to optimize any object allocations and instead had to perform function calls. This has all changed now that we have integration of the GC fastpath into ZJIT. In this article, we’ll be looking at how object allocations work in the GC, what it means to inline the GC fastpath into ZJIT, and the performance implications.

Object Allocation

In Ruby’s default GC, object allocations previously used a free list based allocator. The free list allocator keeps a linked list of all of the empty slots. When it allocates an object, it allocates it at the head of the free list and moves to the next element. We can see an animation of the free list allocator:

In Ruby pseudocode, it looks like this:

def allocate_from_freelist
  obj = @freelist
  if obj # If we are able to allocate at the head of free list
    @freelist = obj.next # Move to next element of free list
    obj # Return allocated object
  else # Free list is empty
    gc_slowpath # Allocate more memory or run GC
  end
end

In contrast, a bump pointer allocator keeps a cursor and a limit. It allocates objects at the cursor and increments, or “bumps”, the cursor by the size of the object. If the allocation pushes the cursor past the limit, then the allocation would not succeed because we do not have enough memory for the allocation. Here’s an animation showing the bump pointer allocator:

In Ruby pseudocode, it looks like this:

def allocate_from_bump_pointer(size)
  obj = @cursor
  @cursor += size # Increment cursor by size of object
  if @cursor <= @limit # Enough space for allocation
    obj # Return allocated object
  else # Not enough space for allocation
    gc_slowpath # Allocate more memory or run GC
  end
end

If you compare the pseudocode of the free list allocator and the bump pointer allocator, it seems like they should both perform roughly the same right? There is one critical difference. A free list allocator calls obj.next, which requires reading from the slot that it has allocated. This extra memory read has significant performance penalties compared to the simple addition of the bump pointer allocator.

This was one of the main reasons why we switched Ruby’s default GC to use a bump pointer allocator from a free list allocator. However, we note that, due to technical limitations, the bump pointer allocator in Ruby’s default GC is not quite a canonical bump pointer allocator as it does not allow the allocation of truly dynamic object sizes. It still maintains several heaps each with their own fixed slot sizes. Each bump pointer is only allowed to bump by a certain fixed size.

Another GC implementation that we’re working on is MMTk. MMTk natively supports bump pointer allocators and the bump pointer allocator can allocate dynamic object sizes. We often use MMTk as a testbed for new techniques since it already has support for many advanced GC features.

GC Fastpath

Even with a bump pointer allocator, every object allocation requires a function call into the GC. This function call has significant overhead for object allocations. So how do we avoid this function call? The answer is for the JIT compiler to generate instructions to inline the fastpath of object allocations and we’ve built that into ZJIT.

But what is the “fastpath”? There are currently three possible paths an object allocation can take. Unsurprisingly, they’re called fast, medium, and slow paths:

• Fastpath: the path shown above in the Object Allocation section. This tries to bump the pointer in the current block to allocate the object.
• Medium path: if the fastpath fails to yield an object, it means that we have exhausted the memory of the block. We need to find the next block to allocate memory into and we can retry the fastpath.
• Slow path: if we have no more blocks, then we are really out of memory. We have to either allocate more memory or trigger a new GC cycle. This will create blocks that are empty, and we can retry the medium path.

We only inline the fastpath in ZJIT. If it fails, we fall back to using a function call into the GC. Since the vast majority of the allocations will hit the fastpath, it will yield the most performance gains without dealing with the complexity of the medium and slow paths.

Implementation

The GC fastpath was introduced in Ruby PR #17277. There are several moving parts that together make it all work, and we’ll look at it piece by piece.

GC Implementations

Support for the GC fastpath starts at the GC implementations themselves. Since the introduction of the modular GC feature there are multiple GC implementations in Ruby. GC fastpath implementation for each GC is different and so we have a GC API function rb_gc_impl_zjit_new_obj_fastpath that is given information about an object that we want to allocate. It will then decide whether it can perform the allocation using the fastpath or not. If it can, it will also emit the data necessary for the allocation, such as which cursor and limit to use.

The implementation here differs depending on whether we are using the default GC or MMTk. In this article, we will focus on the default GC.

ZJIT GC Fastpath

Once rb_gc_impl_zjit_new_obj_fastpath determines if we can use the fastpath allocator and provides us with the necessary data to do so, we can emit ZJIT LIR instructions for the fastpath. This happens in the file zjit/src/codegen/gc_fastpath.rs which defines the functions emit_default_new_obj_fastpath and emit_mmtk_new_obj_fastpath which emits the ZJIT LIR for the default GC and MMTk, respectively.

ZJIT HIR Instructions

Implementing the GC fastpath doesn’t automatically give us any gains because no objects are allocated through it. To take advantage of the GC fastpath, we have to use it when generating LIR for various ZJIT HIR instructions that allocate. For example, PR #17717 implements the GC fastpath for the newhash instruction when the hash is empty. The most important code is in gen_new_hash. I have extracted the relevant code from that function and added some comments:

// Determine the size of the object.
let alloc_size = unsafe { rb_zjit_hash_new_size() };
// Flags for object metadata.
let flags = RUBY_T_HASH as u64;
// Class of the object.
let klass = unsafe { rb_cHash };

// Use the GC fastpath to allocate the object.
let hash = gc_fastpath::gc_fastpath_new_obj(jit, asm, alloc_size, flags, klass, |asm| {
    // Closure for a fallback that gets called when the GC fastpath is not
    // supported by the GC or when it is out of memory.
    // This calls rb_hash_new, which is a C function that allocates a new
    // empty hash.
    asm_ccall!(asm, rb_hash_new,)
});
// Set up the allocated hash with nil as the default value.
asm.store(Opnd::mem(VALUE_BITS, hash, RUBY_OFFSET_RHASH_IFNONE), Qnil.into());
// Return the allocated hash object.
hash

So far, we have implemented GC fastpath on the following instructions:

Performance

Since the GC fastpath API is designed with a fallback closure when the fastpath is not able to allocate, we can easily turn off the fastpath by always executing the fallback. To clearly see the performance improvements, we can write small microbenchmarks that compare the allocation performance of various types. For example, the following benchmark allocates hashes with 3 elements in a loop.

def run(times)
  i = 0
  while i < times
    a = {a: 1, b: 2, c: 3}
    i += 1
  end
end

30.times { run(1) } # Warm up ZJIT
run(10_000_000) # Run the benchmark

We can see from the benchmark results below that the fastpath nearly doubles the performance of hash allocations. Note that this benchmark also includes interpreter and garbage collection overhead, which isn’t optimized by the GC fastpath:

Benchmark 1: fastpath
  Time (mean ± σ):      66.4 ms ±   1.4 ms    [User: 59.4 ms, System: 5.4 ms]
  Range (min … max):    64.7 ms …  73.1 ms    43 runs

Benchmark 2: no fastpath
  Time (mean ± σ):     117.2 ms ±   1.5 ms    [User: 113.1 ms, System: 2.9 ms]
  Range (min … max):   115.0 ms … 121.6 ms    25 runs

Summary
  fastpath ran
    1.77 ± 0.04 times faster than no fastpath

We can write similar benchmarks for other types and also see 2-3x performance gains.

Conclusion

For the first time, the garbage collector is no longer a black box to a Ruby JIT compiler. By having the GC hand ZJIT everything it needs to allocate an object ZJIT can inline the allocation fastpath and skip the C function call entirely. We’ve already brought this to hashes, ranges, arrays, strings, and object instances, with significant speedups seen in microbenchmarks.

Ractors have been experimental for years, but things are changing fast. Last year, a team of four on the Shopify Ruby Infrastructure team tackled Ractor scalability, performance, and stability. Ractors are now viable in production, and Ruby 4.1 will make them even better.

It’s time for Rails to embrace them.

In this post, we’ll explain why: what Ractors unlock for Rails applications, and the first milestone we’re aiming for in the framework.

First, a refresher

A Ractor, or Ruby Actor, brings true multi-core parallelism to Ruby. If you aren’t familiar with CRuby internals, you might expect Threads to already provide this — but they don’t, because of the Global VM Lock (GVL). Threads give you concurrency; Ractors give you parallelism.

For a Rails application, that unlocks two things:

• Speed — CPU-bound work can actually run in parallel instead of taking turns.
• Memory — you get that parallelism inside a single process, instead of forking a full copy of the app per worker.

The cost of forking

When several users hit a threaded Rails server like Puma, their requests run as separate threads in one process — but the GVL lets only one thread execute Ruby at any instant. If the first request does CPU-bound work, the others run only in the gaps. They interleave on a single core instead of running in parallel, so each one’s latency goes up.

To work around this, most Ruby servers fork the application: each worker runs in its own process, with its own memory and its own GVL. Our two users can now be served in parallel by two workers. This works — but it introduces a new trade-off: memory.

Because each forked worker has its own copy of the application, total memory usage goes up. Servers soften this by preloading the application before forking, so workers share those initial pages via Copy-on-Write. But that sharing is only a starting point: as each worker allocates objects, grows caches, and runs GC, its pages diverge and its footprint grows independently.

This is the trade-off Ractors let us sidestep. They run in parallel within a single process, so we get true parallelism without forking a full copy of the app per worker.

Testing the theory on a real application

That’s the theory. To see whether it holds outside microbenchmarks, we decided to put it to the test with an experiment: could we serve requests in a real Rails application using a Ractor-compatible web server, Kino?

We picked Basecamp’s Writebook as an ideal candidate: it’s a standard Rails application, well written and small enough. The questions we wanted to answer were:

• Is this experiment even doable, or are Ractors still fundamentally incompatible with Rails?
• How much memory could we save with a pool of N Ractors versus a clustered Puma setup with N workers?
• Does latency increase?

With the help of an AI agent, we worked our way toward making a few endpoints Ractor-safe:

• GET /up — A super lightweight endpoint that just returns a green webpage.
• GET / — The home page after setting up Writebook. It’s similarly lightweight, but exercises ERB rendering and executes a couple of DB queries.
• POST /first_run — The heaviest endpoint of the three: multiple DB queries, markdown rendering, image upload and analysis, and HTML sanitization.

We selected these endpoints so we could gradually see whether latency increased depending on the type of computation.

Multiple changes had to be made to let Writebook serve requests in a Ractor — inside gems, the Rails framework, and Writebook itself. A few native gem extensions were deliberately left Ractor-incompatible so we could measure the performance tax we pay when using an escape hatch, Ractor Dispatch, which lets us dispatch work to the main Ractor.

The source code of this experiment and the benchmarks are available in this repository if you’d like to run them on your own machine. Note that none of this is meant to be pretty or permanent — it’s just an experiment to gather numbers.

Results

Memory

The memory savings compounded as we increased the number of Ractors / Puma workers. Measured as PSS (which fairly accounts for memory shared between forked workers), a pool of 8 Ractors used nearly 7× less memory than 8 forked Puma workers serving the same load.

Latency

The latency difference on the most trivial endpoint is, unsurprisingly, virtually identical.

Things get more interesting on an endpoint that uses the Ractor Dispatch escape hatch, where a couple of DB queries are funneled to the main Ractor. The latency difference goes up slightly, but in absolute terms it’s negligible.

Finally, on the heaviest endpoint — where many operations are performed to set up a brand-new Writebook instance — latency clearly increases. It’s important to note, however, that most of the overhead comes from dispatching work to the main Ractor multiple times, because of native extensions that aren’t Ractor-compatible. Each dispatch incurs a full cross-Ractor round-trip — making the block shareable, then passing it and its result across the boundary — which ultimately increases latency.

As the Ruby ecosystem’s support for Ractors matures, this performance tax will fade away.

Concluding the experiment

This experiment confirmed that the benefit of Ractors isn’t only theoretical, and that there’s a viable path for Rails to support them. Of course, none of this is free. Making a Rails application Ractor-safe means freezing state, sharing objects explicitly, and — for now — funneling things like database queries to the main Ractor as we figure out how each Ractor can manage its own database connections.

This undeniably adds complexity to the framework, but we’re taking it on deliberately, because we believe the payoff is worth it and most of today’s rough edges are temporary.

Bringing Ractors to Rails

Our first milestone is deliberately modest: generate a brand-new Rails app, scaffold a resource, and serve its requests inside a Ractor. It sounds small, but a freshly scaffolded app already exercises a surprising amount of the framework — routing, the request/response cycle, view rendering, translation, database queries, the asset pipeline, and the many Active Support helpers.

We’re fixing all of these issues and are nearing the end. Getting all of that Ractor-safe is the foundation everything else builds on, and it puts Ractors within reach of anyone who can run rails new.

But it’s only the beginning. Beyond that first milestone, there’s a long list of work ahead — for instance, making Rails Ractor-safe in a non-eager-loaded environment so it works during local development, or extending Ractor support to the rest of the framework, such as Active Job, Active Storage, and Action Cable. There’s a lot left to do, and we’re excited to bring the rest of Rails into the Ractor-age.

We recently enabled a really cool feature in ZJIT that makes it feel like a Real Compiler™: the inliner! We’ll write more about it soon. In this post, we’ll talk about one excellent concrete benefit we are already seeing and how it optimizes blocks in pretty much every Ruby program.

I’ll start off with a refresher on how blocks work in the Ruby interpreter, then show you how ZJIT understands and optimizes that bytecode, and then show you the impact of the inliner.

Ruby: a refresher

In the beginning, there were loops.

arr = [1, 2, 3]
i = 0
sum = 0
while i < arr.length
  sum += arr[i]
  i += 1
end

People used them to navigate and manipulate variable-length structures, like arrays and strings. This was fine.

Then, in the 1970s, a small group of computer scientists at Palo Alto Research Center invented a programming language called Smalltalk. One of the core features of Smalltalk was that everything was an object and computation was done by sending messages to objects.

This meant that iteration wouldn’t do at all. Instead, we would have to send the do: message to the array object and pass it a block object.

#(1 2 3) do: [:a | sum := sum + a].

Then, in the 1990s, Matz, inspired by Smalltalk and Perl, created Ruby. We still have “normal” loops but we also have a very Smalltalk-y way of doing it, too:

arr = [1, 2, 3]
sum = 0
arr.each do |a|
  sum += a
end

When this program gets compiled to Ruby bytecode, it ends up looking like a mostly normal method call to each except that we pass a special kind of argument to it: a block argument.

To see how this works inside CRuby, we’re going to look at a listing of YARV bytecode—CRuby bytecode. For more on YARV, I recommend Kevin Newton’s excellent Advent of YARV.

Ignore most of the bytecode dump below except for the instruction at 0009, the send instruction. We are send-ing (see? a message!) each with the block argument block in <main> (passed a different way than “normal” arguments, hence the argc:0).

$ ruby --dump=insns /tmp/arrayeach.rb
== disasm: #<ISeq:<main>@/tmp/arrayeach.rb:1 (1,0)-(5,3)>
local table (...)
[ 2] arr@0      [ 1] sum@1
0000 duparray                   [1, 2, 3]                 (   1)[Li]
0002 setlocal_WC_0              arr@0
0004 putobject_INT2FIX_0_                                 (   2)[Li]
0005 setlocal_WC_0              sum@1
0007 getlocal_WC_0              arr@0                     (   3)[Li]
0009 send                       <calldata!mid:each, argc:0>, block in <main>
0012 leave
...continued below...

This block in <main> from the bytecode listing above is the generated name of the instruction sequence (bytecode) corresponding to the block we passed to each. Its code, shown below, is the next thing in the bytecode dump. You can see the usage of local variables sum and a via getlocal and setlocal variants and also opt_plus, which represents addition (+). Again, the details are not terribly important:

...continued from above...
== disasm: #<ISeq:block in <main>@/tmp/arrayeach.rb:3 (3,9)-(5,3)>
local table (...)
[ 1] a@0<AmbiguousArg>
0000 getlocal_WC_1              sum@1                     (   4)[LiBc]
0002 getlocal_WC_0              a@0
0004 opt_plus                   <calldata!mid:+, argc:1, ARGS_SIMPLE>[CcCr]
0006 dup
0007 setlocal_WC_1              sum@1
0009 leave                                                (   5)[Br]
$

To make this work, Array has an each method. This each method takes in its optional block argument and calls it once for every element in the array.

As with most of the core data structures, Array’s methods tend to be written in C and each is no different. Here is its nice and short definition with comments added by me:

VALUE
rb_ary_each(VALUE ary)
{
    long i;
    // Some debug-mode-only checks
    ary_verify(ary);
    // If we didn't pass a block in, return an enumerator object
    RETURN_SIZED_ENUMERATOR(ary, 0, 0, ary_enum_length);
    // We passed a block in, so call it for every element
    for (i=0; i<RARRAY_LEN(ary); i++) {
        rb_yield(RARRAY_AREF(ary, i));
    }
    return ary;
}

You can’t really see the block parameter to the C code because it’s passed in a special location on the Ruby VM’s own stack called the block handler. All you need to know is that rb_yield knows where to find that and how to call it.

There’s just one more thing, which is… hang on, weren’t we building a JIT to optimize Ruby code? How are we going to optimize this C code?

JITs: a refresher

Rewriting code from C to Ruby means that the JIT compiler gets a chance to introspect the run-time behavior and code. This means that, over time, as the compiler and the runtime system grow together, more and more code might get rewritten in Ruby. This started happening a couple of years ago with YJIT.

YJIT precipitated some interesting changes to Ruby VM internals. For example, in 2022, Array#each being written in C started to hurt: it was an opaque blob that the JIT couldn’t reason about. So Kokubun submitted a PR to rewrite it in Ruby. After some back and forth, in 2024, Kokubun landed a different PR that everyone was happy with.

class Array
  def each # :nodoc:
    Primitive.attr! :inline_block, :c_trace, :without_interrupts

    unless defined?(yield)
      return Primitive.cexpr! 'SIZED_ENUMERATOR(self, 0, 0, ary_enum_length)'
    end
    i = 0
    until Primitive.rb_jit_ary_at_end(i)
      yield Primitive.rb_jit_ary_at(i)
      i = Primitive.rb_jit_fixnum_inc(i)
    end
    self
  end
end

Ah, finally. A version that JITs can reason about.

I keep saying “reason about” and what that means concretely here is a) that it’s written in a format that the JIT can ingest and optimize: Ruby, and b) mostly a brief rehashing of the key lessons in the venerable Smalltalk (!) paper Efficient Implementation of the Smalltalk-80 System (PDF):

• Though systems may offer very dynamic behavior, people don’t frequently make use of wild features all over the place
  • Most people pass fewer than 4 types of objects through a given method
  • As a corollary, even if there are many classes in a system, code locality is super important, and we can take advantage of that
  • Also, most people do not define, re-define, and otherwise continuously modify method definitions

(And if you don’t believe me that the lessons still apply, check out the excellent paper Who You Gonna Call (PDF) by Sophie Kaleba, Octave Larose, Richard Jones, and Stefan Marr.)

So JITs like to watch what types of objects flow through methods before compiling them. And JITs like to, since they know the types of objects, cache method lookups and specialize method invocations on those objects.

For example, take a look at this code:

def next_capacity(x)
  x.length * 2 + 1
end

x could be anything. There is no way of knowing its type by looking at the code. This means that the length method could be anything. Furthermore, there is no way of knowing what the return type of x.length is, so we can’t specialize the method lookups or invocations of * or + either.

But! Per our lessons above, likely only a few types flow through this code. Say the JIT’s profiler notices that x has historically been an Array. Then we might reasonably assume that it will continue to be an Array, so when we compile the method, we add a run-time type check: if the type is no longer an Array, jump back into the interpreter.

Let’s see what optimized code ZJIT can construct by combining profiling information with the above bytecode. ZJIT operates on its own high-level intermediate representation called, uncreatively, HIR. In the following HIR snippet, we can see this very run-time type check (“guard”) for the Array class:

fn next_capacity@file.rb:8:
...
bb3(v9:BasicObject, v10:BasicObject):  # v9 is self and v10 is x
  PatchPoint NoSingletonClass(Array@0xdecaf1)
  PatchPoint MethodRedefined(Array@0xdecaf2, length@0xdecaf3, cme:0xdecaf4)
  v35:ArrayExact = GuardType v10, ArrayExact recompile
  v36:CInt64 = ArrayLength v35
  v37:Fixnum = BoxFixnum v36
  # ...

(with real pointers replaced by fake ones for readability)

Because classes are, among other things, collections of methods, this type information tells us what the call target of x.length is: it’s Array#length! We have a special fast code snippet to read an array’s length so we do that instead of a method call.

And in case the methods ever get changed out from underneath us, we leave behind these markers called PatchPoint that invalidate the code.

Finally, because we know that the result of Array#length is always a small integer (Fixnum), we can special case the method lookups for * and + as well.

fn next_capacity@file.rb:8:
...
bb3(v9:BasicObject, v10:BasicObject):  # v9 is self and v10 is x
  PatchPoint NoSingletonClass(Array@0xdecaf)
  PatchPoint MethodRedefined(Array@0xdecaf, length@0xdecaf, cme:0xdecaf)
  v35:ArrayExact = GuardType v10, ArrayExact recompile
  v36:CInt64 = ArrayLength v35
  v37:Fixnum = BoxFixnum v36
  v18:Fixnum[2] = Const Value(2)
  PatchPoint MethodRedefined(Integer@0xdecaf, *@0xdecaf, cme:0xdecaf)
  v41:Fixnum = FixnumMult v37, v18
  v23:Fixnum[1] = Const Value(1)
  PatchPoint MethodRedefined(Integer@0xdecaf, +@0xdecaf, cme:0xdecaf)
  v45:Fixnum = FixnumAdd v41, v23
  CheckInterrupts
  Return v45

There you have it. This is how JITs work: observe, assume, specialize. So why am I telling you all this? How does this circle back to blocks?

Well, blocks work not quite the same way, but similarly. Instead of having an object that we call a method on, we just have the target instruction sequence¹. But if we observe that the block argument, in its special location, has been consistently one object, we can specialize the call to it.

This is more or less fine for some cases of code. For example, in the following code snippet, we only have one caller to custom_iterator, so its profiled block will be monomorphic (one observed shape).

def custom_iterator
  yield 1
  yield 2
  yield 3
end

custom_iterator do |x|
  puts x
end

This reinforces what we know and love about the Smalltalk-80 paper: code locality wins! Yes!

But unfortunately this falls apart when we start thinking about all of the core library methods (and potentially the methods and classes you have stashed away in the grab-bag utils.rb in your application). Those methods are probably megamorphic (many many observed shapes). They probably see all sorts of stuff because they are general-purpose utilities that everybody needs, all the time.

One such example in the Ruby core library is the venerable Array#each that we saw earlier. Because there are a million different call sites to Array#each across your application and each probably passes a totally different block, we’re in an unhappy situation. How can we possibly optimize for so many different blocks? What happened to our code locality? How do we fix this?

The code locality is in another tower

It’s okay. Code locality still rules. Look at all the various callers of Array#each. They all pass a different, but constant², block at the call-site:

def method_a(some_array)
  some_array.each do |x|
    puts x
  end
end

def method_b(some_array)
  some_array.each do |x|
    puts x*2
  end
end

def method_c(some_array)
  some_array.each do |x|
    puts x*4
  end
end

So the code locality that we need in Array#each to optimize the code is one level up at the caller. If we can use that call context, we can specialize the code.

YJIT accomplishes this by splitting, which is a very natural transformation for basic block versioning and tracing. Such compilers are good at following code paths as they would be executed and putting together context across method calls. However, YJIT’s heuristic for splitting blocks is based on manual annotations: it will only kick in for certain Ruby library functions specially annotated with Primitive.attr! :inline_block (and the name is a bit of a misnomer).

The team that builds ZJIT, a method JIT, decided not to add splitting facilities. We could split methods (and blocks), but we have an easier time reasoning about larger code units than YJIT does because we optimize an entire method at once. So instead, ZJIT chooses to get call context by inlining.

Inlining

Method inlining refers to copying the body of the callee into the caller. In the above example, it means copying the body of Array#each into each of method_a, method_b, and method_c. I don’t mean the Ruby code and I don’t mean the bytecode: I mean the HIR. ZJIT does this by building the HIR of the callee (Array#each) into the existing HIR of the caller (method_a, …).

The illustrious Kevin Menard wrote ZJIT’s inliner and he’ll write a post about all the details soon. It’s pretty interesting stuff.

For now, we can take a look at the (lightly edited) result of Array#each being inlined into method_a. The details don’t matter, but there are two things I want to call out:

• What used to be a dynamic invokeblock is now what we call InvokeBlockIseqDirect because we know from the call context what block method_a is passing to Array#each.
• What used to be a method call is now a loop: the condition check is in bb10, the body is in bb9, and the stuff after the loop is bb13/bb4. If you’re interested, try to find the iteration variable and where it gets incremented. See the footnote³ for the answer.

This is a massive improvement over the previous very generic operations.

fn method_a@/tmp/arrayeach.rb:2:
# ...
bb3(v9:BasicObject, v10:BasicObject):
  # ...
  v55:Fixnum[0] = Const Value(0)
  Jump bb10(v25, v55)
bb10(v68:ArrayExact, v69:Fixnum):
  v73:CInt64 = ArrayLength v68
  v74:Fixnum = BoxFixnum v73
  v75:BoolExact = FixnumGe v69, v74
  v77:CBool = Test v75
  CondBranch v77, bb13(), bb9(v68, v69)
bb13():
  CheckInterrupts
  Jump bb4(v68)
bb9(v90:ArrayExact, v91:Fixnum):
  v96:CInt64 = UnboxFixnum v91
  v97:BasicObject = ArrayAref v90, v96
  v99:CPtr = GetEP 0
  v100:CInt64 = LoadField v99, :VM_ENV_DATA_INDEX_SPECVAL@-0x8
  v101:CInt64[-4] = Const CInt64(-4)
  v102:CInt64 = IntAnd v100, v101
  v103:BasicObject = InvokeBlockIseqDirect (0x16f3d5968), v102, v97
  v107:Fixnum[1] = Const Value(1)
  v108:Fixnum = FixnumAdd v91, v107
  PatchPoint NoEPEscape(each)
  Jump bb10(v90, v108)
bb4(v114:BasicObject):
  PopInlineFrame
  PatchPoint NoEPEscape(method_a)
  CheckInterrupts
  Return v114

You may notice that there is still one call in the loop: the call to the block (InvokeBlockIseqDirect). That’s next on our list to tackle. We are optimistic that we will soon also be able to inline block calls. Then the whole thing will really be just a loop!

The code that enables us to reason about which block got passed into the inlined callee (Array#each) only landed a couple of days ago (July 10, 2026), written by Luke Gruber. This was one of Luke’s first changes to ZJIT.

So what does performance look like?

Well, for starters, the microbenchmarks that we use as “performance unit tests” for specific aspects of Ruby went wild. Some benchmarks that were using block-based looping got much faster; they were previously bounded by ZJIT’s block invocation performance.

Take a look at the cfunc_itself benchmark, which tests that we can fold away the call to Ruby’s built-in itself method. ZJIT ends up optimizing the times method to invoke the do block directly, and the do block gets optimized to nothing but a guard on the self’s class to make sure it hasn’t changed.

run_benchmark(500) do
  500000.times do |i|
    itself
    itself
    itself
    itself
    itself
    itself
    itself
    itself
    itself
    itself
  end
end

Because we had previously optimized the body away, the result of the direct block invocation is a massive speedup:

Other benchmarks also kind of stop making sense because of the amount of inlining. Our bmethod benchmark, which benchmarked how fast we can call methods defined with define_method, also stopped measuring anything of use. We’re going to have to rework the benchmark to be more fair…

As expected, larger Rails benchmarks don’t see a ton of change; they exercise a diffuse set of features so optimizing any one feature bumps the big benchmarks only a little bit.

I am excited to see what happens when we can fully turn each and friends into call-less loops!

Wrapping up

Code locality rules. The inliner helps inject more of it and reason across method calls. ZJIT, a little over one year old, is growing up! :’)

We’re still tuning the inliner knobs. Some of the code in this post required tweaking --zjit-inline-threshold to convince the compiler to inline Array#each into method_a because of Array#each’s size. It will take some time for the ZJIT developers to figure out reasonable defaults.

Try out our HIR explorer at tryzjit.fly.dev. Try out ZJIT in your application by adding the --zjit flag to a Ruby over 4.0. Thanks for reading and see you next time.

If you’ve ever written a test for your code, you’re probably familiar with typical test framework methods: test/it to define test cases, and assert/expect to make assertions about the behavior of your code.

However, I want to highlight a less commonly used method: in other languages or frameworks it goes by other names, but in Ruby’s minitest it’s called skip. In this post, I’ll cover what skip does, when it may be useful, and, most importantly, when you should probably use something else.

Just skip to the good stuff

Okay, so what does skip do? Put simply, it allows you to not run a test.

More concretely: in minitest, none of the test code after skip is run, an S will be printed instead of the usual ./F/E, and you’ll see it included in the number of skipped tests in the summary:

# test.rb
require "minitest/autorun"

class SkipTest < Minitest::Test
  def test_skip
    skip "This test is skipped."
    assert_equal 1, 2 # Notice that this assertion _would_ fail
  end

  def test_normal
    assert_equal 1, 1
  end
end

$ ruby test.rb
Run options: --seed 9367

# Running:

.S

Finished in 0.000576s, 3472.2225 runs/s, 1736.1112 assertions/s.

2 runs, 1 assertions, 0 failures, 0 errors, 1 skips

You have skipped tests. Run with --verbose for details.

So it’s not quite as simple as “just not running a test”. skip also includes some signals to make sure you know “hey, by the way, this test didn’t actually run”.

skip, don’t run

The most common use of skip is to temporarily disable a test. Let’s say you have a newly failing test, and maybe it’s caused by a dependency upgrade, or maybe you’re just in the middle of a really big refactor. In either case, you know you need to fix the test eventually, but you don’t want to deal with it right now. This is a good use case for skip!

Instead of skip, you could comment out the test and leave a TODO. However, this approach is worse because it’s much easier to forget that the test exists at all. With skip, you get a reminder every time you run your test suite that “you should probably fix these”.

In the rails/rails test suite, we also use skip to indicate something is missing from a developer’s environment. For example, the Active Support test suite contains tests for ActiveSupport::Cache that depend on redis and memcached. If those services aren’t running locally, the tests depending on them are skipped¹ and a message is printed telling the developer why.

This is another good use of skip! It allows developers who aren’t actively working on ActiveSupport::Cache to run the Active Support test suite without requiring them to set up more complex dependencies. But it also signals to those developers that there are more tests to run, they just aren’t currently running.

Don’t skip this next part

We’ve looked at a few good examples of using skip, but I also see it used in places where it shouldn’t be.

Here’s a (not real) example using ActiveSupport::Cache:

module SharedCacheTests
  def test_some_redis_specific_thing
    skip unless cache_store.is_a?(ActiveSupport::Cache::RedisCacheStore)

    # ... test code that only works for RedisCacheStore
  end
end

Don’t use skip for this!

This is bad because it completely ruins the value of the skip signal. If any tests are always skipped, then the test output will always have Ss, and the final skips count will always be nonzero. Both of these signals are useful because of their rarity; if a developer sees them, they know there’s something for them to do. If skip is used for tests where there isn’t anything for a developer to do, then the useful signals gets drowned out by the noise.

Another issue with using skip like this is the runtime cost: minitest’s skip happens at test runtime. That means all the code before the skip call still runs: any setup/teardown hooks in the test’s own class as well as any setup/teardown hooks in the test class’ ancestors. Maybe you’re lucky and your test suite is fast enough that this doesn’t matter, but in a larger codebase this could add up to a significant amount of wasted time.

So, if you shouldn’t use skip in these scenarios, what should you use instead?

There are (at least) three good alternatives.

In the ActiveSupport::CacheStore example above, the skipped test is specific to a particular cache store (redis). So really, it doesn’t belong in the shared tests for all cache stores. Put it where it belongs!

module SharedCacheTests
  # ... shared tests for all cache stores
end

class RedisCacheStoreTest < Minitest::Test
  include SharedCacheTests

  def test_some_redis_specific_thing
    # ... test code that only works for RedisCacheStore
  end
end

Maybe you run your entire test suite with different configurations: instead of a test class per backend you run tests against each backend in a separate process². Since each backend will have different capabilities, some tests may not apply to every backend. Instead of conditionally skipping those tests, you can lift the conditional out of the test so that the test isn’t even defined if it shouldn’t run.

class CacheTest < Minitest::Test
  if cache_store.supports_multi_get?
    def test_multi_get
      # ...
    end
  end
end

Or, if you aren’t using minitest, your test framework may have a way to annotate tests so that they only run in certain scenarios.

class CacheTest < Megatest::Test
  test "only works with redis", store: :redis do
    # ...
  end
end

# Skip tests that only work with redis
$ megatest ! :@store=redis

In all of these cases, the skip signals in the test output remain actionable and the test suite remains fast, while still ensuring only the relevant tests are run in each scenario.

skip to the end…

skip is a powerful tool for signaling to developers that some tests aren’t running. However, it must be used conservatively to ensure the signal retains its value.

Luckily, there are many alternatives to skip for those cases where action isn’t required. Don’t skip out on using them!

We recently landed a new register allocator in ZJIT and I thought I’d write a post about it!

What is a register allocator?

Whenever a compiler generates machine code it needs to decide where to put values. Those values usually take the shape of a variable in your function, though the compiler can also compute intermediate values as well. When we need to perform a calculation on some value, the CPU needs to know how to find the value.

The CPU can typically only compute output based on inputs that are in registers, though some architectures (like x86) allow computations on values stored in memory. That said, reading and writing to registers is much faster than memory, so it behooves the compiler to keep values in registers as long as possible.

Any particular function in your program could have tons of variables, but the number of available registers is finite, and architecture dependent. This is where a register allocator comes in. The register allocator looks at all of the variables, then figures out which registers they should go in, and if there aren’t enough registers figures out how to “spill” those variables to memory.

How does it work?

There are several well-known approaches to register allocation, each with different trade-offs between compile time and code quality. For ZJIT, we chose to implement a linear scan register allocator based on a reduced version of Christian Wimmer’s paper titled “Linear Scan Register Allocation on SSA Form”. The paper isn’t very long so I highly recommend giving it a read when you have time! Alternatively, Max Bernstein wrote a blog post breaking down the paper as well.

ZJIT uses Static single-assignment form, or SSA form in its back end. “SSA form” is a representation of code that only allows a variable to be assigned once.

For example, this Ruby code:

a = 123
a += 1
a

Would be represented in our backend Low-level Intermediate Representation (or “LIR”) kind of like this:

v1 = Const(123)
v2 = Const(1)
v3 = Add v1, v2
CRet v3

In the above pseudocode (it’s not exactly the same as the IR we use in the backend, but quite close), v1, v2, and v3 are all different variables. No variables are allowed to be re-assigned like in the corresponding Ruby code. The generated intermediate representation has exactly the same semantics as the Ruby code, but adds the restriction that a variable can only be written to once. When ZJIT generates an intermediate representation of the Ruby code, it translates all variables to these numeric SSA variables and also uses SSA variables for temporary values. In the example translation above we can think of v1 as being equivalent to a, v2 as a temporary variable with the value 1, and v3 as a new version of a that has been added with v2.

Variables can be written to once, but you can read from the variable as many times as you want. We call the place where the variable was written the “definition” and the place where the variable is read a “use”.

Lifetimes

The first thing a register allocator needs to understand is when each value is alive and for how long. We call this duration a “lifetime” or “live range”. A value’s lifetime (or live range) starts at the point where it’s defined (the definition) and ends at the place it is last used (its “last use”). If two values have overlapping lifetimes they can’t share the same register. If there are more overlapping lifetimes than registers, then we know we need to spill a value to memory.

Since these ranges refer to the “lifetime” of the variable, it’s common to say that the variable “came to life” at its definition, and then “died” at its last use.

Let’s look at an example. Consider this Ruby method that is already in SSA form:

def add_twice(a, b, c)
  d = a + b
  e = d + c
  e
end

Here are the lifetimes for each value:

#  Instruction     |  a  |  b  |  c  |  d  |  e
-------------------+-----+-----+-----+-----+----
1  d = a + b       |  x  |  x  |  .  |  .  |
2  e = d + c       |     |     |  x  |  x  |  .
3  return e        |     |     |     |     |  x

A . character means that the variable is alive at that instruction. An x character means the variable dies, but is still used at that instruction.

We can also express these lifetimes as ranges. The live range for each value is [definition, last use]:

a: [0, 1]
b: [0, 1]
c: [0, 2]
d: [1, 2]
e: [2, 3]

Where instruction 0 represents the function entry (parameter definitions).

• The parameters a, b, and c are all alive at instruction 1 because they were defined before the method body.
• a and b are last used at instruction 1, so they die after that.
• c stays alive through instruction 2 because that’s where it’s last used.
• d is defined at instruction 1 and last used at instruction 2, so it’s alive for both.
• e is defined at instruction 2 and last used at instruction 3.

At instruction 1, a and b die and d comes to life. Since we know that a and b are never used again, we’re free to reuse one of their registers for d. So we only need three registers at instruction 1: one each for a/d, b, and c. Similarly at instruction 2, c and d die as e is born, so we can reuse a register again.

Computing lifetimes involves a backward dataflow analysis over the control flow graph. We walk the instructions in reverse order, tracking which values are currently live. When we see the definition or use of a value, we know it is live at that instruction.

ZJIT has a debugging option to dump live range graphs like this:

$ ruby --zjit-call-threshold=2 --zjit-dump-lir=live_intervals ../test.rb

Here is an example of the output graph from one basic block as an excerpt from many blocks in a larger function:

          v0  v1  v2  v3  v4  v5  v6  v7  v8  v9  v10 v11
          --- --- --- --- --- --- --- --- --- --- --- ---
i0     :   .   .   .   .   .   .   .   .   .   .   .   .  FrameSetup
i2     :   v   .   .   .   .   .   .   .   .   .   .   .  v0 = Load [x21 - 0x28]
i4     :   █   v   .   .   .   .   .   .   .   .   .   .  v1 = Load [x21 - 0x20]
i6     :   █   █   .   .   .   .   .   .   .   .   .   .  Jmp bb3_l2([x19 + 0x18], v0, v1)

As mentioned earlier, in ZJIT, SSA variables names are just numbers. In the block above we have 12 variables numbered 0 through 11. These variables are listed in the first row along the top of the graph, numbered v0 through v11.

The first column lists the instruction numbers. In this case we have 4 instructions (i0 through i6), and for reasons outside the scope of this post, they’ve been numbered with even numbers.

The last column of the graph lists the actual LIR instruction. I’m not going to dive in to exactly what each of these instructions means, but you can see that some of them define variables (like i2 and i4 define v0 and v1), and some of them use variables (like i6 uses v0 and v1).

In the center of the chart, the v character indicates when a variable came to life, the solid block is when a variable is “alive”, and a ^ character indicates last use (though there are no “last uses” in this example).

Interference graphs

Once we know the lifetimes of all values, we need to figure out how many ranges overlap and where. One way to do this is to build an “interference graph”. An interference graph is a straightforward graph data structure where each node in the graph represents a live range, and each edge in the graph represents an “overlap”, or “interference”, with another live range.

Once you have an interference graph, you can treat allocation as a graph coloring problem, where each color represents a physical register available on that CPU. If we have k physical registers, we need a valid k-coloring of the interference graph. This is NP-complete in the general case, but heuristics can help simplify the problem.

While graph coloring produces excellent results, building and manipulating the interference graph can be expensive in terms of both time and memory, especially for large functions. JIT compilers should be fast, so we opted for a different algorithm: Linear Scan.

Linear Scan

As I mentioned earlier, we’re using a linear scan register allocator based on Christian Wimmer’s paper.

The algorithm is fairly straightforward. Once you’ve computed live ranges, iterate over those live ranges in order. When you get to an instruction where a live range starts, “pull” a free register from a pool, and assign the register to the live range. When you get to an instruction where a live range ends, put the register back in the pool.

If, at some point, you run out of registers in the pool, spill either the new live range, or an existing one.

Local vs global allocation

The techniques we’ve discussed so far like lifetimes, interference graphs, and linear scan can all be applied at different scopes. In the register allocator world there are two main types you’ll read about, and the difference is to do with how they process the program’s control flow graph.

First is the “local” register allocator. Every function is broken down in to multiple basic blocks, and program control flows through these blocks. A local register allocator will only ever “see” one single block at a time.

The second allocator is a “global” register allocator. A global register allocator will process the function’s entire graph at once. Confusingly, this allocator is not “global” in the sense of “global variables” in a program, but “global” in the sense that it analyzes an entire function at once rather than one basic block.

ZJIT’s previous register allocator was a local allocator inherited from YJIT. It only tracked lifetimes within a single basic block. This means that at every block boundary, all live values had to be moved to well-known locations (like the stack or fixed registers) so the next block could pick them up.

For a very lazy block-at-a-time compiler like YJIT, this is a reasonable trade-off. YJIT compiles one basic block at a time, so it never has the full picture of the function anyway. Using YJIT’s allocator helped us to bootstrap ZJIT quickly.

But ZJIT compiles entire methods, so we have all the information we need to do better.

A global allocator can keep a variable in the same register across block boundaries. If a value is defined in one block and used in a later block, the allocator can let its live range span both blocks and assign it a single register for the whole duration. This avoids unnecessary stores and loads at block boundaries, which can make a real difference in tight loops.

A global allocator also unlocks features that are difficult or impossible with a local one. For example, splitting and adding new basic blocks in various optimization passes was very tricky due to keeping track of what variables went where. Now basic blocks can be easily manipulated without worrying about which blocks use what variable. It’s also a prerequisite for method inlining, where the inlined callee’s code becomes part of the caller’s control flow graph and its values need to participate in the same allocation.

Where we are now

The new register allocator has landed and is working well. We’re now building on top of it! Method inlining is actively in progress and relies heavily on the global allocation we now have.

There are still improvements to make to the allocator itself. One big one is lifetime holes. Right now, a value’s live range is a single contiguous interval from its definition to its last use. But in practice, a value can be “dead” in the middle of its range.

For example, if a value is used in one branch of an if but not the other, the register allocator could accidentally keep the value “live” for one branch. This can be a problem because the value will end up using valuable resources (physical registers) that should be used for values that are actually alive at that time. The below Ruby program kind of visualizes this issue:

def example(cond, a, b)
  x = a + b

  if cond
    y = a * 2
    use(y)
  else
    use(x)
  end
end

Depending on how the code is linearized, the value x could be considered “alive” inside the true side of the if statement even though we can clearly see it’s not used. In other words, the register assigned to the value x will be reserved inside the true side even though it’s not actually used.

Representing these holes would let us reuse registers more aggressively in those gaps, reducing spills.

We’re excited about the foundation this gives us and looking forward to building on it!

One engine, many tools

A few years ago, the new Ruby parser Prism was released. One of its primary goals was to unify the community since we had multiple implementations of Ruby parsers, each with their own bugs, differences in implementation and portability. By having a single parser, community investments in performance and correctness benefit every single tool built on top of it (including Ruby itself!).

However, the story of repeated implementations of highly complex foundational blocks doesn’t end at the parser level. Move one level up the stack and the pattern repeats. Today, we have multiple tools that implement code indexing and related static analysis algorithms. Consider just a few examples:

• Language servers: tools like Ruby LSP and Solargraph need code indexing to provide go to definition, hover, signature help, completion and so on
• Type checkers: tools like Sorbet and Steep need code indexing for all of the previous reasons plus having the ability to type check code
• Documentation generators: tools like RDoc and YARD need code indexing to aggregate all declarations and their respective documentation for navigating and generating the static website
• Dead code detectors: tools like Spoom and debride need code indexing to match declarations and references, so that they can identify what declarations are dead (i.e.: unused)
• Linters: tools like RuboCop and Standard don’t currently use code indexing, but could provide much more sophisticated linting capabilities given a global knowledge of the codebase

The story we have here is the same. Multiple implementations of code indexing with varying performance, implementation differences and correctness discrepancies. On top of that, none of them are packaged and portable as an API that any other project can use.

It’s another case of our community’s efforts being diluted when we could instead have compounding benefits of working together in a foundational building block. We ought to do something about it.

Introducing Rubydex

Rubydex is a new portable static analysis engine intended to provide features such as code indexing and type analysis through a convenient API.

An important thing to note is that Rubydex is a framework/engine. It is not a tool by itself, but rather the core building block to create other tools. Despite being early in its development, Rubydex can already:

• Collect all definitions in a codebase and its dependencies, including classes, modules, constants, singleton classes, instance variables, class variables, global variables and methods
• Index RBS documents (including the bundled core and stdlib files and any RBS files in the workspace)
• Resolve constant references
• Track constant and instance variable references completely and method references with limitations¹
• Create declarations from the discovered definitions and constant references²
• Linearize ancestor chains
• Track descendants
• Query the resulting graph in many ways

Built for portability

One of our goals with Rubydex is portability. If someone wants to write sophisticated tooling for Ruby using a different language or maybe target the browser through WASM, then they should be able to!

For this reason, Rubydex is built with Rust, C and Ruby to ship 3 distinct components:

• The main Rust crate: this is where the entire logic is implemented. Rust allows for high performance and easy parallelism, which are incredibly valuable when implementing static analysis tooling where the performance constraints are intense. Other Rust projects can use this directly, like creating a Zed extension that can understand Ruby code or writing a linter in Rust.
• A Rust FFI crate: this crate provides C compatible bindings to use the main crate’s logic, allowing other languages to integrate with Rubydex. Developers can use this to write tooling in other languages, like a VS Code extension that can analyze Ruby codebases with no Ruby runtime dependency.
• A Ruby gem: a native extension that provides the Ruby API, which interacts with the underlying Rust implementation through the FFI crate. The gem ships with pre-compiled binaries for macOS (Intel and M series), Linux (x64 and ARM64) and Windows. For any other platforms, rubydex has a dependency on cargo (the Rust package manager) in order to compile correctly when installed.

Impact on existing tools

As of the time of this writing, we have either completed or started migrating our existing tools to use Rubydex. The impact story for all of them is essentially the same: better performance, higher accuracy and a lot less code to maintain.

Tapioca

You may know Tapioca for all of its runtime analysis, which is what allows the tool to output static RBI information for more accurate Sorbet type checking. However, Tapioca also consumes static information. There are two main use cases for static analysis in Tapioca:

• Fetching documentation for a given declaration so that it can be included in RBIs. This is important so that Sorbet can show documentation on hover
• Bootstrapping the initial analysis for generating gem RBIs. We discover the initial set of constants defined in a gem statically and then proceed to uncover the rest of the information by using the runtime

Tapioca adopted Rubydex to handle documentation, replacing the original approach which used YARD. After the change, we saw a massive performance improvement, dropping the total execution time for generating gem RBIs in our Core monolith from ~6 minutes to ~20 seconds, while using much less memory. We also replaced the gem RBI bootstrapping.

In addition to the performance gains, the correctness of Rubydex’s constant resolution algorithm and declaration handling allows comments to be attached in a more predictable way. This ensures that the comments are associated with the declaration they were meant to document, improving the quality of hover results for the Sorbet language server.

You can enjoy the Rubydex improvements starting in Tapioca v0.19.0.

Packwerk

Packwerk relies heavily on constant resolution to identify package violations. The problem is that constant resolution in Ruby is hard. The algorithm is complex and the amount of edge cases for static analysis increases the difficulty in getting it right. Investing the time to get it right in Rubydex means we don’t have to re-invent the wheel multiple times.

In our initial exploration of replacing Packwerk’s logic with Rubydex as the engine, we see the power of being able to reuse battle tested algorithms:

• Considerable reduction in codebase size (about -3000 lines)
• Significantly more violations found thanks to correct constant resolution (about 3x more constant references resolved in our monolith)
• Varying levels of performance improvements depending on the conditions (single-threaded vs parallel mode)

Spoom

We started prototyping the migration of Spoom’s dead code candidate detection to Rubydex. The story repeats once again. Faster analysis and higher accuracy thanks to being able to account for constant resolution and ancestors correctly. However, in the middle of the work, a question struck the team. Why not bundle this into Rubydex? It feels like dead code detection can be a part of the core engine, even if it’s only detecting candidates.

For that reason, we paused the Spoom migration to explore moving it into Rubydex instead. We hope to be able to ship the ability to find dead code candidates directly through our API.

Ruby LSP

For the Ruby LSP, Rubydex replaces the entire old code indexer with a significantly more performant and correct implementation. Several cases that we could not handle correctly are now understood by the language server and performance should be better across the board. The features directly impacted by Rubydex are:

• Go to definition, hover and signature help: better handling of nesting resolution will provide more accurate results
• Completion: the new completion engine in Rubydex provides significantly better results than the previous implementation
• Workspace symbol: new fuzzy search is much faster than the previous implementation
• Type hierarchy: ancestor information is more complete and correct. Additionally, descendant tracker allows us to implement the sub-types feature
• Test discovery for the explorer: faster analysis for test discovery
• Find references and rename: fast reference finding and renaming since Rubydex automatically tracks those
• Foundation: the previous indexer was not fast enough, so we ran indexing asynchronously in a thread. This improved responsiveness, but created several possible race conditions where taking an action in the editor before indexing was complete could result in discrepancies between the features and the actual state of the codebase. Rubydex is fast enough that we can simply index and manage the analysis synchronously, allowing us to eliminate the chances for race conditions and greatly simplifying the code in the Ruby LSP

In addition to the features already migrated to Rubydex, there are other improvements we are now able to deliver that weren’t possible before. For example, in semantic highlighting we treat all constant references with the token namespace. The reasoning behind this was performance. Resolving every constant reference in the old indexer was slow and semantic highlighting runs on every keypress, which would cause the editor to grind to a halt. With Rubydex, we can now solve this issue and correctly classify constant references as class, namespace or variable.constant.

Another possibility to explore is better test discovery. The current implementation of the test explorer relies on glob patterns to find test files in a performant way. With Rubydex, identifying all classes that inherit from Minitest::Test as a test group and all public methods prefixed with test_ is a breeze. Maybe we can finally ditch the glob pattern and achieve higher accuracy in discovery?

Safe to say, there are lots of opportunities for more sophisticated and correct functionality in the Ruby LSP, without compromising performance in any way.

Add-ons

The switch to Rubydex means breaking changes for most add-ons. Please refer to the Ruby LSP’s documentation and release notes for migration instructions.

MCP server

As part of Rubydex, we are also including an experimental MCP server. The purpose is to provide powerful tools for LLMs to explore the codebase more efficiently. Instead of reading files in their entirety and grepping based on text, the LLM can directly request information about declarations, definitions, references, ancestors, descendants (basically anything that you could query through the regular API).

We are still working on improvements and ironing out the best way to distribute the MCP server so that it can find dependencies correctly, integrating with version managers and Bundler (which can mutate where dependencies are installed in the system, making discovering them not so straightforward). For now, the MCP server must be built from source, but we hope to ship it as part of the Ruby gem in a future release.

In early testing, we’re seeing 15 to 80% reduction in token usage, cost and duration for certain LLM tasks thanks to the ability of quickly navigating the codebase. One example where this is easy to understand is asking the LLM to perform a refactor. Instead of scanning the entire codebase for usages of a class/module, it can directly ask Rubydex to get the information.

Current API

The Rubydex API is quite extensive to support all use cases in the Ruby LSP, Tapioca, Packwerk and other tools. Here are a few simple examples of how to use it, but please refer to the repo for more complete API documentation.

For this example, consider the following files present in the workspace:

# foo.rb
class Foo
  def found_me; end
end

# bar.rb
class Bar < Foo
  def found_me; end
end

# qux.rb
module Zip
  class Bar::Qux
    Foo
  end
end

We can use Rubydex to ask several questions about the state of the code:

require "rubydex"

# The graph is Rubydex's representation of the codebase. It's where everything starts and where all data is stored. To
# populate the graph, we first index the workspace (all .rb, .rbs files + dependencies) and then run the resolution step
# to transform definition information into semantic declarations.
#
# Find all of the other methods for managing the state of the graph in the repository's documentation.
graph = Rubydex::Graph.new
graph.index_workspace
graph.resolve

# Once the graph is ready, we can query it for any information we would like. This is what powers all language server's
# features in the new v0.27 and forward for the Ruby LSP.
#
# Find all of the querying features in the repository's documentation.
declaration = graph["Foo"]
puts declaration.ancestors.map(&:name) # => ["Foo", "Object", "Kernel", "BasicObject"]
puts declaration.descendants.map(&:name) # => ["Bar"]

puts graph.search("#found_me()") # => [#<Declaration name=Foo#found_me()>, #<Declaration name=Bar#found_me()>]

# Resolve a constant reference to `Foo` inside the `Zip, Baz::Qux` nesting
puts graph.resolve_constant("Foo", ["Zip", "Baz::Qux"]) # => [#<Declaration name=Foo>]

Future plans

This announcement marks the first iteration on Rubydex. There are a lot of interesting experiments that we want to do and lots to build. Here are some of the future ideas we have in our backlog.

Shared database

One of the ideas we partially experimented with and are eager to build is allowing Rubydex to save the analysis into a database. This benefits tools in a few ways:

• Reducing boot time: since the analysis is saved in the database, Rubydex wouldn’t have to start from scratch every time it is booted. This is regardless of how it gets initiated (language server, linter, MCP server)
• Loading partial data: developers rarely interact with 100% of their codebase all at the same time. Usually, you’re really only working in a small subset of the codebase at a time. This is true of LLMs too, they don’t need to read every single file in the codebase if you’re making a targeted change. Maintaining the entire analysis data in-memory at all times is wasteful. A lot of the information is simply not relevant for the changes being executed. If Rubydex can trace all of the dependencies of a document (and we believe it’s possible), then we can load a subset of the graph into memory and leave the rest in the database. This approach reduces the amount of memory being used by offloading to disk and it makes it so that memory usage does not scale linearly with the number of tools based on Rubydex (since they can share the same database)

If we can get this right, it would mean that you can have a language server, MCP server, linter, type checker all operating at the same time, reusing the same database to avoid multiple rounds of analysis and consuming little memory since most of the data is offloaded to disk.

Semantic linting

Another hope we have for Rubydex is enabling semantic linting. Rubydex can already collect diagnostics such as undefined constants. The idea is to offer a rich set of APIs so that linting rules can be created with the multitude of data available across the different phases of analysis.

The main benefits of semantic linting are reducing false positives and writing more complex rules that simple AST inspection cannot achieve. Here are some example rules that are possible (and even easy to write) with Rubydex, but not possible without a complete code indexing engine:

• Prohibiting inheriting from a certain ancestor: Maybe you’re deprecating the use of a module or parent class and you want to detect all existing uses (direct or transitive)
• Detecting potential dead code: Dead code is basically any declaration for which no references have been found. Of course, with meta-programming and untyped code, we can’t always be 100% sure it’s dead. However, our team has seen great benefit in flagging dead code candidates that then get reviewed by a developer to remove
• Detecting invalid mocks in tests: For example, stubbing a method to return a boolean when in fact it returns a string
• Detecting methods that are always shadowed: For example, a method in a parent class with real behaviour, but all of the subclasses override the method without invoking super
• Detecting incorrectly decomposed modules: For example, a module that depends on two instance variables, but none of the classes that include it define both instance variables, leading to module methods that can only be invoked when included into specific classes

And many, many others.

Type aware analysis

Currently, Rubydex has no concept of types and performs no inference. This limits the framework’s ability to understand method calls. For example:

var = Foo.something
var.other_thing

To know what defines other_thing, we need to know the type of var, and to know the type of var, we need to know the type returned by Foo.something. Only by propagating types through the analysis can we know for sure which methods are being used (or if they exist at all). Without this, Rubydex cannot associate method references to their declarations like it can for constants and instance variables.

The natural next step is to consume type annotations to improve the accuracy of the analysis. For that to work, we need to design the internal type system, consume annotations already used by the community (e.g.: Sorbet sigs and RBS), and experiment with different algorithms. The experimentation lies in the approaches we can take and understanding how well they can fit the Ruby language (e.g.: constraint solving vs set theoretic types).

Conclusion

Prism showed what happens when the Ruby community consolidates around a shared parser: every tool got better and every improvement compounded. We believe the same is possible one layer up. Analyzing Ruby statically is inherently hard, but we don’t have to solve the same problems separately.

Rubydex is our bet on a shared foundation. If you maintain a linter, a language server, a documentation generator, a type checker, a code-mod tool, or something we haven’t imagined yet, we’d love to build this with you.

Try it through the Ruby LSP, Tapioca or Packwerk, or directly use Rubydex’s API. File bugs, open issues, send PRs, tell us what’s missing in the repo. Every improvement made to Rubydex benefits all tools—even the ones that haven’t been built yet.

The story starts with Rust 1.85.0 giving me a surprise SIGTRAP in some test code. For reasons not important here, I had to bump opt-level from 0 to 1 for tests. That made a bunch of tests fail with SIGTRAP, whose default signal handler terminates the process.

The SIGTRAP happened in Rust code code that call C.

Passing a Rust closure through C code

The code that broke passes a Rust closure through a C function and invokes the closure in a callback.

The C API takes a user data argument in addition to the callback function, for passing through context:

// Calls (*proc)(data). VALUE is uintptr_t.
VALUE rb_protect(VALUE (* proc) (VALUE), VALUE data, int *pstate);

Each closure has a distinct anonymous type as they can vary in size depending on what’s captured. There is no way to separate out the captures and the function pointer to fit the C API. But, we can use a trait object to talk about a category of closure types, and the trait object has one uniform size:

fn closure_info(closure: impl FnMut()) {
    use std::mem::size_of_val;
    let trait_object: &dyn FnMut() = &closure;
    println!(
        "closure_size={} trait_object_size={}",
        size_of_val(&closure),
        size_of_val(&trait_object)
    );
}

fn main() {
    let mut int = 0;
    let one_capture = || int = 42;
    let no_capture = || {};
    closure_info(one_capture); // closure_size=8 trait_object_size=16
    closure_info(no_capture);  // closure_size=0 trait_object_size=16
    // Varying closure size, same trait_object_size.
}

The trait object is too big to fit the pointer-sized data argument, but we can solve that by taking a reference of it. That gives &mut &mut dyn FnMut(), a double reference to the trait object. &mut is pointer-sized, unlike &mut dyn. Great, we can now pass the closure through. Now we need to write the callback that invokes the closure.

The transmute

To call the closure, we need to first turn the VALUE back into the double reference to trait object. I used std::mem::transmute for this, and it blew up. Rust 1.85.0 compiles the following function to a single UD2 instruction which raises SIGTRAP.

#[repr(transparent)]
struct VALUE(usize);

extern "C" fn c_callback(obj: VALUE) {
    let closure: &mut &mut dyn FnMut() = unsafe { std::mem::transmute(obj) };
    closure();
}

Surprise SIGTRAPs like these from LLVM makes me think of trap mode in UndefinedBehaviorSanitizer. What rules did I break?

Probably pointer provenance

Integer to pointer transmute is documented to have unspecified behavior, but our struct VALUE(usize) definition is not an integer. Like many parts of Unsafe Rust, it’s hard to say how the transmute should have behaved. Rust 1.78.0 includes a change that seems to draw a clear line. “Lower transmutes from int to pointer type as gep on null” (“transmute patch” from here on) changes how transmute picks the provenance of the target pointer. Before, the transmute acted like an integer-to-pointer as cast, picking a previously exposed provenance. Now, it’s based on the null pointer. The null pointer is invalid for access, and so is the derived pointer. The SIGTRAP is probably trying to say, in an obtuse way, that I’m calling an invalid function pointer.

I had a hunch this is related to pointer provenance changes. I also remember from discussions about provenance something about writing a signature on the Rust side that uses a pointer type to accept an integer argument from the C side. I’m not sure if it’s a good idea to misrepresent types like this, but it’s useful to do it as an experiment to see if the SIGTRAP goes away:

// Experiment: the C side calls this with an integer (VALUE), but we claim it's a pointer
extern "C" fn c_callback(obj: *mut ()) {
    let closure: &mut &mut dyn FnMut() = unsafe { std::mem::transmute(obj) };
    closure();
}

And this version doesn’t SIGTRAP! So this is probably related to pointer provenance, rules that stipulate that in addition to having a good address, a pointer is valid for dereference only when obtained a certain way. The pointer in both the working version and the SIGTRAP version have the same address and in-memory representation, but only one is valid for dereference. The ptr module has documentation about provenance rules.

Searching in rust-lang/rust found me the transmute patch, and experiments on Compiler Explorer with various Rust versions showed 1.78.0 to be the first version that compiles the function to a single UD2 instruction. It’s the first release that includes the transmute patch.

But why does it SIGTRAP only on older Rusts? What changed?

Revert due to LLVM provenance bug

In the rust_has_provenance RFC, the lack of proper treatment of provenance in LLVM is stated as a drawback. Turns out, the transmute patch triggers such bugs and so was reverted in version 1.91.0. In some situations, the existence of a pointer with invalid provenance in the system can have LLVM confused and wrongly decide that an unrelated pointer is invalid for access. To be clear, the LLVM bugs are not the reason my code raises SIGTRAP; the transmute patch is designed to break such code. The unintended breakages from the transmute patch were due to the LLVM bugs giving bad output for code completely absent of Unsafe Rust.

There are many reports of code breakages due to the transmute patch, but the Rust team did not consider it to be a breaking change. I think it deserved a place in the release note as a compatibility issue. It also would have been nice if it panicked with a clear message rather than a nondescript SIGTRAP, but maybe the UD2 comes from a place too deep in LLVM’s pipeline to realistically replace.

With the revert, the SIGTRAP is gone. Absence of problematic runtime behavior does not imply absence of Undefined Behavior, though, and the revert did not change rules of the language. The current documentation for std::mem::transmute is clear about this subject:

Transmuting integers to pointers is a largely unspecified operation. It is likely not equivalent to an as cast. Doing non-zero-sized memory accesses with a pointer constructed this way is currently considered undefined behavior.

Let’s try to follow the rules.

The fix

Using an as cast instead of transmute avoids the SIGTRAP in all Rust versions:

extern "C" fn c_callback(obj: VALUE) {
    let closure = obj.0 as *const *mut dyn FnMut();
    unsafe { (**closure)() };
}

I understand why as cast works here better than transmute as follows: transmute works solely with the bits of the input value, but provenance is not represented in the input integer (integers have no provenance), so the output pointer has no provenance and is invalid for dereference. On the other hand, as casts can add things to the output not in the input. For casting from a smaller integer size to a larger one, it adds bits. Here, it adds provenance.

Ready for round two

I wrote code that triggers Undefined Behavior. We fix the code and move on, having learned a bit about pointer provenance language rules. If and when the transmute patch is reintroduced after the LLVM bugs are fixed or mitigated, our code will work.

Look! A trace of slow events in a benchmark! Hover over the image to see it get bigger.

Now read on to see what the slow events are and how we got this pretty picture.

The rules

The first rule of just-in-time compilers is: you stay in JIT code. The second rule of JIT is: you STAY in JIT code!

When control leaves the compiled code to run in the interpreter—what the ZJIT team calls either a “side-exit” or a “deopt”, depending on who you talk to—things slow down. In a well-tuned system, this should happen pretty rarely. Right now, because we’re still bringing up the compiler and runtime system, it happens more than we would like.

We’re reducing the number of exits over time.

Lies, damned lies, and statistics

We can track our side-exit reduction progress with --zjit-stats, which, on process exit, prints out a tidy summary of the counters for all of the bad stuff we track. It’s got side-exits. It’s got calls to C code. It’s got calls to slow-path runtime helpers. It’s got everything.

Here is a chopped-up sample of stats output for the Lobsters benchmark, which is a large Rails app:

$ WARMUP_ITRS=0 MIN_BENCH_ITRS=20 MIN_BENCH_TIME=0 ruby --zjit-stats benchmarks/lobsters/benchmark.rb
...
***ZJIT: Printing ZJIT statistics on exit***
...
Top-20 side exit reasons (100.0% of total 12,549,876):
                   guard_type_failure: 6,020,734 (48.0%)
                  guard_shape_failure: 5,556,147 (44.3%)
  block_param_proxy_not_iseq_or_ifunc:   445,358 ( 3.5%)
                   unhandled_hir_insn:   215,168 ( 1.7%)
                        compile_error:   181,474 ( 1.4%)
...
compiled_iseq_count:                               5,581
failed_iseq_count:                                     2
compile_time:                                    1,443ms
...
guard_type_count:                            133,425,094
guard_type_exit_ratio:                              4.5%
guard_shape_count:                            49,386,694
guard_shape_exit_ratio:                            11.3%
...
code_region_bytes:                            31,571,968
side_exit_size_ratio:                              33.1%
zjit_alloc_bytes:                             19,329,659
total_mem_bytes:                              50,901,627
...
ratio_in_zjit:                                     82.8%
$

(I’ve cut out significant chunks of the stats output and replaced them with ... because it’s overwhelming the first time you see it.)

The first thing you might note is that the thing I just described as terrible for performance is happening over twelve million times. The second thing you might notice is that despite this, we’re staying in JIT code seemingly a high percentage of the time. Or are we? Is 80% high? Is a 4.5% class guard miss ratio high? What about 11% for shapes? It’s hard to say.

The counters are great because they’re quick and they’re reasonably stable proxies for performance. There’s no substitute for painstaking measurements on a quiet machine but if the counter for Bad Slow Thing goes down (and others do not go up), we’re probably doing a good job.

But they’re not great for building intuition. For intuition, we want more tangible feeling numbers. We want to see things.

Building intuition

The third thing is that you might ask yourself “where are these exits coming from?” Unfortunately, counters cannot tell you that. For that, we want stack traces. This lets us know where in the guest (Ruby) code triggers an exit.

Ideally also we would want some notion of time: we would want to know not just where these events happen but also when. Are the exits happening early, at application boot? At warmup? Even during what should be steady state application time? Hard to say.

So we need more tools. Thankfully, Perfetto exists. Perfetto is a system for visualizing and analyzing traces and profiles that your application generates. It has both a web UI and a command-line UI.

We can emit traces for Perfetto and visualize them there.

A look at Perfetto

Take a look at this sample ZJIT Perfetto trace generated by running Ruby with --zjit-trace-exits¹. What do you see?

I see a couple arrows on the left. Arrows indicate “instant” point-in-time events. Then I see a mess of purple to the right of that until the end of the trace.

Hover over an arrow. Find out that each arrow is a side-exit. Scream silently.

But it’s a friendly arrow. It tells you what the side-exit reason is. If you click it, it even tells you the stack trace in the pop-up panel on the bottom. If we click a couple of them, maybe we can learn more.

We can also zoom by mousing over the track, holding Ctrl, and scrolling. That will get us look closer. But there are so many…

Fortunately, Perfetto also provides a SQL interface to the traces. We can write a query to aggregate all of the side exit events from the slice table and line them up with the topmost method from the backtrace arguments in the args table:

SELECT
  s.name AS reason,
  a.display_value AS method,
  COUNT(*) AS count
FROM slice s
JOIN args a ON a.arg_set_id = s.arg_set_id AND a.key = '0'
GROUP BY s.name, a.display_value
ORDER BY count DESC

This pulls up a query box at the bottom showing us that there are a couple big hotspots:

It even has a helpful option to export the results Markdown table so I can paste (an edited version) into this blog post:

Looks like we should figure out why we’re having shape misses so much and that will clear up a lot of exits. (Hint: it’s because once we make our first guess about what we think the object shape will be, we don’t re-assess… yet.)

This has been a taste of Perfetto. There’s probably a lot more to explore. Please join the ZJIT Zulip and let us know if you have any cool tracing or exploring tricks.

Now I’ll explain how you too can use Perfetto from your system. Adding support to ZJIT was pretty straightforward.

Implementation

The first thing is that you’ll need some way to get trace data out of your system. We write to a file with a well-known location (/tmp/perfetto-PID.fxt), but you could do any number of things. Perhaps you can stream events over a socket to another process, or to a server that aggregates them, or store them internally and expose a webserver that serves them over the internet, or… anything, really.

Once you have that, you need a couple lines of code to emit the data. Perfetto accepts a number of formats. For example, in his excellent blog post, Tristan Hume opens with such a simple snippet of code for logging Chromium Trace JSON-formatted events (lightly modified by me):

event_name = ...
timestamp = ...
duration = ...
f = open('trace.json','a')
f.write("[\n")

# ... emit some events here ...

# Log a single event
f.write('{"name": "%s", "ts": %d, "dur": %d, "cat": "hi", "ph": "X", "pid": 1, "tid": 1, "args": {}},\n' %
  (event_name, timestamp, duration))

# ... emit some events here ...

# ... at process exit, close the file ...
f.write("]") # this closing ] isn't actually required
f.close()

This snippet is great. It shows, end-to-end, writing a stream of one event. It is a complete (X) event, as opposed to either:

• two discrete timestamped begin (B) and end (E) events that book-end something, or
• an instant (i) event that has no duration, or
• a couple other event types in the Chromium Trace Event Format doc

It was enough to get me started. Since it’s JSON, and we have a lot of side exits, the trace quickly ballooned to 8GB large for a several second benchmark. Not great. Now, part of this is our fault—we should side exit less—and part of it is just the verbosity of JSON.

Thankfully, Perfetto ingests more compact binary formats, such as the Fuchsia trace format. In addition to being more compact, FXT even supports string interning. After modifying the tracer to emit FXT, we ended with closer to 100MB for the same benchmark.

We can reduce further by sampling—not writing every exit to the trace, but instead every K exits (for some (probably prime) K). This is why we provide the --zjit-trace-exits-sample-rate=K option.

Check out the trace writer implementation from the point this article was written.

Tracing more things

We could trace:

• When methods get compiled
• How big the generated code is
• How long each compile phase takes
• When (and where) invalidation events happen
• When (and where) allocations happen from JITed code
• Garbage collection events
• and more!

Conclusion

Visualizations are awesome. Get your data in the right format so you can ask the right questions easily. Thanks for Perfetto!

Also, looks like visualizations are now available in Perfetto canary. Time to go make some fun histograms…