One major internal change in Ruby 3.2 was the introduction of object shapes.
In this post, we’ll try to cover why they were introduced, how they work, and what their limitations are.
Because of how dynamic Ruby is, an operation as simple as accessing an instance variable is a lot of work.
In most cases, Ruby objects store their instance variables in an array of references.
For instance, let’s create a simple object with 2 instance variables:
class User
def initialize(name, admin)
@name = name
@admin = admin
end
def admin?
@admin
end
end
User.new("John", true)
In MRI, this object will occupy a 40 bytes object slot that is laid out like this:
+--------+----------+---------+---------+---------+
| flags | class | ivar0 | ivar1 | ivar2 |
+--------+----------+---------+---------+---------+
| 0x01 | User | "John" | true | UNDEF |
+--------+----------+---------+---------+---------+
Each cell is 8 bytes large, The first cell is used for the object’s flags
, it’s used for many things, but it’s all outside of the scope of this post.
The second cell stores a pointer to the object’s class, and then the last 3 cells are used to store the object’s instance variable.
In this case, since the object only has 2 instance variables, the third slot is not used.
Now when Ruby executes code that accesses an instance variable, it needs to be able to translate the variable name into an index.
In Ruby 3.1 and older, each class would keep a Hash that maps instance variable names to indexes, in pseudo-Ruby it worked like this:
class Class
def initialize
@instance_variables_index = {}
end
def ivar_index(ivar_name)
@instance_variables_index[ivar_name] ||= @instance_variables_index.size
end
end
class Object
def initialize
@ivars = []
end
def instance_variable_get(ivar_name)
index = self.class.ivar_index(ivar_name)
value = @ivars[index]
return value == UNDEF ? nil : value
end
def instance_variable_set(ivar_name, value)
index = self.class.ivar_index(ivar_name)
@ivars[index] = value
end
end
user.instance_variable_get(:@admin) # => true
As you can imagine, having to do a Hash lookup for every instance variable access is pretty slow.
To avoid doing this every time, the Ruby Virtual Machine uses what’s called inline caches. They are small memory regions inside the VM bytecode, that can store the result of this kind of computations to hopefully avoid doing it every time.
In pseudo-Ruby, it looks like this:
def ruby_vm_get_instance_variable(object, ivar_name, cache)
if cache.last_class != object.class
cache.last_class = object.class
cache.index = object.class.ivar_index(ivar_name)
end
value = object.ivars[cache.index]
return value == UNDEF ? nil : value
end
With such inline caches, if a method only ever deals with a single class, it only has to look up the instance variable index once. On any future execution, all it has to do is compare the cached class with the current object class, and if it matches it can directly re-use the cached index. This is much much faster than having to lookup the index.
The problem with such an approach though, is that it doesn’t perform well in the presence of simple polymorphism:
require "benchmark/ips"
class Parent
def name
@name
end
def initialize(name)
@name = name
end
end
class Subclass < Parent
end
stable_1 = Parent.new("Odile Deray")
stable_2 = Parent.new("Simon Jérémi")
unstable_1 = Parent.new("Peter")
unstable_2 = Subclass.new("Steven")
puts RUBY_DESCRIPTION
Benchmark.ips do |x|
x.report("stable class") do
stable_1.name
stable_2.name
end
x.report("unstable class") do
unstable_1.name
unstable_2.name
end
x.compare!(order: :baseline)
end
In the above benchmark, we compare the performance of a simple instance variable access on two instances of the same class against the same simple access but with two distinct classes.
ruby 3.1.4p223 (2023-03-30 revision 957bb7cb81) [arm64-darwin22]
stable class 14.222M (± 1.3%) i/s - 71.960M in 5.060612s
unstable class 11.625M (± 3.2%) i/s - 58.238M in 5.015375s
Comparison:
stable class: 14221847.8 i/s
unstable class: 11625350.9 i/s - 1.22x slower
As you can see, even though Subclass
is in every shape and form identical to Parent
because the class is used as a cache key
when it differs we have to go through the slow lookup again.
To solve this problem (and a few others), Ruby 3.2
implemented object shapes, which is a technique pioneered by Smalltalk in the 80’s.
To keep this article relatively brief, I will only offer a basic explanation of what object shapes are and how they work. If you’d like to dig deeper into the topic, I heavily recommend either reading Chris Seaton’s post about the general concept, or watching one of Jemma Issroff’s talks on how she implemented shapes in CRuby.
So without going into too much detail, shapes are a tree-like structure where each node represents an instance variable, for example with the following code:
class Person
def initialize(name)
@name = name
end
end
class Employee < Person
end
class Customer < Person
def initialize(name)
super
@balance = 0
end
end
person = Person.new("Peter")
employee = Employee.new("Steven")
customer = Customer.new(George)
Here the shape tree will look like this:
+----------+----------+-------+-----------+
| shape_id | ivar | index | parent_id |
+----------+----------+-------+-----------+
+ 12 | @name | 0 | 0 |
+----------+----------+-------+-----------+
+ 42 | @balance | 1 | 12 |
+----------+----------+-------+-----------+
And in the above example, the user
and employee
instance will have the same shape (12
), even though they are of different class.
Any two objects that have the same list of instance variables defined in the same order will share a common shape.
Thanks to this, the instance variable access code was updated to use the shape as a cache key:
def ruby_vm_get_instance_variable(object, ivar_name, cache)
if cache.last_shape_id != object.shape_id
cache.last_shape_id = object.shape_id
cache.index = RubyVM.ivar_index(shape_id, ivar_name)
end
if cache.index == IVAR_NOT_SET
return nil
else
return object.ivars[cache.index]
end
end
Running the same benchmark on Ruby 3.2
confirms that now the two cases have identical performance:
ruby 3.2.2 (2023-03-30 revision e51014f9c0) [arm64-darwin22]
stable class 13.994M (± 1.2%) i/s - 70.504M in 5.038920s
unstable class 14.122M (± 1.5%) i/s - 71.367M in 5.054938s
Comparison:
stable class: 13993825.7 i/s
unstable class: 14121540.7 i/s - same-ish: difference falls within error
Thanks to shapes, the inline cache is no longer flip-flopping in the presence of simple polymorphism.
I won’t go into too much detail but note that this isn’t the only advantage of shapes.
However, shapes aren’t faster in all cases.
A very common idiom you have certainly encountered if you ever worked with Ruby, is the memoization pattern:
def dynamic_property
@dynamic_property ||= something_slow_to_compute
end
Unfortunately, this idiom doesn’t play well with shapes, because now instances of the same class will frequently have different shapes, causing the inline caches hit rate to degrade.
A class with all its instance variables defined in initialize
will only ever have one shape.
But when extra instance variables can be defined in semi-random order, the number of shapes can quickly explode. A class with 1 memoized variable will have 2 shapes, a class with 2 memoized variables will have 4 shapes, a class with 5 memoized variables will have up to 121 shapes, and so on.
Yes, this is a factorial growth. The number of shapes a class may have is 1 + !lazily_defined_variables
.
But how bad is it? Let’s measure:
require "benchmark/ips"
class UnstableShape
attr_reader :name
def initialize(name)
@name = name
end
def first_name
@first_name ||= @name.split.first
end
def upcased_name
@upcased_name ||= @name.upcase
end
end
# Generate two objects with the same shape
stable_1 = UnstableShape.new("Odile Deray")
stable_2 = UnstableShape.new("Simon Jérémi")
stable_1.first_name
stable_1.upcased_name
stable_2.first_name
stable_2.upcased_name
# Generate two objects with distinct shapes
unstable_1 = UnstableShape.new("Peter")
unstable_2 = UnstableShape.new("Steven")
unstable_1.first_name
unstable_1.upcased_name
unstable_2.upcased_name
unstable_2.first_name
puts RUBY_DESCRIPTION
Benchmark.ips do |x|
x.report("stable shape") do
stable_1.name
stable_2.name
end
x.report("unstable shape") do
unstable_1.name
unstable_2.name
end
x.compare!(order: :baseline)
end
ruby 3.2.2 (2023-03-30 revision e51014f9c0) [arm64-darwin22]
stable shape 15.341M (± 0.9%) i/s - 76.755M in 5.003687s
unstable shape 13.259M (± 1.1%) i/s - 66.385M in 5.007561s
Comparison:
stable shape: 15340799.2 i/s
unstable shape: 13258541.5 i/s - 1.16x slower
16% slower isn’t too bad, but keep in mind that the overhead directly depends on how deep the shape is, if we perform the same benchmark, but add an arbitrarily large number of variables to the shape, we can make the benchmark arbitrarily slower:
class UnstableShape
def initialize(name)
@name = name
@ivar01 = true
@ivar02 = true
@ivar03 = true
@ivar04 = true
@ivar05 = true
@ivar06 = true
@ivar07 = true
@ivar08 = true
@ivar09 = true
@ivar10 = true
@ivar11 = true
@ivar12 = true
@ivar13 = true
@ivar14 = true
@ivar15 = true
@ivar16 = true
@ivar17 = true
@ivar18 = true
@ivar19 = true
@ivar20 = true
@ivar21 = true
@ivar22 = true
@ivar23 = true
@ivar24 = true
@ivar25 = true
end
end
ruby 3.2.2 (2023-03-30 revision e51014f9c0) [arm64-darwin22]
stable shape 14.253M (± 0.9%) i/s - 71.296M in 5.002615s
unstable shape 5.821M (± 1.8%) i/s - 29.116M in 5.003786s
Comparison:
stable shape: 14253062.2 i/s
unstable shape: 5820686.1 i/s - 2.45x slower
The reason is that on a cache miss, Ruby has to walk the object’s shape tree until it finds the node it needs, so it’s an O(n)
operation where n
is the object’s shape depth
which is more or less its number of instance variables.
If you are curious, the function responsible for that is rb_shape_get_iv_index
,
in pseudo-Ruby, it looks like:
def RubyVM.ivar_index(shape_id, ivar)
shape = SHAPES[shape_id]
while shape
if shape.ivar == ivar
return shape.index
else
shape = SHAPES[shape.parent_id]
end
end
IVAR_NOT_SET
end
You may think that 25
instance variables is a bit over the top, but I chose that number because that is how many instance variables an ActiveRecord::Base
instance has.
So this shows that using the memoization pattern in a single Rails model can significantly degrade the performance of most Active Record operations.
This unstable shape problem is in large part because the virtual machine’s inline caches are monomorphic, meaning they can only record a single shape.
However one of YJIT’s optimizations is that it implements polymorphic inline caches. In Ruby 3.2, YJIT inline caches can record up to 20 different shapes. But that’s a memory usage versus speed tradeoff, so based on production data, this was reduced to 5 in Ruby 3.3.
Still, if we enable YJIT, we can see that we again have identical performance for this simple memoization benchmark:
ruby 3.2.2 (2023-03-30 revision e51014f9c0) +YJIT [arm64-darwin22]
stable shape 40.524M (± 1.4%) i/s - 203.140M in 5.013915s
unstable shape 40.557M (± 1.0%) i/s - 203.056M in 5.007196s
Comparison:
stable shape: 40523693.8 i/s
unstable shape: 40557046.2 i/s - same-ish: difference falls within error
But as explained before, the number of shapes can quickly explode if memoization is used extensively.
SHAPE_TOO_COMPLEX
This problem was spotted quite early during object shapes implementation, so a workaround was implemented
to deal with pathological cases. When the virtual machine detects that a class is generating a lot of different shapes,
it automatically marks that class as “too complex”, and all future instances will have a special unique shape called SHAPE_TOO_COMPLEX
.
Objects with this special shape no longer store their instance variables in an internal array, but instead use a hash map. This uses more memory and makes for slower but more consistent access performance.
This can be demonstrated by modifying our benchmark a bit more:
class StableShape
attr_reader :name
def initialize(name)
@name = name
end
end
class TooComplexShape
attr_reader :name
def initialize(name)
@name = name
end
end
# Cause the class to be marked as too complex
10.times do |i|
TooComplexShape.allocate.instance_variable_set("@iv_#{i}", i)
end
stable = StableShape.new("Test")
too_complex = TooComplexShape.new("George Abitbol")
puts ObjectSpace.dump(too_complex)
puts RUBY_DESCRIPTION
Benchmark.ips do |x|
x.report("stable shape") do
stable.name
end
x.report("too-complex shape") do
too_complex.name
end
x.compare!(order: :baseline)
end
{"address":"0x104d991d0", "type":"OBJECT", ..., "too_complex_shape":true, ...}
ruby 3.2.2 (2023-03-30 revision e51014f9c0) +YJIT [arm64-darwin22]
stable shape 45.207M (± 1.0%) i/s - 226.553M in 5.011953s
too-complex shape 29.952M (± 1.1%) i/s - 150.621M in 5.029290s
Comparison:
stable shape: 45207185.7 i/s
too-complex shape: 29952498.7 i/s - 1.51x slower
So SHAPE_TOO_COMPLEX
is slower than the happy path, but faster than the worst-case scenario where we have to constantly lookup
the variable index in the shape tree.
While working on this blog post, it occurred to me that we could make inline caches handle the memoization pattern better, without changes to any Ruby code.
On a cache miss, we start searching for the variable index from scratch. But if the inline cache is flip-flopping because of memoization, the shape that was stored in the cache and the current shape will have a common ancestor, and checking for it would allow us to stop walking the shape tree much sooner.
In the UnstableShape
example, we had an object with 26
fixed instance variables, and 2
memoized instance variables.
The variable we’re looking for is the very first one, so we have to walk over at least 26
shapes which is slow.
But if we provide the stale shape_id
, we’d only need to check at most 3
shapes, which is much faster.
C code can be a bit cryptic, so here’s the pseudo-Ruby version of it:
def RubyVM.ivar_index(shape_id, ivar, cached_shape_id, cached_index)
shape = SHAPES[shape_id]
while shape
if cached_shape_id == shape.parent_id
# The new shape is a descendant of the one we had in the cache
# so we know the cached index is still valid.
return cached_index
elsif shape.ivar == ivar
return shape.index
else
shape = SHAPES[shape.parent_id]
end
end
IVAR_NOT_SET
end
It’s still an O(n)
operation, but now in most cases n
is the number of lazily defined variables rather than the total number of variables, so n
is significantly reduced.
When I shared this idea with some fellow Ruby committers, they pointed out that John Hawthorn had pretty much the same idea 6 months prior, but even better implemented.
As I’m writing this, this patch isn’t yet merged, but I’m hoping some version of it will be for Ruby 3.3.0 in December.
The above patch would pretty much solve the performance concern of moderate users of the memoization pattern.
However, one pattern it doesn’t help with at all is code that is shared between radically different shapes:
module Synchronized
def initialize
@mutex = Mutex.new
end
def synchronized(...)
@mutex.synchronize(...)
end
end
class Car
include Synchronized
def initialize(brand, ...)
super
@brand = brand
...
end
end
class Animal
include Synchronized
def initialize(breed, ...)
super
@breed = breed
...
end
end
In the above example, the @mutex
access in the synchronized
method will witness radically different shapes, and won’t even see a common ancestor, as such the access will remain O(n)
.
This issue was foreseen before the release of Ruby 3.2, but no solution could be implemented in time.
A possible solution is to keep an index of the parent shapes in the child shapes so that on a miss we can more quickly lookup the attribute index. The downside of course is that such indexes would use more memory.
Aaron Patterson worked on implementing this idea, and chose to use a Red-black tree index because it allows to easily share the parent shape’s index, keeping the memory overhead down. To further minimize the extra memory usage his implementation only generates the index every 10 shapes, so on a miss Ruby would have to at most walk 10 shapes and then do an index lookup.
This patch isn’t yet merged either, but will hopefully make it into Ruby 3.3 too.
As demonstrated in the above benchmarks, over-use of the memoization pattern can cause a degradation of performance in your Ruby programs, so it can be effective to eagerly initialize instance variables in the constructor to avoid this, at least in hot spots.
But if the pattern is used moderately, it’s probably not a big deal and will be handled better by either YJIT or future versions of Ruby.
The one thing that is best avoided though, is SHAPE_TOO_COMPLEX
as it is not only slower but consumes more memory.
To help with that, starting in Ruby 3.3 a warning will be emitted when a class is marked as too complex,
and it can be enabled by setting Warning[:performance] = true
or passing -W:performance
to the command line:
$ ruby -W:performance /tmp/too-complex.rb
/tmp/too-complex.rb:19: warning: Maximum shapes variations (8) reached by TooComplexShape, instance variables accesses will be slower.
But overall, for moderate use of the pattern, outside of hotspots, it’s not a major performance concern.
My rule of thumb is that one or two memoized variables in a class are fine, but more than that likely deserve a quick refactor to eagerly initialize the variables to nil
in the constructor:
class StableShape
def initialize
@expensive_value = nil
end
def expensive_value
@expensive_value ||= compute_expensive_value
end
end
And in the case where the memoized value may be falsy, you can use a token module:
class StableShape
NOT_SET = Module.new
private_constant :NOT_SET
def initialize
@expensive_value = NOT_SET
end
def expensive_value
return @expensive_value unless @expensive_value == NOT_SET
@expensive_value = compute_expensive_value
end
end