Prism in 2024

In Ruby 3.3.0, a new standard library was added to CRuby called Prism. Prism is a parser for the Ruby language, exposed as both a C library (optionally usable by CRuby) and a Ruby library (usable as a Ruby gem). The Prism project represents many person-years worth of effort, and is the result of a collaboration between Shopify, CRuby core contributors, other Ruby implementation authors, and Ruby tooling developers.

Catching Assertionless Tests

In Shopify we have more than 300,000 tests in our core monolith. These tests cover most of our application code and provide us with a great level of confidence when making changes to our app. But do all tests still perform the duties they were intended to even if some of these tests were added more than 10 years ago? In this article we will explore how Shopify revised all tests to reveal and fix ones that were testing nothing while still being executed.

A Packwerk Retrospective

In September, 2020, our team at Shopify released a Ruby gem named Packwerk, a tool to enforce boundaries and modularize Rails applications. Since its release, Packwerk has taken on a life of its own, inspiring blog posts, conference talks, and even an entire gem ecosystem. Its popularity is an indication that Packwerk clearly filled a void in the Rails community.

Shopify at RubyConf 2023

Shopify continues investing in Ruby and Rails to help ensure that they are 100-year tools. Moving them forward and bringing the community with us are a large part of that work, and participating in community events like RubyConf is an important part of that.

Unveiling the big leap in Ruby 3.3's IRB

In this blog post, we will delve into the major enhancements introduced in Ruby 3.3’s IRB, as well as what’s currently planned for the next year.

Ruby 3.3's YJIT: Faster While Using Less Memory

This year, the YJIT team has been working hard to improve and optimize YJIT. We’re proud to say that the version of YJIT to be included with Ruby 3.3 is leaps and bounds ahead of Ruby 3.2’s. It provides better performance across the board, while also warming up faster and using less memory. The 3.3 release is also more robust, including a number of bug fixes along with an improved test suite. In this blog post, we share some early numbers.

YJIT Is the Most Memory-Efficient Ruby JIT

This year, the YJIT team and I have gotten a paper accepted at MPLR 2023 (Managed Programming Languages and Runtimes), which is now freely available through ACM open access. The paper, titled “Evaluating YJIT’s Performance in a Production Context: A Pragmatic Approach”, goes into details of the strategy taken to evaluate YJIT’s performance in a production context. One of our key findings, when comparing YJIT against other existing Ruby JITs such as JRuby and TruffleRuby, is that YJIT is the most memory-efficient Ruby JIT (by a long shot).

Performance impact of the memoization idiom on modern Ruby

Effects of Pitchfork reforking on Shopify's Monolith

Over the last year, I’ve been working on a new Ruby application server called Pitchfork. In most regards it’s extremely similar to the venerable Unicorn, from which it originates, but with one major extra feature: Reforking, which reduce memory usage of Ruby applications.

Enabling Bootsnap in TruffleRuby

One of the places where TruffleRuby is still noticeably slower than CRuby is in finding and loading a lot of files. Do Ruby apps really do this?

Ruby 3.3's YJIT Runs Shopify's Production Code 15% Faster

Ruby Outperforms C: Breaking the Catch-22

We Turned Lobste.rs into a Rails Benchmark for YJIT

We’re very proud of how well YJIT, the default JIT in CRuby, optimizes Rails apps. We’ve been using small Rails benchmarks for a couple of years. But as YJIT improves we need more data from real world apps to help us understand what will speed up Ruby the most. We turned the Lobsters codebase into a nice new benchmark. So how did we do that?

How We Used a SQLite Memory DB for Rails Benchmarking

Here at Shopify, we recently adapted the Lobste.rs source code into a Rails benchmark for YJIT. But we want very little database time in our benchmarks, since we’re trying to optimise Ruby. One way to do that is to use SQLite rather than a separate database like MySQL or PostgresSQL. In fact, faster yet is an in-memory SQLite database. It doesn’t even write to disk. But that’s an unusual setup, and wants some code to set it up. Want to see how we did it?

Two Garbage Collection Improvements Made Our Storefronts 8% Faster

Making Sorbet compatible with Ruby 3.2

On the Ruby Developer Experience team here at Shopify, our goal is to deliver a state-of-the-art development experience to Rubyists both at Shopify and in the broader community. This means keeping our tools up-to-date with the most recent versions of Ruby.

IO#reopen and its surprising side effect

A short debugging story that taught us about the attributes of IO operations.

Open Sourcing Shopify's Ruby Builds

We’ve open sourced shopify-ruby-definitions, which contains definitions to build the Ruby we use at Shopify in development, testing, and production. Our builds of Ruby are API compliant with the vanilla Rubies, but with additional bug fixes and performance improvements we’ve backported from the next major version.

TruffleRuby in Shopify CI

Shopify is deeply committed to the future of Ruby. One of our key investments is in TruffleRuby — a high-performance implementation of the Ruby language based on cutting edge compiler research. While there are now several JIT compilers available for CRuby, they’re somewhat constrained by CRuby’s architecture. The same approach that allows CRuby to start up quickly and run fast during warm-up also makes it challenging to perform Ruby-level optimizations such as function inlining. In contrast, TruffleRuby is authored predominately in Ruby, allowing its JIT compiler to optimize both application code and the core libraries provided by the language. One big trade-off of this approach is TruffleRuby has needed to implement all of CRuby’s extensive core library in Ruby. It has taken several years for TruffleRuby to reach the desired level of compatibility, but it’s now a robust implementation that can run many complex Ruby workloads, including Rails applications and CRuby native extensions.

Rewriting the Ruby parser

At Shopify, we have spent the last year writing a new Ruby parser, which we’ve called YARP (Yet Another Ruby Parser). As of the date of this post, YARP can parse a semantically equivalent syntax tree to Ruby 3.3 on every Ruby file in Shopify’s main codebase, GitHub’s main codebase, CRuby, and the 100 most popular gems downloaded from rubygems.org. We recently got approval to merge this work into CRuby, and are very excited to share our work with the community. This post will take you through the motivations behind this work, the way it was developed, and the path forward.

Automatically Find Memory Leaks in Native Gems

Monitoring YJIT in Production

YJIT is a JIT compiler for Ruby that Shopify has developed. In 2022, we enabled YJIT in all storefront requests and observed ~10% speedups. Do you use an app performance monitor like NewRelic or Scout, or check your daily performance graphs? Here’s how you can make sure YJIT is doing what you want when you monitor.

How to load code efficiently in Ruby

In Ruby, we usually don’t think about memory. In frameworks like Ruby on Rails, we don’t even have to write statements to load our code. While Ruby has tools that abstract these concepts away from us, it is useful to understand how our apps get loaded into memory.

Sorbet as your Ruby mentor

Ruby has a number of ‘gotchas’ which often catch beginners off-guard. Although Sorbet is aimed at large, complex applications, it can also help to catch simple, common mistakes. And even in those large, complex apps, bugs can often be from simple mistakes. Let’s take a look at some.

How We Scaled Maintenance Tasks to Shopify's Core Monolith

In 2020, we built the maintenance_tasks gem as a solution for perfoming data migrations in Rails applications. Adopting the gem in Shopify’s core Rails monolith was not so simple, however! We had to adapt the gem to fit Core’s sharded architecture and to handle data migrations across millions of rows. Let’s take a look at how we did it.

Adopting a New Database Adapter: Trilogy

Trilogy is an open source client library for MySQL database servers. At Shopify, we’ve started to adopt the Trilogy client in our Rails applications. Here’s a quick look at why we’re adopting it, and how we’re getting there.

Farewell to a Friend

Our dear friend and colleague Chris Seaton passed away last Saturday, December 3rd, 2022. Chris was an important part of our team at Shopify, and a notable figure in the Ruby community, and we are deeply saddened by our collective loss.