• Finding Memory Leaks in the Ruby Ecosystem

    Until recently, Ruby lacked a mechanism for detecting native-level memory leaks from within Ruby and native gems. This meant that it was hard to analyze Ruby applications that suffered from memory leaks, causing them to consume increasing amounts of memory until the system runs out and terminates the application. In this blog post, we'll be looking at the RUBY_FREE_AT_EXIT feature, which allows memory leak checkers to find memory leaks in Ruby and in native gems.
  • Fixing a footgun in ActiveRecord::Core#inspect

    The inspect method on Active Record models returns a string including the model’s class and a list of all its attributes and their values. In Rails 7.2, you can configure which attributes are included in the output of inspect. In this post, I’ll discuss the performance issue that led me to implement this feature. I’ll also talk about how this feature can be used to improve developer experience.

  • Autotuner: How to Speed Up Your Rails App

    Ruby's garbage collector is designed to be adaptable, scaling from short Ruby scripts to running apps that serve millions of requests per second. While it's designed to be adaptable, it may not work optimally for every use case. For this reason, Ruby's garbage collector supports many parameters that can be used to tune it. However, the use of these parameters requires knowledge into how the garbage collector works on the inside. Learn how the Autotuner gem can analyze your app's traffic and provide suggestions for tuning the garbage collector.
  • Prism in 2024

    In Ruby 3.3.0, a new standard library was added to CRuby called Prism. Prism is a parser for the Ruby language, exposed as both a C library (optionally usable by CRuby) and a Ruby library (usable as a Ruby gem). The Prism project represents many person-years worth of effort, and is the result of a collaboration between Shopify, CRuby core contributors, other Ruby implementation authors, and Ruby tooling developers.

  • Catching Assertionless Tests

    In Shopify we have more than 300,000 tests in our core monolith. These tests cover most of our application code and provide us with a great level of confidence when making changes to our app. But do all tests still perform the duties they were intended to even if some of these tests were added more than 10 years ago? In this article we will explore how Shopify revised all tests to reveal and fix ones that were testing nothing while still being executed.

  • A Packwerk Retrospective

    In September, 2020, our team at Shopify released a Ruby gem named Packwerk, a tool to enforce boundaries and modularize Rails applications. Since its release, Packwerk has taken on a life of its own, inspiring blog posts, conference talks, and even an entire gem ecosystem. Its popularity is an indication that Packwerk clearly filled a void in the Rails community.

  • Shopify at RubyConf 2023

    Shopify continues investing in Ruby and Rails to help ensure that they are 100-year tools. Moving them forward and bringing the community with us are a large part of that work, and participating in community events like RubyConf is an important part of that.

  • Unveiling the big leap in Ruby 3.3's IRB

    In this blog post, we will delve into the major enhancements introduced in Ruby 3.3’s IRB, as well as what’s currently planned for the next year.

  • Ruby 3.3's YJIT: Faster While Using Less Memory

    This year, the YJIT team has been working hard to improve and optimize YJIT. We’re proud to say that the version of YJIT to be included with Ruby 3.3 is leaps and bounds ahead of Ruby 3.2’s. It provides better performance across the board, while also warming up faster and using less memory. The 3.3 release is also more robust, including a number of bug fixes along with an improved test suite. In this blog post, we share some early numbers.

  • YJIT Is the Most Memory-Efficient Ruby JIT

    This year, the YJIT team and I have gotten a paper accepted at MPLR 2023 (Managed Programming Languages and Runtimes), which is now freely available through ACM open access. The paper, titled “Evaluating YJIT’s Performance in a Production Context: A Pragmatic Approach”, goes into details of the strategy taken to evaluate YJIT’s performance in a production context. One of our key findings, when comparing YJIT against other existing Ruby JITs such as JRuby and TruffleRuby, is that YJIT is the most memory-efficient Ruby JIT (by a long shot).

  • Performance impact of the memoization idiom on modern Ruby

    Ruby 3.2 saw the introduction of object shapes which speedup instance variables access in most cases, but can be slower in some pathological cases.
  • Effects of Pitchfork reforking on Shopify's Monolith

    Over the last year, I’ve been working on a new Ruby application server called Pitchfork. In most regards it’s extremely similar to the venerable Unicorn, from which it originates, but with one major extra feature: Reforking, which reduce memory usage of Ruby applications.

  • Enabling Bootsnap in TruffleRuby

    One of the places where TruffleRuby is still noticeably slower than CRuby is in finding and loading a lot of files. Do Ruby apps really do this?

  • Ruby 3.3's YJIT Runs Shopify's Production Code 15% Faster

    Ruby 3.2 YJIT has optimized the production workloads of Shopify and other companies. We encourage you to enable YJIT in production. Once Ruby 3.3 is released, it should make your application even faster.
  • Ruby Outperforms C: Breaking the Catch-22

    YJIT's ability to improve performance by adapting to run-time behavior can increase the speed of our code in a way that dropping down to C can't. As such, I think we should reconsider the common wisdom that "rewriting our Ruby in C" is the ideal path to performance optimization and take a serious look at "rewriting our C in Ruby" instead.
  • We Turned Lobste.rs into a Rails Benchmark for YJIT

    We’re very proud of how well YJIT, the default JIT in CRuby, optimizes Rails apps. We’ve been using small Rails benchmarks for a couple of years. But as YJIT improves we need more data from real world apps to help us understand what will speed up Ruby the most. We turned the Lobsters codebase into a nice new benchmark. So how did we do that?

  • How We Used a SQLite Memory DB for Rails Benchmarking

    Here at Shopify, we recently adapted the Lobste.rs source code into a Rails benchmark for YJIT. But we want very little database time in our benchmarks, since we’re trying to optimise Ruby. One way to do that is to use SQLite rather than a separate database like MySQL or PostgresSQL. In fact, faster yet is an in-memory SQLite database. It doesn’t even write to disk. But that’s an unusual setup, and wants some code to set it up. Want to see how we did it?

  • Two Garbage Collection Improvements Made Our Storefronts 8% Faster

    At Shopify, we are constantly improving the performance of Ruby for everyone. Based on analysis of and experiments on one of our largest and highest traffic apps, we found two improvements in Ruby's garbage collector that significantly improved our garbage collector performance, which in turn reduced our response times by 8%.
  • Making Sorbet compatible with Ruby 3.2

    On the Ruby Developer Experience team here at Shopify, our goal is to deliver a state-of-the-art development experience to Rubyists both at Shopify and in the broader community. This means keeping our tools up-to-date with the most recent versions of Ruby.

  • IO#reopen and its surprising side effect

    A short debugging story that taught us about the attributes of IO operations.

  • Open Sourcing Shopify's Ruby Builds

    We’ve open sourced shopify-ruby-definitions, which contains definitions to build the Ruby we use at Shopify in development, testing, and production. Our builds of Ruby are API compliant with the vanilla Rubies, but with additional bug fixes and performance improvements we’ve backported from the next major version.

  • TruffleRuby in Shopify CI

    Shopify is deeply committed to the future of Ruby. One of our key investments is in TruffleRuby — a high-performance implementation of the Ruby language based on cutting edge compiler research. While there are now several JIT compilers available for CRuby, they’re somewhat constrained by CRuby’s architecture. The same approach that allows CRuby to start up quickly and run fast during warm-up also makes it challenging to perform Ruby-level optimizations such as function inlining. In contrast, TruffleRuby is authored predominately in Ruby, allowing its JIT compiler to optimize both application code and the core libraries provided by the language. One big trade-off of this approach is TruffleRuby has needed to implement all of CRuby’s extensive core library in Ruby. It has taken several years for TruffleRuby to reach the desired level of compatibility, but it’s now a robust implementation that can run many complex Ruby workloads, including Rails applications and CRuby native extensions.

  • Rewriting the Ruby parser

    At Shopify, we have spent the last year writing a new Ruby parser, which we’ve called YARP (Yet Another Ruby Parser). As of the date of this post, YARP can parse a semantically equivalent syntax tree to Ruby 3.3 on every Ruby file in Shopify’s main codebase, GitHub’s main codebase, CRuby, and the 100 most popular gems downloaded from rubygems.org. We recently got approval to merge this work into CRuby, and are very excited to share our work with the community. This post will take you through the motivations behind this work, the way it was developed, and the path forward.

  • Automatically Find Memory Leaks in Native Gems

    It's not uncommon for Rails developers to encounter memory leaks, which causes the system to run out of memory and kill the app. Often, this is caused by native gems that forgot to clean up memory. Let's fix this problem using ruby_memcheck, a gem that automatically finds memory leaks in native gems.
  • Monitoring YJIT in Production

    YJIT is a JIT compiler for Ruby that Shopify has developed. In 2022, we enabled YJIT in all storefront requests and observed ~10% speedups. Do you use an app performance monitor like NewRelic or Scout, or check your daily performance graphs? Here’s how you can make sure YJIT is doing what you want when you monitor.

  • How to load code efficiently in Ruby

    In Ruby, we usually don’t think about memory. In frameworks like Ruby on Rails, we don’t even have to write statements to load our code. While Ruby has tools that abstract these concepts away from us, it is useful to understand how our apps get loaded into memory.

  • Sorbet as your Ruby mentor

    Ruby has a number of ‘gotchas’ which often catch beginners off-guard. Although Sorbet is aimed at large, complex applications, it can also help to catch simple, common mistakes. And even in those large, complex apps, bugs can often be from simple mistakes. Let’s take a look at some.

  • How We Scaled Maintenance Tasks to Shopify's Core Monolith

    In 2020, we built the maintenance_tasks gem as a solution for perfoming data migrations in Rails applications. Adopting the gem in Shopify’s core Rails monolith was not so simple, however! We had to adapt the gem to fit Core’s sharded architecture and to handle data migrations across millions of rows. Let’s take a look at how we did it.

  • Adopting a New Database Adapter: Trilogy

    Trilogy is an open source client library for MySQL database servers. At Shopify, we’ve started to adopt the Trilogy client in our Rails applications. Here’s a quick look at why we’re adopting it, and how we’re getting there.

  • Farewell to a Friend

    Our dear friend and colleague Chris Seaton passed away last Saturday, December 3rd, 2022. Chris was an important part of our team at Shopify, and a notable figure in the Ruby community, and we are deeply saddened by our collective loss.