CONFERENCE NEWS

The Ruby LSP team will be in attendance at RubyConf 2024 in Chicago (November). Reach out if you want to chat about anything related to Ruby LSP, or the wider Ruby developer experience.

Overview

In this post we introduce the Ruby LSP add-on system. We’ll explain the problem it solves, discuss its architecture, showcase some example add-ons, and share our vision for a future add-on ecosystem that enhances the Ruby development experience.

Introduction

Ruby LSP is a language server implementation designed to streamline writing Ruby code. It uses static analysis to parse your code to provide editor features. However, in the Ruby ecosystem, dynamic programming and DSLs are commonly used to extend Ruby, and this can be a challenge for editor tooling. Rails, in particular, leverages these techniques extensively.

Since any gem can define its own DSL written with meta-programming, it wouldn’t be feasible to add specialized handling for each one in the Ruby LSP static analysis toolkit.

Additionally, each project uses a different set of dependencies. Trying to handle all possible DSLs at the same time could degrade both performance and correctness.

We needed an extensible way to allow other gems to teach the language server about how to properly handle the DSLs that they define, so that we could deliver specialized handling at scale. We also wanted to explore if we could add runtime components to the language server’s knowledge. That way, we could overcome some of the limitations of static analysis by communicating with the application being developed.

The language server protocol (LSP) allows editors to communicate with multiple servers simultaneously, enabling authors to implement their own LSPs. While implementing the basics of a language server isn’t overly complex, ensuring proper encoding handling, extensive specification support and adequate performance is challenging. This approach has several downsides:

  • Repeated effort: Implementing the same functionality, such as handling different Ruby version managers and platform differences.
  • Re-implementing static analysis: Every new language server has to implement its own static analysis tooling.
  • Higher resource usage: Duplicate indexing of the codebase and maintaining duplicate document representations.
  • Less responsive editing: Each LSP server has to process the Ruby code, rather than sharing a parsed representation.
  • More configuration: Using multiple language servers may require users to configure each one separately.
  • Requires a VS Code extensions: Unlike other editors, each LSP server needs a corresponding extension if VS Code is to be supported.

Architecture

To address these issues, we offer an add-on system for developers to enhance Ruby LSP’s behavior. Add-ons can add specialized functionality to a number of language server features, but they can also improve the Ruby LSP’s knowledge of the codebase declarations by contributing indexing enhancements.

Add-ons can be distributed in several ways:

  • As a standalone add-on gem
  • Alongside an existing library
  • As part of an existing app or library, for project-specific behavior

Add-ons are written in Ruby and implemented as classes inheriting from RubyLsp::Addon, and are automatically discovered and loaded by the Ruby LSP server provided they follow the naming convention my_gem/lib/ruby_lsp/my_gem/addon.rb. As with Ruby LSP itself, add-ons are not tied to any particular editor.

The Ruby LSP relies extensively on Prism, a fast, modern parser for Ruby. This is the basis of how the LSP interacts with your code, so familiarity with it is important when building an add-on.

In the example below, global_state is an object supplied by Ruby LSP which provides shared concepts about the current codebase such as the index, configuration and file encoding. The message_queue provides a way for us to send messages to the editor, such as registering for file watching or logging a trace event.

Example: Hover

In this example, we show some information when a developer hovers over a class definition:

# lib/ruby_lsp/my_gem/addon.rb
require "ruby_lsp/addon"
require_relative "my_listener"

module RubyLsp
  module MyAddonGem
    class Addon < ::RubyLsp::Addon
      def activate(global_state, message_queue)
        # Optional: put any activation code that needs to happen
        # once when the language server is booted here
      end

      def deactivate
        # Optional: put any cleanup code that needs to happen
        # when shutting down the server, like terminating a subprocess
      end

      def name
        "My Gem"
      end
    end
  end
end

Within the server, we use an observer pattern, where a listener registers with Prism’s dispatcher to receive events corresponding to particular node events, such as on_class_node_enter. This approach optimizes performance by traversing the AST only once, rather than each add-on doing so separately.

The example below outputs the name of the class when you hover over it:

# lib/ruby_lsp/my_gem/my_listener.rb
class MyListener
  def initialize(dispatcher)
    # Register to listen to `on_class_node_enter` events
    dispatcher.register(self, :on_class_node_enter)
  end

  # Define the handler method for the `on_class_node_enter` event
  def on_class_node_enter(node)
    $stderr.puts "Hello, #{node.constant_path.slice}!"
  end
end

To see this in action, we can pass a snippet of Ruby code:

dispatcher = Prism::Dispatcher.new
MyListener.new(dispatcher)

parse_result = Prism.parse("class Foo; end")
dispatcher.dispatch(parse_result.value)

which will output Hello, Foo!.

To implement an actual LSP feature, we append to @response_builder. This is an abstraction used to compose responses coming from different sources into what is expected by the editor. For example, the code below would add Hover information to each class:

  def on_class_node_enter(node)
    range = range_from_node(node)
    text = "This class is named **#{node.constant_path.slice}**"
    contents = Interface::MarkupContent.new(kind: "markdown", value: text)

    @response_builder << Interface::Hover.new(contents: contents)
  end

(The Interface:: classes come from the language-server-protocol gem that Ruby LSP depends on).

Capabilities

Add-ons are provided with a rich API to deliver their editor integrations:

  • Testing: Utilities such as helpers.
  • Logging: Visibility into what Ruby LSP is doing.
  • Registering formatters and linters: Support features such as format-on-save or quick-fix.
  • Sending notifications to the client: Display information or errors.
  • Listening for file updates: Reloading when a configuration file changes.
  • Static analysis (indexing, type inference, etc.): Provide an accurate representation of the whole codebase’s structure.

As of the date of this post, add-ons are able to enhance the LSP requests for CodeLens, Completion, Definition, DocumentSymbol and Hover.

Indexing Enhancements

There are two ways to enhance Ruby LSP features. One is handling DSLs that occur at a call site and that do not change which declarations exist in the project. An example of this is the Rails validate method, which accepts a symbol that represents a method that is dynamically invoked.

The second is to handle declaration DSLs. These are DSLs that create declarations via metaprogramming. To use another Rails example, belongs_to is a DSL that mutates the current class and adds extra methods based on the arguments passed to it.

These methods would normally be ‘invisible’ to Ruby LSP and so we offer an API for enhancing the index. By ensuring that the index is populated with all declarations, features like go to definition, hover, completion, signature help and workspace symbol will all automatically work.

Limitations

The add-on system is an approach for extending language servers, which is not common to find in other language ecosystems. We are working to understand what is needed from the API with help from the community, so that we can deliver a rich and extensible architecture for the Ruby LSP server.

Note that only the server portion of the language server can currently be enhanced by add-ons. It is not possible to add editor UI elements through this API and it’s unlikely that we would be able to do so as each editor exposes different APIs for managing UI elements.

Case Study: Ruby LSP Rails

Developed by Shopify, Ruby LSP Rails was the first add-on and helped us design the API. It enhances several requests, such as:

  • Hover for showing database schema information.
  • Go To Definition for associations and callbacks.
  • Code Lens for running ‘declarative’ tests inheriting from ActiveSupport::TestCase.

The Rails add-on directly communicates with your Rails app in development, meaning we can offer features that would be very difficult or impossible to build with static analysis, such as navigation from a controller action to the corresponding route:

Demonstration of Ruby LSP Rails

Case Study: Standard

Logo for Standard

Standard is a linter and formatter built on top of RuboCop. Initially, it had its own LSP server and VS Code extension.

We collaborated with Standard’s author, Justin Searls, to make a few necessary changes to Ruby LSP to allow Standard to be used as an add-on.

Check out his post Why I just uninstalled my own VS Code extension for more insights.

We’re grateful to Justin for believing in our vision and collaborating to get the Standard add-on shipped!

Other Add-ons

There are several other third-party add-ons already available, such as:

We encourage you to search for ruby-lsp- on rubygems.org to discover more add-ons.

Future Vision

We envision Ruby LSP becoming not only a fully featured language server for Ruby, but also a platform for delivering editor integrations with a thriving ecosystem of add-ons. We hope to see more gems ship with an add-on included, supporting not only linters and formatters but also more specialized tooling. Ideally, add-ons should be auto-discovered and require zero configuration by the user. Our goal is to stabilize the API at v1.0.

Next Steps

To learn more about building an add-on, check out the Add-ons page in the Ruby LSP documentation.

You can also join the conversation in the #ruby-lsp-addons channel in the Ruby DX Slack workspace. We look forward to seeing how you enhance your Ruby development experience with Ruby LSP add-ons!