123456789_123456789_123456789_123456789_123456789_

YJIT - Yet Another Ruby JIT

YJIT is a lightweight, minimalistic Ruby JIT built inside CRuby. It lazily compiles code using a Basic Block Versioning (BBV) architecture. YJIT is currently supported for macOS, Linux and BSD on x86-64 and arm64/aarch64 CPUs. This project is open source and falls under the same license as CRuby.

If you're using YJIT in production, please share your success stories with us!

If you wish to learn more about the approach taken, here are some conference talks and publications:

To cite YJIT in your publications, please cite the MPLR 2023 paper:

@inproceedings{yjit_mplr_2023,
author = {Chevalier-Boisvert, Maxime and Kokubun, Takashi and Gibbs, Noah and Wu, Si Xing (Alan) and Patterson, Aaron and Issroff, Jemma},
title = {Evaluating YJIT’s Performance in a Production Context: A Pragmatic Approach},
year = {2023},
isbn = {9798400703805},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3617651.3622982},
doi = {10.1145/3617651.3622982},
booktitle = {Proceedings of the 20th ACM SIGPLAN International Conference on Managed Programming Languages and Runtimes},
pages = {20–33},
numpages = {14},
keywords = {dynamically typed, optimization, just-in-time, virtual machine, ruby, compiler, bytecode},
location = {Cascais, Portugal},
series = {MPLR 2023}
}

Current Limitations

YJIT may not be suitable for certain applications. It currently only supports macOS, Linux and BSD on x86-64 and arm64/aarch64 CPUs. YJIT will use more memory than the Ruby interpreter because the JIT compiler needs to generate machine code in memory and maintain additional state information. You can change how much executable memory is allocated using YJIT's command-line options.

Installation

Requirements

You will need to install:

If you don't intend on making code changes to YJIT itself, we recommend obtaining rustc through your OS's package manager since that likely reuses the same vendor which provides the C toolchain.

If you will be changing YJIT's Rust code, we suggest using the first-party installation method for Rust. Rust also provides first class support for many source code editors.

Building YJIT

Start by cloning the ruby/ruby repository:

git clone https://github.com/ruby/ruby yjit
cd yjit

The YJIT ruby binary can be built with either GCC or Clang. It can be built either in dev (debug) mode or in release mode. For maximum performance, compile YJIT in release mode with GCC. More detailed build instructions are provided in the Ruby README.

# Configure in release mode for maximum performance, build and install
./autogen.sh
./configure --enable-yjit --prefix=$HOME/.rubies/ruby-yjit --disable-install-doc
make -j && make install

or

# Configure in lower-performance dev (debug) mode for development, build and install
./autogen.sh
./configure --enable-yjit=dev --prefix=$HOME/.rubies/ruby-yjit --disable-install-doc
make -j && make install

Dev mode includes extended YJIT statistics, but can be slow. For only statistics you can configure in stats mode:

# Configure in extended-stats mode without slow runtime checks, build and install
./autogen.sh
./configure --enable-yjit=stats --prefix=$HOME/.rubies/ruby-yjit --disable-install-doc
make -j && make install

On macOS, you may need to specify where to find some libraries:

# Install dependencies
brew install openssl libyaml

# Configure in dev (debug) mode for development, build and install
./autogen.sh
./configure --enable-yjit=dev --prefix=$HOME/.rubies/ruby-yjit --disable-install-doc --with-opt-dir="$(brew --prefix openssl):$(brew --prefix readline):$(brew --prefix libyaml)"
make -j && make install

Typically configure will choose the default C compiler. To specify the C compiler, use

# Choosing a specific c compiler
export CC=/path/to/my/chosen/c/compiler

before running ./configure.

You can test that YJIT works correctly by running:

# Quick tests found in /bootstraptest
make btest

# Complete set of tests
make -j test-all

Usage

Examples

Once YJIT is built, you can either use ./miniruby from within your build directory, or switch to the YJIT version of ruby by using the chruby tool:

chruby ruby-yjit
ruby myscript.rb

You can dump statistics about compilation and execution by running YJIT with the --yjit-stats command-line option:

./miniruby --yjit-stats myscript.rb

You can see what YJIT has compiled by running YJIT with the --yjit-log command-line option:

./miniruby --yjit-log myscript.rb

The machine code generated for a given method can be printed by adding puts RubyVM::YJIT.disasm(method(:method_name)) to a Ruby script. Note that no code will be generated if the method is not compiled.

Command-Line Options

YJIT supports all command-line options supported by upstream CRuby, but also adds a few YJIT-specific options:

Note that there is also an environment variable RUBY_YJIT_ENABLE which can be used to enable YJIT. This can be useful for some deployment scripts where specifying an extra command-line option to Ruby is not practical.

You can also enable YJIT at run-time using RubyVM::YJIT.enable. This can allow you to enable YJIT after your application is done booting, which makes it possible to avoid compiling any initialization code.

You can verify that YJIT is enabled using RubyVM::YJIT.enabled? or by checking that ruby --yjit -v includes the string +YJIT:

ruby --yjit -v
ruby 3.3.0dev (2023-01-31T15:11:10Z master 2a0bf269c9) +YJIT dev [x86_64-darwin22]

ruby --yjit -e "p RubyVM::YJIT.enabled?"
true

ruby -e "RubyVM::YJIT.enable; p RubyVM::YJIT.enabled?"
true

Benchmarking

We have collected a set of benchmarks and implemented a simple benchmarking harness in the yjit-bench repository. This benchmarking harness is designed to disable CPU frequency scaling, set process affinity and disable address space randomization so that the variance between benchmarking runs will be as small as possible.

Performance Tips for Production Deployments

While YJIT options default to what we think would work well for most workloads, they might not necessarily be the best configuration for your application. This section covers tips on improving YJIT performance in case YJIT does not speed up your application in production.

Increasing --yjit-exec-mem-size

When JIT code size (RubyVM::YJIT.runtime_stats[:code_region_size]) reaches this value, YJIT stops compiling new code. Increasing the executable memory size means more code can be optimized by YJIT, at the cost of more memory usage.

If you start Ruby with --yjit-stats, e.g. using an environment variable RUBYOPT=--yjit-stats, RubyVM::YJIT.runtime_stats[:ratio_in_yjit] shows the ratio of YJIT-executed instructions in %. Ideally, ratio_in_yjit should be as large as 99%, and increasing --yjit-exec-mem-size often helps improving ratio_in_yjit.

Running workers as long as possible

It's helpful to call the same code as many times as possible before a process restarts. If a process is killed too frequently, the time taken for compiling methods may outweigh the speedup obtained by compiling them.

You should monitor the number of requests each process has served. If you're periodically killing worker processes, e.g. with unicorn-worker-killer or puma_worker_killer, you may want to reduce the killing frequency or increase the limit.

Reducing YJIT Memory Usage

YJIT allocates memory for JIT code and metadata. Enabling YJIT generally results in more memory usage. This section goes over tips on minimizing YJIT memory usage in case it uses more than your capacity.

Decreasing --yjit-exec-mem-size

The --yjit-exec-mem-size option specifies the JIT code size, but YJIT also uses memory for its metadata, which often consumes more memory than JIT code. Generally, YJIT adds memory overhead by roughly 3-4x of --yjit-exec-mem-size in production as of Ruby 3.3. You should multiply that by the number of worker processes to estimate the worst case memory overhead.

--yjit-exec-mem-size=48 is the default since Ruby 3.3.1, but smaller values like 32 MiB might make sense for your application. While doing so, you may want to monitor RubyVM::YJIT.runtime_stats[:ratio_in_yjit] as explained above.

Enabling YJIT lazily

If you enable YJIT by --yjit options or RUBY_YJIT_ENABLE=1, YJIT may compile code that is used only during the application boot. RubyVM::YJIT.enable allows you to enable YJIT from Ruby code, and you can call this after your application is initialized, e.g. on Unicorn's after_fork hook. If you use any YJIT options (--yjit-*), YJIT will start at boot by default, but --yjit-disable allows you to start Ruby with the YJIT-disabled mode while passing YJIT tuning options.

Code Optimization Tips

This section contains tips on writing Ruby code that will run as fast as possible on YJIT. Some of this advice is based on current limitations of YJIT, while other advice is broadly applicable. It probably won't be practical to apply these tips everywhere in your codebase. You should ideally start by profiling your application using a tool such as stackprof so that you can determine which methods make up most of the execution time. You can then refactor the specific methods that make up the largest fractions of the execution time. We do not recommend modifying your entire codebase based on the current limitations of YJIT.

You can also use the --yjit-stats command-line option to see which bytecodes cause YJIT to exit, and refactor your code to avoid using these instructions in the hottest methods of your code.

Other Statistics

If you run ruby with --yjit-stats, YJIT will track and return performance statistics in RubyVM::YJIT.runtime_stats.

$ RUBYOPT="--yjit-stats" irb
irb(main):001:0> RubyVM::YJIT.runtime_stats
=>
{:inline_code_size=>340745,
 :outlined_code_size=>297664,
 :all_stats=>true,
 :yjit_insns_count=>1547816,
 :send_callsite_not_simple=>7267,
 :send_kw_splat=>7,
 :send_ivar_set_method=>72,
...

Some of the counters include:

Counters starting with "exit_" show reasons for YJIT code taking a side exit (return to the interpreter.)

Performance counter names are not guaranteed to remain the same between Ruby versions. If you're curious what each counter means, it's usually best to search the source code for it — but it may change in a later Ruby version.

The printed text after a --yjit-stats run includes other information that may be named differently than the information in {RubyVM::YJIT.runtime_stats}.

Contributing

We welcome open source contributions. You should feel free to open new issues to report bugs or just to ask questions. Suggestions on how to make this readme file more helpful for new contributors are most welcome.

Bug fixes and bug reports are very valuable to us. If you find a bug in YJIT, it's very possible be that nobody has reported it before, or that we don't have a good reproduction for it, so please open an issue and provide as much information as you can about your configuration and a description of how you encountered the problem. List the commands you used to run YJIT so that we can easily reproduce the issue on our end and investigate it. If you are able to produce a small program reproducing the error to help us track it down, that is very much appreciated as well.

If you would like to contribute a large patch to YJIT, we suggest opening an issue or a discussion on the Shopify/ruby repository so that we can have an active discussion. A common problem is that sometimes people submit large pull requests to open source projects without prior communication, and we have to reject them because the work they implemented does not fit within the design of the project. We want to save you time and frustration, so please reach out so we can have a productive discussion as to how you can contribute patches we will want to merge into YJIT.

Source Code Organization

The YJIT source code is divided between:

The core of CRuby's interpreter logic is found in:

Generating C bindings with bindgen

In order to expose C functions to the Rust codebase, you will need to generate C bindings:

CC=clang ./configure --enable-yjit=dev
make -j yjit-bindgen

This uses the bindgen tools to generate/update yjit/src/cruby_bindings.inc.rs based on the bindings listed in yjit/bindgen/src/main.rs. Avoid manually editing this file as it could be automatically regenerated at a later time. If you need to manually add C bindings, add them to yjit/cruby.rs instead.

Coding & Debugging Protips

There are multiple test suites:

The tests can be run in parallel like this:

make -j test-all RUN_OPTS="--yjit-call-threshold=1"

Or single-threaded like this, to more easily identify which specific test is failing:

make test-all TESTOPTS=--verbose RUN_OPTS="--yjit-call-threshold=1"

To run a single test file with test-all:

make test-all TESTS='test/-ext-/marshal/test_usrmarshal.rb' RUNRUBYOPT=--debugger=lldb RUN_OPTS="--yjit-call-threshold=1"

It's also possible to filter tests by name to run a single test:

make test-all TESTS='-n /test_float_plus/' RUN_OPTS="--yjit-call-threshold=1"

You can also run one specific test in btest:

make btest BTESTS=bootstraptest/test_ractor.rb RUN_OPTS="--yjit-call-threshold=1"

There are shortcuts to run/debug your own test/repro in test.rb:

make run  # runs ./miniruby test.rb
make lldb # launches ./miniruby test.rb in lldb

You can use the Intel syntax for disassembly in LLDB, keeping it consistent with YJIT's disassembly:

echo "settings set target.x86-disassembly-flavor intel" >> ~/.lldbinit

Running x86 YJIT on Apple's Rosetta

For development purposes, it is possible to run x86 YJIT on an Apple M1 via Rosetta. You can find basic instructions below, but there are a few caveats listed further down.

First, install Rosetta:

$ softwareupdate --install-rosetta

Now any command can be run with Rosetta via the arch command line tool.

Then you can start your shell in an x86 environment:

$ arch -x86_64 zsh

You can double check your current architecture via the arch command:

$ arch -x86_64 zsh
$ arch
i386

You may need to set the default target for rustc to x86-64, e.g.

$ rustup default stable-x86_64-apple-darwin

While in your i386 shell, install Cargo and Homebrew, then hack away!

Rosetta Caveats

  1. You must install a version of Homebrew for each architecture
  2. Cargo will install in $HOME/.cargo by default, and I don't know a good way to change architectures after install

If you use Fish shell you can read this link for information on making the dev environment easier.

Profiling with Linux perf

--yjit-perf allows you to profile JIT-ed methods along with other native functions using Linux perf. When you run Ruby with perf record, perf looks up /tmp/perf-{pid}.map to resolve symbols in JIT code, and this option lets YJIT write method symbols into that file as well as enabling frame pointers.

Call graph

Here's an example way to use this option with Firefox Profiler (See also: Profiling with Linux perf):

# Compile the interpreter with frame pointers enabled
./configure --enable-yjit --prefix=$HOME/.rubies/ruby-yjit --disable-install-doc cflags=-fno-omit-frame-pointer
make -j && make install

# [Optional] Allow running perf without sudo
echo 0 | sudo tee /proc/sys/kernel/kptr_restrict
echo -1 | sudo tee /proc/sys/kernel/perf_event_paranoid

# Profile Ruby with --yjit-perf
cd ../yjit-bench
PERF="record --call-graph fp" ruby --yjit-perf -Iharness-perf benchmarks/liquid-render/benchmark.rb

# View results on Firefox Profiler https://profiler.firefox.com.
# Create /tmp/test.perf as below and upload it using "Load a profile from file".
perf script --fields +pid > /tmp/test.perf

YJIT codegen

You can also profile the number of cycles consumed by code generated by each YJIT function.

# Install perf
apt-get install linux-tools-common linux-tools-generic linux-tools-`uname -r`

# [Optional] Allow running perf without sudo
echo 0 | sudo tee /proc/sys/kernel/kptr_restrict
echo -1 | sudo tee /proc/sys/kernel/perf_event_paranoid

# Profile Ruby with --yjit-perf=codegen
cd ../yjit-bench
PERF=record ruby --yjit-perf=codegen -Iharness-perf benchmarks/lobsters/benchmark.rb

# Aggregate results
perf script > /tmp/perf.txt
../ruby/misc/yjit_perf.py /tmp/perf.txt

Building perf with Python support

The above instructions work fine for most people, but you could also use a handy perf script -s interface if you build perf from source.

# Build perf from source for Python support
sudo apt-get install libpython3-dev python3-pip flex libtraceevent-dev \
  libelf-dev libunwind-dev libaudit-dev libslang2-dev libdw-dev
git clone --depth=1 https://github.com/torvalds/linux
cd linux/tools/perf
make
make install

# Aggregate results
perf script -s ../ruby/misc/yjit_perf.py