Rails Autoloading - Classic and Zeitwerk

Have you noticed the absence of require statements in a Rails codebase? That is due to Rails autoloading feature. Until Rails 5, the framework used the Classic autoloader. Starting with Rails 6.0, Zeitwerk has taken over as the improved method for handling autoloading.

Classic Autoloader

3 key members of the Classic autoloader:

Ruby Constant Lookup
Module#const_missing
autoload_paths

Initially, Ruby tries to resolve a constant using the Ruby Constant Lookup mechanism. This involves

Module.nesting: This determines the scope in which constants are defined and resolved.

module Company
  module Employee
    $a = Module.nesting  # Outputs: [Company::Employee, Company]
  end
end

In this case, constants can be defined either in Company::Employee or Company.

module Company::Employee
  $b = Module.nesting # Outputs: [Company::Employee]
end

So, in this case, defining a constant in Company won't make it accessible in Company::Employee.

Object.ancestors: This method returns the chain of modules included in the class, which helps in resolving constants through inheritance.

class User
end

=> User.ancestors 

# Outputs [User, ActiveSupport::Dependencies::RequireDependency,
 ActiveSupport::ToJsonWithActiveSupportEncoder, Object, PP::ObjectMixin, ActiveSupport::Tryable,
 JSON::Ext::Generator::GeneratorMethods::Object, DEBUGGER__::TrapInterceptor, Kernel, BasicObject]

# See the difference in the case of a module

module ActiveUser
end

=> ActiveUser.ancestors
[ActiveUser]

Here is how Rails autoloading works

Initially, Ruby searches for the constant within the Module.nesting list.
If the constant is not located in this immediate scope or any of its enclosing modules or classes, Ruby then moves on to examine the Object.ancestors chain. This sequence not only includes parent classes but also encompasses any modules that have been mixed into these classes.
If Ruby still cannot locate the constant following these searches, it results in a NameError. At this point, Module#const_missing is triggered, allowing Rails to intervene.
We all know Rails follows convention over configuration, and we see stirct folder structure. Rails, then searches for the constant within its autoload_paths based on conventions. For instance, in the scenario described, Rails would attempt to locate and load Company/employee.rb from the autoload_paths.

There are a bunch of issues related to Classic mode, which are listed here

Common issues are:

Loading Order Matters

One of the primary issues with the classic autoloader is the dependency on the loading order of files. Since Rails does not load all constants at startup, the order in which files are loaded can affect the availability of constants.

Example Scenario:

Suppose you have two classes defined in different files:

# app/models/payment.rb
class Payment < ApplicationRecord
  belongs_to :user
end

# app/models/user.rb
class User < ApplicationRecord
  has_many :payments
end

If the User is referenced somewhere in the application before Payment is loaded (for example, in an initializer), and if the file defining Payment hasn't been loaded yet, you might encounter an uninitialized constant User::Payment error. This happens because Rails tries to load the User constant and all its associations, but the Payment class isn't available yet.

Unintended Constant Loading

This occurs when the Rails autoloader loads a different constant than expected due to naming conflicts or incorrect file paths.

This is the directory structure.

app/
  models/
    financial/
      report.rb
    report.rb

And the class definitions are as follows:

# app/models/financial/report.rb
module Financial
  class Report
    def self.generate
      puts "Generating financial report..."
    end
  end
end

# app/models/report.rb
class Report
  def self.generate
    puts "Generating general report..."
  end
end

If the code elsewhere (say, in a controller) mistakenly refers to Financial::Report before models/report.rb has been autoloaded, and due to some autoloading quirks or manual require misplacements, Rails could end up loading app/models/report.rb instead of app/models/financial/report.rb. This would lead to Financial::Report inadvertently mapping to the wrong Report class, causing unexpected behavior or errors. To avoid issues with the classic autoloader in Rails, you should use require_dependency to explicitly load the necessary files.

Zeitwerk Autoloader

Having explored the details and common pitfalls of the Classic autoloader in Rails, let's now turn our attention to Zeitwerk, the new and improved autoloading mechanism introduced in Rails 6, which addresses many of the limitations of the classic method and streamlines code loading for modern Rails applications. Zeiwerk has no dependencies, meaning it can work with any Ruby application.

3 key members of Zeitwerk autoloader:

Module#autoload
Kernel#require
TracePoint

Yep! no more Module#const_missing

To delve deeper into how Zeitwerk functions, let’s consider an example involving a Rails application structure:

app/
  models/
    comment.rb
    post.rb

Upon application startup, Rails initiates the Zeitwerk autoloader by invoking Zeitwerk#setup.
This method is responsible for preparing the autoloaders for all directories specified in autoload_paths, though it doesn't load the files immediately.
For instance, in the app/models/ directory, which is included in autoload_paths, Zeitwerk sets up an Module#autoload for each Ruby file, mapping them to their respective constants based on the filenames.
Hence, Zeitwerk automatically deduces that comment.rb and post.rb are expected to define the Comment and Post constants, respectively.

This approach is a reverse of the Classic autoloader’s mechanism. While the Classic autoloader starts with a constant and searches the filesystem for the corresponding file, Zeitwerk instead scans the filesystem first and predicts the constants from the filenames.

In contrast to the Classic autoloader, which often struggled with implicit and explicit namespaces, leading to load order issues and unexpected NameErrors, Zeitwerk introduces a robust approach to handling these namespaces.

Lets see how Implicit namespaces are handled using Zeitwerk.

Zeitwerk employs a technique known as autovivification to manage implicit namespaces effectively. For instance, consider a Rails application with the following structure:

app/
  controllers/
    posts_controller.rb // Admin::PostsController

Even though an admin directory does not explicitly exist, Zeitwerk handles this scenario.
It creates a dummy Admin module on the fly using autovivification. This is where Kernel#require is monkey-patched.
Through Kernel#require method, dummy module is then configured to autoload any constants within this namespace—Admin::PostsController in our example.
The setup for autoloading means that whenever the Admin namespace is referenced, Zeitwerk loads this virtual module, and subsequent constants under this namespace, such as PostsController, are automatically loaded based on their declaration in the file system.

Let's explore how Zeitwerk handles Explicit namespaces

/example
  /company
    employee.rb
    /employee/
      manager.rb

Here's the sequence of operations Zeitwerk performs:

When the constant Company::Employee is first referenced, Zeitwerk loads the file employee.rb, which is expected to define the module Company::Employee.
Upon loading employee.rb, Zeitwerk utilizes a Ruby feature called TracePoint to monitor and confirm the definition of Company::Employee.
Once this definition is confirmed, Zeitwerk then recognizes the presence of an employee directory alongside the employee.rb file.
With the directory structure indicating a nested namespace, Zeitwerk sets up autoloading for any constants that might be defined within the Company::Employee namespace.
This preparation includes the directory named employee, suggesting potential additional constants under this namespace.
When Company::Employee::Manager is subsequently referenced, Zeitwerk automatically loads manager.rb from within the employee directory.
This is because it has set up autoloading ahead of time for any constants within the Company::Employee namespace, effectively managing the explicit namespace defined by the directory and file structure.

TracePoint provides real-time feedback that a particular constant has been successfully defined. For Zeitwerk, it is essential to confirm that the file it loaded (like employee.rb in our example) correctly defines the expected module or class (Company::Employee). This confirmation is critical before setting up further autoloads for nested constants.

Rails Autoloading - Classic and Zeitwerk

Classic Autoloader

Zeitwerk Autoloader

References