The Publish-Subscribe Pattern on Rails: An Implementation Tutorial

The publish-subscribe pattern (or pub/sub, for short) is a Ruby on Rails messaging pattern where senders of messages (publishers), do not program the messages to be sent directly to specific receivers (subscribers). Instead, the programmer “publishes” messages (events), without any knowledge of any subscribers there may be.

Similarly, subscribers express interest in one or more events, and only receive messages that are of interest, without any knowledge of any publishers.

To accomplish this, an intermediary, called a “message broker” or “event bus”, receives published messages, and then forwards them on to those subscribers who are registered to receive them.

In other words, pub-sub is a pattern used to communicate messages between different system components without these components knowing anything about each other’s identity.

This design pattern is not new, but it’s not commonly used by Rails developers. There are many tools that help incorporate this design pattern into your code base, such as:

• Wisper (which I personally prefer and will discuss it further)
• EventBus
• EventBGBus (a fork of EventBus)
• RabbitMQ
• Redis

All of these tools have different underlying pub-sub implementations, but they all offer the same major advantages for a Rails application.

Advantages to Pub-Sub Implementation

Reducing Model/Controller Bloat

It is a common practice, but not a best practice, to have some fat models or controllers in your Rails application.

The pub/sub pattern can easily help decompose fat models or controllers.

Fewer Callbacks

Having a lot of intertwined callbacks between the models is a well-known known code smell, and bit by bit it tightly couples the models together, making them harder to maintain or extend.

For example a Post model could look like the following:

# app/models/post.rb
class Post
  # ...
  field: content, type: String
  # ...

  after_create :create_feed, :notify_followers
  # ...

  def create_feed
    Feed.create!(self)
  end

  def notify_followers
    User::NotifyFollowers.call(self)
  end
end

And the Post controller might look something like the following:

# app/controllers/api/v1/posts_controller.rb
class Api::V1::PostsController < Api::V1::ApiController
  # ...
  def create
    @post = current_user.posts.build(post_params)
    if @post.save
      render_created(@post)
    else 
      render_unprocessable_entity(@post.errors)
    end
  end
  # ...
end

As you can see, the Post model has callbacks that tightly couple the model to both the Feed model and the User::NotifyFollowers service or concern. By using any pub/sub pattern, the previous code could be re-factored to be something like the following, which uses Wisper:

# app/models/post.rb
class Post
  # ...
  field: content, type: String
  # ...
  # no callbacks in the models!
end

Publishers publish the event with the event payload object that might be needed.

# app/controllers/api/v1/posts_controller.rb
# corresponds to the publisher in the previous figure
class Api::V1::PostsController < Api::V1::ApiController

  include Wisper::Publisher
  # ...
  def create
    @post = current_user.posts.build(post_params)
    if @post.save
      # Publish event about post creation for any interested listeners
      publish(:post_create, @post)
      render_created(@post)
    else 
      # Publish event about post error for any interested listeners
      publish(:post_errors, @post)
      render_unprocessable_entity(@post.errors)
    end
  end
  # ...
end

Subscribers only subscribe to the events they wish to respond to.

# app/listener/feed_listener.rb
class FeedListener
  def post_create(post)
    Feed.create!(post)
  end
end

# app/listener/user_listener.rb
class UserListener
  def post_create(post)
    User::NotifyFollowers.call(self)
  end
end

Event Bus registers the different subscribers in the system.

# config/initializers/wisper.rb

Wisper.subscribe(FeedListener.new)
Wisper.subscribe(UserListener.new)

In this example, the pub-sub pattern completely eliminated callbacks in the Post model and helped the models to work independently from each other with minimum knowledge about each other, ensuring loose coupling. Expanding the behavior to additional actions is just a matter of hooking to the desired event.

The Single Responsibility Principle (SRP)

The Single Responsibility Principle is really helpful for maintaining a clean code base. The problem in sticking to it is that sometimes the responsibility of the class is not as clear as it should be. This is especially common when it comes to MVCs (like Rails).

Models should handle persistence, associations and not much else.

Controllers should handle user requests and be a wrapper around the business logic (Service Objects).

Service Objects should encapsulate one of the responsibilities of the business logic, provide an entry point for external services, or act as an alternative to model concerns.

Thanks to its power to reduce coupling, the pub-sub design pattern can be combined with single responsibility service objects (SRSOs) to help encapsulate the business logic, and forbid the business logic from creeping into either the models or the controllers. This keeps the code base clean, readable, maintainable and scalable.

Here is an example of some complex business logic implemented using the pub/sub pattern and service objects:

Publisher

# app/service/financial/order_review.rb
class Financial::OrderReview
  include Wisper::Publisher
  # ...
  def self.call(order)
    if order.approved?
      publish(:order_create, order)
    else
      publish(:order_decline, order)
    end
  end
  # ...

Subscribers

# app/listener/client_listener.rb
class ClientListener
  def order_create(order)
    # can implement transaction using different service objects
    Client::Charge.call(order)
    Inventory::UpdateStock.call(order)
  end

  def order_decline(order)
    Client::NotifyDeclinedOrder(order)
  end
end

By using the publish-subscribe pattern, the code base gets organized into SRSOs almost automatically. Moreover, implementing code for complex workflows is easily organized around events, without sacrificing readability, maintainability or scalability.

Testing

By decomposing the fat models and controllers, and having a lot of SRSOs, testing of the code base becomes a much, much easier process. This is particularly the case when it comes to integration testing and inter-module communication. Testing should simply ensure that events are published and received correctly.

Wisper has a testing gem that adds RSpec matchers to ease the testing of different components.

In the previous two examples (Post example and Order example), testing should include the following:

Publishers

# spec/service/financial/order_review.rb
describe Financial::OrderReview do
  it 'publishes :order_create' do
    @order = Fabricate(:order, approved: true)
    expect { Financial::OrderReview.call(@order) }.to broadcast(:order_create)
  end

  it 'publishes :order_decline' do
    @order = Fabricate(:order, approved: false)
    expect { Financial::OrderReview.call(@order) }.to broadcast(:order_decline)
  end
end

Subscribers

# spec/listeners/feed_listener_spec.rb
describe FeedListener do
  it 'receives :post_create event on PostController#create' do
    expect(FeedListner).to receive(:post_create).with(Post.last)
    post '/post', { content: 'Some post content' }, request_headers
  end
end

However, there are some limitations to testing the published events when the publisher is the controller.

If you want to go the extra mile, having the payload tested as well will help maintain an even better code base.

As you can see, pub-sub design pattern testing is simple. It’s just the matter of ensuring that the different events are correctly published and received.

Performance

This is more of a possible advantage. The publish-subscribe design pattern itself doesn’t have a major inherent impact on code performance. However, as with any tool you use in your code, the tools for implementing pub/sub can have a big effect on performance. Sometimes it can be a bad effect, but sometimes it can be very good.

First, an example of a bad effect: Redis is an “advanced key-value cache and store. It is often referred to as a data structure server.” This popular tool supports the pub/sub pattern and is very stable. However, if it’s used on a remote server (not the same server the Rails application is deployed on), it will result in a huge performance loss due to network overhead.

On the other hand, Wisper has various adapters for asynchronous event handling, like wisper-celluloid, wisper-sidekiq and wisper-activejob. These tools support asynchronous events and threaded executions. which if applied appropriately, can hugely increase the performance of the application.

The Bottom Line

If you are aiming for the extra mile in performance, the pub/sub pattern could help you reach it. But even if you don’t find a performance boost with this Rails design pattern, it’ll still help keep code organized and make it more maintainable. After all, who can worry about the performance of code that cannot be maintained, or that doesn’t work in the first place?

Disadvantages of Pub-Sub Implementation

As with all things, there are some possible drawbacks to the pub-sub pattern as well.

Loose Coupling (Inflexible Semantic Coupling)

The greatest of the pub/sub pattern’s strengths are also it’s greatest weaknesses. The structure of the data published (the event payload) must be well defined, and quickly becomes rather inflexible. In order to modify data structure of the published payload, it is necessary to know about all the Subscribers, and either modify them also, or ensure the modifications are compatible with older versions. This makes refactoring of Publisher code much more difficult.

If you want to avoid this you have to be extra cautious when defining the payload of the publishers. Of course, if you have a great test suite, that tests the payload as well as mentioned previously, you don’t have to worry much about the system going down after you change the publisher’s payload or event name.

Messaging Bus Stability

Publishers have no knowledge of the status of the subscriber and vice versa. Using simple pub/sub tools, it might not be possible to ensure the stability of the messaging bus itself, and to ensure that all the published messages are correctly queued and delivered.

The increasing number of messages being exchanged leads to instabilities in the system when using simple tools, and it may not be possible to ensure delivery to all subscribers without some more sophisticated protocols. Depending on how many messages are being exchanged, and the performance parameters you want to achieve, you might consider using services like RabbitMQ, PubNub, Pusher, CloudAMQP, IronMQ or many many other alternatives. These alternatives provide extra functionality, and are more stable than Wisper for more complex systems. However, they also require some extra work to implement. You can read more about how message brokers work here

Infinite Event Loops

When the system is completely driven by events, you should be extra cautious not to have event loops. These loops are just like the infinite loops that can happen in code. However, they are harder to detect ahead of time, and they can bring your system to a standstill. They can exist without your notice when there are many events published and subscribed across the system.

Rails Tutorial Conclusion

The publish-subscribe pattern is not a silver bullet for all your Rails problems and code smells, but it’s a really good design pattern that helps in decoupling different system components, and making it more maintainable, readable, and scalable.

When combined with single responsibility service objects (SRSOs), pub-sub can also really help with encapsulating business logic and preventing different business concerns from from creeping into the models or the controllers.

The gain in performance after using this pattern depends mostly on the underlying tool being used, but the performance gain can be improved significantly in some cases, and in most cases it certainly won’t hurt performance.

However, use of the pub-sub pattern should be studied and planned out carefully, because with the great power of loose coupling comes the great responsibility of maintaining and refactoring loosely coupled components.

Because events could easily grow out of control, a simple pub/sub library may not ensure the stability of the message broker.

And finally, there is the danger of introducing infinite event loops that go unnoticed until it is too late.

I’ve been using this pattern for almost a year now, and it’s hard for me to imagine writing code without it. For me, it’s the glue that makes background jobs, service objects, concerns, controllers and models all communicate with each other cleanly and work together like charm.

I hope you have learned as much as I did from reviewing this code, and that you feel inspired to give the Publish-Subscribe pattern a chance to make your Rails application awesome.

Finally, a huge thank you to @krisleech for his awesome work implementing Wisper.