This is a nice update.
Is it possible to either have multiple ractors dispatching jobs with fibres or to set up multiple queues with different strategies?
E.g. one for IO bound and one for CPU bound?
With Sidekiq I’ve had luck having workers running on Truffleruby but generally don’t use it for my main rails apps.
You can, and Carmine (who coded this update) wrote exactly about this on this blog post: https://paolino.me/solid-queue-doesnt-need-a-thread-per-job/
From his article:
One backend, two modes
Fiber mode isn’t universally better. CPU-bound jobs get nothing from it, and blocking libraries or C extensions that do not cooperate with Ruby’s fiber scheduler stall the reactor. And that’s fine – you don’t have to pick one.
As Trevor Turk pointed out in the PR discussion, that’s the whole point: separately configured worker pools. Here’s what Chat with Work actually runs in production:
workers: - queues: [ chat ] fibers: 10 processes: 2 polling_interval: 0.1 - queues: [ turbo ] fibers: 10 processes: 1 polling_interval: 0.05 - queues: [ notifications, default, maintenance ] fibers: 5 processes: 1 polling_interval: 0.2 - queues: [ cpu ] threads: 1 processes: 1
That post, and the linked post, provide very good context to understand these changes.
https://paolino.me/async-ruby-is-the-future/