0

I'm a little bit frustrated with Magento 2 queue for RabbitMQ.

I have setup RabbitMQ, my env.php has such lines

'queue' => [
    'consumers_wait_for_messages' => 1,
    'amqp' => [
        'host' => 'rabbitmq',
        'port' => '5672',
        'user' => 'magento',
        'password' => 'magento',
        'virtualhost' => '/'
    ]
],
'cron_consumers_runner' => [
    'cron_run' => true,
    'max_messages' => 20000,
    'consumers' => []
],  

but I see only async.operations.all in rabbitmqctl list_queues:

/ # rabbitmqctl list_queues
Timeout: 60.0 seconds ...
Listing queues for vhost / ...
name    messages
async.operations.all

Looks like queue is working, but why I don't see any other Magento 2 records in queue? Like
product_action_attribute.update
product_action_attribute.website.update
Strange that I see new queue messages in queue_message table in Magento DB. I see that some of items have connection="db" in queue_topology.xml. Are they executed using RabbitMQ or using Magento own queue implementation?

2

By default, all of Magento's default queues are defined with a db connection. The only one with a configured amqp connection is the 'async.operations.all' queue.

However, you can move the queues to amqp connection. Check the Magento official documentation on how to change this. The configurations mentioned there should be usable if added to your app/etc/env.php.

Good luck!


LE I can confirm you can do this :) out of curiosity I just moved all my magento instance's queues to RabbitMQ. Here is a gist with my configuration of the Open Source 2.4.1 queues moved to Rabbit. However, this is just the proof of concept about how can you do this and not a complete validation regarding performance for example.

7
  • Can you share how you moved all queues to Rabbit MQ, your env.php config and did you disable cron completely? – paj Feb 11 at 17:01
  • 1
    @paj hey there, i edited my answer above to include the part of the env.php which I edited in order to move the queues to rabbit. I didn't disable the crons in my installation. – Diana Botean Feb 11 at 17:45
  • 1
    that is very useful, thankyou. I gues at some point message q’s will be completely removed from the db (like search). I use docker containers as consumers so will be interesting to try and disable cron completely and use a consumer container to process all messaging... – paj Feb 11 at 18:24
  • i hope you are right, @paj and that queues will be moved to the message broker. would be great! and yes, i also use docker envs and totally makes sense, no need to have supervisor or other monitoring tool to keep the consumer cron up all time. i agree. – Diana Botean Feb 12 at 6:36
  • what i meant here was that it would be nice to 1. have all queues to rabbit, 2. would be also good to have the consumers externalized (by default, the queues are processed in magento), but it would be nice to have them outside magento as well ;) – Diana Botean Feb 12 at 16:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.