RabbitMQ and cron_consumers_runner

Question

If you are running RabbitMQ in M 2.3/2.4 and have the env.php section set up for amqp:

    'queue' => [
    'amqp' => [
        'host' => '127.0.0.1',
        'port' => '5672',
        'user' => 'abc',
        'password' => 'def',
        'virtualhost' => '/',
        'ssl' => 'false',
        'ssl_options' => [

        ]
    ]
],

do you remove the cron_consumers_runner section from env.php? Or set it to false? Or trim it down to only specific consumers? Just trying to understand the relationship between the 2 sections. A full cron_consumers_runner section may include:

    'cron_consumers_runner' => [
    'cron_run' => true,
    'max_messages' => 20000,
    'consumers' => [
        'product_action_attribute.update',
        'product_action_attribute.website.update',
        'codegeneratorProcessor',
        'exportProcessor',
        'inventory.source.items.cleanup',
        'inventory.mass.update',
        'inventory.reservations.cleanup',
        'inventory.reservations.update',
        'quoteItemCleaner',
        'inventoryQtyCounter',
        'async.operations.all'
    ]
],

Answer 1

TL;DR You can completely omit the cron_consumers_runner section in the env.php file as it will fallback to the default Magento values. But if you want to change the default values, you add this section to 'env.php' and update the values as needed (all of them or partially). As for the consumers management, see below the full explanation on why and how to keep it in Magento or outside Magento.

---

AFAIK, the cron_consumers_runner/cron_run config is enabled by default and if you keep it that way, it means that you want Magento to process the messages in the queue (regardless that you publish the messages in a queue connection to db or amqp). Limitations of this approach - the messages are processed as frequent as every minute as per vendor/magento/module-message-queue/etc/crontab.xml (that's the 'fastest' a cron can go :) ):

<group id="consumers">
    <job name="consumers_runner" instance="Magento\MessageQueue\Model\Cron\ConsumersRunner" method="run">
        <schedule>* * * * *</schedule>
    </job>
</group>

Now, if you want a faster processing of the messages (say as fast as the message is published, it should be 'catched' and processed) or simply you don't want the internal Magento cron management to deal with this, then you should disable the cron_consumers_runner/cron_run and start all consumers individually in a service like supervisor by adding the command for each consumer:

php bin/magento queue:consumers:start [--max-messages=<value>] [--batch-size=<value>] [--single-thread] [--area-code=<value>] <consumer_name>

If you want to externalize say only one consumer, then you have to keep cron_consumers_runner/cron_run enabled and tell Magento to avoid running your consumer via internal cronjob. However, OOTB Magento, there is no 'blacklisting' mechanism (you can't tell it 'avoid X consumer'), instead you will have to workaround this and use a 'whitelisting' mechanism - under the cron_consumers_runner/consumers you will have to explicitly list all consumers which you want Magento to deal with internally, except yours. Limitations and downfalls - if new consumers are added (via upgrade or integration), they will never be executed via the internal Magento cronjob, unless you add them to the configuration manually.

So that's about it :) I hope I shed some light over this topic :)

Answer 2

The cron_consumers_runner config section is not in the env.php by default. That means you have likely chosen a message queueing system when your site was setup. And with RabbitMQ in mind, you want to try a different Messsage system.

• The obvious way to go about it would be to remove this config so that you can trial RabbitMQ with having any biased in your experiment.
• also, if you leave both, you may take chance to have both system conflicting with regards to how the messages get dealt with.

Now, these are recommendations that may not be practical if you want to implement RabbitMQ directly in your production environment.

if you'd want to gradually add RabbitMQ in your Magento stack, my suggestion would be to do it the other way round:

• keep both queueing systems working together but do this first on a dev environment. Once you have the confirmation both works and there is no visible conficts/negative impact, then you would be able to reply at this moment to your own question and keep both on live environment. Again, trialling technologies together has its risks and can create complications that are unwanted on production. But this way would mean you would not need to stop your emails and start them with a new technology.