1

If you are running RabbitMQ in M 2.3/2.4 and have the env.php section set up for amqp:

    'queue' => [
    'amqp' => [
        'host' => '127.0.0.1',
        'port' => '5672',
        'user' => 'abc',
        'password' => 'def',
        'virtualhost' => '/',
        'ssl' => 'false',
        'ssl_options' => [

        ]
    ]
],

do you remove the cron_consumers_runner section from env.php? Or set it to false? Or trim it down to only specific consumers? Just trying to understand the relationship between the 2 sections. A full cron_consumers_runner section may include:

    'cron_consumers_runner' => [
    'cron_run' => true,
    'max_messages' => 20000,
    'consumers' => [
        'product_action_attribute.update',
        'product_action_attribute.website.update',
        'codegeneratorProcessor',
        'exportProcessor',
        'inventory.source.items.cleanup',
        'inventory.mass.update',
        'inventory.reservations.cleanup',
        'inventory.reservations.update',
        'quoteItemCleaner',
        'inventoryQtyCounter',
        'async.operations.all'
    ]
],
2
  • Did you get this working? I am confused as to the correct settings required to migrate all cron jobs to RabbitMq.
    – paj
    Feb 11 at 16:56
  • @paj check my answer below regarding the consumers management (as opposed to moving all queues to rabbit magento.stackexchange.com/questions/328213) Feb 12 at 16:17
3

TL;DR You can completely omit the cron_consumers_runner section in the env.php file as it will fallback to the default Magento values. But if you want to change the default values, you add this section to 'env.php' and update the values as needed (all of them or partially). As for the consumers management, see below the full explanation on why and how to keep it in Magento or outside Magento.


AFAIK, the cron_consumers_runner/cron_run config is enabled by default and if you keep it that way, it means that you want Magento to process the messages in the queue (regardless that you publish the messages in a queue connection to db or amqp). Limitations of this approach - the messages are processed as frequent as every minute as per vendor/magento/module-message-queue/etc/crontab.xml (that's the 'fastest' a cron can go :) ):

<group id="consumers">
    <job name="consumers_runner" instance="Magento\MessageQueue\Model\Cron\ConsumersRunner" method="run">
        <schedule>* * * * *</schedule>
    </job>
</group>

Now, if you want a faster processing of the messages (say as fast as the message is published, it should be 'catched' and processed) or simply you don't want the internal Magento cron management to deal with this, then you should disable the cron_consumers_runner/cron_run and start all consumers individually in a service like supervisor by adding the command for each consumer:

php bin/magento queue:consumers:start [--max-messages=<value>] [--batch-size=<value>] [--single-thread] [--area-code=<value>] <consumer_name>

If you want to externalize say only one consumer, then you have to keep cron_consumers_runner/cron_run enabled and tell Magento to avoid running your consumer via internal cronjob. However, OOTB Magento, there is no 'blacklisting' mechanism (you can't tell it 'avoid X consumer'), instead you will have to workaround this and use a 'whitelisting' mechanism - under the cron_consumers_runner/consumers you will have to explicitly list all consumers which you want Magento to deal with internally, except yours. Limitations and downfalls - if new consumers are added (via upgrade or integration), they will never be executed via the internal Magento cronjob, unless you add them to the configuration manually.

So that's about it :) I hope I shed some light over this topic :)

4
  • I don’t think there is any harm letting the consumer cron run even when the queue is handled by an external consumer, if you look at the cron process it first checks if there are any messages in the db table before launching the consumer. With an aqmp q and external rabbit consumer the db message table will probably remain empty...
    – paj
    Feb 12 at 19:27
  • Yep, that's right. That's how i leave it in my projects. But i think it depends on the project's nature and needs. It's still nice that you have an alternative :) Feb 13 at 9:03
  • thank you for the details this is the only explicit answer I have found on the subject! zo by default the cron processes the messages of the queue whether it is stored in the database or in rabbitmq, which gain does one have to set up rabbitmq in this case if one does not seek the need to have the messages of each queue executed by an independent service ? Apr 7 at 12:58
  • hi there, well by publishing messages in a queue in rabbitmq you are basically moving this data outside the Magento database, so you are not storing or managing locks or other things on the queue tables in the database. The gain comes from the fact that you are not loading the database, but another decoupled application. Apr 7 at 13:03
0

The cron_consumers_runner config section is not in the env.php by default. That means you have likely chosen a message queueing system when your site was setup. And with RabbitMQ in mind, you want to try a different Messsage system.

  • The obvious way to go about it would be to remove this config so that you can trial RabbitMQ with having any biased in your experiment.
  • also, if you leave both, you may take chance to have both system conflicting with regards to how the messages get dealt with.

Now, these are recommendations that may not be practical if you want to implement RabbitMQ directly in your production environment.

if you'd want to gradually add RabbitMQ in your Magento stack, my suggestion would be to do it the other way round:

  • keep both queueing systems working together but do this first on a dev environment. Once you have the confirmation both works and there is no visible conficts/negative impact, then you would be able to reply at this moment to your own question and keep both on live environment. Again, trialling technologies together has its risks and can create complications that are unwanted on production. But this way would mean you would not need to stop your emails and start them with a new technology.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.