"Error while checking if topic is synchronous" on setup:upgrade

Question

While upgrading a 2.2.8 database to 2.3.2 I got this error today:

  Schema post-updates:                                                         
  Module 'Magento_Store':                                                      
  Module 'Magento_Directory':                                                  
  Module 'Magento_Theme':                                                      
  Module 'Magento_Eav':                                                        
  Module 'Magento_AdvancedPricingImportExport':                                
  Module 'Magento_Rule':                                                       
  Module 'Magento_Customer':                                                   
  Module 'Magento_Amqp':                                                       
  Error while checking if topic is synchronous                                 

It must have something to do with environment configuration because on the same DB snapshot with the same code base, it worked on a test environment.

We do not use the Magento_Amqp module at all, so there is no AMQP configuration on either system.

I looked where the error message can be thrown and found:

\Magento\Framework\MessageQueue\Topology\Config\QueueConfigItem\DataMapper::isSynchronousTopic

which can be traced back to:

\Magento\Framework\MessageQueue\Topology\Config\QueueConfigItem\Iterator::__construct

And the exception is thrown if in

\Magento\Framework\Communication\Config::getTopic()

a configuration path (topics/$topicName, from cache id communication_config_cache) cannot be found.

But now I'm lost: Why should this configuration be present in one installation and not in the other? Or why would Magento look for the configuration at all during DB schema updates?

The output suggests, that the recurring schema update (\Magento\Amqp\Setup\Recurring) of Magento_Amqp are not even started (No "Running schema recurring"), so I guess the error comes while instantiating it.

Trying to find a dependency path to the class from above, that throws the exception in its constructor:

• \Magento\Framework\Amqp\TopologyInstaller
• \Magento\Framework\MessageQueue\Topology\Config
• \Magento\Framework\MessageQueue\Topology\Config\QueueConfigItem\Iterator

Yes! We have a trace. (but still no idea, why. I'll keep this updated)

Answer 1

Oh dear, I found it. With the update, I've removed the module magento/module-url-rewrite-import-export because Magento yet has to release a version that's compatible with Magento 2.3

This module came with a queue. On the live system, this queue configuration was still present:

         "queue--url_rewrite_import--url.rewrite.import" => [
           "id" => "queue--url_rewrite_import--url.rewrite.import",
           "destinationType" => "queue",
           "destination" => "url_rewrite_import",
           "disabled" => false,
           "topic" => "url.rewrite.import",
           "arguments" => [],
         ],

Apparently still when setup:upgrade was executed during deployment. But the configuration for its "topic" was not found anymore.

Still unclear: Why is that queue configuration present? At that point, we operate on the new code and the cache has been flushed.

New answer, November 2021

With the update to Magento 2.4.3 we occasionally got this error as well.

Adding

    'queue' => [
        'consumers_wait_for_messages' => 1
    ],

to env.php helped. Apparently Magento adds this configuration itself but sometimes not. Without it, the queue does not work and we get this error during recurring updates of the queue modules

Answer 2

Check for communication.xml file (add it if missing) to the module with the consumer.

https://devdocs.magento.com/guides/v2.3/extension-dev-guide/message-queues/config-mq.html

Answer 3

Providing an answer after facing the issue myself.

Explanations

When you run any CLI command, Magento instantiates the \Magento\Framework\Console\CommandList before doing anything. With the DI, it instantiates all the dependencies for all the commands.

In my install (and that's probably why it's not happening all the times), some of my custom CLI command depend on \Magento\Framework\MessageQueue\PublisherInterface

The interface depends on \Magento\Framework\Communication\Config. When this class constructor is called, it retrieves the communication_config_cache data and assigns it to a variable within the class. At that point, the data retrieved is retrieved FROM CACHE.

When running the upgrade, after the above, we then flush the cache. BUT the data is already assigned to the \Magento\Framework\Communication\Config variable so the next time this class is used, the values used are the cached values from before the upgrade.

After the upgrade command flushes the cache, it loops through the new topology definitions (which are the new values from the queue_topology.xml files). For each definition, we retrieve the corresponding topic (which are the old values as explained above). If you introduced a new topic, you will get the error because the topic was not part of the cache before the upgrade.

Workaround

If you want to avoid that issue in your deployment process, you can run php bin/magento cache:clean config before running php bin/magento setup:upgrade

Answer 4

If you running and using Redis cache then try to flush Redis cache and then run upgrade command, It help me to fix this issue.

command I used to flush redis cache redis-cli flushall