34

First of all yes, this is yet another question/topic about the 1.9.1 email queue. But it is not about about any cron problems (like this or this) or about the new queue feature not being used (like this).

In our case we had the problem, that the queue (core_email_queue and core_email_queue_recipients) simply wouldn't get any emails on new orders or order updates and therefore no more emails got sent out for anything order related, also cron is working perfectly and manually adding emails to the queue works and they get sent out.

The strange thing is, in our test environment everything worked. Even when we went live today in the first minutes all emails got processed but after some minutes (without any further modification on the live system of course) no more new emails got added to the queue at all. It seems like this happened (but I cannot tell for sure) when the first customer used PayPal Express, which we didn't test beforehand :-/ And indeed we were using some custom overrides in the PayPal Express logic with the old sendNewOrderEmail() function. But we couldn't get emails to work again even after patching those to use queueNewOrderEmail().
So the first question would be, is it possible that the old function triggered some inconsistency which 'broke' the email queue? Or is this all just a big coincidence and there is a totally different explanation?

As we couldn't find the problem but of course needed emails to work again asap we went for another core override. In Mage_Core_Model_Email_Template_Mailer (of course in a copy in local) we commented out line 76: ->setQueue($this->getQueue())
This seems to bypass the queue and all mails get sent the old way again.

However, as we'd like to keep the number of core overrides to a minimum and we also cannot tell right now if we will face any other side effects, any other tips or solutions from people with a deeper understanding of the magento code and the email queue would be appreciated.

Update for 1.9.2: On the upgrade to 1.9.2 we had a closer look at the e-mail queue again and weren't able to reproduce the problem. But as we still have no real clue what the problem with 1.9.1 was and as overriding Mage_Core_Model_Email_Template_Mailer::send() still works in the here described way we still aren't using the queue. This way we hope not to run in the same problem again after some time in production.

tl;dr: The email queue isn't working in 1.9.1, commenting out line 76 in Mage_Core_Model_Email_Template_Mailer bypasses the email queue and mails get sent again but this doesn't feel like a good solution. How can this be solved better?

Improve this question
5
  • 1
    How many test transactions did you run compared to how many live transactions came through within the first few minutes? Was this an upgrade from an older version and some files are missing or have improper permissions? How about exception.log or possibly system.log, are there any clues there?
    – pspahn
    Dec 4, 2014 at 2:30
  • It was an upgrade from 1.9.0.1 and it was not done via the Connect-Manager but from the Magento code base so I doubt we have missing files (we also diffed core etc to ensure everything which is not customized or an extension is in place and unmodified and it is). Permissions match the old set up and logs/reports are clean.
    – Jey DWork
    Dec 5, 2014 at 0:43
  • Is cron set up the same?
    – pspahn
    Dec 5, 2014 at 0:50
  • As we saw the email change in the change log we changed cron to run every minute from every 5 minutes before (as we understand the change so that otherwise in the worst case customers would have to wait up to 5 minutes for their confirmation mails). Other than that there is no change and as said before and we see from other jobs cron runs with no problems. //edit: I probably should add that we use (and always used) Aoe_Scheduler where we set core_email_queue_send_all to also run every minute and from where we see that it actually gets executed.
    – Jey DWork
    Dec 5, 2014 at 0:55
  • Q4 2015 and I have the same issue, I can confirm that the queue tables are completely missing entries for some orders, a clear sign to confirm reports that no emails were received. Unfortunately logging was turned off in my case, so I have no errors yet to look for. Have you learned anything new since you originally posted that might be helpful to add?
    – Rick Buczynski
    Oct 4, 2015 at 3:19

2 Answers 2

Reset to default
8

My guess is the setting of cron.php to run every minute has caused a lot of things to stand on top of each other, ie, not finish before the next task scheduled of the same nature or similar is executed. Since both cron.php would not be aware of each state. The same record could be attempted twice causing some odd exception breaking the queue email sends.

With that said there are Mage::Log in the exceptions of the Queue Mailer, so making sure the logging is enabled would be the best step to help determine if theres any exceptions. It may be wise to also just run php -f cron.php from CLI to see if it is also throwing any exceptions, you may not be seeing with it running behind the scenes.

I would also start with a simple PHP mail() test to make sure you're not running into any Spam policies or such. Just to be sure its not something lower in the stack causing the issue.

Just some speculation, hope it helps!

* EDIT *

Use cron.sh instead of cron.php as it will do grep ps to look to see if a previous process is already running.

Improve this answer
6
  • Thanks for the input but these problems can pretty sure be excluded. Just because the actual system cron job runs every minute doesn't mean all Magento cron jobs do. In our case only email was set up to also run every minute and from Magentos cron log we can see that all cron jobs finish successfully - even in "stressful" minutes (i.e. every hour or so when more jobs run at once and not just email). Logging is also enabled and exceptions are not thrown at all. Running into spam filters can be excluded as well as after the workaround I posted all emails do reach all customers.
    – Jey DWork
    Jan 10, 2015 at 1:18
  • You may want to try and enable developer mode to see if it helps generate any exceptions not being logged. Be cautious of it if its running in production however. Also any correlating web server logs?
    – B00MER
    Jan 13, 2015 at 0:37
  • 2
    @JeyDWork have you implemented Aoe_Scheduler? This can give good visibility.
    – benmarks
    Jan 19, 2015 at 13:42
  • 2
    Would like to see an update. The email queue is proving to be a challenge for many.
    – benmarks
    Feb 15, 2015 at 16:07
  • 1
    For me I was using paypal standard which need IPN (Instant Payment Notification) data return from paypal just to confirm payment was success and change the order status to processing/completed and finally send email.. but I didn't have actual domain set up for my server and vhosts to access that was the reason paypal couldn't post IPN data to magento. You can check IPN history in the paypal business account profile. Paypal actually showed the data that was suppose to send out and the status was "retrying"..
    – zaw
    Sep 23, 2015 at 10:30
0

Check if core_email_queue and core_email_queue_recipients has AUTO_INCREMENT. If those table don't have AI enable it will not take new entries.

Improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.