Queued (Pending) Notifications are never sent

[raamdev]

I was testing Comment Mail Lite v151224 after importing StCR subscriptions when I tried posting two new replies to existing comments with subscriptions (i.e., I posted new replies to comments that had StCR subscriptions, which were imported into Comment Mail). The new Comment Mail subscriptions were created fine, and were connected to the appropriate imported StCR subscription, however the notifications for those new replies were never sent.

Looking at the Mail Queue, I noticed the two notifications were sitting there, unsent:

Notice the time on them... it had been ~45 minutes since the notifications were created and they still have not been sent. Here's what I tried to get the email queue to be processed:

• Tried manually running the _cron_comment_mail_queue_processor event
• Tried disabling the two email-related plugins that I had installed (WP Mail SMTP and Email Log) and then manually running the _cron_comment_mail_queue_processor event.
• Tried disabling/enabling Comment Mail
• Tried disabling Mail Queue Processing in Comment Mail → Config. Options → Enable/Disable and then enabling it again.

None of these things helped. The queued notifications remained queued and were not sent out.

Also, my PHP error log did not report any errors during this time.

[raamdev]

Note: I don't believe this is related to #194 (Cron event disappears in some scenarios: _cron_comment_mail_queue_processor) because as noted above the notifications in the queue would not send even when I was able to manually run the _cron_comment_mail_queue_processor event.

[jaswrks]

None of these things helped. The queued notifications remained queued and were not sent out.

Also try looking at the Queue Event Log to see if anything turns up there after you run the queue processor. If they were not processed that should show up in the log as having been invalidated for some reason. Also be sure to look over the available Screen Options in the Queue Event Log, because there is a lot of information available there if you tell WordPress to expose it.

Also referencing: https://github.com/websharks/comment-mail-pro/blob/151224/comment-mail-pro/includes/classes/queue-processor.php#L503

[raamdev]

@jaswsinc The Queue Event Log shows nothing:

[raamdev]

@kristineds @renzms @Reedyseth I could use some help attempting to reproduce the issue described above. 😄

[renzms]

@raamdev

Under my Events list it says notified, but I may need to replicate it exactly as you did by using a fresh install of Comment Mail Lite v151224 right after using StCR subscribe and to import users.

I imported users from StCR a week ago, and the Comment Mail subscriptions were working as normal.

[raamdev]

@renzms Any more progress on testing this one?

[renzms]

@raamdev

• Started over from a clean version.
• Installed Comment Mail Lite v151224
• Used same subscribers from StCR import
• Added new comments and replies
• Notifications were sent (some went to Spam, but that's probably SMTP related)
• Wasn't able to replicate what you experienced.

@kristineds @Reedyseth Could you guys please try as well and see if you can replicate what Raam has noted?

[kristineds]

@raamdev: Tested this but wasn't able to reproduce the issue you pointed out.

• Installed Comment Mail Lite v151224 + WP v.4.4.1 + StCR v160115 on a clean install
• Successfully imported StCR subscribers
• Posted new replies to comments that had StCR subscriptions, which were imported into Comment Mail
• Notifications were sent according to the Comment Mail Queue Event Log

• No unsent notifications (nothing under Queued (Pending) Notifications)

[raamdev]

@renzms @kristineds Thanks so much for testing this some more. I'm going to close this as cannot reproduce for now. I was seeing this on my personal raam.org site where there are several other plugins running, so it sounds like what I was seeing might be related to a plugin conflict.

I'll come back and reopen this if I'm able to reproduce it. 😄

[raamdev]

Reopening. Also reported here: https://wordpress.org/support/topic/why-are-some-emails-awaiting-processing?replies=1

[bobinoz]

@raamdev Yes, I'm happy to test it, and I've sent you an email.

[aptharsia]

Tried the dev release, still have the issue of notifications being stuck in processing mode and nothing gets sent.

[raamdev]

@aptharsia writes...

Tried the dev release, still have the issue of notifications being stuck in processing mode and nothing gets sent.

Are these old notifications that were in the queue before installing the dev version of Comment Mail, or did you install the dev version and then generate some new notifications to see if they'd get stuck? (In my test, I generated new notifications and they didn't get stuck like they had in previous tests.)

@bobinoz writes...

Yes, I'm happy to test it, and I've sent you an email.

I replied to your email.

[aptharsia]

I cleared any notifications still waiting, enabled and generated new notifications that were never sent out.

[raamdev]

@aptharsia Thank you for testing and reporting your results. I'll need to run some more tests to see if I can reproduce this again.

@bobinoz When you get the dev version of Comment Mail that I emailed you, it would be great to hear if you're still seeing this issue.

[bobinoz]

So I have installed Comment Mail and tested it briefly this afternoon, comments are still being held in the queued (pending) notifications area with the message 'n/a; awaiting processing'. However, they are getting sent, there just seems to be a long delay.

After I had installed and activated Comment Mail, I started answering comments on various pages with different numbers of subscribers. It sent out 76 emails immediately but then after that it started holding them in the queue for processing. What it seems to be doing since then is sending out little batches of emails every now and then at around 10 or 20 minute intervals. The most emails it seems to send out from the queue any one time is 21. Sometimes it is as few as 5. The maximum gap between sending them appears to be 30 minutes.

In the space of about seven hours, it has managed to send out all 363 emails in the queue which is now empty as it's late at night here and new comments have stopped coming in. If it were a busy day though, I can imagine the queue just building up until eventually it could take seven or eight hours before catching up with itself.

When I stopped answering comments, I think there were about 150 comments stuck in the queue, it has taken about three hours for those to slowly clear. So I'd say it's working now, but for some reason something is stopping it sending out all the comments at once. I am using SparkPost to deliver email, they do not have any hourly limits. As far as I'm aware, they just have a daily limit of 10,000 emails.

So if you have any clues how to fix it so they will get sent out immediately, we might be there.

I have noticed an unrelated problem as well, I hope you don't mind me mentioning it here. When I am logged in as admin but working on the front end, and I click on 'manage my subscriptions', with StCR I would see a list of subscribers for that particular post with tick boxes for them all so I could add, remove or delete them. When I click on 'manage my subscriptions' with Comment Mail, I don't see them at all, I just see that I am subscribed to all posts with my email address.

I can't find anywhere where I can view or amend subscriptions to a particular post or page. All I appear to be able to do is go to comment mail - subscriptions in the backend and view a complete list of subscribers. Is there anywhere I can go to quickly see all subscribers to a particular post or page?

Anyway, hope this information helps, and for now I am keeping Comment Mail activated. Thanks, Bob

[raamdev]

@bobinoz writes...

So I'd say it's working now

Woohoo! That's great to hear!

@bobinoz writes...

for some reason something is stopping it sending out all the comments at once.

Have you reviewed the Queue Processor Adjustments panel to see if those settings are appropriate? Comment Mail → Config. Options → Queue Processor Adjustments. You probably need to adjust those settings so that more emails get sent out at once.

@bobinoz writes...

I can't find anywhere where I can view or amend subscriptions to a particular post or page. All I appear to be able to do is go to comment mail - subscriptions in the backend and view a complete list of subscribers. Is there anywhere I can go to quickly see all subscribers to a particular post or page?

You can see a list of Comment Mail subscriptions for a post by editing the post itself and viewing the Comment Mail Meta Box, which will list all of the Comment Mail subscriptions associated with that post:

Clicking "View All" at the bottom will bring you to the Comment Mail Subscriptions page, with the subscriptions filtered to just the subscriptions for that post.

[aptharsia]

Over about a 15 hour time span, 73 pending notifications only 12 were sent out. Many have been pending for over 12 hours. Nothing looks to be instant (or according to my settings) once moderator approved, for the 12 that got sent out, took 12 or so hours for the ones I timed. I wouldn't say it's fixed it so hope this issue will still be looked at.

[aptharsia]

Adding. The 12 that were sent has a 14 hour difference between the two sets sent yet there are 15 hour waits queued to be sent so it looks random on what gets sent and what queues for processing.

Edited: I noticed it was 6 notifications sent out the two times it happened with 14 hours between the two sets.

[raamdev]

@aptharsia Thank you for the additional information. I'll be keeping this issue open until I can run some more tests and get to the bottom of this. The template parsing changes that Jason made related to this issue should help us with debugging. I just need to work at reproducing this issue in a test install because my test install that previously wasn't working is now suddenly working, so I need to keep testing.

[jaswrks]

@aptharsia After the recent changes in this issue, what we are now looking for to diagnose the why, are your PHP error logs. If you can check with your hosting company or already have access to these logs, that would be much appreciated.

To clarify, with the improved template parsing as of late, we'd like to inspect any PHP errors being reported when this occurs, as that could shed light on a real solution; i.e., help us understand under what conditions this occurs and what error causes the queue to get stuck.

I have followed the steps that you outlined on a few different occasions now, and have been unable to reproduce. So whatever causes this remains a mystery for now. Until we can see PHP error logs that might unveil the underlying cause, I'm afraid this could remain open.

If you check your PHP error logs and find nothing, we'd like to know that also.

[bobinoz]

@raamdev Thanks for the information, yes, I can now see where I can view and amend subscriptions associated with particular posts and pages.

I've also check the settings you mention Comment Mail → Config. Options → Queue Processor Adjustments, but cannot make changes as that's only available in the pro version. However, the settings seem good to me, if I am reading it correctly it should send 100 emails at a time and do that every five minutes.

Any idea why it isn't able to do that?

[raamdev]

@bobinoz Please open a separate GitHub issue for that, so that we don't go off topic in this one.

[aptharsia]

Is the _cron_comment_mail_queue_processor in WP Cron suppose to have a hook or argument or is that suppose to be blank? I've been watching the PHP error logs but nothing has shown since the last run 5 hours ago.

[raamdev]

@aptharsia There are no arguments, no. See screenshot below from WP Crontrol:

[aptharsia]

23 hours since last notifications sent, the rest are still in the queue. Nothing is on the error log so far, does that mean it's not even trying to send them all this time?

[aptharsia]

I spoke too soon. It finally did another batch after 24 hours. I checked the error log and no errors for this plugin. Yesterday it did it twice with 25 minutes between the two batches and then nothing until 24 hours later.

[raamdev]

@aptharsia Could it be that the notifications were notifications that were part of a Daily Digest subscription that were supposed to be held for 24 hours?

[renzms]

@raamdev

Was unable to reproduce original issue. Notifications eventually sent in batches.

[aptharsia]

@raamdev No, I checked and everyone is instant, no digests. There is inconsistency with when they are being sent out. Yesterday was 24 hours, today there was 16 between the most recent and last time sent.

Edited: After 16 hours, 8 emails were sent out. Then after only 8 minutes 5 emails were sent out (although there were many more queued.)

[raamdev]

@aptharsia Thanks for the continued updates.

@jaswsinc Anything come to mind when you review the reports from @aptharsia above? Maybe we need some more detailed logging routines to help diagnose this issue for site owners that seem to have issues with notifications being sent irregularly?

[raamdev]

@aptharsia I just noticed that @bobinoz opened a separate GitHub issue explaining the same sort of irregular notification behavior; could you please post further updates about that in that other GitHub issue so that we can keep better track of this? See #317

@jaswsinc If you could post your thoughts over in #317 instead of here, that would be better. I think the core issue described here, with pending notifications never being sent, has been resolved after changing the way templates are parsed. Now the issue seems to be more about when notifications are sent.

This GitHub issue is getting really long and I think it's starting to stray from the issue that was originally reported, so I'd like to redirect things to #317 so that I can close this issue as of the next release.

[jaswrks]

@aptharsia Thank you.

@raamdev Copy that. I followed up in #317 (comment)

[raamdev]

Comment Mail v161118 has been released and includes changes from this GitHub Issue. See the v161118 announcement for further details.

---

This issue will now be locked to further updates. If you have something to add related to this GitHub Issue, please open a new GitHub Issue and reference this one (#192).