Only got PASSING alert, never received a FAILING alert
Multiple times, I have gotten StillAlive PASSING alerts without getting a FAILING alert before. This is quite disconcerting to me. If the site was failing, shouldn't I have gotten notified? The last such alert was at 2:51PM ET January 23rd (yesterday). The alert states:
"Success: site bestvendor.com successfully ran recipe Check Shared Page after 694 failures"
My alert threshold is 3 -- I should have definitely gotten a FAILING notification. Looking at the history of notifications, I can also see that that email wasn't lost. According to StillAlive's history, it was never sent. Can you help?
Comments are currently closed for this discussion. You can start a new one.
Support Staff 2 Posted by Mikel Lindsaar on 25 Jan, 2012 01:37 AM
This is alarming! I'll get this looked into now. Thanks for reporting.
3 Posted by magnus on 25 Jan, 2012 07:57 PM
Just to be clear though: it would surprise me if the site had actually been down for that long. 694 is a lot of failures and this happened during the working day. I would expect one of us would have noticed if the site had actually been down. So it's also possible that this is just a false alarm?
In any case, it would be awesome to figure out what's going on. Thanks for looking into that.
4 Posted by magnus on 13 Feb, 2012 12:42 AM
Hi Mikel, any news on this? -- it's been a while.
Lately, this doesn't seem to happen anymore. However, what will happen often now is that I receive only a FAILING notification without the PASSING notification when the site comes back up. That seems to happen with our "Log in" recipe quite a bit.
Support Staff 5 Posted by Mikel Lindsaar on 18 Feb, 2012 12:47 AM
Hi there Mangus,
Sorry for the delay on this. We are swapping out our existing testing infrastructure with a more distributed model testing from multiple locations. I'm pretty sure this is going to handle the problem for you as we found a similar problem in our tests as part of doing the refactor.
However, swapping this out unfortunately is a big job. Almost there :)
Mikel
6 Posted by magnus on 29 Feb, 2012 04:19 PM
Hi Mikel,
Now this is happening the other way around: On our Login script, it will send us a failing notification, but never a passing notification. This has happened a couple of times, and looking at the history I can see that we are not just missing the email, the passing email alert is just never sent. Am I missing anything?
Thanks,
-Magnus
Support Staff 7 Posted by Mikel Lindsaar on 29 Feb, 2012 10:51 PM
Something screwed up is happening here. It's an error in our code I am 99% sure. We'll get to the bottom of it.
8 Posted by magnus on 14 Mar, 2012 03:07 PM
Any news on this? This still seems to be happening.
http://cl.ly/2q2Y1W1U0H1y3R103O2Q
Support Staff 9 Posted by sean on 23 Mar, 2012 07:53 AM
Hi Magnus.
I've been able to replicate your problem while working on the email notification system. It seems to be a interaction between the number of attempts at the running the script and the notification threshold for the script.
We should have fix up for it soon.
Support Staff 10 Posted by sean on 28 Mar, 2012 06:49 AM
Hi Magnus,
This fix for this has been deployed into production. If you still have problems, let us know. And thanks again for reporting this. :)
Cheers
Sean
sean closed this discussion on 28 Mar, 2012 06:49 AM.
magnus re-opened this discussion on 28 Mar, 2012 01:54 PM
11 Posted by magnus on 28 Mar, 2012 01:54 PM
Perfect, thanks for letting me know.
-Magnus
sean closed this discussion on 05 Apr, 2012 04:07 AM.