Faulty messages about mobile push limits

Dear Rocket.Chat users,

We sincerely apologize for the inconvenience caused by faulty messages sent about push notification limits through the banner warning and through our bot Rocket.Cat

There was a bug in the deployment and we managed to have it quickly rolled back but unfortunately, every registered server using docker received those messages.

Alerts are fetched for Workspace and then by DeploymentMethod/DeploymentPlatform. The bug was the alerts were created using both workspace and the DeploymentMethod and DeploymentPlatform. Causing those that happen to also match that criteria to also get alerts, and multiple of them.

We will be deducting the mobile pushes caused by this issue from usage, this way it will not count for your mobile push cap.

If you don’t want to receive further notifications and are running version 3.5 or above, you can disable the bot (Rocket.Cat) and banner going under Admin settings > General > Updates > Enable the update checker (turn off).

We recommend keeping it on though since it is an important way for admins to receive alerts. (yes, this one time it went wrong, unfortunately…) If you don’t have that setting, you are probably below v3.5. in which the setting was introduced.

To remove the banners, as a quick fix, you can run a script for automatic dismissal like this one by pasting it in your browser’s developer tools > console.

banners = Object.values(Meteor.user().banners).filter((b) => !b.read && b.id.startsWith('alert-')); function dismiss() { if (!banners.length) { console.log('Finished'); return; } const banner = banners.shift(); Meteor.call('banner/dismiss', {id: banner.id}, (error, result) => { if (!error) { return dismiss(); } console.log('Too many requests, waiting', Math.round(error.details.timeToReset / 1000), 'seconds to continue.'); banners.unshift(banner); setTimeout(dismiss, error.details.timeToReset); }) }; dismiss();

We are reviewing our process of sending communications through the alert system in order to avoid this kind of error from happening again.

Best Regards,

Milton Rucks

1 Like

Hi @milton.rucks, how do we purchase reasonable additional push notifications at a reasonable price?

I think the error is way more serious than just inadvertent banner warnings - all our mobile push notifications seem to be dropped by the server even though we’ve only used 10.5k of our 20k limit. Getting hundreds of these in the log:

server.js:204 Push ➔ error Error sending push to gateway (4 try) -> Error: ESOCKETTIMEDOUT

Going to restart the container later tonight to see if it’s transient, but I don’t see how.

It may be some other issue, seems to be related to communication with our gateway, you would not see that kind of error when the limits are reached, we will never block the connection, but return a correct error message about the limit reached. Let us know if you could solve the issue.

Yeah - I’m going to chalk this up to transient network (routing?) error - restart (and update) of container seems to have resolved this. Thanks @rodrigo.nascimento for the offer.