Events and Notifications

Quick Reference

How can I just stop the system from automatically generating issues? I still want other notifications - for example, when someone adds a comment to an issue - to be sent out.
Open the notifications control panel and make sure the service state is set to "On", and un-check the "Automatic issue generation enabled" option.

How can I just stop the system from automatically generating issues for certain devices only? I still want other notifications to be sent out and issues generated for all other device manufacturers.
Open the notifications control panel and make sure the service state is set to "On" and the "Automatic issue generation enabled" option is checked, and select the manufacturer(s) you don't want issues generated for in the "Disable issue generation for these types of devices" drop-down.

How can I stop notifications from being sent out, including any notifications that were delivered to me but not yet to the user? I may want users to receive emails about some of the events that occur while notifications are off after I turn notifications back on.
Set the notifications service state to Suspended, and on the Pending Emails tab use the Cancel button to stop any pending emails from being delivered. 

How can I stop issues being generated and notifications from being sent out if I don't care about events that occur while notifications are off?
Set the notifications service state to Off, and on the Pending Emails tab use the Cancel button to stop any pending emails from being delivered. No emails will be generated for any events that occur in the future. 

How can have the system only send emails about future events to me?
Set the notifications service state to Test or Debug, and on the Pending Emails tab use the Cancel button to stop any pending emails from being delivered. If you set it to Test mode, the system will only send emails info@manamonitoring.com. If you set it to Debug mode, the system will only email developers about any events that occur while notifications are off.

How can I turn notifications back on if the notifications service state is Suspended?
First check for any events that have not yet been processed on the Suspended Events page and use the Cancel or Cancel All functions to stop them from being queued for delivery. Then set the notifications service state to On.

How can I turn notifications back on if the notifications service state is Off?
Simple, just set the notifications service state to On.

Overview

To understand how notifications work, let's consider the sequence of events that need to occur for someone to be notified when a comment is added to one of their issues:

1. Someone adds a comment to an issue
2. The system stores the comment and records the occurrence of a comment-added event
3. The front-end notifications service regularly checks if there are any events that have not yet been processed, finds our new comment-added event, generates and sends email user notifications for every user that is affiliated with the issue, and marks the event as processed

In this discussion we will be using the term "event" to denote something that occurred that people should be notified about, and "user notification"  to mean an actual email sent to, or notification shown to a specific user on the website. An event is tied to the issue/system/site that it relates to. A user notification is tied to an event and to the user that it is meant for.

In the database, what is described above as an event is called a "notification", and what is described above as a "user notification" is called a "notification of user."

The issue-added event may have any number of user notifications generated for it. A notification currently can be shown to the user through two channels: email or the notifications widget on the site dashboard. 

The Notifications Control Panel

Setting the Notifications Service State

In the Notifications Control Panel we can set the state of the notifications service as follows:

On means that the notifications service is running normally and creating emails from unprocessed events that occurred in the system. If a Client Delivery Delay is set, then these emails are first sent to info@manamonitoring.com and, after the delay has elapsed, sent to the actual user. If no Client Delivery Delay is set or it is set to 0, then email are sent directly to the user without without a delay.

Suspended means that the notifications service is temporarily not processing events. This does not mean that events aren't generated, it only means that no user notifications are created for the event. 

Test mode means that user notifications are being generated, but they are marked as "test" notifications and only delivered to info@manamonitoring.com 

Debug mode means that user notifications are being generated, but they are marked as "debug" notifications and only delivered to developers. 

Off means that no issues are created for system COM_ERR or NOT_PRODUCING errors, and any other event that occurs is discarded; specifically, it is recorded as a deleted event. No user notifications are generated for deleted events.

Use these buttons to turn notifications on or to suspend them etc. Note that changing the notifications service state does not affect user notifications that have already been generated. So, if the service state was On, and you set the service state to Suspended, this will not cancel the delivery of notifications that are pending delivery to the actual user. Similarly, setting the service state to on when previously it was on "Test mode" will not send notifications to users that have already been sent to info@manamonitoring.com. 

Refer to the following table to understand what will happen when you change the service state:

Setting the User Notification Delay

When notifications are on, the system first sends each user notification to info@manamonitoring.com and then, after waiting out a delay, sends the notification to the user (provided it hasn't been cancelled in the meantime). This delay can be set in the Notifications Control Panel:

To configure the system to send notifications to the client without first sending a copy of the pending email to info@manamonitoring.com, and without waiting for a delay to elapse, then set the notifications service state to "On" and set the Client Delivery Delay to 0.

Turning Issue Generation Off

The system is able to automatically generate issues when a system is not producing or not communicating. This issue generation can be turned on or off independently of the state of the notifications service in the notifications control panel:

When notifications are on, but issue generation is off, the website shows the notifications state as "ON!" in orange:

Turning Issue Generation Off for Certain Manufacturer(s)

Issue generation can also be turned off for certain device manufacturer(s). To do so, select the manufacturer(s) from the "Disable issue generation for these types of devices" drop-down:

The website shows the notifications state as "ON!" in orange to warn that this setting is in effect:

Server-side Notifications Kill Switch

There is a kill-switch mechanism to disable notifications in code that can be used by developers when working on the site to make sure that unwanted notifications aren't accidentally sent out. This kill-switch is useful, for example, when aggregation needs to be shut down for a while - we don't want the site to start sending out notifications that systems are down, when really they are not.

When this kill-switch is set, the notifications state shows "DISABLED":

...and the notifications control panel also reflects this:

When the kill switch is set, the following holds true:

• The notifications service state cannot be changed from the website, so all other options are disabled. A developer can set this state by setting \app\models\application\Notification::$sendemails to true or false.
• Send to Me and Send to Dev buttons work
• Debug and Test modes works as usual
• Dev and Test websites still send notifications only internally as usual
• When notifications are "On" but the kill-switch is set, the first email (with the subject ...[Delivery to ... in XX minutes], but the final email that would be sent to the client is actually only sent internally to us (with a subject ... [Simulation / ...])

Deciphering Email Subjects

The subject of a generated email indicates what state the Notifications Service State is in, and if a follow-up email will be sent. 

Managing Events and Notifications

Reviewing Notifications

Administrators can view events and notifications in the site:

Pending Emails shows user email notifications that have been generated and already sent to info@manamonitoring.com but not yet delivered to the actual user. Use this tab to review notifications that will be sent out to users soon. This tab should only contain notifications when the notifications service state is set to "On" or was "On" not long ago. 

Delivered Emails shows all the last 50 user notifications (emails), whether they were delivered to the actual user or sent only to info@manamonitoring.com or to developers. Use this tab to mark test/debug emails for delivery to the user, or to resend an email to the recipient or to yourself.

Suspended Events shows events that have not yet been processed. When the notifications mode is set to "On", "Test", or "Debug", this tab should only show a few events that occurred in the last few minutes, i.e. since when the last time the notification generation service was run (it runs every few minutes). When the notifications service is suspended, you will see here all events that occurred after the service was suspended. Use the Cancel All or the individual Cancel buttons to cancel these events from being processed before turning the notifications state back on.

Canceled Events shows all events that were "canceled". Canceled events are never processed, thus no user notifications are generated from them. An event may be canceled by an administrator using the Cancel All or the individual Cancel buttons, or automatically by the system (for example, if an issue is deleted, any relevant notifications or, for example, a comment being added, are canceled). Use the Restore buttons to restore any canceled events, thus marking them for normal processing. 

User Notification State

The state of the notifications service at the time when a user notification was created determines the type of notification created. The notification type can be seen on the Delivered Emails page, on the top-right corner:

A user notification can be one of the following types:

On: the notification is delivered to info@manamonitoring.com, and after a timeout, delivered to the user. 
Test: the notification is only delivered to info@manamonitoring.com
Dev: the notification is only delivered to the developer(s).

Available Functions for User Notifications

Cancel Delivery: avalable on "On" type notifications that have not yet been delivered to a user, use this to change the notification type to "Test", thus canceling the final delivery of this notification.

Mark for Delivery: available on "Test" type notifications that have not yet been delivered to a user, use this to change the notification type to "On" thus marking it for delivery to the user.

Send to Me/Dev: use this button to immediately send a copy of this notification to yourself or to the developer.

Mark as Delivered: available only on notifications that have not yet been sent to the user, use this button to indicate that the notification has been delivered to the user (i.e. the test email was manually forwarded). The "Sent" date/time of the notification will be set to now.

Resend to Recipient: use this to clear the "sent" date of a notification, thus queuing it to be reprocessed and thus resent to the user.

Available Functions for Events

Cancel / Cancel All: use these buttons to cancel/delete an event. Canceled events are never processed, thus no notification will be generated from the event.

Restore: use this to restore a canceled event. The next time the notifications engine will run, user notifications will be generated for this event.

Types of Notifications

See Mana_MetaData RXX.xlsx

Configuring Notifications

Subscribing to Notification Types

A user can opt to receive various types of notifications generated by the system through the website and/or via email. This needs to be set for the user on the Users worksheet in Initial Data. The possible settings are:

• "ALL": user receives all notifications of the given type (for all sites/issues that he has access to, regardless whether he or she is "subscribed" to the issue or not)
• "Created/Assigned/Watching": user only received notifications for issues that he created, that he is assigned to, or that he subscribed to
• "NONE" or blank: user received no notifications for the given notification type.

Consider the following example for user "John":

This user will:

• Receive "System Status Needs Attention" emails for every system that he/she has access to
• Receive no emails for "System Production Deviance" type alerts
• Receive emails whenever an issue is created for anyone, for every system that he/she has access to
• Receive emails whenever an issue is created for anyone, for every system that he/she has access to
• Receive emails whenever an issue status changes only for those issues that he/she created, is assigned to, or is wacthing
• Receive emails whenever an issue is deleted, for every system that he/she has access to
• Receive emails whenever a comment is added to an issue that he created, is assigned to, or is "watching"

Additionally, the user will be set to automatically start watching any issue that he edits / adds a comment to.

Who gets what?

The following events/actions trigger notifications (see MANA-2919 for a history of how these were determined).

Note that a user who is subscribed to "ALL" notifications of a given type will receive all such notifications regardless of whether he/she is the assignee/creator/is or is not watching the issue. Similarly, a user who is set to receive "NONE" of the notifications of a given type will never receive that type of notification, even if he is the assignee/creator/set to watch the given site. See Subscribing to Notification Types above. 

1. Issue Created for Me Email

Any user who is subscribed to "ALL" "Issue Created for Me Email" notifications (and has access to the given site) will be notified of this event.

Additionally the following users will be notified of this event, provided they are set to receive "Created/Assigned/Watching" "Issue Created for Me Email" notifications (see Subscribing to Notification Types above):

• the issue assignee
• the issue creator
• Anyone who opts to "watch" the issue

2. Issue Assigned to Me Email

Any user who is subscribed to "ALL" "Issue Assigned to Me Email" notifications (and has access to the given site) will be notified of this event.

Additionally the following users will be notified of this event, provided they are set to receive "Created/Assigned/Watching" "Issue Assigned to Me Email" notifications (see Subscribing to Notification Types above):

• issue assignee
• Issue creator
• Anyone who opts to "watch" the issue

3. Issue Status Changed Email

Any user who is subscribed to "ALL" "Issue Status Changed Email" notifications (and has access to the given site) will be notified of this event.

Additionally the following users will be notified of this event, provided they are set to receive "Created/Assigned/Watching" "Issue Status Changed Email" notifications (see Subscribing to Notification Types above):

• issue assignee
• Issue creator
• Anyone who opts to "watch" the issue

4. Issue Deleted Email

Any user who is subscribed to "ALL" "Issue Deleted Email" notifications (and has access to the given site) will be notified of this event.

Additionally the following users will be notified of this event, provided they are set to receive "Created/Assigned/Watching" "Issue Deleted Email" notifications (see Subscribing to Notification Types above):

• issue assignee
• Issue creator
• Anyone who opts to "watch" the issue

5. Issue Comment Added Email

Any user who is subscribed to "ALL" "Issue Comment Added Email" notifications (and has access to the given site) will be notified of this event.

Additionally the following users will be notified of this event, provided they are set to receive "Created/Assigned/Watching" "Issue Comment Added Email" notifications (see Subscribing to Notification Types above):

• issue assignee
• Issue creator
• Anyone who opts to "watch" the issue

6. System Status Needs Attention Email

Any user who is subscribed to "ALL" "System Status Needs Attention Email" notifications (and has access to the given site) will be notified of this event.

Additionally the following users will be notified of this event, provided they are set to receive "Created/Assigned/Watching" "System Status Needs Attention Email" notifications (see Subscribing to Notification Types above):

• "Primary User" of "System Owner" company of the site

7. System Production Deviance Email

Any user who is subscribed to "ALL" "System Production Deviance Email" notifications (and has access to the given site) will be notified of this event.

Additionally the following users will be notified of this event, provided they are set to receive "Created/Assigned/Watching" "System Production Deviance Email" notifications (see Subscribing to Notification Types above):

• "Primary User" of "System Owner" company of the site

Watching an Issue

A user can manually start to watch an issue by clicking the "Start watching this issue" link on the issue for which he wishes to receive notifications:

Note: as specified in "Who gets what?" above, some users receive notifications about an issue regardless whether they are watching it or not. For example, the issue assignee always gets an email about an "Issue Created for Me" event unless this event is turned off for him in initial data.

Autowatch

The "Autowatch" feature was implemented in MANA-2919 / MANA-3538 and works as follows: a user who has "Autowatch" activated will automatically start watching an issue if he adds a comment or edits the issue etc.

This setting can be set in Initial Data, on the "Users" table, by setting the "Automatically start watching issues I interact with?" column to "X":

Configuring the Default Email Subject and Body

This can be set in the Notifications worksheet of the current Mana_MetaData RXX.xlsx workbook.

Configuring Custom Email Subject and Body for a Client

This can be done on the Notifications by Client worksheet of the current Mana_MetaData RXX.xlsx workbook.

Technical Reference

In this discussion we will be using the term "notification" to denote something that occurred that people should be notified about, and "notification of user" to mean an actual email sent to, or notification shown to a specific user on the website.

What types of notifications are recorded by the system?

• When an issue is created (either automatically or by a user), a SystemStatusComErr, SystemStatusNotProducing, or an IssueCreatedForMe type notification is recorded. See sp_entity_insert_issue
• When a user changes the status of a notification using the Confirm/Close/Reopen buttons, a IssueStatusChanged type notification is recorded. See sp_issue_action
• When a user assigns an issue to someone, an IssueAssignedToMe type notification is recorded. See sp_issue_assign
• When a user comments on an issue, or when a user edits an existing comment, an IssueCommentAdded type notification is recorded. See sp_issue_comment_add and sp_issue_comment_update
• When an issue is automatically closed, for example, when a NOT PRODUCING issue is closed because the system determines that communications have been restored, a SystemStatusComErrResolved, SystemStatusNotProducingResolved, or IssueStatusChanged type notification is recorded. See sp_autoclose_issues
• When an issue is deleted, the user give a reason. This reason is attached to the issue as a comment, and an IssueDeleted type notification is recorded. See sp_entity_delete_issue