Integrate Opsgenie with ITRS OP5 Monitor

Overview

What does the integration offer?

Opsgenie provides a powerful bidirectional integration with OP5. When a new alert - host or service - is created in OP5, a corresponding Opsgenie alert is automatically created, containing rich information about the alert. Opsgenie provides detailed notifications with on-call rotations, scheduling features, and alert escalations. Users select any of the alert actions of Opsgenie to map to the Acknowledge action of OP5.

How does the integration work?

OP5 to Opsgenie

When a host or service alert is created on OP5, an alert is created in Opsgenie.
When a host or service alert is closed on OP5, the corresponding alert is closed in Opsgenie.
When a host or service alert is acknowledged on OP5, the corresponding alert is acknowledged in Opsgenie.

Opsgenie to OP5

If Send alert updates back to OP5 is selected, actions that are run on OP5 alerts are sent to OP5 as acknowledgment actions.

Set up the integration

OP5 is a bidirectional integration. Setting it up involves the following steps:

Add an OP5 integration in Opsgenie
Map alert actions
Configure the integration in OP5

Install the Opsgenie plugin for OP5

Opsgenie Edge Connector (abbreviated as OEC) is a prerequisite for configuring the outgoing authentication of OP5 integration. Download the latest version of the OP5 package from this repository.

For OP5 Monitor's Red Hat-based distributions

Either share the OP5 file to the remote OP5 monitor or copy the link on the opening page.

Run the following command:
wget https://linkTo.the.OP5Package
Run the following command :
rpm -i opsgenie-op5-<your_version>.rpm

Before rpm upgrades, back up your configuration files.

During upgrades, the rpm package does not overwrite the existing configuration. It saves the new default configuration file as integration.conf.rpmnew. To update from version 201X-XX-XX to 2.X.X, add the --force parameter. For example: rpm -U --force opsgenie-integration-<your_version>.rpm

Learn more about rpm upgrade config file handling.

Add an OP5 integration

If you're using Opsgenie's Free or Essentials plan or if you’re using Opsgenie with Jira Service Management's Standard plan, you can add this integration from your team dashboard only. The Integrations page under Settings is not available in your plan.

Go to Teams and select your team.
Select Integrations on the left navigation and then select Add integration.

Adding the integration from your team dashboard will make your team the owner of the integration. This means Opsgenie will assign the alerts received through this integration to your team only. Follow the rest of the steps in this section to set up the integration.

Go to Settings > Integrations.
Select Add integration.
Run a search and select “OP5”.
On the next screen, enter a name for the integration.
Optional: Select a team in Assignee team if you want a specific team to receive alerts from the integration.
Select Continue.
The integration is saved at this point.
Expand the Steps to configure the integration section and copy the integration API key.
You will use this key while configuring the integration in OP5 later.
Select Turn on integration.
The rules you create for the integration will work only if you turn on the integration.

Configure OP5 package in OP5 Monitor

The plugin uses a golang-executable file (included in the plugin as send2opsgenie) to create, acknowledge, and close alerts in Opsgenie. Configure OP5 to execute this file on events to create, acknowledge, and close alerts in Opsgenie. Setting the apiKey is mandatory. Other configuration parameters are set to defaults that work with OP5.

The following table lists the parameters and their locations:

Configuration Parameters		Location
apiKey	Copy the API key from the OP5 integration created above. send2opsgenie uses this key to authenticate to Opsgenie. API key is also used to identify the right integration configuration that should be used to process alerts.	/home/opsgenie/oec/conf/config.json
baseUrl	If you're using Opsgenie from another domain(eg. EU, sandbox), you should update this configuration.Opsgenie OP5 integration, Advanced Settings page.	/home/opsgenie/oec/conf/config.json
responders	Responders field is used to specify which teams should be notified for the OP5 alerts. This field is used to set the default teams field value. Modify to route different alerts to different teams in Opsgenie OP5 integration's Advanced Settings page.	/home/opsgenie/oec/conf/opsgenie-integration.conf
tags	Tags field is used to specify the tags of the alert that created in Opsgenie.	/home/opsgenie/oec/conf/opsgenie-integration.conf
logPath	Specifies the full path of the log file. (Default value is /var/log/opsgenie/send2opsgenie.log)	/home/opsgenie/oec/conf/opsgenie-integration.conf
nagios2opsgenie.http.proxy.enabled	nagios2opsgenie.http.proxy.enabled field is to enable/disable external proxy configuration. The default value is false.	/home/opsgenie/oec/conf/opsgenie-integration.conf
nagios2opsgenie.http.proxy.host	It is the host of the proxy.	/home/opsgenie/oec/conf/opsgenie-integration.conf
nagios2opsgenie.http.proxy.port	It is the port of the proxy.	/home/opsgenie/oec/conf/opsgenie-integration.conf
nagios2opsgenie.http.proxy.scheme	It is the proxy connection protocol. It may be http or https depending on your proxy servers. Its default value is http.	/home/opsgenie/oec/conf/opsgenie-integration.conf
nagios2opsgenie.http.proxy.username	It is the Proxy authentication username.	/home/opsgenie/oec/conf/opsgenie-integration.conf
nagios2opsgenie.http.proxy.password	It is the Proxy authentication password.	/home/opsgenie/oec/conf/opsgenie-integration.conf

Configure the golang-executable file in any of the following three methods:

Method 1: Configure from conf file

Configure from the /home/opsgenie/oec/conf/integration.conf file. This overwrites any configuration you previously made in the script.

Method 2: Configure by using Golang flags

Configure by entering flags into the command in the opsgenie.cfg file. Use -apiKey flag for your apiKey and -ns flag for nagios_server name. If multiple OP5 servers are not in use, there is no need to define the nagios server. Using flags overwrites all the other configuration methods discussed earlier in the article.
Configure the apiKey from the cfg file as follows:

define command {
    command_name    notify-service-by-opsgenie
    command_line    /home/opsgenie/oec/scripts/send2opsgenie -apiKey="apiKey1" -entityType=service ...
}

When apiKey is added to the cfg file, it overrides the apiKey in the config.json file.

To send additional custom arguments, add them after the flags. For example, customArgName1 customArgValue1 customArgName2 customArgValue2. Parse custom arguments by adding {{_payload.customArgName}} to wherever is needed in the input fields. To learn more about using raw parameters, see dynamic fields.

Method 3: Configure from script

Configure apiKey and nagios_server from send2opsgenie.go script. To use this option, build the script again and put the new executable to /usr/bin directory. Find information about the location of the send2opsgenie.go and how to build a go script in the "Source for send2opsgenie and recompiling" section.

Define Opsgenie as Contact

Log in to your OP5 monitor.
Go to the /opt/monitor/etc directory to find nagios.cfg.
cd /opt/monitor/etc
Add the following line to the main Nagios configuration file (nagios.cfg).
1 2 3... cfg_file=opsgenie.cfg ...
Reboot the monitor to see the "opsgenie" contact in the contact list.
Add the contact "opsgenie" to the OP5 Monitor’s main contact group.

If everything goes well, alerts are created in Opsgenie for every notification created in OP5.

Map alert actions

You can define mappings between Opsgenie actions and OP5 actions (also when the source of the alert is OP5), which requires additional authentication for your OP5 account.

Username: The username of your OP5 account
Password: The password of your OP5 account
OP5 Monitor URL: This is the domain name of your OP5 Monitor. For example: https://19.167.1.143

For alerts created by OP5

In the Send alert updates back to OP5 section, map Opsgenie actions to OP5 actions when the source of the alert is OP5 (when the alert is created by OP5 integration itself). Map different Opsgenie actions to different OP5 actions. For example, acknowledge the alert in OP5, when the alert is acknowledged from OP5. To do this, define If alert is acknowledged in Opsgenie, acknowledge in OP5 mapping in Send alert updates back to OP5 section.

Configure Opsgenie to update OP5

This step is optional.

Use Opsgenie’s OEC and the OP5 script to update alerts on OP5. This enables deployment of your own scripts/ability to modify the ones provided and execute customized actions on OP5. To execute actions in OP5, OEC gets the configuration parameters from the configuration file. The configuration file is found under /home/opsgenie/oec/conf/oec-config.json.

Configuration parameters

url – OEC posts alert actions to an endpoint which is constructed using this URL.
username – OEC uses your OP5 account's username to authenticate.
password – OEC uses your OP5 account's password to authenticate.

The package that you downloaded also includes OEC utility which is located under /usr/local/bin and the script that is needed to be run by OEC which is under /home/opsgenie/oec/scripts. After the configuration of OEC is done, be sure to run it. Learn more about running OEC.

Source for and recompiling send2opsgenie

The source for the executable send2opsgenie is found in /usr/bin/ and send2opsgenie.go, in /home/opsgenie/oec/scripts respectively and is also available in this repository. To change the behavior of the executable, edit send2opsgenie.go and build it by using the following command: go build send2opsgenie.go

For installing go, refer to http://golang.org/doc/install. Note that the executable in the plugin is built for linux/amd64 systems.

FAQ and troubleshooting

If the integration is not working, review this section and follow the guidance prescribed:

1. OP5 alerts are not getting created in Opsgenie

Run the following test command from the shell. Check if the test alert is created in Opsgenie:

/home/opsgenie/oec/scripts/send2opsgenie -entityType=host -t=PROBLEM -hs=DOWN -hn=test_host

If "Trace/breakpoint trap" error occurs: It means the send2opsgenie plugin isn't compatible with the server distribution. Follow the "Source for and recompiling send2opsgenie" section on this page and rebuild send2opsgenie.go according to the specific server environment.

If the alert is created in Opsgenie: It means the integration is configured correctly. The problem might be that OP5 is not notifying the Opsgenie contact for alerts. Check the OP5 alert notifications log.

If not: Check the log files under /var/log/oec. Look for the following errors in the log file:

If a "RestException[Could not authenticate.]" error appears in the logs, it means Opsgenie couldn't identify your API key. Check if the API key is set correctly, as explained in "Configure OP5 Package in OP5 Monitor" on this page.
If "Could not execute this action with apiKey of [OP5] integration" appears in the logs, the wrong integration package may have been downloaded. Make sure to download the OP5 integration package.
If unsure of the problem, set the plugin's log level to debug, try again, and contact us and send the logs. If there is no log files under /var/log/oec file, or there are no logs in it, check the following:
- First, make sure the nagios user has permission to write to /var/log/oec directory. The installation package automatically does this for you. If you encounter problems, execute chown -R nagios:nagios:opsgenie /var/log/oec.
- Now check the OP5 server logs under /var/log/oec. See if there are error logs regarding send2opsgenie, and contact us with them.

Set log level of send2opsgenie plugin to DEBUG

Change the line send2opsgenie.logger=warning to nagios2opsgenie.logger = debug in /home/opsgenie/oec/conf/integration.conf file.

Configure Opsgenie to update Zenoss

Set the send2opsgenie plugin's log level to DEBUG. Open the /home/opsgenie/oec/conf/integration.conf file and change the line zenoss2opsgenie.logger=warning to zenoss2opsgenie.logger=debug.

2. The OP5 alert is not acknowledged when acknowledged in Opsgenie

First, check your alert logs.

If a "Posted [Acknowledge] action to OP5.." error does not appear in the log, it means Opsgenie didn't send the Acknowledge action to OP5. Check the integration configuration, it might not have matched the alert action.
If a "Executed [Acknowledge] action via OEC with errors." error appears in the log, it means the op5ActionExecutor.groovy script in OEC has encountered an error. Check the log files under /var/log/opsgenie/ for error logs.
If the "Posted [Acknowledge] action to OP5.." is the only error to appear in the log and no related log after that, it might mean OEC is having connection problems. Check the log files under /var/log/opsgenie/ for error logs.

If no logs exist, restart OEC and try sending an Acknowledge action again.

3. Could not open OP5 RPM package

If you figure out while installing the rpm package that the package is obsolete, use rpm -i opsgenie-op5-1.0.4-rpm-x86-64.rpm --nodeps instead.
If you get "is already installed" error, use rpm -i opsgenie-op5-1.0.4-rpm-x86-64.rpm --force instead.

If unsure of the problem, set the OEC's script log level to “debug” and try again. Contact us with the log files found under /var/log/opsgenie/.

Was this helpful?

It wasn't accurate

It wasn't clear

It wasn't relevant

Still need help?

The Atlassian Community is here for you.

Ask the Community