Replication Setup and Configuration Guide

opHA-MB 5.0 introduces the concept of a Primary & multiple Poller servers and a message bus (with replication):

• The Primary is the server that keeps the information of all the pollers and it is where we can read all the information from. The Consumer (ophad running on Primary) reads data continuously off the message bus and stores it in the database

• The Producer (ophad running on Pollers) has a change stream setup for the collections and receives update from the database if there is any. This change stream data is pushed onto the message bus.

• With Replication and Nats if the Main Primary goes down the secondary primary will take over also if a poller goes down the mirror will take over.

Open

Here are terms used in this project together with their meanings.

Software Installation

opHA-MB 5.1.0 can be installed on infrastructure

• New to opHA/opHA-MB ----- Install to new servers

• Running opHA 4.X ----- Upgrade existing servers, running opHA 4.x

Installation of opHA-5.1.0 that does not have opHA software installed

Prerequisites

IMPORTANT: All the servers should be setup with the following

c. The role of the server on which opHA-5.1.0 needs to be installed should be known and appropriately set (opha_role) in opCommon.json

Editing the Role of the instance in opCommon.json

The default value of opha_role in opCommon.json is Standalone

Edit /usr/local/omk/conf/opCommon.json to change in to one of

• “Main Primary”

• “Primary”

• “Poller”

• “Mirror”

Installing opHA-MB license

Install the opHA-MB for NMIS & opCharts license on the Main Primary server only.

To obtain a license please contact FirstWave Sales.

Currently there are 2 available licenses and Message Bus will not run without a license.

• opHA-MB for NMIS & opCharts (This license is required for Message Bus to run).

Open

• opHA-MB for opEvents (This license is an add-on for opEvents).

Open

You need to enter a License Key in the Modules → opLicensing as below on the Main Primary.

Open

Download Software

Download the required software onto each server.

Install Software

We will be using the below example host names throughout this guide.

You would replace with your own where applicable. For this example we have the required 6 servers for the setup.

IMPORTANT:

One server should be set as an Arbiter and in this example we are using Poller1 Server

Main Primary Server: opha-dev1.opmantek.net

Poller1 Server: opha-dev2.opmantek.net [Arbiter Server]

Mirror1 Server: opha-dev3.opmantek.net

Poller2 Server: opha-dev4.opmantek.net

Mirror2 Server: opha-dev5.opmantek.net

Second Primary: opha-dev6.opmantek.net

Run the installation commands on each server as needed.

All the following setup commands in this setup should be run as the root user

e.g

During the installation of the arbiter server you must answer 'n' at the following prompt

Open

NATS Setup

This configuration should be applied to the Main Primary, Arbiter and Secondary Primary servers ONLY.

1. Edit Configuration: Update the /etc/nats-server.conf file with the following settings:

   1. On Main Primary:

      1. server_name: opha-dev1.opmantek.net

      2. host: "opha-dev1.opmantek.net"

      3. routes: [
         # secondary primary
         "nats://opha-dev6.opmantek.net:6222"
         # arbiter
         "nats://opha-dev2.opmantek.net:6222"
         ]

   2. On Secondary Primary:

      1. server_name: opha-dev6.opmantek.net

      2. host: "opha-dev6.opmantek.net"

      3. routes: [
         # main primary
         "nats://opha-dev1.opmantek.net:6222"
         # arbiter
         "nats://opha-dev2.opmantek.net:6222"
         ]

   3. On Arbiter:

      1. server_name: opha-dev2.opmantek.net

      2. host: "opha-dev2.opmantek.net"

      3. routes: [
         # main primary
         "nats://opha-dev1.opmantek.net:6222"
         # secondary primary
         "nats://opha-dev6.opmantek.net:6222"
         ]

2. Sample file for Main Primary:-

   server_name: "opha-dev1.opmantek.net" #The local server http_port: 8222 listen: 4222 jetstream: enabled #tls { # cert_file: "<path>" # key_file: "<path>" # #ca_file: "<path>" # verify: true #} log_file: "/var/log/nats-server.log" accounts { $SYS { users: [ { user: "admin", pass: "password" } ] } ophad: { users: [ { user: "omkadmin", password: "op42opha42" } ] jetstream: enabled } } jetstream { store_dir: "/opt/nats/storage" max_memory_store: 1028M max_file_store: 1028M } cluster { name: "C1" host: "opha-dev1.opmantek.net" #The current host # the current server listen: "0.0.0.0:6222" routes: [ # secondary primary "nats://opha-dev6.opmantek.net:6222" # server with the arbiter "nats://opha-dev2.opmantek.net:6222" # other servers ] }

    

Common Configuration for All Servers

Update /usr/local/omk/conf/opCommon.json

1. Backup Configuration: Make a backup copy of the opCommon.json file before making any changes.

2. Add NATS Cluster Configuration: Ensure the following configuration is included in opCommon.json on all servers. In the below example we have added the Main Primary, arbiter and Secondary Primary FQDNs and port/s to be utilized in the nats_cluster attribute, you can find this in the database section.

   "nats_cluster": [ "opha-dev1.opmantek.net", "opha-dev2.opmantek.net", "opha-dev6.opmantek.net" ]

    

Server-Specific Configuration

Update /usr/local/omk/conf/opCommon.json for Main Primary and Secondary Primary only. For key “database” add the following information:

db_mongo_cluster is cluster of ‘Main Primary’ and ‘Secondary Primary’

MongoDB Setup (Main Primary Only)

The following steps should be performed only on the Main Primary MongoDB instance.

Initialize MongoDB Replica Set

Connect to MongoDB using the mongosh command and configure the replica set:

Run the following command

Add the main-primary and secondary-primary to the below command and run it

Set Default Write Concern

Run the following command to set the default write concern:

Add Arbiter

Run the following command to add an arbiter to the replica set:

Restart mongod on the Main Primary, Second Primary and Arbiter

Run the following command on the Main Primary, Secondary Primary, and Arbiter servers ONLY:

Starting Services

Start NATS

Start the NATS service on the Main Primary, Secondary Primary, and Arbiter servers ONLY:

Restart mongod on Pollers and Mirrors only

After configuration changes, restart mongod on all Mirrors and Pollers servers ONLY:

Restart services on all servers

After configuration changes have been made, you will need to restart the relevant FirstWave module daemons applicable to your server.

For example if you have NMIS9, opCharts, opEvents and opHA 5 installed you would execute as follows:

Start the Poller and Mirrors

Execute the following command on the poller and mirror servers.

Final Steps

1. Accept EULAs: Make sure when you login to each server to confirm that all End User License Agreements (EULAs) are accepted.

Discover the Peers

Click on Peers in opHA-MB portal on the Main Primary (http://<fqdn of Main Primary>/en/omk/opHA/peers)

Open

Proceed to next step after discovering all the Peers.

Upgrade to opHA-5.1.0 from opHA 4.X

Prerequisites

IMPORTANT: All the servers should be setup with the following

c. The role of the vm on which opHA-5.1.0 needs to be installed should be known and appropriately set (opha_role) in opCommon.json

Editing the Role of the instance in opCommon.json

The default value of opha_role in opCommon.json is Standalone

Edit /usr/local/omk/conf/opCommon.json to change in to one of

• “Main Primary”

• “Primary”

• “Poller”

• “Mirror”

Installing opHA-MB license

Install the opHA-MB for NMIS & opCharts license on the Main Primary server only.

To obtain a license please contact FirstWave Sales.

Currently there are 2 available licenses and Message Bus will not run without a license.

• opHA-MB for NMIS & opCharts (This license is required for Message Bus to run).

Open

• opHA-MB for opEvents (This license is an add-on for opEvents).

Open

You need to enter a License Key in the Modules → opLicensing as below on the Main Primary.

Open

Download Software

Download the required software onto each server.

Install Software

IMPORTANT: All the servers should be setup with the following

We will be using the below example host names throughout this guide.

You would replace with your own where applicable. For this example we have the required 6 servers for the setup.

IMPORTANT:

One server should be set as an Arbiter and in this example we are using Poller1 Server

Main Primary Server: opha-dev1.opmantek.net

Poller1 Server: opha-dev2.opmantek.net [Arbiter Server]

Mirror1 Server: opha-dev3.opmantek.net

Poller2 Server: opha-dev4.opmantek.net

Mirror2 Server: opha-dev5.opmantek.net

Second Primary: opha-dev6.opmantek.net

Run the installation commands on each server as needed.

All the following setup commands in this setup should be run as the root user

e.g

During the installation of the arbiter server you must answer 'n' at the following prompt

Open

NATS Setup

This configuration should be applied to the Main Primary, Arbiter and Secondary Primary servers ONLY.

1. Edit Configuration: Update the /etc/nats-server.conf file with the following settings:

   server_name: "opha-dev1.opmantek.net" #The local server http_port: 8222 listen: 4222 jetstream: enabled #tls { # cert_file: "<path>" # key_file: "<path>" # #ca_file: "<path>" # verify: true #} log_file: "/var/log/nats-server.log" accounts { $SYS { users: [ { user: "admin", pass: "password" } ] } ophad: { users: [ { user: "omkadmin", password: "op42opha42" } ] jetstream: enabled } } jetstream { store_dir: "/opt/nats/storage" max_memory_store: 1028M max_file_store: 1028M } cluster { name: "C1" host: "opha-dev1.opmantek.net" #The current host # the current server listen: "0.0.0.0:6222" routes: [ # secondary primary "nats://opha-dev6.opmantek.net:6222" # server with the arbiter "nats://opha-dev2.opmantek.net:6222" # other servers ] }

    

Common Configuration for All Servers

Update /usr/local/omk/conf/opCommon.json

1. Backup Configuration: Make a backup copy of the opCommon.json file before making any changes.

2. Add NATS Cluster Configuration: Ensure the following configuration is included in opCommon.json on all servers. In the below example we have added the Main Primary, arbiter and Secondary Primary FQDNs and port/s to be utilized in the nats_cluster attribute, you can find this in the database section.

   "nats_cluster": [ "opha-dev1.opmantek.net", "opha-dev2.opmantek.net", "opha-dev6.opmantek.net" ]

    

Server-Specific Configuration

Update /usr/local/omk/conf/opCommon.json for Main Primary and Secondary Primary only

MongoDB Setup (Main Primary Only)

The following steps should be performed only on the Main Primary MongoDB instance.

Initialize MongoDB Replica Set

Connect to MongoDB using the mongosh command and configure the replica set:

Run the following command

Add the main-primary and secondary-primary to the below command and run it

Set Default Write Concern

Run the following command to set the default write concern:

Add Arbiter

Run the following command to add an arbiter to the replica set:

Restart mongod on the Main Primary, Second Primary and Arbiter

Run the following command on the Main Primary, Secondary Primary, and Arbiter servers ONLY:

Starting Services

Start NATS

Start the NATS service on the Main Primary, Secondary Primary, and Arbiter servers ONLY:

Restart mongod on Pollers and Mirrors only

After configuration changes, restart mongod on all Mirrors and Pollers servers ONLY:

Restart services on all servers

After configuration changes have been made, you will need to restart the relevant FirstWave module daemons applicable to your server.

For example if you have NMIS9, opCharts, opEvents and opHA 5 installed you would execute as follows:

Discover the Peers

Click on Peers in opHA-MB portal on the Main Primary (http://<fqdn of Main Primary>/en/omk/opHA/peers)

Open

Proceed to next step after discovering all the Peers.

Restart opHA service on the Main Primary

Start the Poller and Mirrors

Execute the following command on the poller and mirror servers.

Final Steps

1. Accept EULAs: Make sure when you login to each server to confirm that all End User License Agreements (EULAs) are accepted.