Content Comparison

Table of Contents
maxLevel 3

...

cp install/01opmantek.conf /etc/apache2/conf.d/
service apache2 restart

...

Data

...

Storage Size 

Anchor
Considerations_for_NetFlow_Data_Storage Considerations_for_NetFlow_Data_Storage

NetFlow data can get away on you, there could be several Gigabytes of NetFlow data each hour, day or month, this is all very dependant on where you are generating netflow from, the number of active users and the types of applications they are using.  A longer discussion on this will be written up, but in summary, if you are expecting large amounts of NetFlow data, then you , you will want to consider storing the MongoDB database and the NetFlow data into a filesystem with 50 gigabytes or more.  During development we found 20 gigabytes was enough for our purposes, but caused problems with the flow files mainly. If you are using the Opmantek Opmantek NMIS8 Virtual Machine please  please check out our instructions on on Resizing NMIS VMs.

Size

opFlow is highly configurable and customisable, so it is easy to just have the flowd data be a seperate filesystem and update the configuration accordingly. 

'<opflow_dir>' => '/data/opflow',

mongodbpath=/data/mongodb

logfile "/data/opflow/flowd" 

Alternate Installation Directory

opFlow can be installed into another directory if required, e.g. /opt/opmantek, the same process applies, but a few files will need to be changed.

Edit opFlow.nmis and opCommon.nmis and change the <omk_base> to be the new, e.g.

...

uses fixed sizes for raw flows and conversations, when the setup tool (opflow_setup.pl) is run the database files will immediately be pre-allocated to the configured size and will never grow after this.  It is important that the sizes you choose fit on the partition you choose, the setup tool will warn you if they will not fit.  The default settings are: (found in /usr/local/opmantek/conf/opFlow.nmis)

'opflow_db_conversation_collection_size' => 16106127360, #15G
'opflow_db_flow_collection_size' => 5368709120 # 5G

Please adjust them appropriately before running the setup tool. (which is done later in the instructions)

The output from the setup tool may tell you to run it again after adjusting your config with force=1 (opflow_setup.pl setup=db_config force=1) # again, only if required

Location

opFlow is highly configurable and customisable, so it is easy to just have the flowd data be a seperate filesystem and update the configuration accordingly. 

'<opflow_dir>' => '/data/opflow',

mongodbpath=/data/mongodb

logfile "/data/opflow/flowd" 

Alternate Installation Directory

opFlow can be installed into another directory if required, e.g. /opt/opmantek, the same process applies, but a few files will need to be changed.

Edit opFlow.nmis and opCommon.nmis and change the <omk_base> to be the new, e.g.

'<omk_base>' => '/opt/opmantek',

...

adduser: Please enter a username matching the regular expression configured
via the NAME_REGEX[_SYSTEM] configuration variable. Use the `--force-badname'
option to relax this check or reconfigure NAME_REGEX.

adduser --force _flowd

#change this line: 
. /etc/init.d/functions
#to this: 
. /lib/lsb/init-functions

sysv-rc-conf flowd on
service flowd start

...

cp /usr/local/opmantek/install/mongod.init.d /etc/init.d/mongod
chkconfig mongod on
service mongod start  

#change this line: 
. /etc/init.d/functions
#to this: 
. /lib/lsb/init-functions

sysv-rc-conf mongod on
service mongod start

...

1. Ensure mongod is not running with the --auth switch, if you are using the mongod.init.d script included in opFlow run  run:

   Code Block
   /etc/init.d/mongod stop; /etc/init.d/mongod start_no_auth;

2. Create the user, currently the opFlow user requires access to both the admin database and it's own. Start up the mongo shell and type these commands:

   Code Block
   use admin; db.addUser('opUserRW', 'op42flow42'); // these are the defaults, change them as well as your opFlow.nmis file use nmis; // again this is a default, it doesn't need to be changed db.addUser('opUserRW', 'op42flow42'); // these are the defaults, change them as well as your opFlow.nmis file, it should match the above user command

3. Restart the mongo server with authentication, --auth 

   Code Block/

   auth (again, only if you are using the mongod.init.d script included in opFlow)

   Code Block
   /etc/init.d/mongod stop; /etc/init.d/mongod start;

...

cp /usr/local/opmantek/install/opflowd.init.d /etc/init.d/opflowd
chkconfig opflowd on
service opflowd start

#change this line: 
. /etc/init.d/functions
#to this: 
. /lib/lsb/init-functions

sysv-rc-conf opflowd on
service opflowd start

...

cd ~
# get release and untar it
wget opFlow-release.tgz
tar -zxvf opFlow-release.tgz
# make a backup of existing installation
cp -r /usr/local/opmantek /usr/local/opmantek-DAY-MON-YEAR
# stop the existing daemon so update can happen
service opflowd stop
# update the files
cp -r ./opmantek/* / files
cp -r ./opmantek/* /usr/local/opmantek/*

opFlow database now uses capped sizes, please adjust the following config settings to make sure they fit on your partition and are the size you want: (the defaults are listed)

'opflow_db_conversation_collection_size' => 16106127360, #15G
'opflow_db_flow_collection_size' => 5368709120 # 5G

Now run the setup tool:

/usr/local/opmantek/bin/opflow_setup.pl setup=db_config

Inspect the output and be sure you have configured the sizes the way you want. If you are unsure of the current size running opflow_setup.pl setup=db_config will inform you.

When you are happy with your configuration, re-run the setup tool with force=1: (this may take some time to complete if your existing database is large)

/usr/local/opmantek/*

...

/bin/opflow_setup.pl setup=db_config force=1

Now update the configuration files run the following commands to do that.

/usr/local/opmantek/bin/opupdateconfig.pl /usr/local/opmantek/install/opFlow.nmis /usr/local/opmantek/conf/opFlow.nmis
/usr/local/opmantek/bin/opupdateconfig.pl /usr/local/opmantek/install/opCommon.nmis /usr/local/opmantek/conf/opCommon.nmis

/usr/local/opmantek/bin/opflow_setup.pl setup=all  
service opflowd start

#change this line: 
. /etc/init.d/functions
#to this: 
. /lib/lsb/init-functions