Skip to content

0. Quick Start Guide

Jump to bottom Edit New page
Christopher Dunavan edited this page Apr 12, 2017 · 6 revisions

Setting up Node, MongoDB and DCIT

This is an example to setup a fresh install of Node.js and MongoDB on a single host. At the heart of it, DCIT is just a Node.js application that uses MongoDB 3.2 as a back end so there are certainly many other ways you could get this up and running.

This will:

  • Install Node.js and run it as a non-privileged user
  • Install MongoDB as a service and run as a non-privileged user
  • Install DCIT and use PM2 to run it as a clustered service
  • Setup port forwarding to 80 and 443
  • Backup and Restore the DCIT db
  • Update DCIT code from the git repo

The following assumes RHEL/CentOS/OEL linux 6.5 or 7 and you are logged in as a normal user with sudo access.

[TOC]

Prerequisites

Note: pay close attention to the user context of the commands. This install switches back and forth between root, appjs and mongodb users depending on who will own what and what access level is needed for that particular section. If you see exit you are exiting that user and going back to root. Nothing is installed using your login, so if you exit all the way back to your user account, you may have gone too far. You will see as (user)$ above each block of code. That is the user you should be using entering that block of code. If you are following along, you are probably already be using that user when you get there just keep that in mind as you go.

as yourself$

# switch to root user
sudo -i

#install pre-reqs
yum update -y
yum install git -y

Note: Because we are installing from binaries we do not need a compliler. If we wanted to install Node and/or MongoDB from source, we would also install gcc-c++ glibc-devel.

Adding the following directories: /var/appjs/dcit - DCIT application files /var/appjs/userdata - csv files from file upload utility /var/appjs/log - app and db logs /space/db/mongodb - MongoDB database files /space/db/backup - database backup files (should be a remote mount) /opt/nodejs - Node.js application files /opt/mongodb - MongoDB application files /opt/bin - MongoDB service files /opt/config - MongoDB configuration files

as root$

#setup app directories
mkdir -p /var/appjs/{dcit,userdata,log}     
mkdir -p /space/db/{mongodb,backup}                
mkdir /opt/{nodejs,mongodb,bin,config}

Adding non-privileged users for Node.js and MongoDB. Running our services as non-privileged users contains them to their own user space. When working with Node.js files, we will be using the appjs user who will own the appjs directory. The appjs user will also own the Node.js install so global npm packages can be installed without sudo or root.

When working with MongoDB, we will be using the mongodb user who will own those directories.

as root$

#add non priv user for nodejs and mongodb
useradd -mrU appjs

chown appjs -R /var/appjs
chgrp appjs -R /var/appjs
chown appjs -R /opt/nodejs
chgrp appjs -R /opt/nodejs

useradd -M -r --home-dir /opt/mongodb mongodb
chown mongodb -R /opt/mongodb
chgrp mongodb -R /opt/mongodb
chown mongodb:mongodb -R /space/db/

groupadd logwriters
usermod -aG logwriters mongodb
usermod -aG logwriters appjs
chgrp -R logwriters /var/appjs/log
chmod -R 774 /var/appjs/log

Downlaod and install Node.js server binaries

as root$

# switch to appjs user
su appjs

as appjs$

cd ~/
mkdir ~/download
cd download
wget https://nodejs.org/dist/v6.2.2/node-v6.2.2-linux-x64.tar.xz    

#adjust this line and later lines with correct version
unxz node-v6.2.2-linux-x64.tar.xz
tar -xf node-v6.2.2-linux-x64.tar

# cleanup
rm node-v6.2.2-linux-x64.tar                        
cd /opt/nodejs/

# move contents of node download to /opt/nodejs
mv /home/appjs/download/node-v6.2.2-linux-x64/* .
ls   #Should read $ bin  CHANGELOG.md  include  lib  LICENSE  README.md  share
export PATH=/opt/nodejs/bin:$PATH
export NODE_PATH=opt/nodejs/lib/node_modules

# add autocomplete to shell
npm completion >> ~/.bashrc

# refresh shell                         
source ~/.bashrc                                    
cd ~

# check node version $ v6.2.2 (or whatever version you are installing)
node -v

#cleanup                                       
rm -r /home/appjs/download/node-v6.2.2-linux-x64

# switch back to root user
exit

Downlaod and install mongodb server binaries

as root$

#switch to mongodb user 
su mongodb

as mongodb$

cd ~ # should be in /opt/mongodb                                             
wget http://fastdl.mongodb.org/linux/mongodb-linux-x86_64-rhel62-3.2.7.tgz
tar -zxf mongodb-linux-x86_64-rhel62-3.2.7.tgz
rm mongodb-linux-x86_64-rhel62-3.2.7.tgz
mv mongodb-linux-x86_64-rhel62-3.2.7/* .
rm -r mongodb-linux-x86_64-rhel62-3.2.7

# switch back to root user
exit

Setup Mongodb files

Here we are creating new files and adding the code. You can use whatever text editor you like. If you are new to text editors in linux, I recomend Nano. To start nano and create/edit a file, the command would look like this. nano /opt/bin/mongodb-start Once you are done adding your contents, you can press CTRL + O to save and CTRL + X to exit that file. Create the following files and paste the commands shown:

All of these files are added as root$ /opt/bin/mongodb-start

#!/bin/sh

/opt/mongodb/bin/mongod --config /opt/config/mongodb \
## --upgrade \ ##runs a database upgrade option if needed \

/opt/bin/mongodb-stop

#!/bin/bash

pid=`ps -o pid,command ax | grep mongod | awk '!/awk/ && !/grep/ {print $1}'`;
if [ "${pid}" != "" ]; then
    kill -2 ${pid};
fi

/opt/config/mongodb

# Configuration Options for MongoDB
#
# For More Information, Consider:
# - Configuration Parameters: http://www.mongodb.org/display/DOCS/Command+Line+Parameters
# - File Based Configuration: http://www.mongodb.org/display/DOCS/File+Based+Configuration

dbpath = /space/db/mongodb
logpath = /var/appjs/log/mongodb.log
logappend = true

bind_ip = 127.0.0.1
port = 27017
fork = true

# auth = true
noauth = true

/etc/init.d/mongodb

#! /bin/sh
#
# mongodb – this script starts and stops the mongodb daemon
#
# chkconfig: - 85 15
# description: MongoDB is a non-relational database storage system.
# processname: mongodb
# config: /opt/config/mongodb
# pidfile: /opt/mongodb/mongo.pid

PATH=/opt/mongodb/bin:/sbin:/bin:/usr/sbin:/usr/bin
NAME=mongodb

test -x $DAEMON || exit 0

set -e

case "$1" in
  start)
        echo -n "Starting MongoDB... "
        su - mongodb -c "/opt/bin/mongodb-start"
        ;;
  stop)
        echo -n "Stopping MongoDB"
        /opt/bin/mongodb-stop
        ;;
      *)
            N=/etc/init.d/$NAME
            echo "Usage: $N {start|stop}" >&2
            exit 1
            ;;
    esac

    exit 0

/etc/init.d/disable-transparent-hugepages

#!/bin/sh
### BEGIN INIT INFO
# Provides:          disable-transparent-hugepages
# Required-Start:    $local_fs
# Required-Stop:
# X-Start-Before:    mongod mongodb-mms-automation-agent
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Disable Linux transparent huge pages
# Description:       Disable Linux transparent huge pages, to improve
#                    database performance.
### END INIT INFO

case $1 in
  start)
    if [ -d /sys/kernel/mm/transparent_hugepage ]; then
      thp_path=/sys/kernel/mm/transparent_hugepage
    elif [ -d /sys/kernel/mm/redhat_transparent_hugepage ]; then
      thp_path=/sys/kernel/mm/redhat_transparent_hugepage
    else
      return 0
    fi

    echo 'never' > ${thp_path}/enabled
    echo 'never' > ${thp_path}/defrag

    unset thp_path
    ;;
esac
Fix Hugepages

MongoDB needs hugepages disabled, see [this] (https://docs.mongodb.com/manual/tutorial/transparent-huge-pages/) for more information.

as root$

chmod 755 /etc/init.d/disable-transparent-hugepages
chkconfig --add disable-transparent-hugepages
Set MongoDB as a service

as root$

chmod +x /opt/bin/*
chmod +x /etc/init.d/mongodb
chkconfig --add mongodb
chkconfig mongodb on
# start mongodb
service mongodb start
#switch to mongodb user
su mongodb
cd ~/
echo 'export PATH=/opt/mongodb/bin/:$PATH' >> ~/.bashrc
. ~/.bashrc

mongo

The mongo command starts the MongoDB interactive shell. It should return MongoDB shell version: 3.2.7 and a prompt > At the mongo promt

use admin
db.runCommand({logRotate:1})
// exit mongo shell
exit
# switch to root
exit

####Install DCIT as root$

# switch to apppjs
su appjs
echo 'export PATH=/opt/nodejs/bin/:$PATH' >> ~/.bashrc
echo 'export NODE_PATH=opt/nodejs/lib/node_modules' >> ~/.bashrc
. ~/.bashrc
cd /var/appjs
git clone https://github.com/HyperSprite/dcit.git

cd /dcit
npm install --production

Create and edit the credentials.js file. Use the credentials-file-example.js as an example. See this for more information.

Install PM2, this is a process manager for Node applications.

as appjs$

npm install pm2@latest -g

pm2 start pm2env.json --env production
pm2 save
exit
# as root
env PATH=$PATH:/opt/nodejs/bin pm2 startup redhat -u appjs --hp /home/appjs

Setup port forward to 80 and 443 on 6.5

Edit the /etc/sysctl.conf and update the following line to =1 net.ipv4.ip_forward = 1 Then apply them as root$

sysctl -p
sysctl --system

Add the rules to port forward to iptables as root$

iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 443 -j REDIRECT --to-port 3000
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 3080
service iptables save # saved in /etc/sysconfig/iptables
service iptables restart
chkconfig --level 345 iptables on

Setup port forward to 80 and 443 on 7.0

as root$

yum install firewalld -y
systemctl restart firewalld
firewall-cmd --zone=public --add-masquerade --permanent
firewall-cmd --zone=public --add-service=http --permanent
firewall-cmd --zone=public --add-service=https --permanent
firewall-cmd --zone=public --add-forward-port=port=80:proto=tcp:toport=3080 --permanent
firewall-cmd --zone=public --add-forward-port=port=443:proto=tcp:toport=3000 --permanent
systemctl restart firewalld
firewall-cmd --state
firewall-cmd --get-zones
# check ports
ss -antlup | grep

Backup and Restore

Earlier we created a directory /space/db/backup, replace this with an NFS mount or other remote storage option if you want to have these backups land on a filer. #####Backup Create a file called /opt/bin/dcit-backup.sh and add the following. Create this file as root$

#!/bin/sh
su mongodb -c "cd /space/db/backup&& mkdir `date +%m%d%y` && /opt/mongodb/bin/mongodump -h localhost -d dcit -o `date +%m%d%y`"

This will:

  • su to mongodb user
  • switch to the backup location
  • make a new directory with the date
  • run mongodump on the local host instance of dcit and put it in that date directory

Set the script as an executable and open up crontab and add the job as root$

chmod +x /opt/bin/dcit-backup.sh
crontab -e
# put this in crontab
45 1 * * * /opt/bin/dcit-backup.sh
# verify the job
crontab -l
Restore

Note: This assumes you are restoring a file from today's date. If you are restoring an older file, change the date +%m%d%y to the folder name.

as root$

su mongodb
cd /space/db/backup
rm `date +%m%d%y`/dcit/sessions*
rm `date +%m%d%y`/dcit/fileinfo*
mongorestore --host localhost --port 27017 --db dcit --drop /space/db/backup/`date +%m%d%y`/dcit
exit
su appjs
pm2 restart dcit
pm2 status dcit

This will:

  • su to mongodb user
  • switch to /space/db/backup directory
  • delete the sessions file
  • delete the file upload file
  • restore the database to the

We delete the sessions and fileinfo before restoring. A new sessions and fileinfo db will be created on startup.

Update DCIT to the latest code

When you want to update DCIT to the latest code, run the following commands as root$

su appjs
cd /var/appjs/dcit
git pull origin master
npm install
pm2 restart dcit
pm2 status

This will:

  • Pull down the latest code
  • Install any new modules that may be needed
  • Restart DCIT web service
  • Show the web service status

Look at the web service status and see that the application is running for more than a few seconds. Also, look at the restarts and make sure it has only restarted once. If it looks like it is looping through restarts, there may be an issue.

Add a custom footer
Add a custom sidebar
Clone this wiki locally