This article builds on our article from a few years ago on basic Linux server setups.  You can read the article “here.”  For our build, we are creating a server on KVM.  The server currently has mounted the following drives:

freespace-beginning-mongodb1

The VM was built with 1GB of RAM for now on Centos 7.

Our first order of operations is to check the DNS settings for the server.  As Network Manager is installed by default and common on Ubuntu systems as well, we won’t disable it.  However, we want to check where our DNS is pointed and make a change to our internal DNS if possible.  For this, run dig from our shell pointing to the server name we just created (this works for systems added to Bind on creation or Active Directory as an A record before creation).  You should see a response similar to the following

dnscheck-mongo1

As you can see, the DNS server in our test environment has responded with information on our local mongo server.  Now that we know DNS is operational and we can see our server, we need to look at security.

Security

Before we get started, let’s talk about firewalld, iptables, and ufw.  Firewalld and ufw are front-ends for iptables which uses the netfilter framework.  Firewalld is the default installed firewall on Centos 7 and as far as I can tell, on Ubuntu 18.04 as well.  This leads to the debate over whether to disable it and simply use iptables or go ahead and use firewalld.  I’ve been a hard-core iptables person for many years, but as the defaults start getting replaced in distributions in favor of firewalld, it might be better to move forward using it.  In an earlier article, I detailed using iptables (although the configuration I used opened a lot of ports).  If you wish to use iptables, skip this section and use that document to guide you.  When you’re done, we’ll be here waiting..

For those of us who decided to brave the new world of firewalld, let’s get started.  Make sure firewalld is running first:

firewalld-check

Use systemctl status firewalld to see if everything is running.  If it isn’t, type systemctl start firewalld and you should be good to go.  We now need to add a couple of rules to make Mongo run once it’s installed.  Other rules will be added as we complete different sections of this installation.

Mongo runs on port 27017 by default and can use port 28017 if you have a REST interface to connect to it.  Let’s add both for safe measure:

firewall-cmd –permanent –zone=public –add-port=27017/tcp

firewall-cmd –permanent –zone=public –add-port=27017/udp

firewall-cmd –permanent –zone=public –add-port=28017/tcp

firewall-cmd –permanent –zone=public –add-port=28017/udp

mongo-firewall-rules

Once you have four successes, run firewall-cmd –reload.  This will load the new rules and make them active.  We can check to make sure the rules are set by running iptables -L.  You should see the following in the rule list:

firewalld-rules-verify

Selinux

To truly make a system secure, especially a database server, we need to set up security policies.  If you have ever used Active Directory, you know how important group policies are.  Centos and many other distributions do this through Selinux.  Let’s begin!

Centos will install several Selinux packages by default, but just to make sure, let’s install most of the Selinux family of packages:

yum install selinux-policy policycoreutils policycoreutils-python libselinux-utils setroubleshoot-server setools setools-console mcstrans

It should install at least five packages and perhaps more:

selinux-install-packages

We need to check out Selinux’s mode.  To do this, type sestatus at the shell prompt:

sestatus

For now, we want to change the mode from “enforcing” to “Permissive” in order to make sure Mongo can start running without Selinux interfering.  Moreover, issues found when Selinux is set back to “enforcing” mode will demonstrate policy changes that need to be fixed.  Open /etc/selinux/config and change the SELINUX=enforced to SELINUX=permissive.

Reboot the server.

Once we have the server up and running, we will change the mode back and set contexts for Mongo.  You might be wondering why we don’t simply set the “permissive” context at the command line.  That’s a great question!  However, when we install other products and update the system during the installation, we have to reboot.  A reboot puts the server back into the mode set in the config file.

Antivirus

Most Linux/Debian installations come with some sort of antivirus product.  Since ClamAV is available on most distributions, we will install it here.  We will not be installing the server version as I’ve discovered it likes to ping at least one core of the CPU at 100% and not let go of it.  Given an opportunity, the clamav daemon will peg multiple cores and cause issues with files (in terms of opening them).  For this to work well, we need to start by installing the epel repository:

yum install epel-release

Once installed, let’s install several other packages:

yum install clamav clamav-scanner clamav-update clamav-filesystem clamav-devel

After the install completes, you will need to edit /etc/freshclam.conf and comment out the word “Example” on line 8:

freshclam.conf

The line may already be commented out.  In that case, simply exit vi.

Type “freshclam” at the command line to update the virus definitions.

Once this is complete, you will want to run “clamscan -r /” to verify everything is clean and clamscan runs correctly.

clamscan-firstrun

A summary of the scan should show no infected files – hopefully:

clamscan-summary

We now need to alter the crontab for our root user (note that in production environments, you would want to install clam or any antivirus system as its own user).  Running crontab -u root -e will allow us to add the following:

antivirus-crontab

Antivirus programs are important to update and run on your servers.  The two lines above do the following:

At 11:12 PM on every Wednesday, Friday, and Sunday, the virus defintions will be updated.  Twelve minutes later every day of the week, a full virus scan will be conducted.

We now need to ensure that SeLinux accepts the virus scans when its policies are set back to enforced:

setsebool -P antivirus_can_scan_system 1

You may need to modify the file /etc/sysconfig/freshclam to remove the line that begins:

FRESHCLAM_DELAY=

If the line is commented out, this can be disregarded.

Mail System

Postfix

A mail system such as Postfix should already be installed by default on your system.  You will want to run vi /etc/postfix/main.cf and change the following parameters:

myhostname = {enter FQDN of your host}

relayhost = {enter your smtp server here}

Do a search for the variable names above and either add a line for each or modify one of the existing ones like so:

postfix-variables

As you can see, we added the hostname in our test domain.  Save the file.  However, do not restart the server or service until you modify the /etc/aliases file.  At the bottom of the file, you will want to replace the commented out line with something like this:

postfix-aliases

In the scenario above, we want me to receive all system emails going through the local domain.  I’ll get into setting up a mail server in another post.  Once you save and quit the file, run newaliases to update the system.

You can now run service postfix restart or restart the server (your choice).

If you want to test postfix, you will want to run yum install mailx and then create a command line email to make sure you are getting system emails.

SNMP

In a production network, you will need to allow network monitoring for a variety of reasons.  Many products work with the SNMP (Simple Network Management Protocol) to provide CPU, disk, memory, and network traffic monitoring.  Here we are enabling it as a demonstration, but you may need to depending on the size of your network.

First, install the packages:

yum -y install net-snmp net-snmp-tools

Next, let’s get the snmpd.conf file out of the way to create a new one:

mv /etc/snmp/snmpd.conf /etc/snmp/snmpd.conf.orig

vi /etc/snmp/snmpd.conf

Add lines for your read-only and read-write communities.  Here we simply made up some for demonstration purposes:

snmpd-communities

The final steps are to start the daemon and make sure it comes up when the system boots:

systemctl enable snmpd.service

systemctl start snmpd.service

YUM

We’ve now got some basic services running, so let’s go ahead and update the system:

yum -y update

Once the update is complete, we will need to add a line in the crontab that updates the system every week.  Updates will include kernel updates as well.  This will require a reboot of the system.  You can add this to crontab as well or manually handle it – it’s personal preference.  I tend to do manual reboots in case something goes wrong and, more importantly, so that I don’t interrupt services for my customers.

Add a line like:

# Weekly system updates
25 22 * * 5 /usr/bin/yum -y update

The above line will run yum every Friday at 10:25 pm and not confirm you want to update.  The odd times ensure other processes aren’t running.  Your crontab should look something like this now:

yum-crontab

Additional Accounts

When we set up systems on Centos, a good security measure is to not run commands as root.  To accomplish this we will need to create a user or users with “sudo” capability:

useradd -c “LITD System Administrator” -G root,wheel -s /bin/bash -m systemadmin

You may want to add additional users for this server depending on your needs.  Run passwd for each user including the systemadmin user above.  You may also want to create groups that encompass user roles on the server.  I will write another blog entry on this in the near future.

Hardening your system is the priority for most system administrators.  To do this, we want to set the minimum password length and complexity for all NEW users created on the system.  You will want to change the /etc/pam.d/system-auth file for a password minimum length of 8, to require upper and lower case in the password along with one number.  You also want to limit the number of retries.

Add this line to the top:

auth required pam_tally2.so deny=3 unlock_time=1800 even_deny_root

Change the lines in the third session for “password” to show the following:

password requisite pam_pwquality.so local_users_only retry=3 minlength=19 minclass=3 gecoscheck maxrepeat=3
password requisite pam_pwhistory.so remember=400 use_authtok
password sufficient pam_unix.so sha512 shadow use_authtok

It should look like this:

pam-changes

NTP

Let’s set up a time server so that functions like rsync or clustering work more effectively:

yum -y install ntp

If you have an NTP server set up already, you will want to point to it or an Active Directory server if that is dominant on your network. If neither are true, you can pull a list of servers from ntp.org and test which ones work best for you.  Change the /etc/ntp.conf file to reflect these changes:

server 0.centos.pool.ntp.org iburst
server 1.centos.pool.ntp.org iburst
server 2.centos.pool.ntp.org iburst
server 3.centos.pool.ntp.org iburst

Change these to something that works:

server time.litd.prv iburst
server adserver.litd.prv iburst
server 0.us.pool.ntp.org iburst
server 0.centos.pool.ntp.org iburst

Since ntpdate has been depreciated, we will want to run ntpd -g prior to starting up the daemon (ntpd -g must be run without the daemon running or can create problematic results).  Note that you will need to kill the ntpd -g command after a minute or two as it does not stop on its own.

Finish it up by running:

systemctl enable ntpd.service

systemctl start ntpd.service

Even though ntpdate is depreciated, we still have it to use, so let’s check our time status:

ntpdate -s us.pool.ntp.org

It should return with a blank line.  Add something like the below to your crontab:

ntp-crontab

Finishing up Part 1

Applications like Mongo can open and close thousands of files at a time depending on many factors.  As we stated above, our security objective is to start installed programs under their own user space by creating separate users for each application.  To help us control the number of files, memory, etc.. used by the system, changing the ulimit number is a good idea.  Add the following lines to the /etc/security/limits.conf file:

root soft nofile 500000
root hard nofile 500000

You will want to add others per the applications you install.  We may see this later with the Mongo installation.  One of the benefits of using ulimit is that it protects the operating system from crashing in many circumstances.  The bottom half of your file should look something like this:

ulimits-limits.conf

ACL (Access Control Lists) support is also important in our security minded system.  Although it should be there by default, it is a good idea to run setfacl on a touched file and check it with getfacl to make sure everything looks correct.

In our next installment, we will look at the Firewall more closely, SeLinux, and adding Tripwire as part of a HIDS (Hardware Intrusion Detection System) before we start the application installation and configuration.