There are plenty of times that you need to stop Qmail for maintainence purposes.

The qmailctl script generally does a good job of stopping qmail, but sometimes it does not quite shutdown all of the qmail processes.

[root@here]# qmailctl stop
Stopping qmail…
qmail-smtpd
qmail-send

[root@here]# qmailctl stat
/service/qmail-send: up (pid 32436) 358918 seconds, want down
/service/qmail-send/log: down 3 seconds, normally up
/service/qmail-smtpd: down 3 seconds, normally up
/service/qmail-smtpd/log: down 3 seconds, normally up
messages in queue: 142
messages in queue but not yet preprocessed: 0

As you can see from the output of qmailctl stat, the qmail-send program is still running. If you do a^[1]

ps waux | grep qmail | grep -v grep

you will often find output like

[root@here]# psq
root 55033 0.0 0.1 1196 480 ?? I Tue05PM 0:00.01 supervise qmail-send
qmails 55035 0.0 0.1 1340 760 ?? I Tue05PM 1:28.27 qmail-send
root 55036 0.0 0.1 1196 480 ?? I Tue05PM 0:00.01 supervise qmail-smtpd
qmaill 55040 0.0 0.1 1224 532 ?? I Tue05PM 0:29.48 multilog t s100000 n20 /var/log/qmail/qmail-send
root 55043 0.0 0.1 1216 500 ?? I Tue05PM 0:00.03 tcpserver -H -R -v -c100 0 110 qmail-popup
root 55048 0.0 0.1 1216 548 ?? I Tue05PM 0:00.02 qmail-lspawn ./Maildir
qmailr 55049 0.0 0.1 1284 684 ?? I Tue05PM 0:22.64 qmail-rspawn
qmailq 55050 0.0 0.1 1212 544 ?? I Tue05PM 0:04.69 qmail-clean
qmailr 83484 0.0 0.2 2432 1196 ?? I 2:38AM 0:00.00 qmail-remote yahoo.com webmaster@here.com there@yahoo.com
qmailr 83498 0.0 0.2 2432 1196 ?? I 2:38AM 0:00.01 qmail-remote hotmail.com webmaster@here.com you@yahoo.com

If you then run qmailctl start, you will often have the situation with qmail-send showing up as running for 1 second while the other qmail processes are up for much longer. Basically a second qmail-send is trying to start and is dying after a second. The first instance of qmail-send is still running and needs to be stopped.

The output often contains a mixture of qmail-send, qmail-remote, qmail-clean and others depending on how your mail has been setup and what is in the process of being sent/received when you issue the qmailctl stop command.

At this point, we can wait a bit, to see if qmail-send eventually goes down or whether we will have to kill it explicitly. It is worth trying another qmailctl stop at this point. If qmailctl stat shows qmail-send is still up, which is often the case, it is probably time to kill qmail-send directly.

I usually do a kill -9 qmail-send. Then follow it with psq to see what is still running. Often I have to run a number of kills and killalls, such as killall qmail-remote. We’re not really shutting down qmail very cleanly, but sometimes, especially when you’re in a hurry it makes sense to kill things rather than wait for them to terminate.

Eventually only the supervise scripts should be running,

[root@work]#psq
root 561 0.0 0.0 1148 84 con- I Sat02AM 0:00.05 supervise qmail-send
root 563 0.0 0.0 1148 84 con- I Sat02AM 0:00.06 supervise qmail-smtpd

At this stage I usually call /usr/local/etc/rc.d/svscan.sh stop. The location of your actual svscan.sh script may be different from mine.

A psq shows nothing running. A sockstat -4 | grep 25 shows nothing listening on the smtp port. Now is the time to go in and do maintainance - such as recompile qmail, dequeue mail or repair the queue.

Once you’re done, starting qmail is as simple as /usr/local/etc/rc.d/svscan.sh start.

---

[1] Actually I use this snippet quite a bit so to save time, I’ve copied it to a file called psq under /usr/local/bin

echo "ps waux | grep qmail | grep -v grep" > /usr/local/bin/psq

so that simply typing psq gives me a list of all the running qmail processes.

To get awstats to work with qmail log files I use this little script from Bob Hutchinson called qlogs4awstats.

Download the file and edit the lines below to reflect your local paths. Most likely only the third line below
will need to be edited, to reflect where you have installed awstats.

chdir(”/var/log/qmail/qmail-send”);

my $taiprog = “/usr/local/bin/tai64nlocal”;

my $mailconvprog = “/usr/local/www/awstats/tools/maillogconvert.pl standard”;

Once edited move it to /usr/local/bin and chmod it so that it is executable.

Next copy awstats.model.conf to awstats.mail.conf or whatever you want to call your conf file.
You will have to edit your awstats.mail.conf in the following places. Note that in the first line there is a space after the pipe character | .

LogFile="/usr/local/bin/qlogs4awstats 3600 | "

LogType=M

LogFormat="%time2 %email %email_r %host %host_r %method %url %code %bytesd"

HostAliases="localhost 127.0.0.1 @locals"

LevelForBrowsersDetection=0 # 0 disables Browsers detection.

LevelForOSDetection=0 # 0 disables OS detection.

LevelForRefererAnalyze=0 # 0 disables Origin detection.

LevelForRobotsDetection=0 # 0 disables Robots detection.

LevelForSearchEnginesDetection=0 # 0 disables Search engines detection.

LevelForKeywordsDetection=0 # 0 disables Keyphrases/Keywords detection.

LevelForFileTypesDetection=0 # 0 disables File types detection.

ShowSummary=HB

ShowMonthStats=HB

ShowDaysOfMonthStats=HB

ShowDaysOfWeekStats=HB

ShowHoursStats=HB

ShowDomainsStats=0

ShowHostsStats=HBL

ShowRobotsStats=0

ShowEMailSenders=HBML

ShowEMailReceivers=HBML

ShowSessionsStats=0

ShowPagesStats=0

ShowFileTypesStats=0

ShowOSStats=0

ShowBrowsersStats=0

ShowOriginStats=0

ShowKeyphrasesStats=0

ShowKeywordsStats=0

ShowMiscStats=0

ShowHTTPErrorsStats=0

ShowSMTPErrorsStats=1

You can download my sample awstats.mail.conf with the above changes from here .

When installing ports from the ports tree, I was intrigued by this message for a long time:

root@t2# make
===> Vulnerability check disabled, database not found

I never really had time to check out what the vulnerability check was and what the database it refered to was. Some time later, in an unrelated context I came across portaudit under /usr/ports/security/portaudit

The package description reads

portaudit provides a system to check if installed ports are listed in a
database of published security vulnerabilities.

After installation it will update this security database automatically and
include its reports in the output of the daily security run.

…

This sounded like a good idea, I especially liked the idea of getting the vulnerabilities reported along with the daily output run.
Once installed the message about the vulnerability check being disabled goes away, which makes sense.

Installation from ports

root@t2# cd /usr/ports/security/portaudit
root@t2# make install
===> Installing for portaudit-0.5.10
===> Generating temporary packing list
===> Checking if security/portaudit already installed

===> To check your installed ports for known vulnerabilities now, do:

/usr/local/sbin/portaudit -Fda

===> Compressing manual pages for portaudit-0.5.10
===> Registering installation for portaudit-0.5.10

Interestingly and usefully, if you try to make a port that is vulnerable, portaudit complains loudly and won’t let you build it. It suggests you cvsup the ports tree and get the latest patched version of what you are trying to install.

Here’s a sample of a mailout from one of the machine:

Checking for a current audit database:

Downloading fresh database.

32063 bytes transferred in 0.0 seconds (712.35 kBps)
New database installed.
Database created: Thu Nov 24 02:40:19 CST 2005

Checking for packages with security vulnerabilities:

Affected package: squid-2.5.11_2
Type of problem: squid — FTP server response handling denial of service.
Reference: <http ://www.FreeBSD.org/ports/portaudit/1c3142a3-4ab2-11da-932d-00055d790c25.html>

Affected package: lynx-2.8.5
Type of problem: lynx — remote buffer overflow.
Reference: <http ://www.FreeBSD.org/ports/portaudit/c01170bf-4990-11da-a1b8-000854d03344.html>

Affected package: ruby-1.8.2_4
Type of problem: ruby — vulnerability in the safe level settings.
Reference: <http ://www.FreeBSD.org/ports/portaudit/1daea60a-4719-11da-b5c6-0004614cc33d.html>

Affected package: openssl-0.9.8_1
Type of problem: openssl — potential SSL 2.0 rollback.
Reference: <http ://www.FreeBSD.org/ports/portaudit/60e26a40-3b25-11da-9484-00123ffe8333.html>

As you can see, portaudit advises of vulnerabilities in your installed software on a daily basis, allowing you to patch things in a timely manner.

Part of my daily security routine is to check machine logs for signs of suspicious activity. Recently I found a few shell commands showing up in the Apache error log of one of the machines. It took a long time to track down the problem. In the end I found it was due to a poorly written and insecure script. This script was on the server when I inherited it from the previous administrator (figuring out which script was causing the problems is a story in itself).

After securing the script I couldn’t help but worry that something might have been dropped onto the server. In fact, after looking around I found a few files in /tmp and a couple in some places where apache had write access. I decided to look around for a tool to help me figure out if anything important had been tampered with. A quick look through the ports turned up rkhunter under /usr/ports/security/rkhunter.

The package description reads:

Rootkit Hunter is scanning tool to ensure you for about 99.9% you’re clean of nasty tools.

This tool scans for rootkits, backdoors and local exploits by running tests like:

- MD5/SHA1 hash compare
- Look for default files used by rootkits
- Wrong file permissions for binaries
- Look for suspected strings in LKM and KLD modules
- Look for hidden files
- Optional scan within plaintext and binary files

WWW: http://www.rootkit.nl/

Installing from ports is easy:
cd /usr/ports/security/rkhunter
followed by
make install && make clean
and rkhunter is installed (you may also wish to install portaudit¹).
The post-install blurb suggests a typing rkhunter as a first step. This gives rkhunter’s command line options.

The –update option updates rkhunter’s databases and is a good first step.

rkhunter --update

root@t2# rkhunter –update
Running updater…

Mirrorfile /usr/local/lib/rkhunter/db/mirrors.dat rotated
Using mirror http://www.rootkit.nl/rkhunter
[DB] Mirror file : Up to date
[DB] MD5 hashes system binaries : Update available
Action: Database updated (current version: 2005051900, new version 2005121400)
[DB] Operating System information : Update available
Action: Database updated (current version: 2005052200, new version 2005102800)
[DB] MD5 blacklisted tools/binaries : Up to date
[DB] Known good program versions : Update available
Action: Database updated (current version: 2005041700, new version 2005111500)
[DB] Known bad program versions : Update available
Action: Database updated (current version: 2005041700, new version 2005111500)

Ready.

Now that the database is up to date it is time to run rkhunter in check mode (–checkall or -c)
rkhunter -c

Here is a sample of the output:

root@t2# rkhunter -c

Rootkit Hunter 1.2.7 is running

Determining OS... Ready

Checking binaries
* Selftests
     Strings (command)                                        [ OK ]

* System tools
  Performing 'known good' check...
   /bin/cat                                                   [ OK ]
   /bin/chmod                                                 [ OK ]
   /bin/kill                                                  [ OK ]
   /bin/ls                                                    [ OK ]
   ...

Check rootkits
* Default files and directories
   Rootkit '55808 Trojan - Variant A'...                      [ OK ]
   ADM Worm...                                                [ OK ]
   Rootkit 'AjaKit'...                                        [ OK ]
   Rootkit 'aPa Kit'...                                       [ OK ]
   Rootkit 'Apache Worm'...                                   [ OK ]
   ...

* Suspicious files and malware
   Scanning for known rootkit strings                         [ OK ]
   Scanning for known rootkit files                           [ OK ]
   ...

It does seem to do a fairly thorough job in checking for a lot of things. It can be run as a cron job (the –cronjob switch takes out the colorized output).

Note: RK Hunter copies /etc/group and /etc/passwd to root’s home directory. So don’t have a coronary if you spot them in there (I nearly did).

[1] Portaudit is used by rkhunter to check for vulnerable applications, it is useful but not actually necessary to have it installed. It can be found under /usr/ports/security/portaudit.

Welcome to BSDSpot. We specialise in FreeBSD systems administration. If you’re looking for long term systems administration or one off systems adminstration you’ve come to the right place.

BSDSpot has been doing systems administration for a number of private clients for the past 3 years. During this time we have configured base systems from scratch, installing software like Apache, MySQL, Postgres, PHP, Webmin, Rsync, ProFTPd, AWStats just to name a few.

We are very experinced in doing Qmail installs, including spamassassin and clam-av mail filtering. A standard Qmail install also includes qmailadmin and vqadmin, so that you can manage your email accounts yourself.

Software can be installed via ports, packages or installed from source for maximum flexibility.