November 20, 2009

Centos 5.4 mysql-server install problem

I ran into a problem installing mysql-server on Centos 5.4. I did the usual

[root@myserver]# yum install mysql-server

The installation works, but this error appears when trying to start mysqld

[root@myserver]# service mysqld start
Initializing MySQL database: Installing MySQL system tables…
ERROR: 1004 Can’t create file ‘/tmp/#sql30e0_1_0.frm’ (errno: 13)
091119 20:43:10 [ERROR] Aborting

091119 20:43:10 [Note] /usr/libexec/mysqld: Shutdown complete

Installation of system tables failed!

Examine the logs in /var/lib/mysql for more information.
You can try to start the mysqld daemon with:
/usr/libexec/mysqld –skip-grant &
and use the command line tool
/usr/bin/mysql to connect to the mysql
database and look at the grant tables:

shell> /usr/bin/mysql -u root mysql
mysql> show tables

Try ‘mysqld –help’ if you have problems with paths. Using –log
gives you a log in /var/lib/mysql that may be helpful.

The latest information about MySQL is available on the web at

http://www.mysql.com

Please consult the MySQL manual section: ‘Problems running mysql_install_db’,
and the manual section that describes problems on your OS.
Another information source is the MySQL email archive.
Please check all of the above before mailing us!
And if you do mail us, you MUST use the /usr/bin/mysqlbug script!
[FAILED]

The problem was with SELinux. Issuing

[root@myserver]# sestatus

shows SELinux is enabled and in enforcing mode.

[root@myserver]# sestatus
SELinux status: enabled
SELinuxfs mount: /selinux
Current mode: enforcing
Mode from config file: enforcing
Policy version: 21
Policy from config file: targeted

Disabling SELinux enforcing temporarily allowed the install to proceed.

First we take out the old mysql-server install (the tables weren’t written to correctly):

[root@myserver]# yum remove mysql-server

Make sure mysqld is not running:

[root@myserver]# service mysqld stop

Then we move the corrupt mysql tables out of the way:

[root@myserver]# mv /var/lib/mysql/* /home/some-where-safe/

Next, we set SELinux to permissive mode:

[root@myserver]# setenforce 0

and check that it is so

[root@myserver]# sestatus
SELinux status: enabled
SELinuxfs mount: /selinux
Current mode: permissive
Mode from config file: enforcing
Policy version: 21
Policy from config file: targeted

Now we reinstall mysql-server

[root@myserver]# yum install mysql-server

and start it:

[root@melstar mysql]# service mysqld start
Initializing MySQL database: Installing MySQL system tables…
OK
Filling help tables…
OK

To start mysqld at boot time you have to copy
support-files/mysql.server to the right place for your system

PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER !
To do so, start the server, then issue the following commands:
/usr/bin/mysqladmin -u root password ‘new-password’
/usr/bin/mysqladmin -u root -h melstar.theplanet.host password ‘new-password’

Alternatively you can run:
/usr/bin/mysql_secure_installation

which will also give you the option of removing the test
databases and anonymous user created by default. This is
strongly recommended for production servers.

See the manual for more instructions.

You can start the MySQL daemon with:
cd /usr ; /usr/bin/mysqld_safe &

You can test the MySQL daemon with mysql-test-run.pl
cd mysql-test ; perl mysql-test-run.pl

Please report any problems with the /usr/bin/mysqlbug script!

The latest information about MySQL is available on the web at

http://www.mysql.com

Support MySQL by buying support/licenses at http://shop.mysql.com
[ OK ]
Starting MySQL: [ OK ]

The initialisation proceeded correctly, so we can re-enable SELinux enforcing mode.

[root@myserver]# setenforce 1

and then move on to securing and configuring mysql.

April 30, 2009

Id “x” respawning too fast

Testing an openvz container (Debian Lenny), noticed this in the logs:

Apr 29 15:53:10 shadow init: Id “1″ respawning too fast: disabled for 5 minutes
Apr 29 15:53:10 shadow init: Id “2″ respawning too fast: disabled for 5 minutes
Apr 29 15:53:10 shadow init: Id “3″ respawning too fast: disabled for 5 minutes
Apr 29 15:53:11 shadow init: Id “4″ respawning too fast: disabled for 5 minutes
Apr 29 15:53:11 shadow init: Id “5″ respawning too fast: disabled for 5 minutes
Apr 29 15:53:11 shadow init: Id “6″ respawning too fast: disabled for 5 minutes

I followed the Debian template creation guide at openvz.org

but forgot this step:

sed -i -e '/getty/d' /etc/inittab

as a casual look at /etc/inittab shows:

# /etc/inittab: init(8) configuration.
# $Id: inittab,v 1.91 2002/01/25 13:35:21 miquels Exp $

# /sbin/getty invocations for the runlevels.
#
# The “id” field MUST be the same as the last
# characters of the device (after “tty”).
#
# Format:
# <id>:<runlevels>:<action>:<process>
#
# Note that on most Debian systems tty7 is used by the X Window System,
# so if you want to add more getty’s go ahead but skip tty7 if you run X.
#
1:2345:respawn:/sbin/getty 38400 tty1
2:23:respawn:/sbin/getty 38400 tty2
3:23:respawn:/sbin/getty 38400 tty3
4:23:respawn:/sbin/getty 38400 tty4
5:23:respawn:/sbin/getty 38400 tty5
6:23:respawn:/sbin/getty 38400 tty6

So, the virtual was trying to respawn ttys (which aren’t needed on a virtual), and unable to do so.

Deleting the offending lines in /etc/innittab gets rid of the problem (I ran sed -i -e '/getty/d' /etc/inittab from inside the virtual, as I should have done in the first place).

On a side note, Lenny does not use syslog but rather rsyslog, so the instruction to

Disable sync() for syslog

Turn off doing sync() on every write for syslog’s log files, to improve I/O performance:

sed -i -e ’s@\([[:space:]]\)\(/var/log/\)@\1-\2@’ /etc/syslog.conf

in the Debian template creation guide at openvz.org isn’t strictly necessary[1], since rsyslog doesn’t do a sync() on every write by default.

From http://www.rsyslog.com/doc-v3compatibility.html

Output File Syncing

Rsyslogd tries to keep as compatible to stock syslogd as possible. As such, it retained stock syslogd’s default of syncing every file write if not specified otherwise (by placing a dash in front of the output file name). While this was a useful feature in past days where hardware was much less reliable and UPS seldom, this no longer is useful in today’s worl. Instead, the syncing is a high performace hit. With it, rsyslogd writes files around 50 *times* slower than without it. It also affects overall system performance due to the high IO activity. In rsyslog v3, syncing has been turned off by default. This is done via a specific configuration directive “$ActionFileEnableSync on/off” which is off by default. So even if rsyslogd finds sync selector lines, it ignores them by default. In order to enable file syncing, the administrator must specify “$ActionFileEnableSync on” at the top of rsyslog.conf. This ensures that syncing only happens in some installations where the administrator actually wanted that (performance-intense) feature. In the fast majority of cases (if not all), this dramatically increases rsyslogd performance without any negative effects.


[1] Running this command does nothing on a new Lenny install since /etc/syslog.conf does not exist.

June 3, 2007

FreeBSD Firefox Flash Java

I’ve tried to get Flash working with Firefox on FreeBSD a number of times, but haven’t been able to get it just right until recently.

I’m using,
FreeBSD 6.1-RELEASE #0: Sun May 7 04:32:43 UTC 2006
with
firefox-2.0_2,1.

The following worked for me (Thanks to a post from Lluis López: here).

1. First install linuxpluginwrapper from the ports tree:

cd usr/ports/www/linuxpluginwrapper
make
make install
make clean

2. Now go to /usr/src and get this patch and patch

cd /usr/src
patch < /home/work/rtld_dlsym_hack.diff
cd libexec/rtld-elf
make clean
make
make install

Substitute the correct path above !! (I downloaded the patch to /home/work/).

This assumes you have sources installed. If the tree under /usr/src is empty, then

sysinstall

[configure >> distributions >> src >> all ]

See The FreeBSD handbook if you’ve forgotten how to use sysinstall.

3. Install linux-flashplugin7

cd /usr/ports/www/linux-flashplugin7
make
make install
make clean

4. Copy /usr/local/share/examples/linuxpluginwrapper/libmap.conf-FreeBSD6 to /etc/libmap.conf

cp /usr/local/share/examples/linuxpluginwrapper/libmap.conf-FreeBSD6 /etc/libmap.conf

No need to edit /etc/libmap.conf.

5. Add symlinks in /usr/local/lib/browser_plugins to libflashplayer.so and flashplayer.xpt to

cd /usr/local/lib/browser_plugins
ln -s /usr/X11R6/lib/linux-mozilla/plugins/libflashplayer.so .
ln -s /usr/X11R6/lib/linux-mozilla/plugins/flashplayer.xpt .

6. Restart your browser, and navigate to about:plugins, you should see shockwave_flash listed:

Shockwave Flash
File name:
libflashplayer.so
Shockwave Flash 7.0 r69
MIME Type Description Suffixes Enabled
application/x-shockwave-flash Shockwave Flash swf Yes
application/futuresplash FutureSplash Player spl Yes

7. Install javavmwrapper

Before we can install Java (I’m using the diablo-jdk from the FreeBSD Foundation in this example) we need to install javavmwrapper from ports.

cd /usr/ports/java/javavmwrapper/
make
make install
make clean

8. Now we can install Java. Download the appropriate JDK package from The FreeBSD foundation and simply issue a pkg_add.

In my case,

pkg_add diablo-jdk-freebsd6.i386.1.5.0.07.01.tbz

8. We need to add a symlink in /usr/local/lib/browser to /usr/local/diablo-jdk1.5.0/jre/plugin/i386/ns7/libjavaplugin_oji.so

cd /usr/local/lib/browser_plugins
ln -s /usr/local/diablo-jdk1.5.0/jre/plugin/i386/ns7/libjavaplugin_oji.so .

9. Restart your browser and navigate to about:plugins you should see

Java(TM) Plug-in diablo-1.5.0-b01
File name:

libjavaplugin_oji.so

Java(TM) Plug-in 1.5.0
MIME Type Description Suffixes Enabled
application/x-java-vm Java Yes
application/x-java-applet Java Yes
application/x-java-applet;version=1.1 Java Yes
application/x-java-applet;version=1.1.1 Java Yes
application/x-java-applet;version=1.1.2 Java Yes
… snipped …
application/x-java-bean;jpi-version=1.5 Java Yes

I tested the Java install by trying the Rubik’s Cube Java Applet. It worked without problems.

Verifiying the Java installation at Sun’s site gives a (soft) error:

Oops! You don’t have the recommended Java installed.
Your Java version is 1.5.0. Please click the button below to get the recommended Java for your computer.

It does however verify that Java version 1.5.0 is installed.


Addendum: An alternative way to install Flash for Firefox, that bypasses the Linux compatibility layer altogether is to use the GNU Flash movie playergnash.

cd /usr/ports/graphics/gnash/
make
make install
make clean

Thanks to Matteo at Matteo’s Wasps’ Nest for this one. See his post for more details.

May 4, 2006

Java on FreeBSD

I just installed Java on my FreeBSD box. In the past it has been a nightmare to install Java due to licensing issues – BUT – the FreeBSD Foundation has negotiated a license with Sun to distribute JRE and JDK binaries for FreeBSD.

Installation is now very easy! Download the appropriate package from here and simply issue a pkg_add.

In my case it was

pkg_add diablo-jdk-freebsd6.i386.1.5.0.07.01.tbz

(I am currently running FreeBSD 6.1).

My installation baulked because javavmwrapper was not installed. A simple install from ports was all that was needed (it is a small package).

cd /usr/ports/java/javavmwrapper/
make
make install
make clean

After this I ran pkg_add again and the install went through without incident. It sure beats compiling the whole thing from sources.

March 16, 2006

Stopping qmail

There are plenty of times that you need to stop Qmail for maintainence purposes.

The qmailctl script generally does a good job of stopping qmail, but sometimes it does not quite shutdown all of the qmail processes.

[root@here]# qmailctl stop
Stopping qmail…
qmail-smtpd
qmail-send

[root@here]# qmailctl stat
/service/qmail-send: up (pid 32436) 358918 seconds, want down
/service/qmail-send/log: down 3 seconds, normally up
/service/qmail-smtpd: down 3 seconds, normally up
/service/qmail-smtpd/log: down 3 seconds, normally up
messages in queue: 142
messages in queue but not yet preprocessed: 0

As you can see from the output of qmailctl stat, the qmail-send program is still running. If you do a[1]

ps waux | grep qmail | grep -v grep

you will often find output like

[root@here]# psq
root 55033 0.0 0.1 1196 480 ?? I Tue05PM 0:00.01 supervise qmail-send
qmails 55035 0.0 0.1 1340 760 ?? I Tue05PM 1:28.27 qmail-send
root 55036 0.0 0.1 1196 480 ?? I Tue05PM 0:00.01 supervise qmail-smtpd
qmaill 55040 0.0 0.1 1224 532 ?? I Tue05PM 0:29.48 multilog t s100000 n20 /var/log/qmail/qmail-send
root 55043 0.0 0.1 1216 500 ?? I Tue05PM 0:00.03 tcpserver -H -R -v -c100 0 110 qmail-popup
root 55048 0.0 0.1 1216 548 ?? I Tue05PM 0:00.02 qmail-lspawn ./Maildir
qmailr 55049 0.0 0.1 1284 684 ?? I Tue05PM 0:22.64 qmail-rspawn
qmailq 55050 0.0 0.1 1212 544 ?? I Tue05PM 0:04.69 qmail-clean
qmailr 83484 0.0 0.2 2432 1196 ?? I 2:38AM 0:00.00 qmail-remote yahoo.com webmaster@here.com there@yahoo.com
qmailr 83498 0.0 0.2 2432 1196 ?? I 2:38AM 0:00.01 qmail-remote hotmail.com webmaster@here.com you@yahoo.com

If you then run qmailctl start, you will often have the situation with qmail-send showing up as running for 1 second while the other qmail processes are up for much longer. Basically a second qmail-send is trying to start and is dying after a second. The first instance of qmail-send is still running and needs to be stopped.

The output often contains a mixture of qmail-send, qmail-remote, qmail-clean and others depending on how your mail has been setup and what is in the process of being sent/received when you issue the qmailctl stop command.

At this point, we can wait a bit, to see if qmail-send eventually goes down or whether we will have to kill it explicitly. It is worth trying another qmailctl stop at this point. If qmailctl stat shows qmail-send is still up, which is often the case, it is probably time to kill qmail-send directly.

I usually do a kill -9 qmail-send. Then follow it with psq to see what is still running. Often I have to run a number of kills and killalls, such as killall qmail-remote. We’re not really shutting down qmail very cleanly, but sometimes, especially when you’re in a hurry it makes sense to kill things rather than wait for them to terminate.

Eventually only the supervise scripts should be running,

[root@work]#psq
root 561 0.0 0.0 1148 84 con- I Sat02AM 0:00.05 supervise qmail-send
root 563 0.0 0.0 1148 84 con- I Sat02AM 0:00.06 supervise qmail-smtpd

At this stage I usually call /usr/local/etc/rc.d/svscan.sh stop. The location of your actual svscan.sh script may be different from mine.

A psq shows nothing running. A sockstat -4 | grep 25 shows nothing listening on the smtp port. Now is the time to go in and do maintainance – such as recompile qmail, dequeue mail or repair the queue.

Once you’re done, starting qmail is as simple as /usr/local/etc/rc.d/svscan.sh start.


[1] Actually I use this snippet quite a bit so to save time, I’ve copied it to a file called psq under /usr/local/bin

echo "ps waux | grep qmail | grep -v grep" > /usr/local/bin/psq

so that simply typing psq gives me a list of all the running qmail processes.