Sent from my iPhone

Begin forwarded message:

From: "R. G. Newbury" <newbury@mandamus.org>
Date: 1 July, 2012 12:50:57 PM EDT
To: Fedora-List <users-request@lists.fedoraproject.org>
Subject: Systemctl hates mysqld: Fails writing to /tmp

I am stumped. I have a puzzler which needs the brain-power and knowledge which hands around here.

Systemctl on fedora 16 will NOT start mysqld.

Startup fails with a permissions error, attempting to write to /tmp, which is chmod 1777, that is, world writeable and readable! There is nothing wrong with the mysql setup. If I run each command from the command line, mysqld starts without problem. That is to say: there is nothing wrong with the mysqld setup. I repeat: there is nothing wrong with the mysqld setup.

In order to avoid unhelpful responses/questions relating to the many many many other ways in which one can have mysqld fail to start, I list some of the sanity checks which I have used to ensure that everything lines up properly: I have checked my.cnf and use the same command line switches, and folders and permissions throughout. Everything run as root. Selinux disabled. And mysqld *will* start properly from the commandline, just NOT by way of systemctl.

*** Proof of Setup
# mkdir /var/log/mysqld
mkdir: cannot create directory `/var/log/mysqld': File exists
# chown -R mysql:mysql /var/log/mysqld
# chmod -R 777 /var/log/mysqld
# chown -R mysql:mysql /run/mysqld
# chmod -R 777 /run/mysqld
# chmod -R 1777 /tmp
# killall mysqld
mysqld: no process found

**** Replicate mysqld.service commands:

# /usr/libexec/mysqld-prepare-db-dir
# /usr/bin/mysqld_safe --nowatch --datadir=/var/lib/mysql --socket=/var/lib/mysql/mysql.sock --pid-file=/run/mysqld/mysqld.pid --basedir=/usr/ --log-error=/var/log/mysqld/mysqld.log
# /usr/libexec/mysqld-wait-ready $MAINPID

**** and mysqld STARTS
# ps -ae | grep mysqld
20853 pts/3    00:00:00 mysqld

At this point, I can use the console mysql client, etc. etc.
So the mysqld setup is correct. But this is not what happens under systemctl

******** COMPARE AND CONTRAST *******
# killall mysqld

# systemctl start mysqld.service
Job failed. See system logs and 'systemctl status' for details.

*** Edited *** Failure is:
        Process: 25762 ExecStartPost=/usr/libexec/mysqld-wait-ready $MAINPID (code=exited, status=1/FAILURE)

*** Comment out the exec-post command (which is not supposed to be required) and reload:

# systemctl start mysqld.service

******* Note that it gives no error....as appears above
******** Yes the service IS disabled atm, because of this testing
# systemctl status mysqld.service
mysqld.service - MySQL database server
         Loaded: loaded (/lib/systemd/system/mysqld.service; disabled)
         Active: failed since Sun, 01 Jul 2012 12:05:22 -0400; 5s ago
        Process: 26943 ExecStart=/usr/bin/mysqld_safe --nowatch --datadir=/var/lib/mysql --socket=/var/lib/mysql/mysql.sock --pid-file=/run/mysqld/mysqld.pid --basedir=/usr/ --log-error=/var/log/mysqld/mysqld.log (code=exited, status=0/SUCCESS)
        Process: 26927 ExecStartPre=/usr/libexec/mysqld-prepare-db-dir (code=exited, status=0/SUCCESS)
       Main PID: 27128 (code=exited, status=1/FAILURE)
         CGroup: name=systemd:/system/mysqld.service

SO, the operation was a success but the patient died...

**** mysqld.log

**** First the command line startup and shutdown:

120701 12:15:30 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
120701 12:15:30 [Note] Plugin 'FEDERATED' is disabled.
120701 12:15:30 InnoDB: The InnoDB memory heap is disabled
120701 12:15:30 InnoDB: Mutexes and rw_locks use GCC atomic builtins
120701 12:15:30 InnoDB: Compressed tables use zlib 1.2.5
120701 12:15:30 InnoDB: Using Linux native AIO
120701 12:15:30 InnoDB: Initializing buffer pool, size = 128.0M
120701 12:15:30 InnoDB: Completed initialization of buffer pool
120701 12:15:30 InnoDB: highest supported file format is Barracuda.
120701 12:15:30  InnoDB: Waiting for the background threads to start
120701 12:15:31 InnoDB: 1.1.8 started; log sequence number 5950198631
120701 12:15:31 [Note] Server hostname (bind-address): '127.0.0.1'; port: 3306
120701 12:15:31 [Note]   - '127.0.0.1' resolves to '127.0.0.1';
120701 12:15:31 [Note] Server socket created on IP: '127.0.0.1'.
120701 12:15:31 [Note] Event Scheduler: Loaded 0 events
120701 12:15:31 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.24'  socket: '/var/lib/mysql/mysql.sock'  port: 3306 MySQL Community Server (GPL)
120701 12:16:08 [Note] /usr/libexec/mysqld: Normal shutdown

120701 12:16:08 [Note] Event Scheduler: Purging the queue. 0 events
120701 12:16:08  InnoDB: Starting shutdown...
120701 12:16:09  InnoDB: Shutdown completed; log sequence number 5950198631
120701 12:16:09 [Note] /usr/libexec/mysqld: Shutdown complete
120701 12:16:08 [Note] Event Scheduler: Purging the queue. 0 events
120701 12:16:08  InnoDB: Starting shutdown...

**** All as expected
**** Systemctl startup:

20701 12:23:23 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
120701 12:23:23 [Note] Plugin 'FEDERATED' is disabled.
120701 12:23:23 InnoDB: The InnoDB memory heap is disabled
120701 12:23:23 InnoDB: Mutexes and rw_locks use GCC atomic builtins
120701 12:23:23 InnoDB: Compressed tables use zlib 1.2.5
120701 12:23:23 InnoDB: Using Linux native AIO

********** HERE IS THE ERROR

^G/usr/libexec/mysqld: Can't create/write to file '/tmp/ibVcsIip' (Errcode: 13)
120701 12:23:23  InnoDB: Error: unable to create temporary file; errno: 13
120701 12:23:23 [ERROR] Plugin 'InnoDB' init function returned error.
120701 12:23:23 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
120701 12:23:23 [ERROR] Unknown/unsupported storage engine: InnoDB
120701 12:23:23 [ERROR] Aborting

120701 12:23:23 [Note] /usr/libexec/mysqld: Shutdown complete

I can find NO other error messages. /tmp is 1777: completely world writeable, but systemctl/mysqld cannot write there.

Sorry for the length. Can anyone shed ANY light on this?

(And for all our sakes when replying, please PLEASE PLEASE chop off at the least everything above the 'HERE IS THE ERROR LINE'. It is all background and setup and not relevant to the actual error. It only shows *where* the error appears.

ATM, my workaround is to disable mysqld.service, and start mysqld as an /etc/rc.d/init.d service by calling mysqld-prepare-db-dir and mysqld_safe from there. It's a bodge (and that's what I call it!) And without mysql, my mythbox is useless. And Spain v Italy is this afternoon!

I want to know why mysqld cannot write to /tmp when started by systemctl. (Anyone know if Lennart reads this list?)


Geoff