Postfix smtpd killed by signal 11: bad command startup

[Jonas De Vuyst]

Hi,

I was hoping someone could help me out with a Postfix-problem that I
ran into after upgrading from FC1 to FC2. I did Google around first
but found nothing that really got me closer to a fix. Below are some
of my findings.

rpm -q postfix: postfix-2.0.18-4

What works:
- echo Message | mail someone@otherserver
- echo Message | mail someone@myserver

What broke:
- incoming e-mail
- outgoing e-mail when using smtpd

The /var/log/maillog error:

Oct 31 00:28:24 foobar postfix/smtpd[25296]: running:
PATH=/bin:/usr/bin:/usr/local/bin; export PATH;         (echo cont;
echo where) | gdb /usr/libexec/postfix/smtpd 25296 2>&1
> /etc/postfix/smtpd.25296.log & sleep 5 
Oct 31 00:28:29
foobar postfix/master[25291]: warning: process
/usr/libexec/postfix/smtpd pid 25296 killed by signal 11
Oct 31 00:28:29 foobar postfix/master[25291]: warning:
/usr/libexec/postfix/smtpd: bad command startup -- throttling

All this tells me that smtpd is the one failing component.

So I Googled around and stumbled on some mailing list conversations
wherein similar smtpd segfaults were discussed. I found two cruelpits
were mentioned in particular:

1. db4, in combination with nsswitch and openldap. However, I don't
use OpenLDAP and I don't have nss_db(-compat) installed.
/etc/nsswitch.conf also doesn't contain any 'db' lines.
2. sasl. In the past there seem to have been problems with a
Postfix/Sasl1 combination but FC2 uses sasl2. Some Google results laid
blame on sasl version mismatches between different e-mail components,
but they seem not to be applicable to my system.

That's not to say they're no longer suspect though.

With db4, for example, I ran into problems earlier with respect to my
Subversion repositories (in addition to the change in db schemas). A
dump/load cycle fixed this though. Because of this I rebuilt the alias
and virtual map databases but this didn't help.

That nss could be to blame shows from the gdb results that I'll paste
below. The exact command, which I took from a Postfix debug help file,
was:

debugger_command =
         PATH=/bin:/usr/bin:/usr/local/bin; export PATH;
         (echo cont; echo where) | gdb $daemon_directory/$process_name
$process_id 2>&1 >$config_directory/$process_name.$process_id.log &
sleep 5

Notice though, that the 'cont' and 'where' commands didn't seem to be
executed. I don't know why that is. I found one e-mail online from
someone with a similar problem where the result of cont and where gave
a hint at sasl. I don't know if this can be generalised to my case,

-----

GNU gdb Red Hat Linux (6.0post-0.20040223.19rh)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux-gnu"...(no debugging
symbols found)...Using host libthread_db library
"/lib/tls/libthread_db.so.1".

Attaching to program: /usr/libexec/postfix/smtpd, process 25296
Reading symbols from /usr/lib/libldap.so.2...(no debugging symbols
found)...done.
Loaded symbols for /usr/lib/libldap.so.2
Reading symbols from /usr/lib/liblber.so.2...(no debugging symbols
found)...done.
Loaded symbols for /usr/lib/liblber.so.2
Reading symbols from /lib/libpcre.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib/libpcre.so.0
Reading symbols from /usr/lib/libsasl2.so.2...(no debugging symbols
found)...done.
Loaded symbols for /usr/lib/libsasl2.so.2
Reading symbols from /lib/libssl.so.4...(no debugging symbols found)...done.
Loaded symbols for /lib/libssl.so.4
Reading symbols from /lib/libcrypto.so.4...(no debugging symbols found)...done.
Loaded symbols for /lib/libcrypto.so.4
Reading symbols from /usr/lib/libgssapi_krb5.so.2...(no debugging
symbols found)...done.
Loaded symbols for /usr/lib/libgssapi_krb5.so.2
Reading symbols from /usr/lib/libkrb5.so.3...(no debugging symbols
found)...done.
Loaded symbols for /usr/lib/libkrb5.so.3
Reading symbols from /lib/libcom_err.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libcom_err.so.2
Reading symbols from /usr/lib/libk5crypto.so.3...(no debugging symbols
found)...done.
Loaded symbols for /usr/lib/libk5crypto.so.3
Reading symbols from /lib/libresolv.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libresolv.so.2
Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /usr/lib/libz.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libz.so.1
Reading symbols from /lib/tls/libdb-4.2.so...(no debugging symbols
found)...done.
Loaded symbols for /lib/tls/libdb-4.2.so
Reading symbols from /lib/libnsl.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libnsl.so.1
Reading symbols from /lib/tls/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/tls/libc.so.6
Reading symbols from /lib/libcrypt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libcrypt.so.1
Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /lib/tls/libpthread.so.0...(no debugging symbols
found)...done.
[Thread debugging using libthread_db enabled]
[New Thread -151037152 (LWP 25296)]
Loaded symbols for /lib/tls/libpthread.so.0
Reading symbols from /lib/libnss_files.so.2...(no debugging symbols
found)...done.
Loaded symbols for /lib/libnss_files.so.2
Loaded symbols for /lib/libnss_files.so.2
Loaded symbols for /lib/libnss_files.so.2
0x008e27a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
(gdb) Hangup detected on fd 0
error detected on stdin
Detaching from program: /usr/libexec/postfix/smtpd, process 25296

-----

So if anyone could help, I'd appreciate it much.

Thanks,
Jonas

[Jonas De Vuyst]

Hi,

A kind person sent me an e-mail explaining how they solved a Postfix
problem similar to one that I had last month when upgrading from FC1
to FC2 and about which I sent an e-mail to this list at the time. As I
myself had spent quite some time looking for the problem before (with
help from postfix-users) I figured out how to fix it, I think that it
would only be polite to update this list with an explanation. So,
forgive the tardiness, here it is.

The problem that causes the symptoms quoted below turned out to be
some missing files in /var/spool/postfix/ that Postfix needs if smtpd
is run in chroot mode.

One solution (used by the person whose e-mail prompted me to write
this) is to disable the chroot option in /etc/postfix/master.cf as
follows:

smtp    inet    n       -       n       -       -       smtpd

An alternative solution, which I used, is to copy the missing files to
/var/spool/postfix. In particular this meant deleting
/var/spool/postfix/etc which was a regular file (this somehow worked
fine on FC1), copying host.conf and resolv.conf from /etc/ to
/var/spool/postfix/etc, and lastly copying /usr/lib/libnss_dns.so.2 to
/var/spool/postfix/usr/lib.

Regards

On Sun, 31 Oct 2004 01:10:35 +0200, Jonas De Vuyst <jdevuyst(a)gmail.com&gt; wrote:
>  Hi,
>  
>  I was hoping someone could help me out with a Postfix-problem that I
>  ran into after upgrading from FC1 to FC2. I did Google around first
>  but found nothing that really got me closer to a fix. Below are some
>  of my findings.
>  
>  rpm -q postfix: postfix-2.0.18-4
>  
>  What works:
>  - echo Message | mail someone@otherserver
>  - echo Message | mail someone@myserver
>  
>  What broke:
>  - incoming e-mail
>  - outgoing e-mail when using smtpd
>  
>  The /var/log/maillog error:
>  
>  Oct 31 00:28:24 foobar postfix/smtpd[25296]: running:
>  PATH=/bin:/usr/bin:/usr/local/bin; export PATH;         (echo cont;
>  echo where) | gdb /usr/libexec/postfix/smtpd 25296 2>&1
>  >/etc/postfix/smtpd.25296.log & sleep 5
>  Oct 31 00:28:29 foobar postfix/master[25291]: warning: process
>  /usr/libexec/postfix/smtpd pid 25296 killed by signal 11
>  Oct 31 00:28:29 foobar postfix/master[25291]: warning:
>  /usr/libexec/postfix/smtpd: bad command startup -- throttling
>  
>  All this tells me that smtpd is the one failing component.
>  
>  So I Googled around and stumbled on some mailing list conversations
>  wherein similar smtpd segfaults were discussed. I found two cruelpits
>  were mentioned in particular:
>  
>  1. db4, in combination with nsswitch and openldap. However, I don't
>  use OpenLDAP and I don't have nss_db(-compat) installed.
>  /etc/nsswitch.conf also doesn't contain any 'db' lines.
>  2. sasl. In the past there seem to have been problems with a
>  Postfix/Sasl1 combination but FC2 uses sasl2. Some Google results laid
>  blame on sasl version mismatches between different e-mail components,
>  but they seem not to be applicable to my system.
>  
>  That's not to say they're no longer suspect though.
>  
>  With db4, for example, I ran into problems earlier with respect to my
>  Subversion repositories (in addition to the change in db schemas). A
>  dump/load cycle fixed this though. Because of this I rebuilt the alias
>  and virtual map databases but this didn't help.
>  
>  That nss could be to blame shows from the gdb results that I'll paste
>  below. The exact command, which I took from a Postfix debug help file,
>  was:
>  
>  debugger_command =
>           PATH=/bin:/usr/bin:/usr/local/bin; export PATH;
>           (echo cont; echo where) | gdb $daemon_directory/$process_name
>  $process_id 2>&1 >$config_directory/$process_name.$process_id.log &
>  sleep 5
>  
>  Notice though, that the 'cont' and 'where' commands didn't seem to
> be
>  executed. I don't know why that is. I found one e-mail online from
>  someone with a similar problem where the result of cont and where gave
>  a hint at sasl. I don't know if this can be generalised to my case,
>  
>  -----
>  
>  GNU gdb Red Hat Linux (6.0post-0.20040223.19rh)
>  Copyright 2004 Free Software Foundation, Inc.
>  GDB is free software, covered by the GNU General Public License, and you are
>  welcome to change it and/or distribute copies of it under certain conditions.
>  Type "show copying" to see the conditions.
>  There is absolutely no warranty for GDB.  Type "show warranty" for details.
>  This GDB was configured as "i386-redhat-linux-gnu"...(no debugging
>  symbols found)...Using host libthread_db library
>  "/lib/tls/libthread_db.so.1".
>  
>  Attaching to program: /usr/libexec/postfix/smtpd, process 25296
>  Reading symbols from /usr/lib/libldap.so.2...(no debugging symbols
>  found)...done.
>  Loaded symbols for /usr/lib/libldap.so.2
>  Reading symbols from /usr/lib/liblber.so.2...(no debugging symbols
>  found)...done.
>  Loaded symbols for /usr/lib/liblber.so.2
>  Reading symbols from /lib/libpcre.so.0...(no debugging symbols found)...done.
>  Loaded symbols for /lib/libpcre.so.0
>  Reading symbols from /usr/lib/libsasl2.so.2...(no debugging symbols
>  found)...done.
>  Loaded symbols for /usr/lib/libsasl2.so.2
>  Reading symbols from /lib/libssl.so.4...(no debugging symbols found)...done.
>  Loaded symbols for /lib/libssl.so.4
>  Reading symbols from /lib/libcrypto.so.4...(no debugging symbols found)...done.
>  Loaded symbols for /lib/libcrypto.so.4
>  Reading symbols from /usr/lib/libgssapi_krb5.so.2...(no debugging
>  symbols found)...done.
>  Loaded symbols for /usr/lib/libgssapi_krb5.so.2
>  Reading symbols from /usr/lib/libkrb5.so.3...(no debugging symbols
>  found)...done.
>  Loaded symbols for /usr/lib/libkrb5.so.3
>  Reading symbols from /lib/libcom_err.so.2...(no debugging symbols found)...done.
>  Loaded symbols for /lib/libcom_err.so.2
>  Reading symbols from /usr/lib/libk5crypto.so.3...(no debugging symbols
>  found)...done.
>  Loaded symbols for /usr/lib/libk5crypto.so.3
>  Reading symbols from /lib/libresolv.so.2...(no debugging symbols found)...done.
>  Loaded symbols for /lib/libresolv.so.2
>  Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done.
>  Loaded symbols for /lib/libdl.so.2
>  Reading symbols from /usr/lib/libz.so.1...(no debugging symbols found)...done.
>  Loaded symbols for /usr/lib/libz.so.1
>  Reading symbols from /lib/tls/libdb-4.2.so...(no debugging symbols
>  found)...done.
>  Loaded symbols for /lib/tls/libdb-4.2.so
>  Reading symbols from /lib/libnsl.so.1...(no debugging symbols found)...done.
>  Loaded symbols for /lib/libnsl.so.1
>  Reading symbols from /lib/tls/libc.so.6...(no debugging symbols found)...done.
>  Loaded symbols for /lib/tls/libc.so.6
>  Reading symbols from /lib/libcrypt.so.1...(no debugging symbols found)...done.
>  Loaded symbols for /lib/libcrypt.so.1
>  Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done.
>  Loaded symbols for /lib/ld-linux.so.2
>  Reading symbols from /lib/tls/libpthread.so.0...(no debugging symbols
>  found)...done.
>  [Thread debugging using libthread_db enabled]
>  [New Thread -151037152 (LWP 25296)]
>  Loaded symbols for /lib/tls/libpthread.so.0
>  Reading symbols from /lib/libnss_files.so.2...(no debugging symbols
>  found)...done.
>  Loaded symbols for /lib/libnss_files.so.2
>  Loaded symbols for /lib/libnss_files.so.2
>  Loaded symbols for /lib/libnss_files.so.2
>  0x008e27a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
>  (gdb) Hangup detected on fd 0
>  error detected on stdin
>  Detaching from program: /usr/libexec/postfix/smtpd, process 25296
>  
>  -----
>  
>  So if anyone could help, I'd appreciate it much.
>  
>  Thanks,
>  Jonas
>