[Bug 1337216] New: p1_pam fails to start - ejabberd barfs

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1337216

            Bug ID: 1337216
           Summary: p1_pam fails to start - ejabberd barfs
           Product: Fedora
           Version: rawhide
         Component: ejabberd
          Severity: high
          Assignee: lemenkov(a)gmail.com
          Reporter: alan.milligan(a)last-bastion.net
        QA Contact: extras-qa(a)fedoraproject.org
                CC: erlang(a)lists.fedoraproject.org, jeremy(a)jcline.org,
                    lemenkov(a)gmail.com, martin(a)laptop.org,
                    rbarlow(a)redhat.com



Description of problem:
p1_pam fails to start because ejabberd is looking in the incorrect place for
epam.beam (in ejabberd file path instead of p1_pam) - sorry haven't looked at
code to provide patch


Version-Release number of selected component (if applicable):
Rawhide 16.04


How reproducible:
Always

Steps to Reproduce:
1. Ensure PAM turned on in ejabberd.yml
2. start ejabberd
3. it fails: check /var/log/ejabberd/server.log

Actual results:
/var/log/ejabberd/server.log
2016-05-18 23:40:47.128 [info] <0.38.0>@cyrsasl_digest:start:60 FQDN used to
check DIGEST-MD5 SASL authentication: jenkins-dev.last-bastion.net
2016-05-18 23:40:47.154 [error] <0.392.0> Can't open file
"/usr/lib64/erlang/lib/ejabberd/priv/bin/epam": enoent
2016-05-18 23:40:47.154 [error] <0.392.0> CRASH REPORT Process epam with 0
neighbours exited with reason: bad return value: error in gen_server:init_it/6
line 356
2016-05-18 23:40:47.154 [error] <0.391.0> Supervisor epam_sup had child epam
started with epam:start_link() at undefined exit with reason bad return value:
error in context start_error
2016-05-18 23:40:47.154 [error] <0.389.0> CRASH REPORT Process <0.389.0> with
0
neighbours exited with reason:
{{shutdown,{failed_to_start_child,epam,{bad_return_value,error}}},{epam_app,start,[normal,[]]}}
in application_master:init/4 line 134
2016-05-18 23:40:47.154 [critical] <0.38.0>@ejabberd:exit_or_halt:133 failed to
start application 'p1_pam': {error,
                                       {{shutdown,
                                         {failed_to_start_child,epam,
                                          {bad_return_value,error}}},
                                        {epam_app,start,[normal,[]]}}}
2016-05-18 23:40:47.155 [info] <0.7.0> Application p1_pam exited with reason:
{{shutdown,{failed_to_start_child,epam,{bad_return_value,error}}},{epam_app,start,[normal,[]]}}

Expected results:


Additional info:

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 1337216] p1_pam fails to start - ejabberd barfs

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1337216



--- Comment #1 from Randy Barlow <rbarlow(a)redhat.com&gt; ---
I can confirm this issue, though I'm not yet sure why it happens. Thanks for
the report, I'll keep digging to see if I can figure out what's happening.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 1337216] p1_pam fails to start - ejabberd barfs

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1337216

Randy Barlow <rbarlow(a)redhat.com&gt; changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
           Assignee|lemenkov(a)gmail.com          |rbarlow(a)redhat.com



--- Comment #2 from Randy Barlow <rbarlow(a)redhat.com&gt; ---
I think this may be an issue in p1_pam, though I'm not completely sure yet. I
was poking around trying to figure out how this works, and I noticed this
function which seems to be generating that particular path:

https://github.com/processone/epam/blob/1.0.0/src/epam.erl#L142-L153

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 1337216] p1_pam fails to start - ejabberd barfs

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1337216



--- Comment #3 from Alan Milligan <alan.milligan(a)last-bastion.net&gt; ---
Hmmm,
That's certainly a little strange.  This *should* just be a the local module
path surely.
But we're not explicitly setting EJABBERD_BIN_PATH out of systemd nor
ejabberdctl.cfg - so wouldn't it behave itself??

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 1337216] p1_pam looks into ejabberd' s install path to find epam instead of its own path

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1337216

Randy Barlow <rbarlow(a)redhat.com&gt; changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|ejabberd                    |erlang-p1_pam
            Summary|p1_pam fails to start -     |p1_pam looks into
                   |ejabberd barfs              |ejabberd's install path to
                   |                            |find epam instead of its
                   |                            |own path



--- Comment #4 from Randy Barlow <rbarlow(a)redhat.com&gt; ---
Hello Alan!

I've written a patch that basically uses only the false clause of that case
statement and tested it on my Rawhide box and now ejabberd does at least start.
I'm going to make a new release of erlang-p1_pam with this patch for now, and
then I'll file an upstream issue about this so we can get it solved in a more
general way for everyone (not just Fedora).

I agree that it's strange. Perhaps ejabberd itself is setting that variable,
but I've honestly not done any investigation to determine where that is coming
from. I did talk to Holger in the official ejabberd MUC and he confirmed that
he had seen a similar issue.

Thanks for the report, and I'll drop a link here when I report it upstream. Oh,
and please feel free to let me know how your deployment goes! Jeremy Cline and
I did a whole lot of work on ejabberd to upgrade to a recent version and split
out the dependencies over the winter. It was much more work than we bargained
for, so I hope it pays off well for our users!

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 1337216] p1_pam looks into ejabberd' s install path to find epam instead of its own path

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1337216



--- Comment #5 from Alan Milligan <alan.milligan(a)last-bastion.net&gt; ---
Just another thought - and I haven't really looked into this: but if
priv/bin/epam starts up as ejabberd user, will it have enough perms et al to
actually do pam-based authentication (ie read /etc/shadow)??

This is an issue with ipsilon and Apache-based PAM auth which requires an
suexec if I recall.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 1337216] p1_pam looks into ejabberd' s install path to find epam instead of its own path

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1337216

Randy Barlow <rbarlow(a)redhat.com&gt; changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Priority|unspecified                 |high
            Version|rawhide                     |24



--- Comment #6 from Randy Barlow <rbarlow(a)redhat.com&gt; ---
Hello Alan!

I don't know the answer to your question as I've (obviously) not used the epam
module before. It does sound like a reasonable guess that that might be a
problem. Once you test the patch I'm making I suppose we'll quickly find out if
that is an issue as well. If it is, we can open a new issue to track that
problem separately.

This issue also affects Fedora 24, so I'm going to just mark the ticket as 24
and fix it on both since they pretty much have the same package.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 1337216] p1_pam looks into ejabberd' s install path to find epam instead of its own path

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1337216



--- Comment #7 from Randy Barlow <rbarlow(a)redhat.com&gt; ---
I have filed this issue upstream:

https://github.com/processone/epam/issues/4

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 1337216] p1_pam looks into ejabberd' s install path to find epam instead of its own path

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1337216

Randy Barlow <rbarlow(a)redhat.com&gt; changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Fixed In Version|                            |erlang-p1_pam-1.0.0-2.fc25



--- Comment #8 from Randy Barlow <rbarlow(a)redhat.com&gt; ---
erlang-p1_pam-1.0.0-2.fc25

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 1337216] p1_pam looks into ejabberd' s install path to find epam instead of its own path

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1337216



--- Comment #9 from Randy Barlow <rbarlow(a)redhat.com&gt; ---
Hello Alan! When you have a moment, give erlang-p1_pam-1.0.0-2.fc25 a try. It
will probably take a day or so before it's in the mirrors, but you can get it
from Koji directly if you want:

http://koji.fedoraproject.org/koji/taskinfo?taskID=14215898

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 1337216] p1_pam looks into ejabberd' s install path to find epam instead of its own path

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1337216



--- Comment #10 from Fedora Update System <updates(a)fedoraproject.org&gt; ---
erlang-p1_pam-1.0.0-2.fc24 has been submitted as an update to Fedora 24.
https://bodhi.fedoraproject.org/updates/FEDORA-2016-20999100c6

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 1337216] p1_pam looks into ejabberd' s install path to find epam instead of its own path

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1337216

Fedora Update System <updates(a)fedoraproject.org&gt; changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |MODIFIED



-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 1337216] p1_pam looks into ejabberd' s install path to find epam instead of its own path

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1337216



--- Comment #11 from Alan Milligan <alan.milligan(a)last-bastion.net&gt; ---
Ok.  This patch indeed fixes the reported problem.

But a cursory test suggests that there may well be PAM authentication issues.  

Using mcabber connecting to localhost/ejabberd, if I use an /etc/passwd-based
user (ie root), ejabberd fails authentication.

This vm has 389DS/nslcd set up, and if I use an LDAP-based user, then (PAM)
authentication works fine.

I somehow suspect that if one delves further this will prove to be a 'known
issue' with PAM - and one should configure ejabberd/ldap directly - unless one
runs ejabberd as root (which I didn't try...).

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 1337216] p1_pam looks into ejabberd' s install path to find epam instead of its own path

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1337216

Fedora Update System <updates(a)fedoraproject.org&gt; changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|MODIFIED                    |ON_QA



--- Comment #12 from Fedora Update System <updates(a)fedoraproject.org&gt; ---
erlang-p1_pam-1.0.0-2.fc24 has been pushed to the Fedora 24 testing repository.
If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here:
https://bodhi.fedoraproject.org/updates/FEDORA-2016-20999100c6

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 1337216] p1_pam looks into ejabberd' s install path to find epam instead of its own path

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1337216



--- Comment #13 from Fedora Update System <updates(a)fedoraproject.org&gt; ---
erlang-p1_pam-1.0.0-2.fc24 has been pushed to the Fedora 24 stable repository.
If problems still persist, please make note of it in this bug report.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 1337216] p1_pam looks into ejabberd' s install path to find epam instead of its own path

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1337216

Fedora Update System <updates(a)fedoraproject.org&gt; changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ON_QA                       |CLOSED
   Fixed In Version|erlang-p1_pam-1.0.0-2.fc25  |erlang-p1_pam-1.0.0-2.fc25
                   |                            |erlang-p1_pam-1.0.0-2.fc24
         Resolution|---                         |ERRATA
        Last Closed|                            |2016-05-27 23:33:46



-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 1337216] p1_pam looks into ejabberd' s install path to find epam instead of its own path

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1337216

Jason Spangler <jasons(a)wumple.com&gt; changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jasons(a)wumple.com



--- Comment #14 from Jason Spangler <jasons(a)wumple.com&gt; ---
I just upgraded to Fedora 24 and ejabberd does not seem to be using epam
correctly for auth - all attempts using PAM fail, even .   This same
configuration used to work under previous Fedora versions.

pamtester succeeds with the same auth attempt:
pamtester -I ruser=jasons ejabberd jasons authenticate

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 1337216] p1_pam looks into ejabberd' s install path to find epam instead of its own path

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1337216



--- Comment #15 from Jason Spangler <jasons(a)wumple.com&gt; ---
According to ejabberd epam docs, epam must be setuid root to function:
https://docs.ejabberd.im/admin/guide/configuration/#pam-authentication

ejabberd was running epam from the erlang-p1_pam package at:
/usr/lib/erlang/lib/p1_pam-1.0.0/priv/bin/epam

Making epam setuid root made ejabberd PAM auth work again:
chmod 4750 /usr/lib/erlang/lib/p1_pam-1.0.0/priv/bin/epam

I would not be surprised if there are negative security implications with this
step.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 1337216] p1_pam looks into ejabberd' s install path to find epam instead of its own path

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1337216

Randy Barlow <randy(a)electronsweatshop.com&gt; changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|CLOSED                      |NEW
                 CC|                            |randy(a)electronsweatshop.com
         Resolution|ERRATA                      |---
           Keywords|                            |Reopened



--- Comment #16 from Randy Barlow <randy(a)electronsweatshop.com&gt; ---
Thanks for the notes Jason, I'm going to reopen this bug for further
consideration. As you note, I have some hesitation to setuid the module but
I'll open the bug so I can think about it some more.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 1337216] p1_pam looks into ejabberd' s install path to find epam instead of its own path

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1337216

Randy Barlow <randy(a)electronsweatshop.com&gt; changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |CLOSED
         Resolution|---                         |ERRATA
        Last Closed|2016-05-27 23:33:46         |2016-08-31 11:27:56



--- Comment #17 from Randy Barlow <randy(a)electronsweatshop.com&gt; ---
I've decided to track the setuid epam thing as a separate bug, since this bug
was really about an install path problem. Thanks for reporting it!

https://bugzilla.redhat.com/show_bug.cgi?id=1371984

-- 
You are receiving this mail because:
You are on the CC list for the bug.