hitch is broken in epel7, fix in fedora may or may not break production systems (bz #1731420)

[Ingvar Hagelund]

Hello 

hitch is a TLS terminating network proxy, made to be lean and mean and do nothing else
than terminating TLS. It fits hand-in-glove with varnish cache. I maintain hitch in Fedora
and EPEL. 

There is a bug in the current epel7 config that is fixed in the latest rawhide update. In
short, the bug is that with the default config, hitch forks a daemon, while the systemd
hitch service says Type=simple. See Bugzilla bug #1731420. 

The fedora update fixes the problem by changing the systemd service to Type=forking. 

There were two ways to get around the bug: 

- Set daemon=off in hitch.conf. That file is marked with noreplace, so the update will not
overwrite this fix. As this does not match the updated Type=forking in hitch.service,
hitch will not start after the update. 

- Set Type=forking in hitch.service. This is the same fix as in the update, so this should
be safe. 

Also, the Fedora update adds a systemd limits.conf including LimitNOFILE=10240 that is
important, as the default value (1024) would trig network problems on a medium busy site
(true story). 

Is it safe to push this update to epel7? 

Ingvar

[Stephen John Smoogen]

On Tue, 12 Nov 2019 at 17:06, Ingvar Hagelund <ingvar(a)redpill-linpro.com&gt; wrote:
>  Hello
>
>  hitch is a TLS terminating network proxy, made to be lean and mean and do nothing else
> than terminating TLS. It fits hand-in-glove with varnish cache. I maintain hitch in Fedora
> and EPEL.
>
>  There is a bug in the current epel7 config that is fixed in the latest rawhide update. In
> short, the bug is that with the default config, hitch forks a daemon, while the systemd
> hitch service says Type=simple. See Bugzilla bug #1731420.
>
>  The fedora update fixes the problem by changing the systemd service to Type=forking.
>
>  There were two ways to get around the bug:
>
>  - Set daemon=off in hitch.conf. That file is marked with noreplace, so the update will
> not overwrite this fix. As this does not match the updated Type=forking in hitch.service,
> hitch will not start after the update.
>
>  - Set Type=forking in hitch.service. This is the same fix as in the update, so this
> should be safe.
>
>  Also, the Fedora update adds a systemd limits.conf including LimitNOFILE=10240 that is
> important, as the default value (1024) would trig network problems on a medium busy site
> (true story).
>
>  Is it safe to push this update to epel7? 
This was discussed at today's EPEL meeting and approved. Please push
this to epel-testing and let users know in any tickets that it can be
used there. After that, push to stable after regular feedback time.



-- 
Stephen J Smoogen.

[Leon Fauster]

Am 13.11.19 um 20:44 schrieb Stephen John Smoogen:
>  On Tue, 12 Nov 2019 at 17:06, Ingvar Hagelund
> <ingvar(a)redpill-linpro.com&gt; wrote:
> >
> > Hello
> >
> > hitch is a TLS terminating network proxy, made to be lean and mean and do nothing
> else than terminating TLS. It fits hand-in-glove with varnish cache. I maintain hitch in
> Fedora and EPEL.
> >
> > There is a bug in the current epel7 config that is fixed in the latest rawhide
> update. In short, the bug is that with the default config, hitch forks a daemon, while the
> systemd hitch service says Type=simple. See Bugzilla bug #1731420.
> >
> > The fedora update fixes the problem by changing the systemd service to Type=forking.
> >
> > There were two ways to get around the bug:
> >
> > - Set daemon=off in hitch.conf. That file is marked with noreplace, so the update
> will not overwrite this fix. As this does not match the updated Type=forking in
> hitch.service, hitch will not start after the update.
> >
> > - Set Type=forking in hitch.service. This is the same fix as in the update, so this
> should be safe.
> >
> > Also, the Fedora update adds a systemd limits.conf including LimitNOFILE=10240 that
> is important, as the default value (1024) would trig network problems on a medium busy
> site (true story).
> >
> > Is it safe to push this update to epel7?
>  
>  This was discussed at today's EPEL meeting and approved. Please push
>  this to epel-testing and let users know in any tickets that it can be
>  used there. After that, push to stable after regular feedback time.
>  
>  
>   

I'd really appreciate if it could also be branched into EPEL8?

-- 
Thanks
Leon


PS: Could a list admin help me, please. I subscribed with a @gmail.com
address some weeks ago. google's SMTP rewrites it to @googlemail.com
when trying to send a mail to the list (it fails to be accepted by
mailman of course). I can not unsubscribe leonfauster(a)gmail.com now.
The @googlemail.com variant is subscribed now (to be able to post to the
list).

Rephrased: Could a admin unsubscribe leonfauster(a)gmail.com please?
Thanks and sorry for the noise.

[Leon Fauster]

Am 14.11.19 um 14:01 schrieb Leon Fauster:
>  Am 13.11.19 um 20:44 schrieb Stephen John Smoogen:
> > On Tue, 12 Nov 2019 at 17:06, Ingvar Hagelund 
> > <ingvar(a)redpill-linpro.com&gt; wrote:
> >>
> >> Hello
> >>
> >> hitch is a TLS terminating network proxy, made to be lean and mean 
> >> and do nothing else than terminating TLS. It fits hand-in-glove with 
> >> varnish cache. I maintain hitch in Fedora and EPEL.
> >>
> >> There is a bug in the current epel7 config that is fixed in the 
> >> latest rawhide update. In short, the bug is that with the default 
> >> config, hitch forks a daemon, while the systemd hitch service says 
> >> Type=simple. See Bugzilla bug #1731420.
> >>
> >> The fedora update fixes the problem by changing the systemd service 
> >> to Type=forking.
> >>
> >> There were two ways to get around the bug:
> >>
> >> - Set daemon=off in hitch.conf. That file is marked with noreplace, 
> >> so the update will not overwrite this fix. As this does not match the 
> >> updated Type=forking in hitch.service, hitch will not start after the 
> >> update.
> >>
> >> - Set Type=forking in hitch.service. This is the same fix as in the 
> >> update, so this should be safe.
> >>
> >> Also, the Fedora update adds a systemd limits.conf including 
> >> LimitNOFILE=10240 that is important, as the default value (1024) 
> >> would trig network problems on a medium busy site (true story).
> >>
> >> Is it safe to push this update to epel7?
> >
> > This was discussed at today's EPEL meeting and approved. Please push
> > this to epel-testing and let users know in any tickets that it can be
> > used there. After that, push to stable after regular feedback time.
> >
> >
>  
>  I'd really appreciate if it could also be branched into EPEL8? 
Thanks for pushing it into fedora-epel/testing/8/ !

--
Leon

[Matthew Blissett]

On 13/11/2019 20:44, Stephen John Smoogen wrote:
>  On Tue, 12 Nov 2019 at 17:06, Ingvar Hagelund
> <ingvar(a)redpill-linpro.com> wrote:
> > Hello
> >
> > hitch is a TLS terminating network proxy, made to be lean and mean and do nothing
> else than terminating TLS. It fits hand-in-glove with varnish cache. I maintain hitch in
> Fedora and EPEL.
> >
> > There is a bug in the current epel7 config that is fixed in the latest rawhide
> update. In short, the bug is that with the default config, hitch forks a daemon, while the
> systemd hitch service says Type=simple. See Bugzilla bug #1731420.
> >
> > The fedora update fixes the problem by changing the systemd service to
> Type=forking.
> >
> > There were two ways to get around the bug:
> >
> > - Set daemon=off in hitch.conf. That file is marked with noreplace, so the update
> will not overwrite this fix. As this does not match the updated Type=forking in
> hitch.service, hitch will not start after the update.
> >
> > - Set Type=forking in hitch.service. This is the same fix as in the update, so this
> should be safe.
> >
> > Also, the Fedora update adds a systemd limits.conf including LimitNOFILE=10240 that
> is important, as the default value (1024) would trig network problems on a medium busy
> site (true story).
> >
> > Is it safe to push this update to epel7?
>  This was discussed at today's EPEL meeting and approved. Please push
>  this to epel-testing and let users know in any tickets that it can be
>  used there. After that, push to stable after regular feedback time.
>  
As anticipated by Ingvar, this update broke my production systems.

The EPEL site [1] says

>  it is possible that occasionally an incompatible update will be
> released
>  such that administrator action is required. By policy these are announced
>  in advance in order to give administrators time to test and provide 
>  suggestions.
>
>  It is strongly recommended that if you make use of EPEL, and especially
>  if you rely upon it, that you subscribe to the epel-announce list.
>  Traffic on this list is kept to a minimum needed to notify administrators
>  of important updates.
>  
This update wasn't announced there -- is that an oversight, or should I
change my approach to updates to account for possible incompatible
updates in the future?

Thanks,

Matthew Blissett

[1]https://fedoraproject.org/wiki/EPEL#Can_I_rely_on_these_packages.3F

[Stephen John Smoogen]

On Thu, 12 Dec 2019 at 05:28, Matthew Blissett <mblissett(a)gbif.org&gt; wrote:
>  On 13/11/2019 20:44, Stephen John Smoogen wrote:
>  > On Tue, 12 Nov 2019 at 17:06, Ingvar Hagelund <ingvar(a)redpill-linpro.com>
> wrote:
>  >> Hello
>  >>
>  >> hitch is a TLS terminating network proxy, made to be lean and mean and do
> nothing else than terminating TLS. It fits hand-in-glove with varnish cache. I maintain
> hitch in Fedora and EPEL.
>  >>
>  >> There is a bug in the current epel7 config that is fixed in the latest rawhide
> update. In short, the bug is that with the default config, hitch forks a daemon, while the
> systemd hitch service says Type=simple. See Bugzilla bug #1731420.
>  >>
>  >> The fedora update fixes the problem by changing the systemd service to
> Type=forking.
>  >>
>  >> There were two ways to get around the bug:
>  >>
>  >> - Set daemon=off in hitch.conf. That file is marked with noreplace, so the
> update will not overwrite this fix. As this does not match the updated Type=forking in
> hitch.service, hitch will not start after the update.
>  >>
>  >> - Set Type=forking in hitch.service. This is the same fix as in the update, so
> this should be safe.
>  >>
>  >> Also, the Fedora update adds a systemd limits.conf including LimitNOFILE=10240
> that is important, as the default value (1024) would trig network problems on a medium
> busy site (true story).
>  >>
>  >> Is it safe to push this update to epel7?
>  > This was discussed at today's EPEL meeting and approved. Please push
>  > this to epel-testing and let users know in any tickets that it can be
>  > used there. After that, push to stable after regular feedback time.
>  As anticipated by Ingvar, this update broke my production systems.
>
>  The EPEL site [1] says
>
>  > it is possible that occasionally an incompatible update will be released
>  > such that administrator action is required. By policy these are announced
>  > in advance in order to give administrators time to test and provide
>  > suggestions.
>  >
>  > It is strongly recommended that if you make use of EPEL, and especially
>  > if you rely upon it, that you subscribe to the epel-announce list.
>  > Traffic on this list is kept to a minimum needed to notify administrators
>  > of important updates.
>
>  This update wasn't announced there -- is that an oversight, or should I
>  change my approach to updates to account for possible incompatible
>  updates in the future?
>  
So by policy an update should have been announced to epel-announce. It
wasn't, it was an oversight but there isn't anything we can do about
that :(. This is very much a volunteer effort and things like this
will happen. I apologize for the problems it caused you, and will
reword that paragraph however it should be.


>  Thanks,
>
>  Matthew Blissett
>
>  [1]https://fedoraproject.org/wiki/EPEL#Can_I_rely_on_these_packages.3F
>
>  _______________________________________________
>  epel-devel mailing list -- epel-devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to epel-devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]... 


-- 
Stephen J Smoogen.