Schedule concern next week: holidays ahead!

[Matthew Miller]

Next week is the crunch week before F26 final, but it's also holidays
in two countries with a lot of key Fedora contributors. In the US,
Tuesday July 4th is Independence Day (and many people may take a long
weekend for family stuff including Monday). In the Czech Republic, both
July 5th and July 6th are public holidays (Saints Cyril and Methodius'
Day, and Jan Hus Day).

This makes me worried key people needed to clear blockers and run
validation may be harder to get hold of than usual. I don't have any
answers, just some general worry. :) Are there current blockers that
could use some early pushing to avoid a crisis next week?


-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[Matthew Miller]

On Wed, Jun 28, 2017 at 01:39:33PM -0400, Matthew Miller wrote:
>  Next week is the crunch week before F26 final, but it's also
> holidays
>  in two countries with a lot of key Fedora contributors. In the US,
>  Tuesday July 4th is Independence Day (and many people may take a long
>  weekend for family stuff including Monday). In the Czech Republic, both
>  July 5th and July 6th are public holidays (Saints Cyril and Methodius'
>  Day, and Jan Hus Day).
>  
>  This makes me worried key people needed to clear blockers and run
>  validation may be harder to get hold of than usual. I don't have any
>  answers, just some general worry. :) Are there current blockers that
>  could use some early pushing to avoid a crisis next week? 
And particularly, Dennis Gilmore tells me that it's likely no one from
Release Engineering able to do the composes will be available Tuesday
at all.

-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[Jan Kurik]

On Wed, Jun 28, 2017 at 7:53 PM, Matthew Miller
<mattdm(a)fedoraproject.org&gt; wrote:
>  On Wed, Jun 28, 2017 at 01:39:33PM -0400, Matthew Miller wrote:
> > Next week is the crunch week before F26 final, but it's also holidays
> > in two countries with a lot of key Fedora contributors. In the US,
> > Tuesday July 4th is Independence Day (and many people may take a long
> > weekend for family stuff including Monday). In the Czech Republic, both
> > July 5th and July 6th are public holidays (Saints Cyril and Methodius'
> > Day, and Jan Hus Day).
> >
> > This makes me worried key people needed to clear blockers and run
> > validation may be harder to get hold of than usual. I don't have any
> > answers, just some general worry. :) Are there current blockers that
> > could use some early pushing to avoid a crisis next week?
>
>  And particularly, Dennis Gilmore tells me that it's likely no one from
>  Release Engineering able to do the composes will be available Tuesday
>  at all. 
I will be on leave for the upcoming tree weeks, so I miss this as
well. Fortunately Jaroslav Reznik will substitute for Go/No-Go and
Readiness meetings, even there are public holidays in CZ (Brno).
However, it does not answer your question. If there is no one from
RelEng on Tuesday, we can request compose on Monday. Unfortunately
there are currently several blockers [1] which might not be fixed by
Monday. It will consequently lead to missing RC compose and automatic
No-Go. As I see it, we can pro-actively slip for one week to make time
lines more predictable, or we can leave it as it is and try to do the
best. I am adding Dennis and Mohan on CC to check whether Releng will
be able to do the compose on Monday 2017-July-03 if requested.

So, yes. From my point of view the F26 final release on 2017-July-11
is jeopardized as well.

[1] https://qa.fedoraproject.org/blockerbugs/milestone/26/final/buglist

>  --
>  Matthew Miller
>  <mattdm(a)fedoraproject.org&gt;
>  Fedora Project Leader
>  _______________________________________________
>  test mailing list -- test(a)lists.fedoraproject.org
>  To unsubscribe send an email to test-leave(a)lists.fedoraproject.org 


-- 
Jan Kuřík
Platform & Fedora Program Manager
Red Hat Czech s.r.o., Purkynova 99/71, 612 45 Brno, Czech Republic

[Josh Boyer]

On Wed, Jun 28, 2017 at 2:36 PM, Jan Kurik <jkurik(a)redhat.com&gt; wrote:
>  On Wed, Jun 28, 2017 at 7:53 PM, Matthew Miller
>  <mattdm(a)fedoraproject.org&gt; wrote:
> > On Wed, Jun 28, 2017 at 01:39:33PM -0400, Matthew Miller wrote:
> >> Next week is the crunch week before F26 final, but it's also holidays
> >> in two countries with a lot of key Fedora contributors. In the US,
> >> Tuesday July 4th is Independence Day (and many people may take a long
> >> weekend for family stuff including Monday). In the Czech Republic, both
> >> July 5th and July 6th are public holidays (Saints Cyril and Methodius'
> >> Day, and Jan Hus Day).
> >>
> >> This makes me worried key people needed to clear blockers and run
> >> validation may be harder to get hold of than usual. I don't have any
> >> answers, just some general worry. :) Are there current blockers that
> >> could use some early pushing to avoid a crisis next week?
> >
> > And particularly, Dennis Gilmore tells me that it's likely no one from
> > Release Engineering able to do the composes will be available Tuesday
> > at all.
>
>  I will be on leave for the upcoming tree weeks, so I miss this as
>  well. Fortunately Jaroslav Reznik will substitute for Go/No-Go and
>  Readiness meetings, even there are public holidays in CZ (Brno).
>  However, it does not answer your question. If there is no one from
>  RelEng on Tuesday, we can request compose on Monday. Unfortunately
>  there are currently several blockers [1] which might not be fixed by
>  Monday. It will consequently lead to missing RC compose and automatic
>  No-Go. As I see it, we can pro-actively slip for one week to make time
>  lines more predictable, or we can leave it as it is and try to do the
>  best. I am adding Dennis and Mohan on CC to check whether Releng will
>  be able to do the compose on Monday 2017-July-03 if requested.
>
>  So, yes. From my point of view the F26 final release on 2017-July-11
>  is jeopardized as well. 
I agree with Jan.  I think a week slip is the only realistic action,
given the blockers and holidays, without resorting to a huge amount of
hero testing.  Frankly, I'd be kind of angry if Fedora as a project
expected that to happen.

I thought we highlighted the holiday situation as a concern in
previous discussions but I can't find them at the moment.  If we did,
I'm surprised we didn't proactively slip at that point.

josh

[Matthew Miller]

On Wed, Jun 28, 2017 at 03:26:08PM -0400, Josh Boyer wrote:
>  I thought we highlighted the holiday situation as a concern in
>  previous discussions but I can't find them at the moment.  If we did,
>  I'm surprised we didn't proactively slip at that point. 
I was mostly concerned with slipping *to* the holiday but didn't give
enough thought to this now-obvious conflict.

-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[Mohan Boddu]

Hi,

I think we can have RC compose once all the blockers are resolved and QA
files the ticket even during the holidays, even though I am not
guaranteeing it but I think someone will get to it sooner or later.

But my major concern is blockers, I am not sure if we can get all the
blockers fixed by next Tue.

Also, there are few proposed blockers and I dont know how many people will
show up for Blocker Review meeting.

On Wed, Jun 28, 2017 at 4:11 PM, Matthew Miller <mattdm(a)fedoraproject.org&gt;
wrote:

>  On Wed, Jun 28, 2017 at 03:26:08PM -0400, Josh Boyer wrote:
>  > I thought we highlighted the holiday situation as a concern in
>  > previous discussions but I can't find them at the moment.  If we did,
>  > I'm surprised we didn't proactively slip at that point.
>
>  I was mostly concerned with slipping *to* the holiday but didn't give
>  enough thought to this now-obvious conflict.
>
>  --
>  Matthew Miller
>  <mattdm(a)fedoraproject.org&gt;
>  Fedora Project Leader
>  


-- 
Thanks,
Mohan Boddu

[Adam Williamson]

On Wed, 2017-06-28 at 21:41 -0400, Mohan Boddu wrote:
>  Hi,
>  
>  I think we can have RC compose once all the blockers are resolved and QA
>  files the ticket even during the holidays, even though I am not
>  guaranteeing it but I think someone will get to it sooner or later.
>  
>  But my major concern is blockers, I am not sure if we can get all the
>  blockers fixed by next Tue.
>  
>  Also, there are few proposed blockers and I dont know how many people will
>  show up for Blocker Review meeting. 
So here's a very quick and informal blocker status mail:

There are four accepted blockers.

#1462825 doesn't look like it should be particularly difficult for the
right people to fix - i.e. folks who know libreport and anaconda's
interface with it. We just need to make sure we have their eyes on it;
I've attempted to CC all appropriate people.

We have an approach for fixing #1449752 agreed and it just needs the
maintainer to go ahead and do it. I've tried to explain the urgency of
this in the bug this afternoon.

#1436873 is in a state of slight disagreement about whether it's really
happening, I think. ;) The openQA test is still failing quite often
(although not always). I will try and reproduce the issue manually
tomorrow, since there appears to be scepticism among the KDE folks. But
in general the KDE folks are quite fast to fix issues once we do pin
them down specifically, I would be quite optimistic about getting this
one fixed.

#1404285 is a GNOME crash quite a lot of people seem to be encountering
in different ways (possibly because it's a tracker issue and tracker is
hooked into lots of things). In at least some cases it causes GNOME as
a whole to crash back to GDM, which is of course bad. There are various
different reproduction steps documented in the bug. I'm hoping the
desktop team is looking at this, and will be able to provide us with
some more assessment.

There are five proposed blockers. My professional guesstimate *at this
point* is that at least four of them will probably be rejected, though
that could change with more data (attention pjones: if #1418360 and
#1451071 are more serious than they seem to us so far, please do let us
know). #1462444 is the most unclear one, but it doesn't seem like a lot
of people are running into it, and we may wind up rejecting it also on
that basis.

Given the holiday situation, it might make sense to do a blocker
meeting tomorrow (Thursday) or Friday, depending on how much notice
folks need, to give us at least a shot at doing an RC on Friday or over
the weekend, if all the accepted blockers happen to get resolved.

There are also a *ton* of proposed FEs for broken dependencies; we
might also want to blow through those quickly at a meeting.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

[Matthew Miller]

On Wed, Jun 28, 2017 at 06:53:58PM -0700, Adam Williamson wrote:
>  Given the holiday situation, it might make sense to do a blocker
>  meeting tomorrow (Thursday) or Friday, depending on how much notice
>  folks need, to give us at least a shot at doing an RC on Friday or over
>  the weekend, if all the accepted blockers happen to get resolved. 
Thanks Adam! I know Paul Frields is really concerned for the
infrastructure engineering team if we slip another week, because it
pushes the freeze back into some necessary maintenance and might even
run up against Flock.

-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[Stephen Gallagher]

On Wed, Jun 28, 2017 at 9:54 PM Adam Williamson <adamwill(a)fedoraproject.org&gt;
wrote:

>  On Wed, 2017-06-28 at 21:41 -0400, Mohan Boddu wrote:
>  > Hi,
>  >
>  > I think we can have RC compose once all the blockers are resolved and QA
>  > files the ticket even during the holidays, even though I am not
>  > guaranteeing it but I think someone will get to it sooner or later.
>  >
>  > But my major concern is blockers, I am not sure if we can get all the
>  > blockers fixed by next Tue.
>  >
>  > Also, there are few proposed blockers and I dont know how many people
>  will
>  > show up for Blocker Review meeting.
>
>  So here's a very quick and informal blocker status mail:
>
>  There are four accepted blockers.
>
>  #1462825 doesn't look like it should be particularly difficult for the
>  right people to fix - i.e. folks who know libreport and anaconda's
>  interface with it. We just need to make sure we have their eyes on it;
>  I've attempted to CC all appropriate people.
>
>  We have an approach for fixing #1449752 agreed and it just needs the
>  maintainer to go ahead and do it. I've tried to explain the urgency of
>  this in the bug this afternoon.
>
>  #1436873 is in a state of slight disagreement about whether it's really
>  happening, I think. ;) The openQA test is still failing quite often
>  (although not always). I will try and reproduce the issue manually
>  tomorrow, since there appears to be scepticism among the KDE folks. But
>  in general the KDE folks are quite fast to fix issues once we do pin
>  them down specifically, I would be quite optimistic about getting this
>  one fixed.
>
>  #1404285 is a GNOME crash quite a lot of people seem to be encountering
>  in different ways (possibly because it's a tracker issue and tracker is
>  hooked into lots of things). In at least some cases it causes GNOME as
>  a whole to crash back to GDM, which is of course bad. There are various
>  different reproduction steps documented in the bug. I'm hoping the
>  desktop team is looking at this, and will be able to provide us with
>  some more assessment.
>
>  There are five proposed blockers. My professional guesstimate *at this
>  point* is that at least four of them will probably be rejected, though
>  that could change with more data (attention pjones: if #1418360 and
>  #1451071 are more serious than they seem to us so far, please do let us
>  know). #1462444 is the most unclear one, but it doesn't seem like a lot
>  of people are running into it, and we may wind up rejecting it also on
>  that basis.
>
>  Given the holiday situation, it might make sense to do a blocker
>  meeting tomorrow (Thursday) or Friday, depending on how much notice
>  folks need, to give us at least a shot at doing an RC on Friday or over
>  the weekend, if all the accepted blockers happen to get resolved.
>
>  There are also a *ton* of proposed FEs for broken dependencies; we
>  might also want to blow through those quickly at a meeting.
>  
Given the urgency of the situation, please schedule a blocker meeting for
today (formally or informally).

[Mike Ruckman]

>  
>  Given the urgency of the situation, please schedule a blocker meeting for
>  today (formally or informally). 
I'll be ready to run a meeting at the usual 1600 UTC time in #fedora-blocker-review.

[Peter Jones]

On Wed, Jun 28, 2017 at 06:53:58PM -0700, Adam Williamson wrote:
>  There are five proposed blockers. My professional guesstimate *at
> this
>  point* is that at least four of them will probably be rejected, though
>  that could change with more data (attention pjones: if #1418360 and
>  #1451071 are more serious than they seem to us so far, please do let us
>  know). 
They absolutely are: basically Secure Boot doesn't trigger kmod
signature checking, read-only /dev/mem, etc., in the current trees.
This update fixes a grub bug that's triggering that behavior in the
newer kernels, but was not triggering it in the older ones.

So yes, I very much think these should be blockers.

-- 
  Peter

[Adam Williamson]

On Thu, 2017-06-29 at 10:19 -0400, Peter Jones wrote:
>  On Wed, Jun 28, 2017 at 06:53:58PM -0700, Adam Williamson wrote:
>  > There are five proposed blockers. My professional guesstimate *at this
>  > point* is that at least four of them will probably be rejected, though
>  > that could change with more data (attention pjones: if #1418360 and
>  > #1451071 are more serious than they seem to us so far, please do let us
>  > know).
>  
>  They absolutely are: basically Secure Boot doesn't trigger kmod
>  signature checking, read-only /dev/mem, etc., in the current trees.
>  This update fixes a grub bug that's triggering that behavior in the
>  newer kernels, but was not triggering it in the older ones.
>  
>  So yes, I very much think these should be blockers. 
Ah, from the description I thought it was purely an informational thing
(just the user couldn't tell whether SB was enabled, but if it was in
fact enabled, it was working properly). So basically the appropriate
protections aren't put in place when SB is active, making it quite easy
to subvert SB?
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net