FBR: Make new-updates-sync print what it did and email to releng-cron

[Patrick Uiterwijk]

Any +1s?


From a4b55b290d475769c6cdb076d56fca762a0ed289 Mon Sep 17 00:00:00 2001
From: Patrick Uiterwijk <puiterwijk(a)redhat.com&gt;
Date: Wed, 25 Apr 2018 21:53:17 +0200
Subject: [PATCH] Make new-updates-sync always print status and email to
 releng-cron

Signed-off-by: Patrick Uiterwijk <puiterwijk(a)redhat.com&gt;
---
 roles/bodhi2/backend/files/new-updates-sync      | 7 +++++++
 roles/bodhi2/backend/files/new-updates-sync.cron | 3 +++
 roles/bodhi2/backend/tasks/main.yml              | 6 ++----
 3 files changed, 12 insertions(+), 4 deletions(-)
 create mode 100644 roles/bodhi2/backend/files/new-updates-sync.cron

diff --git a/roles/bodhi2/backend/files/new-updates-sync
b/roles/bodhi2/backend/files/new-updates-sync
index de6dca78c..b90c89384 100755
--- a/roles/bodhi2/backend/files/new-updates-sync
+++ b/roles/bodhi2/backend/files/new-updates-sync
@@ -170,6 +170,7 @@ def sync_ostree(dst, ref):
     if src_commit == dst_commit:
         logger.info('OSTree at %s, ref %s in sync', dst, ref)
     else:
+        print('Syncing ostree ref %s -> %s' % (ref, src_commit, dst_commit))
         logger.info('Syncing OSTree to %s, ref %s: %s -> %s',
                     dst, ref, src_commit, dst_commit)
         cmd = ['ostree', 'pull-local', '--verbose',
'--repo',
@@ -177,6 +178,7 @@ def sync_ostree(dst, ref):
         out = run_command(cmd)
         cmd = ['ostree', 'summary', '--verbose',
'--repo', dst, '--update']
         run_command(cmd)
+        print('Ostree ref %s now at %s' % (ref, dst_commit))


 def update_fullfilelist(modules):
@@ -324,10 +326,15 @@ def sync_single_release(release):
             if curstate and curstate == target:
                 logger.info('This repo has already been synced')
             else:
+                print('Syncing %s %s from %s -> %s' % (release,
+                                                       repo,
+                                                       curstate,
+                                                       target))
                 sync_single_repo(release, repo)
                 with open(curstatefile, 'w') as f:
                     f.write(target)
                 needssync = True
+                print('Synced %s %s to %s' % (release, repo, target))

         for ostree in RELEASES[release]['repos'][repo].get('ostrees',
[]):
             pairs = []
diff --git a/roles/bodhi2/backend/files/new-updates-sync.cron
b/roles/bodhi2/backend/files/new-updates-sync.cron
new file mode 100644
index 000000000..ec7955827
--- /dev/null
+++ b/roles/bodhi2/backend/files/new-updates-sync.cron
@@ -0,0 +1,3 @@
+MAILTO=releng-cron(a)lists.fedoraproject.org
+*/5 * * * * ftpsync /usr/local/bin/lock-wrapper updates-push
'/usr/local/bin/new-updates-sync'
+
diff --git a/roles/bodhi2/backend/tasks/main.yml
b/roles/bodhi2/backend/tasks/main.yml
index e33f710ff..ff5117710 100644
--- a/roles/bodhi2/backend/tasks/main.yml
+++ b/roles/bodhi2/backend/tasks/main.yml
@@ -268,10 +268,8 @@
   - bodhi
   - cron

-- name: Updates sync cron job.
-  cron: name="updates-sync" minute="*/5" user="ftpsync"
-        job="/usr/local/bin/lock-wrapper updates-push
'/usr/local/bin/new-updates-sync'"
-        cron_file=updates-sync
+- name: Set the updates sync cron job
+  copy: src=new-updates-sync.cron dest=/etc/cron.d/updates-sync
   when: inventory_hostname.startswith('bodhi-backend01') and env ==
"production"
   tags:
   - config
-- 
2.17.0

[Dusty Mabe]

On 04/25/2018 03:55 PM, Patrick Uiterwijk wrote:
>  Any +1s?
>  
>  
>  From a4b55b290d475769c6cdb076d56fca762a0ed289 Mon Sep 17 00:00:00 2001
>  From: Patrick Uiterwijk <puiterwijk(a)redhat.com&gt;
>  Date: Wed, 25 Apr 2018 21:53:17 +0200
>  Subject: [PATCH] Make new-updates-sync always print status and email to
>   releng-cron
>  
>  Signed-off-by: Patrick Uiterwijk <puiterwijk(a)redhat.com&gt;
>  ---
>   roles/bodhi2/backend/files/new-updates-sync      | 7 +++++++
>   roles/bodhi2/backend/files/new-updates-sync.cron | 3 +++
>   roles/bodhi2/backend/tasks/main.yml              | 6 ++----
>   3 files changed, 12 insertions(+), 4 deletions(-)
>   create mode 100644 roles/bodhi2/backend/files/new-updates-sync.cron
>  
>  diff --git a/roles/bodhi2/backend/files/new-updates-sync
>  b/roles/bodhi2/backend/files/new-updates-sync
>  index de6dca78c..b90c89384 100755
>  --- a/roles/bodhi2/backend/files/new-updates-sync
>  +++ b/roles/bodhi2/backend/files/new-updates-sync
>  @@ -170,6 +170,7 @@ def sync_ostree(dst, ref):
>       if src_commit == dst_commit:
>           logger.info('OSTree at %s, ref %s in sync', dst, ref)
>       else:
>  +        print('Syncing ostree ref %s -> %s' % (ref, src_commit, dst_commit))
>           logger.info('Syncing OSTree to %s, ref %s: %s -> %s',
>                       dst, ref, src_commit, dst_commit) 

Could we not just piggyback off the logger.info statements rather than using
separate print statements? 

Dusty

[Patrick マルタインアンドレアス Uiterwijk]

No, because the logger.info statements also output things like what the current state is,
and we explicitly set logging to ERROR to only catch the errors.
I want these specific output messages to print whatever the log level is set to.

[Dusty Mabe]

On 04/25/2018 04:00 PM, Patrick  マルタインアンドレアス  Uiterwijk wrote:
>  No, because the logger.info statements also output things like what
> the current state is, and we explicitly set logging to ERROR to only catch the errors.
>  I want these specific output messages to print whatever the log level is set to. 
yeah I understand. It's just not clear to someone coming into the code later why they
exist and $newuser would probably just convert them to logger.info calls and not know any
better.

I'd prefer if we converted any logger.info calls that we don't want to print out
by default to 
logger.debug and then make just these be logger.info and then set the loglevel to INFO.
Would
probably be the best approach to prevent a new person from coming in and
deleting/converting the
print statements.

Dusty

[Stephen John Smoogen]

I am +! to this.

On 25 April 2018 at 16:06, Dusty Mabe <dusty(a)dustymabe.com&gt; wrote:
>  On 04/25/2018 04:00 PM, Patrick  マルタインアンドレアス  Uiterwijk wrote:
> > No, because the logger.info statements also output things like what the current state
> is, and we explicitly set logging to ERROR to only catch the errors.
> > I want these specific output messages to print whatever the log level is set to.
>
>  yeah I understand. It's just not clear to someone coming into the code later why
> they
>  exist and $newuser would probably just convert them to logger.info calls and not know any
> better.
>
>  I'd prefer if we converted any logger.info calls that we don't want to print out
> by default to
>  logger.debug and then make just these be logger.info and then set the loglevel to INFO.
> Would
>  probably be the best approach to prevent a new person from coming in and
> deleting/converting the
>  print statements.
>
>  Dusty
>  _______________________________________________
>  infrastructure mailing list -- infrastructure(a)lists.fedoraproject.org
>  To unsubscribe send an email to infrastructure-leave(a)lists.fedoraproject.org 


-- 
Stephen J Smoogen.

[Stephen John Smoogen]

Undo. Wrong message to +1 as I haven't read this part yet.

On 25 April 2018 at 16:11, Stephen John Smoogen <smooge(a)gmail.com&gt; wrote:
>  I am +! to this.
>
>  On 25 April 2018 at 16:06, Dusty Mabe <dusty(a)dustymabe.com&gt; wrote:
> >
> >
> > On 04/25/2018 04:00 PM, Patrick  マルタインアンドレアス  Uiterwijk wrote:
> >> No, because the logger.info statements also output things like what the current
> state is, and we explicitly set logging to ERROR to only catch the errors.
> >> I want these specific output messages to print whatever the log level is set to.
> >
> > yeah I understand. It's just not clear to someone coming into the code later why
> they
> > exist and $newuser would probably just convert them to logger.info calls and not know
> any better.
> >
> > I'd prefer if we converted any logger.info calls that we don't want to print
> out by default to
> > logger.debug and then make just these be logger.info and then set the loglevel to
> INFO. Would
> > probably be the best approach to prevent a new person from coming in and
> deleting/converting the
> > print statements.
> >
> > Dusty
> > _______________________________________________
> > infrastructure mailing list -- infrastructure(a)lists.fedoraproject.org
> > To unsubscribe send an email to infrastructure-leave(a)lists.fedoraproject.org
>
>
>
>  --
>  Stephen J Smoogen. 


-- 
Stephen J Smoogen.

[Stephen John Smoogen]

+1 to this part. Coffee not working.

On 25 April 2018 at 15:55, Patrick Uiterwijk <puiterwijk(a)redhat.com&gt; wrote:
>  Any +1s?
>
>
>  From a4b55b290d475769c6cdb076d56fca762a0ed289 Mon Sep 17 00:00:00 2001
>  From: Patrick Uiterwijk <puiterwijk(a)redhat.com&gt;
>  Date: Wed, 25 Apr 2018 21:53:17 +0200
>  Subject: [PATCH] Make new-updates-sync always print status and email to
>   releng-cron
>
>  Signed-off-by: Patrick Uiterwijk <puiterwijk(a)redhat.com&gt;
>  ---
>   roles/bodhi2/backend/files/new-updates-sync      | 7 +++++++
>   roles/bodhi2/backend/files/new-updates-sync.cron | 3 +++
>   roles/bodhi2/backend/tasks/main.yml              | 6 ++----
>   3 files changed, 12 insertions(+), 4 deletions(-)
>   create mode 100644 roles/bodhi2/backend/files/new-updates-sync.cron
>
>  diff --git a/roles/bodhi2/backend/files/new-updates-sync
>  b/roles/bodhi2/backend/files/new-updates-sync
>  index de6dca78c..b90c89384 100755
>  --- a/roles/bodhi2/backend/files/new-updates-sync
>  +++ b/roles/bodhi2/backend/files/new-updates-sync
>  @@ -170,6 +170,7 @@ def sync_ostree(dst, ref):
>       if src_commit == dst_commit:
>           logger.info('OSTree at %s, ref %s in sync', dst, ref)
>       else:
>  +        print('Syncing ostree ref %s -> %s' % (ref, src_commit, dst_commit))
>           logger.info('Syncing OSTree to %s, ref %s: %s -> %s',
>                       dst, ref, src_commit, dst_commit)
>           cmd = ['ostree', 'pull-local', '--verbose',
> '--repo',
>  @@ -177,6 +178,7 @@ def sync_ostree(dst, ref):
>           out = run_command(cmd)
>           cmd = ['ostree', 'summary', '--verbose',
> '--repo', dst, '--update']
>           run_command(cmd)
>  +        print('Ostree ref %s now at %s' % (ref, dst_commit))
>
>
>   def update_fullfilelist(modules):
>  @@ -324,10 +326,15 @@ def sync_single_release(release):
>               if curstate and curstate == target:
>                   logger.info('This repo has already been synced')
>               else:
>  +                print('Syncing %s %s from %s -> %s' % (release,
>  +                                                       repo,
>  +                                                       curstate,
>  +                                                       target))
>                   sync_single_repo(release, repo)
>                   with open(curstatefile, 'w') as f:
>                       f.write(target)
>                   needssync = True
>  +                print('Synced %s %s to %s' % (release, repo, target))
>
>           for ostree in RELEASES[release]['repos'][repo].get('ostrees',
> []):
>               pairs = []
>  diff --git a/roles/bodhi2/backend/files/new-updates-sync.cron
>  b/roles/bodhi2/backend/files/new-updates-sync.cron
>  new file mode 100644
>  index 000000000..ec7955827
>  --- /dev/null
>  +++ b/roles/bodhi2/backend/files/new-updates-sync.cron
>  @@ -0,0 +1,3 @@
>  +MAILTO=releng-cron(a)lists.fedoraproject.org
>  +*/5 * * * * ftpsync /usr/local/bin/lock-wrapper updates-push
>  '/usr/local/bin/new-updates-sync'
>  +
>  diff --git a/roles/bodhi2/backend/tasks/main.yml
>  b/roles/bodhi2/backend/tasks/main.yml
>  index e33f710ff..ff5117710 100644
>  --- a/roles/bodhi2/backend/tasks/main.yml
>  +++ b/roles/bodhi2/backend/tasks/main.yml
>  @@ -268,10 +268,8 @@
>     - bodhi
>     - cron
>
>  -- name: Updates sync cron job.
>  -  cron: name="updates-sync" minute="*/5" user="ftpsync"
>  -        job="/usr/local/bin/lock-wrapper updates-push
>  '/usr/local/bin/new-updates-sync'"
>  -        cron_file=updates-sync
>  +- name: Set the updates sync cron job
>  +  copy: src=new-updates-sync.cron dest=/etc/cron.d/updates-sync
>     when: inventory_hostname.startswith('bodhi-backend01') and env ==
>  "production"
>     tags:
>     - config
>  --
>  2.17.0
>  _______________________________________________
>  infrastructure mailing list -- infrastructure(a)lists.fedoraproject.org
>  To unsubscribe send an email to infrastructure-leave(a)lists.fedoraproject.org 


-- 
Stephen J Smoogen.

[Ricky Elrod]

+1

On Wed, Apr 25, 2018 at 4:12 PM, Stephen John Smoogen <smooge(a)gmail.com&gt; wrote:
> +1 to this part. Coffee not working.
>
> On 25 April 2018 at 15:55, Patrick Uiterwijk <puiterwijk(a)redhat.com&gt; wrote:
>> Any +1s?
>>
>>
>> From a4b55b290d475769c6cdb076d56fca762a0ed289 Mon Sep 17 00:00:00 2001
>> From: Patrick Uiterwijk <puiterwijk(a)redhat.com&gt;
>> Date: Wed, 25 Apr 2018 21:53:17 +0200
>> Subject: [PATCH] Make new-updates-sync always print status and email to
>>  releng-cron
>>
>> Signed-off-by: Patrick Uiterwijk <puiterwijk(a)redhat.com&gt;
>> ---
>>  roles/bodhi2/backend/files/new-updates-sync      | 7 +++++++
>>  roles/bodhi2/backend/files/new-updates-sync.cron | 3 +++
>>  roles/bodhi2/backend/tasks/main.yml              | 6 ++----
>>  3 files changed, 12 insertions(+), 4 deletions(-)
>>  create mode 100644 roles/bodhi2/backend/files/new-updates-sync.cron
>>
>> diff --git a/roles/bodhi2/backend/files/new-updates-sync
>> b/roles/bodhi2/backend/files/new-updates-sync
>> index de6dca78c..b90c89384 100755
>> --- a/roles/bodhi2/backend/files/new-updates-sync
>> +++ b/roles/bodhi2/backend/files/new-updates-sync
>> @@ -170,6 +170,7 @@ def sync_ostree(dst, ref):
>>      if src_commit == dst_commit:
>>          logger.info('OSTree at %s, ref %s in sync', dst, ref)
>>      else:
>> +        print('Syncing ostree ref %s -> %s' % (ref, src_commit,
dst_commit))
>>          logger.info('Syncing OSTree to %s, ref %s: %s -> %s',
>>                      dst, ref, src_commit, dst_commit)
>>          cmd = ['ostree', 'pull-local', '--verbose',
'--repo',
>> @@ -177,6 +178,7 @@ def sync_ostree(dst, ref):
>>          out = run_command(cmd)
>>          cmd = ['ostree', 'summary', '--verbose',
'--repo', dst, '--update']
>>          run_command(cmd)
>> +        print('Ostree ref %s now at %s' % (ref, dst_commit))
>>
>>
>>  def update_fullfilelist(modules):
>> @@ -324,10 +326,15 @@ def sync_single_release(release):
>>              if curstate and curstate == target:
>>                  logger.info('This repo has already been synced')
>>              else:
>> +                print('Syncing %s %s from %s -> %s' % (release,
>> +                                                       repo,
>> +                                                       curstate,
>> +                                                       target))
>>                  sync_single_repo(release, repo)
>>                  with open(curstatefile, 'w') as f:
>>                      f.write(target)
>>                  needssync = True
>> +                print('Synced %s %s to %s' % (release, repo, target))
>>
>>          for ostree in
RELEASES[release]['repos'][repo].get('ostrees', []):
>>              pairs = []
>> diff --git a/roles/bodhi2/backend/files/new-updates-sync.cron
>> b/roles/bodhi2/backend/files/new-updates-sync.cron
>> new file mode 100644
>> index 000000000..ec7955827
>> --- /dev/null
>> +++ b/roles/bodhi2/backend/files/new-updates-sync.cron
>> @@ -0,0 +1,3 @@
>> +MAILTO=releng-cron(a)lists.fedoraproject.org
>> +*/5 * * * * ftpsync /usr/local/bin/lock-wrapper updates-push
>> '/usr/local/bin/new-updates-sync'
>> +
>> diff --git a/roles/bodhi2/backend/tasks/main.yml
>> b/roles/bodhi2/backend/tasks/main.yml
>> index e33f710ff..ff5117710 100644
>> --- a/roles/bodhi2/backend/tasks/main.yml
>> +++ b/roles/bodhi2/backend/tasks/main.yml
>> @@ -268,10 +268,8 @@
>>    - bodhi
>>    - cron
>>
>> -- name: Updates sync cron job.
>> -  cron: name="updates-sync" minute="*/5"
user="ftpsync"
>> -        job="/usr/local/bin/lock-wrapper updates-push
>> '/usr/local/bin/new-updates-sync'"
>> -        cron_file=updates-sync
>> +- name: Set the updates sync cron job
>> +  copy: src=new-updates-sync.cron dest=/etc/cron.d/updates-sync
>>    when: inventory_hostname.startswith('bodhi-backend01') and env ==
>> "production"
>>    tags:
>>    - config
>> --
>> 2.17.0
>> _______________________________________________
>> infrastructure mailing list -- infrastructure(a)lists.fedoraproject.org
>> To unsubscribe send an email to infrastructure-leave(a)lists.fedoraproject.org
>
>
>
> --
> Stephen J Smoogen.
> _______________________________________________
> infrastructure mailing list -- infrastructure(a)lists.fedoraproject.org
> To unsubscribe send an email to infrastructure-leave(a)lists.fedoraproject.org

[Mohan Boddu]

+1