to batch or not to batch?

[Zbigniew Jędrzejewski-Szmek]

Bodhi currently provides "batched updates" [1] which lump updates of
packages that are not marked urgent into a single batch, released once
per week. This means that after an update has graduated from testing,
it may be delayed up to a week before it becomes available to users.

Batching is now the default, but maintainers can push theirs updates
to stable, overriding this default, and make the update available the
next day.

Batching is liked by some maintainers, but hated by others
Unfortunately, the positive effects of batching are strongly
decreased when many packages are not batched. Thus, we should settle
on a single policy — either batch as much as possible, or turn
batching off. Having the middle ground of some batching is not very
effective and still annoys people who don't like batching.

To summarize the ups (+) and downs (-):

+ batching reduces the number of times repository metadata is updated.
  Each metadata update results in dnf downloading about 20-40 mb,
  which is expensive and/or slow for users with low bandwidth.

+ a constant stream of metadata updates also puts strain on our mirrors.

+ a constant stream of updates feels overwhelming to users, and a
  predictable once-per-week batch is perceived as easier. In
  particular corporate users might adapt to this and use it to
  schedule an update of all machines at fixed times.

+ a batch of updates may be tested as one, and, at least in principle,
  if users then install this batch as one, QA that was done on the
  batch matches the user systems more closely, compared to QA testing
  package updates one by one as they come in, and users updating them
  at a slightly different schedule.

- batching delays updates of packages between 0 and 7 days after
  they have reached karma and makes it hard for people to immediately
  install updates when they graduate from testing.

- some users (or maybe it's just maintainers?) actually prefer a
  constant stream of small updates, and find it easier to read
  changelogs and pinpoint regressions, etc. a few packages at a time.

- batching (when done on the "server" side) interferes with clients
  applying their own batching policy. This has two aspects:
  clients might want to pick a different day of the week or an
  altogether different schedule,
  clients might want to pick a different policy of updates, e.g. to
  allow any updates for specific packages to go through, etc.

  In particular gnome-software implements its own style of batching, where
  it will suggest an update only once per week, unless there are security
  updates.

Unfortunately there isn't much data on the effects of batching.
Kevin posted some [2], as did the other Kevin [3] ;), but we certainly
could use more detailed stats.

One of the positive aspects of batching — reduction in metadata downloads,
might be obsoleted by improving download efficiency through delta downloads.
A proof-of-concept has been implemented [4].

Second positive aspect of batching — doing updates in batches at a
fixed schedule, may just as well be implemented on the client side,
although that does not recreate the testing on the whole batch, since
now every client it doing it at a different time. It's not clear though
if this additional testing is actually useful.

There's an open FESCo ticket to "adjust/drop/document" batching [5].
That discussion has not been effective, because this issue has many
aspects, and depending on priorities, the view on batching is likely to
be different. FESCo is trying to gather more data and get a better
understanding of what maintainers consider more important.

Did I miss something on the plus or minus side? Or some good statistics?
Does patching make Fedora seem more approachable to end-users?
(this is a question in particular for Matthew Miller who pushed for batching.)
Do the benefits of batching outweigh the downsides?
Should we keep batching as an interim measure until delta downloads are implemented?
Should dnf offer smart batched updates like gnome-software?
Should we encourage maintainers to allow their updates to be batched?

[1] https://github.com/fedora-infra/bodhi/issues/1157,
   
https://lists.fedoraproject.org/archives/list/[e-mail].o...
[2]
https://lists.fedoraproject.org/archives/list/[e-mail].o...
[3]
https://lists.fedoraproject.org/archives/list/[e-mail].o...
[4] http://lists.rpm.org/pipermail/rpm-ecosystem/2018-February/000534.html
[5] https://pagure.io/fesco/issue/1820

Zbyszek