Re: Announcing start of DNF 5 development

On Thu, 2020-03-05 at 07:44 +0100, Fabio Valentini wrote:
>  On Thu, Mar 5, 2020, 00:55 Martin Kolman <mkolman(a)redhat.com&gt;
> wrote:
>  > 
>  > 
>  > ----- Original Message -----
>  > 
>  > > From: "Neal Gompa" <ngompa13(a)gmail.com&gt;
>  > 
>  > > To: "Development discussions related to Fedora" <
>  > devel(a)lists.fedoraproject.org&gt;
>  > 
>  > > Sent: Wednesday, March 4, 2020 11:01:43 PM
>  > 
>  > > Subject: Re: Announcing start of DNF 5 development
>  > 
>  > > 
>  > 
>  > > On Wed, Mar 4, 2020 at 4:37 PM Zbigniew Jędrzejewski-Szmek
>  > 
>  > > <zbyszek(a)in.waw.pl&gt; wrote:
>  > 
>  > > >
>  > 
>  > > > On Wed, Mar 04, 2020 at 07:03:01PM +0100, Daniel Mach wrote:
>  > 
>  > > > > Hello everyone,
>  > 
>  > > > > I'm pleased to announce start of DNF 5 development. We are
>  > planning
>  > 
>  > > > > to deliver a module stream or a COPR repo during Fedora 33
>  > 
>  > > > > development for early adopters and tool developers and we're
>  > hoping
>  > 
>  > > > > in getting a stable version into Fedora 34.
>  > 
>  > > > >
>  > 
>  > > > >
>  > 
>  > > > > More details follow.
>  > 
>  > > > >
>  > 
>  > > > >
>  > 
>  > > > > We've managed to drop a lot of redundant code across the
>  > whole DNF
>  > 
>  > > > > stack in the past years, but we have reached a point when
>  > it's
>  > 
>  > > > > nearly impossible to consolidate the code any further without
>  > 
>  > > > > breaking the API/ABI. Especially with PackageKit being
>  > dead[1], we
>  > 
>  > > > > can't move with the old "libhif" API in libdnf,
> because
>  > making any
>  > 
>  > > > > bigger changes to PackageKit is clearly out of scope.
>  > 
>  > > > >
>  > 
>  > > > > [1]
>  > 
>  > > > > 
>  >
> https://blogs.gnome.org/hughsie/2019/02/14/packagekit-is-dead-long-live-w...
>  > 
>  > > > >
>  > 
>  > > > >
>  > 
>  > > > > That's why we decided to start working on a new version of
>  > the DNF
>  > 
>  > > > > stack: DNF 5. And this is the plan:
>  > 
>  > > > >
>  > 
>  > > > >
>  > 
>  > > > > Priorities
>  > 
>  > > > > ----------
>  > 
>  > > > > 1. Consistency, documentation and user experience is the top
>  > priority.
>  > 
>  > > > > 2. Compatibility on the command line level.
>  > 
>  > > > > 3. Compatibility on the API level.
>  > 
>  > > > >
>  > 
>  > > > >
>  > 
>  > > > > Maintenance
>  > 
>  > > > > -----------
>  > 
>  > > > > The existing DNF 4 stack stays in the current Fedoras and Red
>  > Hat
>  > 
>  > > > > Enterprise Linux 8. We'll keep maintaining it in dnf-4-master
>  > 
>  > > > > branches on GitHub. PackageKit and rpm-ostree will stay on
>  > libdnf
>  > 
>  > > > > from the DNF 4 stack.
>  > 
>  > > > >
>  > 
>  > > > >
>  > 
>  > > > > The existing Python API in DNF
>  > 
>  > > > > ------------------------------
>  > 
>  > > > > The Python API in DNF stays. We'll do our best to keep it
>  > working.
>  > 
>  > > > > If there is an incompatible change, we'll communicate and
>  > document
>  > 
>  > > > > it properly.
>  > 
>  > > > >
>  > 
>  > > > >
>  > 
>  > > > > The new API in libdnf
>  > 
>  > > > > ---------------------
>  > 
>  > > > > All business logic will move from DNF (Python) to libdnf
>  > (C++). This
>  > 
>  > > > > is the only way to ensure that package managers work
>  > identically
>  > 
>  > > > > across the whole distribution. We'll start with C++ API and
>  > 
>  > > > > auto-generated Python bindings via SWIG. We'll focus on the
>  > Python
>  > 
>  > > > > bindings which are required by DNF and we will do our best to
>  > 
>  > > > > provide bindings for Go, Perl5 and Ruby as well. C API will
>  > be
>  > 
>  > > > > created later when the C++ API is stable. At that moment rpm-
>  > ostree
>  > 
>  > > > > will be ported to the new C API.
>  > 
>  > > > >
>  > 
>  > > > >
>  > 
>  > > > > hawkey
>  > 
>  > > > > ------
>  > 
>  > > > > Hawkey Python API is going away and will be replaced with
>  > libdnf Python
>  > 
>  > > > > API.
>  > 
>  > > > >
>  > 
>  > > > >
>  > 
>  > > > > DNF
>  > 
>  > > > > ---
>  > 
>  > > > > DNF stays as the primary command-line package manager. The
>  > overall
>  > 
>  > > > > functionality remains. We don't anticipate any negative
>  > impact of
>  > 
>  > > > > the API rewrite on the end-users. We have built an extensive
>  > test
>  > 
>  > > > > suite (over 1400 scenarios) that will help us to ensure that.
>  > The
>  > 
>  > > > > argument parser and outputs may slightly change in some cases
>  > to
>  > 
>  > > > > provide a more consistent user-experience. All such cases
>  > will be
>  > 
>  > > > > properly documented.
>  > 
>  > > > >
>  > 
>  > > > >
>  > 
>  > > > > microdnf
>  > 
>  > > > > --------
>  > 
>  > > > > Microdnf is becoming important because it's part of many
>  > containers
>  > 
>  > > > > due to its small footprint. We're getting feedback that users
>  > would
>  > 
>  > > > > appreciate something closer to DNF. We'll focus on
>  > implementing a
>  > 
>  > > > > subset of DNF's functionality and improving the user
>  > experience.
>  > 
>  > > > > 100% feature parity with DNF is currently out of scope.
>  > 
>  > > > >
>  > 
>  > > > >
>  > 
>  > > > Hi,
>  > 
>  > > >
>  > 
>  > > > the roadmap is ambitious, but not impossible. Good luck!
>  > 
>  > > >
>  > 
>  > > > > Roadmap (tentative)
>  > 
>  > > > > -------------------
>  > 
>  > > > > * Mar 2020 - making the bigger API changes, upstream code
>  > barely compiles
>  > 
>  > > > > * May 2020 - COPR repo with first development snapshots
>  > 
>  > > > > * Jun 2020 - F33 module available for early adopters and tool
>  > developers
>  > 
>  > > > > * Oct 2020 - DNF 5 landing in F34 Rawhide
>  > 
>  > > > > * Feb 2021 - DNF 5 replacing DNF 4 in stable Fedora
>  > 
>  > > >
>  > 
>  > > > > DBus service
>  > 
>  > > > > ------------
>  > 
>  > > > > DNF team has decided to create a new DBus service replacing
>  > 
>  > > > > PackageKit to provide an interface to GUI applications. It's
>  > 
>  > > > > probably going to take a while because we're planning to
>  > start from
>  > 
>  > > > > scratch.
>  > 
>  > > >
>  > 
>  > > > Do you plan to make normal 'dnf' calls go through the dbus api?
>  > 
>  > > 
>  > 
>  > > This would be interesting, but wouldn't that make using DNF in
>  > rescue
>  > 
>  > > situations impossible?
>  > 
>  > You mean due to the regular DBus daemon & system + session bus not
>  > running
>  > 
>  > in some system rescue scenarios, right ?
>  > 
>  > 
>  > 
>  > While possibly a bit tricky I could imagine some fallback mechanism
>  > where
>  > 
>  > invocation of the CLI tool starts it's own DBus session & instance
>  > of the DNF
>  > 
>  > service when it detects that the regular system & session buses are
>  > not available.
>  > 
>  > 
>  > 
>  > Anaconda does something similar when it starts in Mock during some
>  > phases of the
>  > 
>  > compose process & finds no system bus is available - it starts it's
>  > own DBus daemon
>  > 
>  > process and uses that.
>  
>  Wow, using dbus to communicate between CLI binary and the shared
>  library sounds like an awful idea. Why not do a simple shim around
>  the shared library instead? (And not introduce another 2 moving parts
>  into a critical system component: dnf dbus service and dnf dbus
>  client)
>  
>  Fabio 
Aside of others, it will help to solve a dead locks when DNF is
accessing system resources and DB. Right now, Anaconda is launching
DNF/YUM in a separate process because otherwise we will be locked by
Python GIL (Global Interpreter Lock) on every C call in the DNF library
from python. Another reason is to replace PackageKit by something which
is not dead or have a possibility to replace it.
Second benefit is DBus is language agnostic so you could use any
programming language you want to use DNF.
Jirka
>  > 
>  > > 
>  > 
>  > > > (And e.g. provide a single cache and privilege escalation
>  > through
>  > 
>  > > > packagekit)?
>  > 
>  > > >
>  > 
>  > > 
>  > 
>  > > We can do the single cache thing *today* for PackageKit. The APIs
>  > 
>  > > exist in libdnf _now_, it's just that they're not used
>  > 
>  > > PackageKit-side.
>  > 
>  > > 
>  > 
>  > > > Apart from the dbus api, do you plan to provide some graphical
>  > 
>  > > > application that uses this api?
>  > 
>  > > >
>  > 
>  > > 
>  > 
>  > > I would expect that dnfdragora will be the first consumer of this
>  > new
>  > 
>  > > API, since this plan would essentially involve taking over the
>  > role of
>  > 
>  > > my dnfdaemon.
>  > 
>  > > 
>  > 
>  > > > Are you going to use sd-bus for the dbus library?
>  > 
>  > > >
>  > 
>  > > 
>  > 
>  > > I'd hope not, given that we have cross-distro usage of DNF now,
>  > and a
>  > 
>  > > couple of them don't have systemd.
>  > 
>  > > 
>  > 
>  > > 
>  > 
>  > > 
>  > 
>  > > --
>  > 
>  > > 真実はいつも一つ！/ Always, there's only one truth!
>  > 
>  > > _______________________________________________
>  > 
>  > > devel mailing list -- devel(a)lists.fedoraproject.org
>  > 
>  > > To unsubscribe send an email to 
>  > devel-leave(a)lists.fedoraproject.org
>  > 
>  > > Fedora Code of Conduct:
>  > 
>  > > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  > 
>  > > List Guidelines: 
>  > https://fedoraproject.org/wiki/Mailing_list_guidelines
>  > 
>  > > List Archives:
>  > 
>  > > 
>  > https://lists.fedoraproject.org/archives/list/[e-mail]
>  > 
>  > > 
>  > 
>  > _______________________________________________
>  > 
>  > devel mailing list -- devel(a)lists.fedoraproject.org
>  > 
>  > To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  > 
>  > Fedora Code of Conduct: 
>  > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  > 
>  > List Guidelines: 
>  > https://fedoraproject.org/wiki/Mailing_list_guidelines
>  > 
>  > List Archives: 
>  > https://lists.fedoraproject.org/archives/list/[e-mail]
>  > 
>  
>  _______________________________________________devel mailing list -- 
>  devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: 
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: 
>  https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives: 
>  https://lists.fedoraproject.org/archives/list/[e-mail]