Re: Announcing start of DNF 5 development

5 Mar 2020

      On Thu, 2020-03-05 at 08:56 +0100, jkonecny@redhat.com wrote:
...
On Thu, 2020-03-05 at 07:44 +0100, Fabio Valentini wrote:
...
On Thu, Mar 5, 2020, 00:55 Martin Kolman mkolman@redhat.com wrote:
...
----- Original Message -----
...
From: "Neal Gompa" ngompa13@gmail.com
...
To: "Development discussions related to Fedora" devel@lists.fedoraproject.org
...
Sent: Wednesday, March 4, 2020 11:01:43 PM
...
Subject: Re: Announcing start of DNF 5 development
...
...
On Wed, Mar 4, 2020 at 4:37 PM Zbigniew Jędrzejewski-Szmek
...
zbyszek@in.waw.pl wrote:
...
...
...
...
On Wed, Mar 04, 2020 at 07:03:01PM +0100, Daniel Mach wrote:
...
...
...
Hello everyone,
...
...
...
I'm pleased to announce start of DNF 5 development. We are planning
...
...
...
to deliver a module stream or a COPR repo during Fedora 33
...
...
...
development for early adopters and tool developers and we're hoping
...
...
...
in getting a stable version into Fedora 34.
...
...
...
...
...
...
...
...
...
More details follow.
...
...
...
...
...
...
...
...
...
We've managed to drop a lot of redundant code across the whole DNF
...
...
...
stack in the past years, but we have reached a point when it's
...
...
...
nearly impossible to consolidate the code any further without
...
...
...
breaking the API/ABI. Especially with PackageKit being dead[1], we
...
...
...
can't move with the old "libhif" API in libdnf, because making any
...
...
...
bigger changes to PackageKit is clearly out of scope.
...
...
...
...
...
...
[1]
...
...
...
https://blogs.gnome.org/hughsie/2019/02/14/packagekit-is-dead-long-live-well...
...
...
...
...
...
...
...
...
...
That's why we decided to start working on a new version of the DNF
...
...
...
stack: DNF 5. And this is the plan:
...
...
...
...
...
...
...
...
...
Priorities
...
...
...

...
...
...

Consistency, documentation and user experience is the top priority.

...
...
...

Compatibility on the command line level.

...
...
...

Compatibility on the API level.

...
...
...
...
...
...
...
...
...
Maintenance
...
...
...

...
...
...
The existing DNF 4 stack stays in the current Fedoras and Red Hat
...
...
...
Enterprise Linux 8. We'll keep maintaining it in dnf-4-master
...
...
...
branches on GitHub. PackageKit and rpm-ostree will stay on libdnf
...
...
...
from the DNF 4 stack.
...
...
...
...
...
...
...
...
...
The existing Python API in DNF
...
...
...

...
...
...
The Python API in DNF stays. We'll do our best to keep it working.
...
...
...
If there is an incompatible change, we'll communicate and document
...
...
...
it properly.
...
...
...
...
...
...
...
...
...
The new API in libdnf
...
...
...

...
...
...
All business logic will move from DNF (Python) to libdnf (C++). This
...
...
...
is the only way to ensure that package managers work identically
...
...
...
across the whole distribution. We'll start with C++ API and
...
...
...
auto-generated Python bindings via SWIG. We'll focus on the Python
...
...
...
bindings which are required by DNF and we will do our best to
...
...
...
provide bindings for Go, Perl5 and Ruby as well. C API will be
...
...
...
created later when the C++ API is stable. At that moment rpm-ostree
...
...
...
will be ported to the new C API.
...
...
...
...
...
...
...
...
...
hawkey
...
...
...

...
...
...
Hawkey Python API is going away and will be replaced with libdnf Python
...
...
...
API.
...
...
...
...
...
...
...
...
...
DNF
...
...
...

...
...
...
DNF stays as the primary command-line package manager. The overall
...
...
...
functionality remains. We don't anticipate any negative impact of
...
...
...
the API rewrite on the end-users. We have built an extensive test
...
...
...
suite (over 1400 scenarios) that will help us to ensure that. The
...
...
...
argument parser and outputs may slightly change in some cases to
...
...
...
provide a more consistent user-experience. All such cases will be
...
...
...
properly documented.
...
...
...
...
...
...
...
...
...
microdnf
...
...
...

...
...
...
Microdnf is becoming important because it's part of many containers
...
...
...
due to its small footprint. We're getting feedback that users would
...
...
...
appreciate something closer to DNF. We'll focus on implementing a
...
...
...
subset of DNF's functionality and improving the user experience.
...
...
...
100% feature parity with DNF is currently out of scope.
...
...
...
...
...
...
...
...
Hi,
...
...
...
...
the roadmap is ambitious, but not impossible. Good luck!
...
...
...
...
...
Roadmap (tentative)
...
...
...

...
...
...

Mar 2020 - making the bigger API changes, upstream code barely compiles

...
...
...

May 2020 - COPR repo with first development snapshots

...
...
...

Jun 2020 - F33 module available for early adopters and tool developers

...
...
...

Oct 2020 - DNF 5 landing in F34 Rawhide

...
...
...

Feb 2021 - DNF 5 replacing DNF 4 in stable Fedora

...
...
...
...
...
DBus service
...
...
...

...
...
...
DNF team has decided to create a new DBus service replacing
...
...
...
PackageKit to provide an interface to GUI applications. It's
...
...
...
probably going to take a while because we're planning to start from
...
...
...
scratch.
...
...
...
...
Do you plan to make normal 'dnf' calls go through the dbus api?
...
...
This would be interesting, but wouldn't that make using DNF in rescue
...
situations impossible?
You mean due to the regular DBus daemon & system + session bus not running
in some system rescue scenarios, right ?
While possibly a bit tricky I could imagine some fallback mechanism where
invocation of the CLI tool starts it's own DBus session & instance of the DNF
service when it detects that the regular system & session buses are not available.
Anaconda does something similar when it starts in Mock during some phases of the
compose process & finds no system bus is available - it starts it's own DBus daemon
process and uses that.
Wow, using dbus to communicate between CLI binary and the shared library sounds like an awful idea. Why not do a
simple shim around the shared library instead? (And not introduce another 2 moving parts into a critical system
component: dnf dbus service and dnf dbus client)
Fabio
Aside of others, it will help to solve a dead locks when DNF is accessing system resources and DB. Right now, Anaconda
is launching DNF/YUM in a separate process because otherwise we will be locked by Python GIL (Global Interpreter Lock)
on every C call in the DNF library from python.
I'll add one more reason why we are running DNF in a separate process during the installation - RPM apparently chroots
its process intermittently when it installs packages. 
This is not really a problem when you use DNF over CLI - it already running in a separate process. But when you use the
DNF Python API & RPM does it's chroot dance, it would effectively chroot the Anaconda process into the generally almost
empty at that time target system chroot. 
Let's say GTK and other libraries do not play well with the files they want to read or write suddenly vanishing in split
second only to show up back a second later, repeatedly. This effectively demonstrated as pretty strange deadlocks,
randomly missing icons or even crashes before we tracked it down to the likely unlikely culprit of RPM.
So at the moment we use the DNF Python API from a Python sub-process, but that's really hard from ideal, even just
because I'm pretty sure we are likely the only users of the DNF Python API doing so. 
Really looking forward to the planed DNF DBus API we could use instead, like we already do with other system-level
services and tooling, such as Network Manager.
...
Another reason is to replace PackageKit by something which is not dead or have a possibility to replace it.
Second benefit is DBus is language agnostic so you could use any programming language you want to use DNF.
Jirka
...
...
...
...
...
(And e.g. provide a single cache and privilege escalation through
...
...
packagekit)?
...
...
...
...
We can do the single cache thing *today* for PackageKit. The APIs
...
exist in libdnf _now_, it's just that they're not used
...
PackageKit-side.
...
...
...
Apart from the dbus api, do you plan to provide some graphical
...
...
application that uses this api?
...
...
...
...
I would expect that dnfdragora will be the first consumer of this new
...
API, since this plan would essentially involve taking over the role of
...
my dnfdaemon.
...
...
...
Are you going to use sd-bus for the dbus library?
...
...
...
...
I'd hope not, given that we have cross-distro usage of DNF now, and a
...
couple of them don't have systemd.
...
...
...
...
--
...
真実はいつも一つ！/ Always, there's only one truth!
...

...
devel mailing list -- devel@lists.fedoraproject.org
...
To unsubscribe send an email to devel-leave@lists.fedoraproject.org
...
Fedora Code of Conduct:
...
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
...
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
...
List Archives:
...
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
...

devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
_______________________________________________devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
_______________________________________________devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Re: Announcing start of DNF 5 development