Fedora crda To wireless-regdb Upgrade

[John W. Linville]

A mostly "behind the scenes" aspect of the Linux wireless LAN
(IEEE802.11) subsystem has changed upstream with some system-level
effects. Replacing an existing package with a cleaner but functionally
equivalent package is an attractive option, but there are questions
regarding the update path, how signing the the regulatory database
should be handled, and how to communicate these changes outward to the
greater community.

BACKGROUND

Usage of Radio Frequency (RF) spectrum is generally regulated around
the world. Even unlicensed use of RF spectrum (as with wireless LANs)
is still subject to some level of local regulation. In order to support
lawful use of wireless LAN technology with Linux around the world, the
Linux kernel uses an externally maintained wireless regulatory database
that encodes relevant information about known regulatory (i.e.
legal/governmental) domains. The kernel makes use of this information
in order to enforce wireless regulations for devices under its control,
although some devices further apply their own limitations at the driver
or firmware level as well.

For a long time, the Linux kernel relied on a piece of software known
as the Central Regulatory Domain Agent (CRDA) to feed the kernel with
regulatory information in response to UDEV events that indicate what
regulatory domain's rules are to be enforced. Also, within Fedora, the
setregdomain script runs when a wireless netdev is instantiated. By
default this script attempts to relate a system’s TZ (i.e. Time Zone)
setting to a corresponding country code, which is then used to set the
wireless regulatory domain. The setregdomain script can also use
alternate means to determine which regulatory domain to request, as
detailed in the setregdomain.1 man page.

CRDA is maintained upstream in the crda git repository. The wireless
regulatory database is maintained upstream in the wireless-regdb git
repository. In Fedora, snapshots of both of the above repositories are
built within the single crda RPM. This is done to enable build-time
generation of a key used to sign the wireless-regdb database which is
then embedded in the concurrently built crda binary in order to
validate the integrity of the database at runtime.

As of about the upstream 4.15 version of Linux, the kernel's wireless
developers had determined that future updates to the wireless
regulatory database may require format changes that would be
incompatible with the existing CRDA implementation. Further, Linux
kernel changes over the years now enabled the kernel to load firmware
images without requiring a userland helper like CRDA. So, the Linux
kernel wireless developers implemented an in-kernel replacement for
CRDA. This includes code to validate the wireless regulatory database
against a signature that is built into the kernel itself.

SITUATION

At present, Fedora kernels are configured to use the in-kernel wireless
regulatory database loader. If the database is not found or is not
signed with the key built into the kernel, then boot time messages
appear, creating the appearance of a problem. For now, there is no
actual problem. Fedora systems continue to ship with CRDA, which
continues to perform its original duties with no effective problem as
of yet. The harmless "error" messages during boot are the only symptom.
(NOTE: the latest Fedora CRDA package works around this issue,
eliminating the spurious "error" messages at boot time.)

For now, there is no emergency. We could continue to ship CRDA as-is.
My concern is that eventually, CRDA may no longer be a viable option
for future regulatory database updates. In that case, it would seem to
be better to change now than to do so in a rush later.

Also, replacing the combined CRDA/wireless-regdb "crda" package with a
new "wireless-regdb" package removes a wart from our package collection
and replaces it with a package that is simpler and easier to understand
how it is built. That will be one less "why did they do that?" item
hanging around in Fedora.

ACTION IN PROGRESS

I have created a wireless-regdb package for Fedora. It simply installs
the wireless regulatory database binary into /usr/lib/firmware,
alongside it's pre-existing detached signature. This package includes a
version of the setregdomain script that is virtually identical to what
is in the existing crda package. This package has been approved for
Fedora, but I recently imported the SRPM to the Fedora repository. (htt
ps://bugzilla.redhat.com/show_bug.cgi?id=1598921)

I have been testing with the above package for a couple of weeks, and
the user experience is identical to using the crda solution. Given this
success, I believe that the best option will be to push this package
with Obsoletes and Provides for crda. This will facilitate a painless
and nearly silent upgrade for existing Fedora users. I will follow-up
with retiring the crda package itself a bit later. Other required
actions will include replacing any packaging references to the crda
package with an equivalent wireless-regdb reference (or simply
eliminating such a reference) and making the requisite changes to
comps.xml or any future equivalent.

QUESTIONS

Are there reasons to oppose the Obsolete/Provides upgrade path from
crda to wireless-regdb? Would it be desirable to require users to
manually intervene by installing wireless-regdb by hand? FWIW, I do not
see any benefit from requiring any manual steps.

Is it acceptable to trust the upstream signature of the wireless
regulatory database? Or do we need to use some sort of Fedora
signature? If the latter, can it be a (semi-)permanent (e.g. per
release) signature, which could be maintained in the kernel sources? Or
must it be a per build signature? How can that be accommodated, other
than through another combined package? (My personal preference is to
simply trust the upstream signature that is already being built into
the kernel.)

Does the crda to wireless-regdb upgrade effort need a Feature page (or
similar) in Fedora?

Since crda is a "critical path" component, what special considerations
need to be accommodated in order to replace it with wireless-regdb?

Thoughts?

John
-- 
John W. Linville	“The worst pain...to have insight into much
linville(a)redhat.com 		and power over nothing.” ― Herodotus

[Jeff Johnson]

Package renaming using Obsoletes/Provides will be a problem *ONLY* if applications confuse
the version of the package with the CRDB format version: these are intrinsically different
versions even if there is a single package renaming event that is coincident with the CRDB
format change.

Personally, I would add a CRDB format tracking version as a virtual Provides: CRDB-N =
X:Y-Z where. all of {CRDB-N, X, Y, Z} are merely placeholders for whatever values seem
natural and "appropriate". YMMV.

[Peter Robinson]

On Fri, Jul 20, 2018 at 7:47 PM, John W. Linville <linville(a)redhat.com&gt; wrote:
>  A mostly "behind the scenes" aspect of the Linux wireless
> LAN
>  (IEEE802.11) subsystem has changed upstream with some system-level
>  effects. Replacing an existing package with a cleaner but functionally
>  equivalent package is an attractive option, but there are questions
>  regarding the update path, how signing the the regulatory database
>  should be handled, and how to communicate these changes outward to the
>  greater community.
>
>  BACKGROUND
>
>  Usage of Radio Frequency (RF) spectrum is generally regulated around
>  the world. Even unlicensed use of RF spectrum (as with wireless LANs)
>  is still subject to some level of local regulation. In order to support
>  lawful use of wireless LAN technology with Linux around the world, the
>  Linux kernel uses an externally maintained wireless regulatory database
>  that encodes relevant information about known regulatory (i.e.
>  legal/governmental) domains. The kernel makes use of this information
>  in order to enforce wireless regulations for devices under its control,
>  although some devices further apply their own limitations at the driver
>  or firmware level as well.
>
>  For a long time, the Linux kernel relied on a piece of software known
>  as the Central Regulatory Domain Agent (CRDA) to feed the kernel with
>  regulatory information in response to UDEV events that indicate what
>  regulatory domain's rules are to be enforced. Also, within Fedora, the
>  setregdomain script runs when a wireless netdev is instantiated. By
>  default this script attempts to relate a system’s TZ (i.e. Time Zone)
>  setting to a corresponding country code, which is then used to set the
>  wireless regulatory domain. The setregdomain script can also use
>  alternate means to determine which regulatory domain to request, as
>  detailed in the setregdomain.1 man page.
>
>  CRDA is maintained upstream in the crda git repository. The wireless
>  regulatory database is maintained upstream in the wireless-regdb git
>  repository. In Fedora, snapshots of both of the above repositories are
>  built within the single crda RPM. This is done to enable build-time
>  generation of a key used to sign the wireless-regdb database which is
>  then embedded in the concurrently built crda binary in order to
>  validate the integrity of the database at runtime.
>
>  As of about the upstream 4.15 version of Linux, the kernel's wireless
>  developers had determined that future updates to the wireless
>  regulatory database may require format changes that would be
>  incompatible with the existing CRDA implementation. Further, Linux
>  kernel changes over the years now enabled the kernel to load firmware
>  images without requiring a userland helper like CRDA. So, the Linux
>  kernel wireless developers implemented an in-kernel replacement for
>  CRDA. This includes code to validate the wireless regulatory database
>  against a signature that is built into the kernel itself.
>
>  SITUATION
>
>  At present, Fedora kernels are configured to use the in-kernel wireless
>  regulatory database loader. If the database is not found or is not
>  signed with the key built into the kernel, then boot time messages
>  appear, creating the appearance of a problem. For now, there is no
>  actual problem. Fedora systems continue to ship with CRDA, which
>  continues to perform its original duties with no effective problem as
>  of yet. The harmless "error" messages during boot are the only symptom.
>  (NOTE: the latest Fedora CRDA package works around this issue,
>  eliminating the spurious "error" messages at boot time.)
>
>  For now, there is no emergency. We could continue to ship CRDA as-is.
>  My concern is that eventually, CRDA may no longer be a viable option
>  for future regulatory database updates. In that case, it would seem to
>  be better to change now than to do so in a rush later.
>
>  Also, replacing the combined CRDA/wireless-regdb "crda" package with a
>  new "wireless-regdb" package removes a wart from our package collection
>  and replaces it with a package that is simpler and easier to understand
>  how it is built. That will be one less "why did they do that?" item
>  hanging around in Fedora.
>
>  ACTION IN PROGRESS
>
>  I have created a wireless-regdb package for Fedora. It simply installs
>  the wireless regulatory database binary into /usr/lib/firmware,
>  alongside it's pre-existing detached signature. This package includes a
>  version of the setregdomain script that is virtually identical to what
>  is in the existing crda package. This package has been approved for
>  Fedora, but I recently imported the SRPM to the Fedora repository. (htt
>  ps://bugzilla.redhat.com/show_bug.cgi?id=1598921)
>
>  I have been testing with the above package for a couple of weeks, and
>  the user experience is identical to using the crda solution. Given this
>  success, I believe that the best option will be to push this package
>  with Obsoletes and Provides for crda. This will facilitate a painless
>  and nearly silent upgrade for existing Fedora users. I will follow-up
>  with retiring the crda package itself a bit later. Other required
>  actions will include replacing any packaging references to the crda
>  package with an equivalent wireless-regdb reference (or simply
>  eliminating such a reference) and making the requisite changes to
>  comps.xml or any future equivalent.
>
>  QUESTIONS
>
>  Are there reasons to oppose the Obsolete/Provides upgrade path from
>  crda to wireless-regdb? Would it be desirable to require users to
>  manually intervene by installing wireless-regdb by hand? FWIW, I do not
>  see any benefit from requiring any manual steps. 
Querying dnf it looks like the only package that explicitly depends on
the package or any of the binaries is wireless-tools so I think that
should be straight forward with Obsolete/Provides. It also means the
old crda is removed on upgrade so it won't hang around to
conflict/cause issues later.

>  Is it acceptable to trust the upstream signature of the wireless
>  regulatory database? Or do we need to use some sort of Fedora
>  signature? If the latter, can it be a (semi-)permanent (e.g. per
>  release) signature, which could be maintained in the kernel sources? Or
>  must it be a per build signature? How can that be accommodated, other
>  than through another combined package? (My personal preference is to
>  simply trust the upstream signature that is already being built into
>  the kernel.) 
I don't see an issue with the upstream signing, we do have the
infrastructure in Fedora for signing kernels (for secure boot) and
kernel modules with Fedora keys, presumably being loaded into the
kernel the signing mechanism would be the same as for modules (I'm
making wild assumptions here) so it's possible to have Fedora infra
sign it too if it's desirable. We just need to make sure you're in the
right ACLs.

>  Does the crda to wireless-regdb upgrade effort need a Feature page
> (or
>  similar) in Fedora? 
Personally I don't think it does, it's nothing of real note that we
need to notify people about as it should basically be seamless to the
end user or there's no great added wireless functionality the user
would care about.

>  Since crda is a "critical path" component, what special
> considerations
>  need to be accommodated in order to replace it with wireless-regdb? 
The "critical path" packages are generated based on dependencies of
blocking components so the new package should replace the old one once
it's landed and in a compose replacing it so there should be nothing
manual needed here.

[Nicolas Chauvet]

2018-07-20 20:47 GMT+02:00 John W. Linville <linville(a)redhat.com&gt;:
[...]
>  QUESTIONS
>
>  Are there reasons to oppose the Obsolete/Provides upgrade path from
>  crda to wireless-regdb? Would it be desirable to require users to
>  manually intervene by installing wireless-regdb by hand? FWIW, I do not
>  see any benefit from requiring any manual steps. 
The main issue with Obsoletes/Provides of crda is that if any user
want to maintain a self compiled kernel < 4.15. Then this kernel will
suddenly loose wireless support whereas it was previously working.
This is also a problem if using various kernel version to bissect a
wireless regression.

What I would recommend instead is to :
- have everything set for wireless-regdb to be installed by default in
f29+ (comps,ks,package dependency).
- eventually obsoletes/provides crda only on f29+
- for fedora < 29 have crda to require wireless-regdb, so current
users will migrate to the new method. (that way they can even manually
remove crda  and keep wireless-regdb if they like to).

This doesn't seem that complicated to handle as a transition that way.

-- 
-

Nicolas (kwizart)

[Zbigniew Jędrzejewski-Szmek]

On Sat, Jul 21, 2018 at 11:53:51AM +0200, Nicolas Chauvet wrote:
>  2018-07-20 20:47 GMT+02:00 John W. Linville
> <linville(a)redhat.com&gt;:
>  [...]
>  > QUESTIONS
>  >
>  > Are there reasons to oppose the Obsolete/Provides upgrade path from
>  > crda to wireless-regdb? 
+1 to the upgrade.

>  Would it be desirable to require users to
>  > manually intervene by installing wireless-regdb by hand? FWIW, I do not
>  > see any benefit from requiring any manual steps.
>  
>  The main issue with Obsoletes/Provides of crda is that if any user
>  want to maintain a self compiled kernel < 4.15. Then this kernel will
>  suddenly loose wireless support whereas it was previously working.
>  This is also a problem if using various kernel version to bissect a
>  wireless regression.
>  
>  What I would recommend instead is to :
>  - have everything set for wireless-regdb to be installed by default in
>  f29+ (comps,ks,package dependency).
>  - eventually obsoletes/provides crda only on f29+
>  - for fedora < 29 have crda to require wireless-regdb, so current
>  users will migrate to the new method. (that way they can even manually
>  remove crda  and keep wireless-regdb if they like to).
>  
>  This doesn't seem that complicated to handle as a transition that way. 
Not doing Obsoletes in F<29 is a good idea. There *are* users who runs
older kernels for various reasons (nvidia, various bugs, etc), and this
might break the network for them.

Considering that F29 will be out soon, maybe push the package with
Provides/Obsoletes to rawhide now. Once this gets a bit of testing,
you can consider pushing it out to F28 as an update. Since there's no
urgency to the upgrade, simply ignoring F28- and not doing the upgrade
there sounds like a reasonable option too.

Zbyszek

[Kevin Kofler]

John W. Linville wrote:
>  Is it acceptable to trust the upstream signature of the wireless
>  regulatory database? Or do we need to use some sort of Fedora
>  signature? If the latter, can it be a (semi-)permanent (e.g. per
>  release) signature, which could be maintained in the kernel sources? Or
>  must it be a per build signature? How can that be accommodated, other
>  than through another combined package? (My personal preference is to
>  simply trust the upstream signature that is already being built into
>  the kernel.) 
I think the only way to ship really Free Software is to disable the 
signature enforcement entirely.

        Kevin Kofler

[Howard Howell]

On Fri, 2018-07-20 at 14:47 -0400, John W. Linville wrote:
>  A mostly "behind the scenes" aspect of the Linux wireless
> LAN
>  (IEEE802.11) subsystem has changed upstream with some system-level
>  effects. Replacing an existing package with a cleaner but
>  functionally
>  equivalent package is an attractive option, but there are questions
>  regarding the update path, how signing the the regulatory database
>  should be handled, and how to communicate these changes outward to
>  the
>  greater community.
>  
>  BACKGROUND
>  
>  Usage of Radio Frequency (RF) spectrum is generally regulated around
>  the world. Even unlicensed use of RF spectrum (as with wireless LANs)
>  is still subject to some level of local regulation. In order to
>  support
>  lawful use of wireless LAN technology with Linux around the world,
>  the
>  Linux kernel uses an externally maintained wireless regulatory
>  database
>  that encodes relevant information about known regulatory (i.e.
>  legal/governmental) domains. The kernel makes use of this information
>  in order to enforce wireless regulations for devices under its
>  control,
>  although some devices further apply their own limitations at the
>  driver
>  or firmware level as well.
>  
>  For a long time, the Linux kernel relied on a piece of software known
>  as the Central Regulatory Domain Agent (CRDA) to feed the kernel with
>  regulatory information in response to UDEV events that indicate what
>  regulatory domain's rules are to be enforced. Also, within Fedora,
>  the
>  setregdomain script runs when a wireless netdev is instantiated. By
>  default this script attempts to relate a system’s TZ (i.e. Time Zone)
>  setting to a corresponding country code, which is then used to set
>  the
>  wireless regulatory domain. The setregdomain script can also use
>  alternate means to determine which regulatory domain to request, as
>  detailed in the setregdomain.1 man page.
>  
>  CRDA is maintained upstream in the crda git repository. The wireless
>  regulatory database is maintained upstream in the wireless-regdb git
>  repository. In Fedora, snapshots of both of the above repositories
>  are
>  built within the single crda RPM. This is done to enable build-time
>  generation of a key used to sign the wireless-regdb database which is
>  then embedded in the concurrently built crda binary in order to
>  validate the integrity of the database at runtime.
>  
>  As of about the upstream 4.15 version of Linux, the kernel's wireless
>  developers had determined that future updates to the wireless
>  regulatory database may require format changes that would be
>  incompatible with the existing CRDA implementation. Further, Linux
>  kernel changes over the years now enabled the kernel to load firmware
>  images without requiring a userland helper like CRDA. So, the Linux
>  kernel wireless developers implemented an in-kernel replacement for
>  CRDA. This includes code to validate the wireless regulatory database
>  against a signature that is built into the kernel itself.
>  
>  SITUATION
>  
>  At present, Fedora kernels are configured to use the in-kernel
>  wireless
>  regulatory database loader. If the database is not found or is not
>  signed with the key built into the kernel, then boot time messages
>  appear, creating the appearance of a problem. For now, there is no
>  actual problem. Fedora systems continue to ship with CRDA, which
>  continues to perform its original duties with no effective problem as
>  of yet. The harmless "error" messages during boot are the only
>  symptom.
>  (NOTE: the latest Fedora CRDA package works around this issue,
>  eliminating the spurious "error" messages at boot time.)
>  
>  For now, there is no emergency. We could continue to ship CRDA as-is.
>  My concern is that eventually, CRDA may no longer be a viable option
>  for future regulatory database updates. In that case, it would seem
>  to
>  be better to change now than to do so in a rush later.
>  
>  Also, replacing the combined CRDA/wireless-regdb "crda" package with
>  a
>  new "wireless-regdb" package removes a wart from our package
>  collection
>  and replaces it with a package that is simpler and easier to
>  understand
>  how it is built. That will be one less "why did they do that?" item
>  hanging around in Fedora.
>  
>  ACTION IN PROGRESS
>  
>  I have created a wireless-regdb package for Fedora. It simply
>  installs
>  the wireless regulatory database binary into /usr/lib/firmware,
>  alongside it's pre-existing detached signature. This package includes
>  a
>  version of the setregdomain script that is virtually identical to
>  what
>  is in the existing crda package. This package has been approved for
>  Fedora, but I recently imported the SRPM to the Fedora repository.
>  (htt
>  ps://bugzilla.redhat.com/show_bug.cgi?id=1598921)
>  
>  I have been testing with the above package for a couple of weeks, and
>  the user experience is identical to using the crda solution. Given
>  this
>  success, I believe that the best option will be to push this package
>  with Obsoletes and Provides for crda. This will facilitate a painless
>  and nearly silent upgrade for existing Fedora users. I will follow-up
>  with retiring the crda package itself a bit later. Other required
>  actions will include replacing any packaging references to the crda
>  package with an equivalent wireless-regdb reference (or simply
>  eliminating such a reference) and making the requisite changes to
>  comps.xml or any future equivalent.
>  
>  QUESTIONS
>  
>  Are there reasons to oppose the Obsolete/Provides upgrade path from
>  crda to wireless-regdb? Would it be desirable to require users to
>  manually intervene by installing wireless-regdb by hand? FWIW, I do
>  not
>  see any benefit from requiring any manual steps.
>  
>  Is it acceptable to trust the upstream signature of the wireless
>  regulatory database? Or do we need to use some sort of Fedora
>  signature? If the latter, can it be a (semi-)permanent (e.g. per
>  release) signature, which could be maintained in the kernel sources?
>  Or
>  must it be a per build signature? How can that be accommodated, other
>  than through another combined package? (My personal preference is to
>  simply trust the upstream signature that is already being built into
>  the kernel.)
>  
>  Does the crda to wireless-regdb upgrade effort need a Feature page
>  (or
>  similar) in Fedora?
>  
>  Since crda is a "critical path" component, what special
>  considerations
>  need to be accommodated in order to replace it with wireless-regdb?
>  
>  Thoughts?
>  
>  John
>  -- 
>  John W. Linville	“The worst pain...to have insight into much
>  linville(a)redhat.com 		and power over nothing.” ―
>  Herodotus 
>  _______________________________________________
>   
Your idea suffers from one thing... Who will build, maintain and ensure
that the database is accurate.  World RF spectrum is a tough nut.  It
flunctuates due to the birth and death of nations, ideology and
technology.  Moreover some of the standards are less standards than
suggestions.  Legally it is a minefield.

I have worked on RF devices for many decades, and I literally have no
idea how one would manage what you describe.  Moreover I ultimately
decided to only try to find the regulations when I needed to develop a
new test algorithm (I worked in the field of test) or when I had to
port a solution to some other country, and then I would look up their
regulations and find out how to manage that.  Even test sources are
often treated as emitters in some countries.

regards,
Lesh

[Samuel Sieb]

On 07/21/2018 09:46 AM, Howard Howell wrote:
>  Your idea suffers from one thing... Who will build, maintain and
> ensure
>  that the database is accurate.  World RF spectrum is a tough nut.  It
>  flunctuates due to the birth and death of nations, ideology and
>  technology.  Moreover some of the standards are less standards than
>  suggestions.  Legally it is a minefield. 
This database is not a new thing.  The only change here is how it is 
being stored and packaged for Fedora.  The kernel people manage the 
database and keep it up to date.  If you look on the linux-wireless 
mailing list you can see the occasional emails about updates to the 
database.