F38 proposal: IPP-USB as a weak dependency of CUPS and sane-airscan (Self-Contained Change proposal)

[Ben Cotton]

https://fedoraproject.org/wiki/Changes/IPPUSBasPrintScanDependency

This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.


== Summary ==
Add `ipp-usb` as a weak dependency of packages which provide support
for driverless printing (`cups`), driverless scanning (`sane-airscan`)
and driverless fax for USB devices capable of using driverless
functionality (how to find out whether your USB device is driverless
[https://docs.fedoraproject.org/en-US/quick-docs/cups-useful-tricks/#_how_...
here]), so such devices will work without a specific driver. `ipp-usb`
design conflicts with the way how drivers work with the device, so a
user intervention is required after upgrade.


== Owner ==
* Name: [[User:Zdohnal | Zdenek Dohnal]]
* Email: zdohnal(a)redhat.com


== Detailed Description ==
Nowadays most printers and scanners from common vendors provide a way
how to work without a specific driver. This way relies on driverless
standards implemented in device's firmware and their support in the
operating system user has. At first (in 2010) network driverless
standards were implemented in devices and widely spread, such as
AirPrint, [https://www.pwg.org/ipp/everywhere.html IPP Everywhere] for
printers or eSCL (sometimes called AirScan) and WSD (Windows Service
Discovery) for scanners. Those network driverless standards and others
are already supported in `cups` (for printing) and `sane-airscan`
packages. Two years later USB devices got their own driverless
standard -
[https://robots.org.uk/IPPOverUSB?action=AttachFile&do=view&target...
IPP over USB] - which is now implemented by `ipp-usb`. The package
itself has been in Fedora for more than two years available for users
to opt-in USB driverless support.

The `ipp-usb` package contains `ipp-usb` daemon, which works as an
HTTP proxy over the claimed USB devices, provides printing (via
[http://ftp.pwg.org/pub/pwg/standards/std-ipp20-20151030-5100.12.pdf
IPP 2.0+] protocol), scanning (via eSCL) and fax (via
[http://ftp.pwg.org/pub/pwg/candidates/cs-ippfaxout10-20140618-5100.15.pdf
IPP Faxout]) support and advertises them on localhost by mDNS
protocol. mDNS is the key feature for automatic discovery, but it is
not required to use it - the virtual printer device provided by
`ipp-usb` can be accessed at URI `ipp://localhost:60000/ipp/print` and
the URI can be used for permanent queue installation -
[https://docs.fedoraproject.org/en-US/quick-docs/cups-useful-tricks/#_how_...
here] is the manual. However if mDNS is enabled, the virtual device
shared by `ipp-usb` can be automatically picked up by other services
(as `cups` or `sane-airscan`), so no further configuration is required
to get the device installed. The feature is called ''temporary queue''
in CUPS and it is supported in applications using the up-to-date CUPS
API or toolkits using up-to-date CUPS API - f.e. GTK3+ based
applications, Libreoffice and Firefox. The fax functionality is
available at URI `ipp://localhost:60000/ipp/faxout`, but the automatic
installation doesn't work for it and it has to be installed manually.

As mentioned above, the `ipp-usb` daemon claims the USB interface of
the device which supports IPP over USB standard. This behavior
conflicts with the previous driver approach, where the discovery
mechanism only scans USB ports for available devices, but doesn't try
to claim the USB interface, which is unavailable because `ipp-usb` has
claimed it already. The result is the device can be discovered by
classic driver tools, but it won't work once user wants to print, scan
or fax. In such cases user intervention is needed, where user has to
make a decision whether to use driverless USB support or classic
support with drivers. The way how to do it will be explained later in
this proposal.

Based on the current `ipp-usb` design the following specific setups
aren't expected to work, because they are not common with USB device
usage:

* combining driverless and classic driver's support doesn't work on
the same device - driverless or classic driver has to be used for
whole device's functionality.
* if user has several devices of the same model, all of them has to be
supported by a single solution - driverless or classic driver -
because quirks and SANE backends use model name, vendor ID or product
ID, which are the same for all devices of the same model, for denying
the support.
* if scanner backend does not support disabling support for a specific
device (f.e. `hpaio`, `pixma` are such backends), the whole backend
can be disabled to prevent discovering broken scanners - it results in
the scanner support provided by the backend will be disabled for all
other devices which are in the user's environment - both network and
USB.

To provide a possibility to opt-out from driverless USB printing the
`ipp-usb` package will be added as a weak dependency of `cups` and
`sane-airscan`, so it can be uninstalled without losing the rest of
the printing and scanning stack, and `ipp-usb check` will be added
into `%post` scriptlet of `ipp-usb` package for users to see whether
they have a device claimed by `ipp-usb` during upgrade process.

== Feedback ==
I've investigated possible solutions for two biggest concerns which
adding `ipp-usb` has raised and consulted it upstream:

1. The current design of claiming the interface of IPP-over-USB device
and not releasing it until `ipp-usb` daemon stops and whether it can
be changed - I talked with upstream about it at
[https://github.com/OpenPrinting/ipp-usb/issues/50#issuecomment-1122248609
upstream issue] - the reason for it is to prevent potential race
conditions/bugs in device's firmware, which is sometimes flaky and
updates are not coming to it in regular manner, due combined access
for printing or scanning.

2. The design makes impact to an existing setups as well, so the
upgrade path has to be taken into consideration - this was mentioned
in the [https://github.com/OpenPrinting/ipp-usb/issues/50 upstream
ticket] too. Based on the following expectations regarding devices:

* might have firmware bugs
* might have lesser functionality than classic driver

and the expectations above differ from device to device, we can't
automatically prefer one solution over the other at the moment and
user manual intervention after upgrade is needed.

This feature will explain how to create a quirk for the `ipp-usb`
daemon to ignore the device or disable them in classic drivers to
reduce this conflict.

== Benefit to Fedora ==
Modern USB devices will work out of the box without a specific driver,
so users don't need to install a driver or the device at all.


== Scope ==
* Proposal owners:
The proposal owner will add `Recommends: ipp-usb` to `cups` and
`sane-airscan` packages, `ipp-usb check` call into `%post` scriptlet
of `ipp-usb` and rebuild all changed packges. The owner will update
Fedora Quick Docs with the manual how to create the `ipp-usb` quirk
and how to disable the device in classic drivers if possible.

* Other developers:
* Release engineering:
* Policies and guidelines: N/A (not needed for this Change)
* Trademark approval: N/A (not needed for this Change)
* Alignment with Objectives:


== Upgrade/compatibility impact ==
`ipp-usb` is incompatible with classic printing and scanning drivers
for IPP-over-USB devices, so a manual intervention depending on user's
choice is required after upgrade. The steps which users have to do are
described below.

Choices:
# IPP-USB
# Classic drivers


=== Prerequisites: Checking the IPP-over-USB device and its capabilities ===

* update device's firmware if possible
* stop and disable `cups-browsed` service if it is not used for
installing printers from remote print server (which means `BrowsePoll
<server>` is used in `/etc/cups/cups-browsed.conf`)
* check if the device is recognized by `ipp-usb`:

<pre>
$ sudo ipp-usb check
Configuration files: OK
IPP over USB devices:
 Num  Device              Vndr:Prod  Model
   1. Bus 001 Device 005  04a9:2823  "Canon MF440 Series"
</pre>

* if the device has a printing functionality:
:* check whether the device is seen by CUPS among existing
destinations, but not among installed printers
(''Canon_MF440_Series_USB'' is the CUPS temporary queue for
IPP-over-USB device called ''Canon MF430''):

<pre>
$ lpstat -e
Canon_MF440_Series_USB
Canon_MF440_Series
$ lpstat -a
Canon_MF440_Series accepting requests since Wed 16 Mar 2022 11:27:02 AM CET
</pre>

::'''Note''': If the ''Canon_MF440_Series_USB'' is
shown here, but not
in your application, report the issue to the application.

:* check whether it provides the options you use during printing:

<pre>
$ ipptool -tv ipp://localhost:60000/ipp/print get-printer-attributes.test
$ lpoptions -p Canon_MF440_Series_USB -l
</pre>

::'''Note''': `ipptool` command will return all IPP attributes
which
the device supports - currently common PPD options are generated from
some of the attributes, so if your printing option is present in IPP
response, but not in `lpoptions` output, then it is a CUPS issue.
`lpoptions` returns available PPD options, which are used by print
dialogs in most cases.
* If the device has a scanning functionality:
:* check whether the device is seen by `airscan` backend (the HP
LaserJet MFP M130fw device here is used for illustration, it does not
show its real IPP-over-USB compatibility or its real options shared
via AirScan from `ipp-usb`):

<pre>
$ scanimage -L
...
device `airscan:e0:HP LaserJet MFP M130fw (E700D6)' is a eSCL HP
LaserJet MFP M130fw (E700D6) ip=127.0.0.1
</pre>

:* check the device capabilities:

<pre>
$ scanimage --help -d 'airscan:e0:HP LaserJet MFP M130fw (E700D6)'
...
Options specific to device `airscan:e0:HP LaserJet MFP M130fw (E700D6)':
  Standard:
    --resolution 75|150|200|300|600|1200dpi [300]
        Sets the resolution of the scanned image.
    --mode Color|Gray [Color]
        Selects the scan mode (e.g., lineart, monochrome, or color).
    --source Flatbed|ADF [Flatbed]
        Selects the scan source (such as a document-feeder).
  Geometry:
    -l 0..215.9mm [0]
        Top-left x position of scan area.
    -t 0..297.011mm [0]
        Top-left y position of scan area.
    -x 0..215.9mm [215.9]
        Width of scan-area.
    -y 0..297.011mm [297.011]
        Height of scan-area.
  Enhancement:
    --brightness -100..100% (in steps of 1) [0]
        Controls the brightness of the acquired image.
    --contrast -100..100% (in steps of 1) [0]
        Controls the contrast of the acquired image.
    --shadow 0..100% (in steps of 1) [0]
        Selects what radiance level should be considered "black".
    --highlight 0..100% (in steps of 1) [100]
        Selects what radiance level should be considered "white".
    --analog-gamma 0.0999908..4 [1]
        Analog gamma-correction
    --negative[=(yes|no)] [no]
        Swap black and white
...
</pre>

* if the device has a fax functionality and user requires it:
:* check its capabilities via `ipptool` command:

<pre>
$ ipptool -tv ipp://localhost:60000/ipp/faxout get-printer-attributes.test
</pre>

User can see the available printing, scanning and fax capabilities
with the commands above and can decide which solution -
'''driverless''' or '''classic drivers'''
- he wants to use. The
recommendation is to use driverless if it provides the set of options
required by user - the device is not bound to a driver being available
in the distribution and, if mDNS works, the device is automatically
installed and no other intervention will be needed in the future, when
classic drivers will be covered by printer applications and permanent
queues will turn into printer profiles on desktops.


=== Choice 1: IPP-USB is chosen to support the device ===

* if the device has a printing functionality:

:* Remove any existing printers installed for the device in the past:
::* find out the printer name:

<pre>
$ lpstat -a
Canon_MF440_Series accepting requests since Wed 16 Mar 2022 11:27:02 AM CET
</pre>

::* remove the printer:

<pre>
$ lpadmin -x Canon_MF440_Series
</pre>

* if the device has a scanning functionality:
:* Disable the device in a SANE backend, disable the backend as whole
or uninstall the driver:

<pre>
$ scanimage -L
device `hpaio:/usb/laserjet_mfp_m129-m134?serial=XXXX' is a
Hewlett-Packard laserjet_mfp_m129-m134 all-in-one
device `airscan:e0:HP LaserJet MFP M130fw (E700D6)' is a eSCL HP
LaserJet MFP M130fw (E700D6) ip=127.0.0.1
$ sudo sed -i 's,^\s*hpaio$,#hpaio,' /etc/sane.d/dll.d/hpaio
                                       # disables backend
</pre>

* if the device has a fax functionality:

:* remove the old fax queue if exists:

<pre>
$ lpadmin -x <fax_queue>
</pre>

:* install a new fax queue:

<pre>
$ lpadmin -p <fax_name> -v ipp://localhost:60000/ipp/faxout -m
driverless-fax:ipp://localhost:60000/ipp/faxout -E
</pre>

After this user is able to send fax via `lp` command and the chosen fax queue.


=== Choice 2: A classic driver is chosen to support the device ===

* create a quirk for `ipp-usb`:

:* get device's model name:

<pre>
$ sudo ipp-usb check
$ sudo ipp-usb check
Configuration files: OK
IPP over USB devices:
 Num  Device              Vndr:Prod  Model
   1. Bus 001 Device 005  04a9:2823  "Canon MF440 Series"
</pre>

:* use the name in new quirk file at `/etc/ipp-usb/quirks` directory.
The `.conf` suffix is required.

<pre>
$ cat /etc/ipp-usb/quirks/canon.conf
[Canon MF440 Series]
  blacklist = true
</pre>

:* restart `ipp-usb` service:

<pre>
$ sudo systemctl restart ipp-usb
</pre>

This quirk will deny device's support in `ipp-usb` and classic drivers
will work.

== How To Test ==
USB device capable of supporting IPP-over-USB is required to test this
change. `ipp-usb` starts once IPP-over-USB device is connected and
then do the following steps:

* follow prerequisites mentioned in the
[https://fedoraproject.org/wiki/Changes/IPPUSBasPrintScanDependency#Upgrad...
Upgrade/compatibility impact] and choose IPP-over-USB support,
* open a document in applications you use for printing/scanning/fax
* check whether the device is seen in the application (in print
dialog, or in scanner list) - if the device is not seen, report the
issue to the application's bugzilla component,
* check which options are available in the dialog/settings - if some
required (for your use cases) options are missing in comparison to
`lpoptions` and `scanimage` outputs (details how to find out device's
capabilities in
[https://fedoraproject.org/wiki/Changes/IPPUSBasPrintScanDependency#Upgrad...
Upgrade/compatibility impact]), report the issue to the application's
bugzilla component,
* try the actions you usually do on your device (print/scan/fax) with
your common options set:
:* in case of printers and fax if the printout is not in expected
format, do report the issue to `cups` bugzilla component together with
additional info (described
[https://docs.fedoraproject.org/en-US/quick-docs/how-to-debug-printing-pro...
here]),
:* in case of problem with scanning output do report the issue to
`sane-airscan` bugzilla component together with additional info
acquired by following steps from this
[https://docs.fedoraproject.org/en-US/quick-docs/how-to-debug-scanning-pro...
link],
* once you disable the device or backend for scanning, check whether
one scanner's disappeared from scanner application's dialog
(`simple-scan`, `xsane`, `scanimage`)

In case user chooses to have classic driver support instead of
driverless because driverless support does not work or it misses some
options which user requires, it would be great if the user reported
such case by filing an issue to `golang-github-openprinting-ipp-usb`
bugzilla component, explaining which required options are missing or
whether driverless does not print/scan at all and it will reviewed by
the component's maintainer. If the model has the driverless support
broken in general, the model can be disabled in `ipp-usb` on system
level by quirk, which is located in `/usr/share/ipp-usb/quirks`.

Once the quirk is set and `ipp-usb` restarted, previously installed
printers and scanners will work as before - the printing/scanning/fax
will end with error otherwise.

== User Experience ==
A new printer and a new scanner will appear in applications and
settings for IPP-over-USB devices  by default. Previously installed
printer and discovered scanner for the device will stop working and
manual intervention is required to remove the broken instances, or to
create a quirk for `ipp-usb`.

== Dependencies ==


== Contingency Plan ==
* Contingency mechanism:  N/A (not a System Wide Change)
* Contingency deadline: N/A (not a System Wide Change)
* Blocks release? N/A (not a System Wide Change)


== Documentation ==
* Printing and scanning
[https://docs.fedoraproject.org/en-US/quick-docs/cups-terminology/
terminology]
* [https://docs.fedoraproject.org/en-US/quick-docs/how-to-debug-printing-pro...
Printing] debugging
* [https://docs.fedoraproject.org/en-US/quick-docs/how-to-debug-scanning-pro...
Scanning] debugging
* [https://docs.fedoraproject.org/en-US/quick-docs/cups-useful-tricks/
Tips and tricks]
* [https://docs.fedoraproject.org/en-US/quick-docs/cups-known-issues/
Known issues]

== Release Notes ==
Driverless USB printing/scanning/fax support is present by default
with printing and scanning packages, providing the support for devices
capable of using IPP-over-USB. The manual intervention is required
after upgrade, which is described
[https://fedoraproject.org/wiki/Changes/IPPUSBasPrintScanDependency#Upgrad...
here].


-- 
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis