Re: Using %verify and %ghost and other issues for mass cleanups

[Pavel Valena]

----- Original Message -----
>  From: "Tomasz Kłoczko" <kloczko.tomasz(a)gmail.com&gt;
>  To: "Development discussions related to Fedora"
> <devel(a)lists.fedoraproject.org&gt;
>  Sent: Thursday, April 4, 2019 3:38:11 PM
>  Subject: Using %verify and %ghost and other issues for mass cleanups
>  
>  Hi,
>  
>  Looks like I found something to do for someone with proven packager
>  privileges (which allow straight modify of any Fedora package git repo
>  without asking package maintainer to do modification).
>  Submitting hundreds of separated PRs for all below cases does not make to
>  much sense and totally will consume few man/days and with proven packager
>  privs it should take minutes. 
Hello,
I'm not a provenpackager, but I want to help a bit.
Commands inline should do the trick. I've tested them, so it should be smooth.

>  
>  !) (over)use %veryfy() with %ghost
>  
>  Just results of two commands:
>  
>  [tkloczko@domek SPECS.fedora]$ grep %ghost * | grep %verify | awk -F: '{print
>  $1}' | sort | uniq | wc -l
>  45
>  [tkloczko@domek SPECS.fedora]$ grep %ghost * | grep %verify | awk -F: '{print
>  $1}' | sort | uniq -c
>  1 arpwatch.spec
>  1 at.spec
>  8 bind.spec
>  2 certmaster.spec
>  2 clamav.spec
>  1 community-mysql.spec
>  1 cone.spec
>  3 cronie.spec
>  1 cyrus-imapd.spec
>  2 dovecot.spec
>  2 efont-unicode-bdf.spec
>  2 elinks.spec
>  2 exim.spec
>  1 fail2ban.spec
>  1 freeipa.spec
>  3 func.spec
>  14 glibc.spec
>  2 hitch.spec
>  1 initscripts.spec
>  1 japanese-bitmap-fonts.spec
>  2 jwhois.spec
>  1 kde-workspace.spec
>  1 libXvMC.spec
>  2 links.spec
>  1 logrotate.spec
>  1 mariadb.spec
>  5 monotone.spec
>  1 nessus-libraries.spec
>  3 openvswitch.spec
>  2 PackageKit.spec
>  1 pam.spec
>  2 pax.spec
>  2 setup.spec
>  1 sgml-common.spec
>  3 sssd.spec
>  2 star.spec
>  1 system-config-printer.spec
>  2 t1lib.spec
>  1 texlive-base.spec
>  2 tog-pegasus.spec
>  4 urw-base35-fonts.spec
>  2 util-linux.spec
>  2 uw-imap.spec
>  2 whois.spec
>  72 xorg-x11-fonts.spec 
find -maxdepth 2 -name '*.spec' -exec sed -i '/%ghost/
s/%verify\s*([^)]*)//g' "{}" \;

>  
>  What exactly is wrong with those specs files?
>  In all those specs %verify() token can be dropped without any consequences.
>  
>  Example fragment from last whois.spec:
>  
>  %ghost %verify(not md5 size mtime) %{_bindir}/%{name}
>  %{_mandir}/man1/%{name}.%{alternative}.*
>  %ghost %verify(not md5 size mtime) %{_mandir}/man1/%{name}.1.gz
>  
>  In all those cases looks like packagers don't know that %ghost disables md5,
>  size, mtime verification automatically . Fragment from rpm C code from:
>  https://github.com/rpm-software-management/rpm/blob/master/lib/verify.c#L117
>  
>  /* Content checks of %ghost files are meaningless. */
>  if (fileAttrs & RPMFILE_GHOST)
>  flags &= ~(RPMVERIFY_FILEDIGEST | RPMVERIFY_FILESIZE |
>  RPMVERIFY_MTIME | RPMVERIFY_LINKTO);
>  
>  +2 years ago I've submitted PR for glibc.spec patch to simplify (already
>  horrible and hard to read spec) but even above URL to the rpm code was
>  not able convince glibc maintainers.
>  
>  2) Using .gz suffix with man and info pages %files entries
>  
>  That is possible to see in already quoted whois.spec part.
>  
>  Number of affected packages:
>  
>  [tkloczko@domek SPECS.fedora]$ grep "^%{_mandir}/.*.gz" * -l | wc -l
>  618
>  
>  List of the packages which needst to be corrected:
>  
>  [tkloczko@domek SPECS.fedora]$ grep "^%{_mandir}/.*.gz" * -l | awk -F.
>  '{print $1}' | xargs
>  389-ds-base 3proxy abiword abrt-java-connector acpid agedu AGReader ahcpd
>  alacarte alsa-utils amoebax amora amtterm anyremote anyterm
>  api-sanity-checker archivemail arduino-ctags aria2 arm-none-eabi-binutils-cs
>  arm-none-eabi-gcc-cs artha asc aterm atf atop audit autogen avr-binutils
>  avrdude avr-gcc awesfx aws balance barcode barman batctl bats bchunk beaker
>  bibtex2html biosig4c++ bip bluez-hcidump bluez boinc-client bombardier
>  boomaga botan2 bspwm btrfs-progs busybox byobu ca-certificates cairo-clock
>  caja-extensions calcurse calendar ccd2iso cduce certmaster cfv CGAL cgdb
>  check-mk checkpolicy chirp chocolate-doom chromium-bsu cifs-utils cjdns ck
>  clamsmtp clang ClanLib06 clazy clive clpbar cmark codeblocks colord-gtk
>  colorhug-client colrdx connman conserver console-image-viewer corkscrew
>  cqrlog crack-attack cryptlib cryptobone cryptsetup cups-filters cups
>  curblaster cwdaemon daemonize dahdi-tools danmaq darktable dasher datamash
>  davfs2 dconf debmirror deepin-mutter deja-dup desktop-file-utils detox
>  device-mapper-multipath device-mapper-persistent-data devilspie2 devilspie
>  dex-autostart dhcpcd dillo disper dmtcp dnscap dnssec-tools docker-latest
>  docker dogtag-pki doxy2man dpkg drawtiming drbd dt dxcc dynamite
>  ecryptfs-utils efibootmgr electric emelfm2 enchant endless-sky
>  environment-modules esound espeak-ng espeak expect ezstream fastd fbterm
>  fedrepos ffgtk Field3D fishpoll fldigi fmtools fntsample focuswriter foo2zjs
>  fpaste fpm2 fprintd freecad freecol freedroidrpg freight-tools fribidi frysk
>  fts func fuse-emulator fuse-emulator-utils fuse-sshfs fuse-zip fwrestart
>  gammaray gammu gbrainy gconf-editor gcstar gdigi genwqe-tools gftp
>  ggz-base-libs ggz-gtk-client gigolo git-review glaxium glmark2 gmic gmsh
>  gnome-mud gnome-multi-writer gnome-phone-manager gnome-search-tool
>  gnome-shell gnome-software gnucap gnupg1 gnuplot gnushogi
>  gobject-introspection gource gramps graphviz gretl grfcodec grinder
>  gshutdown gspiceui gst123 gstreamer1-plugins-base gtk2 gtkterm guake gyazo
>  hdf hexchat hfsplus-tools highlight hunspell hydrogen hylafax+ i2c-tools ibp
>  iipsrv imapfilter incron infiniband-diags inkscape inn input-pad
>  intel-cmt-cat iodine iok iperf3 ipe iputils irsim iscsi-initiator-utils
>  isight-firmware-tools istatd itcl itk jflex jnettop josm kakasi
>  kde-cli-tools kdiff3 kea kernel-tools kexec-tools kf5-kjs kimchi
>  knot-resolver KoboDeluxe krazy2 krb5 kronosnet krusader kstart kst kvirc lcm
>  ldapvi ldm ldns lecm libappstream-glib libavc1394 libcbor libcryptui
>  libdc1394 libdwarf libewf libffi libgsf libhugetlbfs libipt libiscsi libldb
>  libldm liblogging libmicrohttpd libnih libpano13 libpreludedb libprelude
>  libreport libsepol libtalloc libtevent libuninameslist libvoikko libyubikey
>  liferea limnoria link-grammar linux-user-chroot liquidwar llvm5 llvm6 llvm
>  lm_sensors logiweb logjam lsvpd lttng-tools lttng-ust lumina-desktop lvm2 lv
>  lxde-common lxqt-session marsshooter mate-notification-daemon mate-utils
>  mathgl mcstrans mdbtools meld memchan memstomp memtest86+ memtester
>  menulibre metacity metis mindless miniupnpd mkelfimage moarvm mono-addins
>  mono mono-tools moreutils mosh mot-adms mpdscribble mpssh msynctool munge
>  mupdf musca myman nagi nautilus-sendto nbdkit ncl ndjbdns netstat-nat
>  NetworkManager-libreswan nfs-ganesha nfs-utils nicotine+ nicstat nilfs-utils
>  nip2 njam nkf nmap nng nodejs-html-minifier nqc nss_updatedb nut nvme-cli
>  nvmetcli nvml nwipe OCE ocfs2-tools oct2spec omniORB onedrive opendbx
>  OpenImageIO openmpi openmsx openscap-daemon openscap opensmtpd openvas-cli
>  orangefs orange osmium-tool ostree otter-browser p11-kit pam_afs_session
>  pam_yubico paperkey parrot parted pdfcrack pdns-recursor pdns
>  percona-xtrabackup perl-Cflow perl-CPAN-Inject perl-Crypt-RC4-XS
>  perl-DateTime-Precise perl-DBICx-AutoDoc perl-Devel-CheckOS
>  perl-Eval-LineNumbers perl-File-BaseDir perl-Finance-YahooQuote
>  perl-Flickr-Upload perl-Glib perl-Goo-Canvas perl-HTML-FromText
>  perl-HTML-PrettyPrinter perl-JavaScript-Beautifier perl-Jemplate
>  perl-Linux-Pid perl-Module-Starter-Plugin-CGIApp perl-MojoX-JSON-RPC
>  perl-Net-eBay perl-Net-Server-Coro perl-Net-SSH-Expect
>  perl-Parse-ErrorString-Perl perl-PerlIO-gzip perl-Pod-Abstract
>  perl-Pod-PseudoPod-LaTeX perl-Pod-Xhtml perl-POE-man Test-Loops perl-SQL-Shell
>  perl-Term-ReadLine-Gnu perl-User perl-WWW-Search perl-XML-Handler-YAWriter
>  perl-XML-Merge perl-XML-Tidy pfstools pgcenter pgdbf pioneers
>  pipenightdreams pki-core pkpgcounter planner plplot pmdk-convert
>  policycoreutils postgresql_autodoc postsrsd ppl procinfo-ng psiconv
>  pulseaudio pungi puppet pv pwsafe pybugz pykickstart pynag python-ansi2html
>  python-anyconfig python-arc python-bloom python-bugzilla python-catkin_tools
>  python-filelock python-glanceclient python-heatclient python-libsass
>  python-murano-pkg-check python-mygpoclient python-novaclient
>  python-osrf-pycommon python-pycodestyle python-rosdep python-rosinstall
>  python-rospkg python-rtslib python-scss python-tilestache python-vcstools
>  python-wstool python-yolk q4wine qsynth QtDMM R2spec rakudo ratpoison
>  rawtherapee realmd recap rfcdiff rho rkward rootsh rpl rpmrebuild rsyslog
>  rubber rubygem-rdiscount sage samba sawfish sayonara sblim-sfcc scamp
>  schedtool schroot scl-utils scmxx sdcv sdlhack seahorse-sharing secilc
>  sendmail seren setroubleshoot setuptool sflphone simple-mtpfs slirp4netns
>  snap SoQt spectrwm speedtest-cli spicctrl starplot stoken strongswan sudo
>  suil sunpinyin sxhkd synce-trayicon sysfsutils system-config-language
>  tagsoup targetcli tboot tcpjunk tcputils tdom telepathy-gabble
>  telepathy-idle telepathy-rakia telepathy-salut teseq tikzit tilp2 tinyproxy
>  tipcutils tkdnd tkgate tnftp tntnet tokyocabinet tomboy torque tpm2-tools
>  tpm2-tss trackballs transfig transmission-remote-cli transmission-remote-gtk
>  transtats-cli trickle trophy ttmkfdir tunir tuxpuck txt2rss udpxy uisp
>  unifdef unshield upslug2 usbguard usb_modeswitch usnic-tools util-linux vala
>  validns vcftools verilator vhostmd vile vim-vimoutliner virt-who vrq wammu
>  wdiff whois wipe WoeUSB wordnet wsjtx x2goclient x2godesktopsharing
>  x2goserver xarchon xblast xcompmgr xconvers xen xfce4-dict
>  xfce4-sensors-plugin xfmpc xgalaxy xhotkeys xiphos xkb-switch xmoto
>  xorg-x11-drv-openchrome xpilot-ng xpsk31 xscope xskat yacpi yamllint
>  ykclient ykpers yubico-piv-tool zanata-util zfs-fuse
>  
>  This will block for example quick switching to use xz or lz4 to compress all
>  those files.
>  
>  The same is with info pages:
>  
>  [tkloczko@domek SPECS.fedora]$ grep "^%{_infodir}/.*.gz" * -l | wc -l
>  57
>  
>  List of affected packages:
>  
>  [tkloczko@domek SPECS.fedora]$ grep "^%{_infodir}/.*.gz" * -l | awk -F.
>  '{print $1}' | xargs
>  adime adplug am-utils autogen avrdude barcode binutils cflow cgdb datalog
>  datamash DevIL emacs-goodies emacs-magit emacs-mmm emacs-rinari emacs-slime
>  findutils foxtrotgps ghdl giac gnubik gnupg1 gnuplot gnushogi grep
>  guile-cairo guile-lib isight-firmware-tools jwhois libbinio libchewing
>  libffi libIDL libmicrohttpd librep libtool lightning lilypond macchanger
>  mathgl moon-buggy mpop mutt mydns nettle openocd ORBit parted psacct pwmd
>  python2-docs ratpoison teseq tinc wdiff wol 
find -maxdepth 2 -name '*.spec' -exec sed -i '/%{_mandir}/ s/\.gz/.*/g'
"{}" \;
find -maxdepth 2 -name '*.spec' -exec sed -i '/%{_infodir}/ s/\.gz/.*/g'
"{}" \;

>  
>  3) (over)use %doc in case of man pages entries in %files
>  
>  [tkloczko@domek SPECS.fedora]$ grep "^%doc %{_mandir}" * -l | wc -l
>  192
>  
>  Man pages are by definition %doc because:
>  
>  $ rpm -E %__docdir_path
>
> /usr/share/doc:/usr/share/man:/usr/share/info:/usr/share/gtk-doc/html::/usr/share/man:/usr/share/info:/usr/share/javadoc:/usr/doc:/usr/man:/usr/info:/usr/X11R6/man
>  
>  BTW: currently it is possible to shorten that list of paths because it has
>  duplicates and no longer used by any fedora package directories.
>  
>  And list of affected packages:
>  
>  [tkloczko@domek SPECS.fedora]$ grep "^%doc %{_mandir}" * -l | awk -F.
> '{print
>  $1}' | xargs
>  adcli annobin ansible argtable asciidoc augeas auto-destdir bcrypt boom-boot
>  boxes brltty castxml cdargs c-graph CGSI-gSOAP cherokee cinnamon-session
>  cloud-utils cockpit collectd criu cstream ctpl dbus-java dbxtool dietlibc
>  diffoscope dnssec-system-tray dracut e2tools ebtables eot-utils fbida fdupes
>  fedora-upgrade fetch-crl firefox freeradius freeze freight fwknop-gui
>  fwupdate gajim gcc-python-plugin gdcm geany general-purpose-preprocessor
>  gitstats globus-authz-callout-error globus-authz globus-callout
>  globus-common globus-ftp-client globus-ftp-control globus-gass-cache
>  globus-gass-copy globus-gass-transfer globus-gatekeeper globus-gram-audit
>  globus-gram-client globus-gram-client-tools
>  globus-gram-job-manager-callout-error globus-gram-job-manager-fork
>  globus-gram-job-manager-scripts globus-gram-job-manager globus-gram-protocol
>  globus-gridftp-server globus-gridmap-callout-error globus-gsi-callback
>  globus-gsi-cert-utils globus-gsi-credential globus-gsi-openssl-error
>  globus-gsi-proxy-core globus-gsi-proxy-ssl globus-gsi-sysconfig
>  globus-gssapi-error globus-gssapi-gsi globus-gss-assist globus-net-manager
>  globus-openssl-module globus-proxy-utils globus-rsl
>  globus-scheduler-event-generator globus-simple-ca globus-xio-gridftp-driver
>  globus-xio-gsi-driver globus-xio gnome-desktop gnome-screensaver
>  gnome-session gnome-system-log gnugo gofer grig gsm-ussd
>  gst-editing-services gst-entrans gstreamer1 gtick hash-slinger hunt icemon
>  igraph ipset iucode-tool jwm lbdb lcdtest lcgdm lde libcsv librabbitmq
>  librecad libreswan libxslt liquibase lookup loopabull lsyncd mod_mono
>  mousetweaks mpg123 mpich munin ndisc6 nethogs NLopt nml nodejs nomarch
>  nordugrid-arc nss numad nut obs-signd ocaml-tplib OpenCoarrays pacemaker
>  pam_shield papi parfait pass perl-App-PFT perl-PFT planarity pngcrush
>  python-behave python-bitmath python-clint python-ethtool
>  python-networkmanager qpid-cpp rear retrace-server rktime rssh
>  rubygem-bundler rubygem-chake rubygem-clockwork rubygem-mustache
>  rubygem-rake rubygem-sdoc rubygem-shotgun rubygem-tilt rubygem-treetop salt
>  sbd scap-workbench scapy scponly sepolicy_analysis serd snake socat
>  squeezelite sslh ssss star stow subscription-manager tcpreplay tito
>  transmission twinkle txt2tags udns ursa-major whowatch xlockmore xwax ytree
>  yuicompressor 
find -maxdepth 2 -name '*.spec' -exec sed -i '/%{_mandir}/ s/%doc //g'
"{}" \;

>  
>  4) Above issues should be cached by rpmlint so it is yet another small point
>  on TODO list.
>  
>  Who will take care at least those 3 first points?
>  Volunteers?
>  
>  kloczek
>  --
>  Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH
>  
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>   
HIH,

-- 
Pavel Valena
Software Engineer, Red Hat
Brno, Czech Republic