On 05/14/2018 10:51 AM, Florian Weimer wrote:
On 03/06/2018 10:26 PM, Carlos O'Donell wrote:
> We need ABI testing at each stage to provide confidence that
> allows developers to work quicker, trusting in the regression tests
> to help them make sure fewer mistakes are made.
This may be the case, but I'm even less convinced now that ABI test
artifacts belong into the source package itself. We need external
ABI test tooling, with proper test case management and review tools
for failures. If we can't consume something that already exists, I
don't think we have the resources to implement that, whether we put
it into the package build itself or not.
How do we make incremental progress to get us there?
- External ABI test tooling.
= Today this is abidw.
- Test case management.
= There is just one test today. Compare all DSOs.
= We could manually make one shell script per DSO and run that
as a single test case.
= pagure.io for glibc-abi is the place we could have ABI defects
reported.
- Review tools for failures.
= In the case of a failure you get full abidw output for the failed
DSO in the glibc build logs. At that point, set save_abi to 1
and warn_abi to 1, and do a scratch build, fetch the results and
compare offline with abidw (we may need to save a little more data)
Notes:
- We could roll abidw as a *patch* against the upstream sources.
I could do that, but I'm not yet ready to post upstream until we
have some more experience with the tooling. It would effectively
be a abidw-type test per directory that can install a DSO, and
the test would expect an unpacked ABI snapshot in top of the
glibc src directory or in a --with-abi-baseline=/path directory
kind of configure option. This is just a refactoring of where
tests go and how ready our patches are for inclusion upstream.
The current proposal looks like this:
* glibc-abi project:
- Contains curated ABI artifacts for all arches
for all upstream branches of interest.
- Distinct pull-request process.
- Distinct bug tracking.
[ This project is live on pagure.io ]
* glibc project:
- Consumes a copy of the branch of glibc-abi.
- Can produce as output a list of ABI artifaces.
- We should refactor the glibc.spec parts to become scripts
that are in the glibc-abi project instead of the spec file.
- glibc.spec uses glibc-abi sources to verify ABI.
Steps:
fedpkg clone glibc
git clone glibc-abi
cd glibc
[do your work]
[set glibc.spec to save_abi and warn_abi]
fedpkg build --scratch --srpm glibc.src.rpm
[look for ABI warnings, if there are none, no ABI work required, skip to (2)]
[if there are ABI changes, fetch binary glibc.rpms (where ABI artifacting is saved)]
cd ../glibc-abi
tar zxvf ../glibc/[downloaded ABI artifact files]*
[review ABI changes, commit, push]
make dist
cp glibc-abi*.tar.gz ../glibc/
[add new ABI artifacts file as new source]
(2) [download logs, verify, set save_abi=0 warn_abi=0 verify_abi=1]
[commit changes]
fedpkg build
--
Cheers,
Carlos.