Re: Meeting agenda: RATS vs. librat
On 04/16/2018 10:49 AM, Randy Barlow wrote:
Alternatively, we could discuss it on
this mailing list if desired.
Perhaps it would be useful if I started the discussion here so we could
be more informed during Thursday's meeting.
pingou and I have been debating whether the project that allows
packagers to rerun failed tests should be a service or a library. I am
of the position that it should be a library.
RATS
====
RATS (Run Another Test Service) is a project that pingou has been
working on. It is a web service that allows API callers to ask for a
test to be run again. pingou, please do reply if I have misrepresented
or omitted anything; I will attempt to outline the benefits that pingou
argues for here.
* Being a service allows it to keep track of which tests have recently
been requested for re-run, which allows it to make sure a single test
doesn't get re-requested too many times.
* Being a service makes it easy to call from any language (Python, Ruby,
etc.).
* Being a service makes it possible to update in one place and have all
callers get the new behavior without themselves needing to update (as
long as the API is kept stable).
librat
======
I would like to make the case for using a library (perhaps called librat
- lib run another test?) here. Benefits:
* A library creates less work for the infrastructure development and ops
subteams. Adding a service means adding workload to a thinly stretched
devops team, as a service needs monitoring and requires intervention
when it goes down.
* A library is inherently more reliable than a service. Services must
obviously contain the library code, but now also add network
dependencies to projects that use them. It's one more piece in the
system that can fail and bring down Bodhi.
* A library is significantly less code than a service. For example,
libraries don't need to authenticate their callers, don't have to
serialize/deserialize inputs and return values, and don't need to
process human input (like config files). As stated, any service would
need the library's code anyway, but will also need much more code to
do all of the above. This also means less code to write tests and
documentation for.
* A library is able to meet all of our known requirements, and is
simple. I believe in the "keep it simple principle" - we should pick
the simplest solution that meets the requirements.
I would like to address the benefits of RATS outlined above, one by one
here:
* RATS can keep track of which tests have been run to prevent too many
re-runs
I believe that a library could do this locally for Bodhi and for Pagure
too once Pagure gets more test integration, by using a local cache.
Bodhi and Pagure will likely not gate on the same kinds of tests, and so
they don't need a central authority to make sure they each aren't
requesting the same test to be re-run. Furthermore, I think we don't
need to do a perfect job of making sure tests aren't re-run at all,
which is why I think having a library that caches recent re-run requests
will be "good enough" under the "perfect is the enemy of good"
mantra.
* Calling from multiple languages
This is a theoretical requirement at this point - we don't have a real
use case for this at this time. Further, a library would make it easy to
write a CLI which can be used if there ever does become a requirement
for other languages. A REST API is not inherently easier to use than a
CLI - in fact, I would make the case that it is harder to use.
* Updating
While it is true that a service can be updated in one place such that
all callers get the update immediately, we do have a big ansible project
that can just as easily deploy a library update out to all the machines
that use it, so I would argue that we practically have the same benefit
with a library.
Thoughts?