Brain-dump on the topic: $SUBJ. Comments please.
-- Martian
# Tortilla - Custom Wrappers for Harness
Current harness uses few wrappers scattered all over the code and there is a need for few more. Goal of this extension is to provide a better way how to define and apply wrappers.
## What's a wrapper?
Wrapper is a small script which modifies environment in which the task runs.
Usage: `wrapper [wrapper-options] command [options] [arguments]`
Example: `nice(1)`
## Use-cases
*Show me where are they useful?*
### What we currently have
- selinux context and setting groups - defined in `bin/beah-initgroups.py` and `bin/beah-unconfined.sh` - used in `bin/beah-rhts-runner.sh` and `bin/rhts-compat-runner.sh` - undocumented: defaults and overrides for parameters - defined as `/etc/profile.d/task-defaults-rhts.sh` and `/etc/profile.d/task-overrides-rhts.sh` - used: - defaults in wrapper created by `beakerlc.py` - overrides in `bin/beah-rhts-runner.sh` and `bin/rhts-compat-runner.sh` - `bin/beah-rhts-runner.sh` and `bin/rhts-compat-runner.sh` - these are not wrappers but should be. Ideally they will be removed...
### What we need
- [Bug 773259 - tests don't run in root cpu cgroup with systemd](https://bugzilla.redhat.com/show_bug.cgi?id=773259) - [Bug 658455 - Implement global parameters in harness](https://bugzilla.redhat.com/show_bug.cgi?id=658455) - [Bug 755407 - SIGXFSZ signal behaviour is different while running under harness](https://bugzilla.redhat.com/show_bug.cgi?id=755407) - [Bug 618322 - Add option to enable MALLOC_PERTURB_ and MALLOC_CHECK_ on test system and use it as default](https://bugzilla.redhat.com/show_bug.cgi?id=618322) - [Bug 790881 - Software Collection support in the Test Harness](https://bugzilla.redhat.com/show_bug.cgi?id=790881) - nice(?) - login shell
### What else could be moved to wrappers
- test installation - yum or git? Just use different wrapper. - metadata processing - handle defaults, test metadata and recipe/task metadata. - rhts-compat - this should be ideally removed. - use wrappers as RPC-filters - filter/adjust messages - watchdog-handling - task stdout/stderr collector - on result: collect additional logs: messages and AVC errors
## To Feature or not to feature
- known order of execution - some wrappers must execute before others - add/remove/override wrappers - provide predefined wrappers - allow user to use own chains and override/reuse parts of predefined chains - configure defaults at wrapper instantiation time - a task will define/add wrapper to a chain using params to customize the defaults - modify default chain - override by parameters/metadata at run time - allow chosing chain and/or overriding wrapper params
## Implementation(?)
### Components
#### Wrapper
Individual wrapper with some "metadata"
Operations on wrapper:
- instantiation - create instance of wrapper with defaults provided by environment. - application - apply wrapper processing params passed in environment.
Predefined wrappers:
- `TORTILLA_QUEUE=QUEUE_NAME apply-queue` - apply named queue - `TORTILLA=SET_NAME apply-set` - apply named set - `${PREFIX}_CGROUPS=cpu:/ cgroup-wrapper` - change cgroups
#### Wrap
This is basically a linked-list of instantiated wrappers.
Members:
- instantiated wrapper - `__call__` - next wrap - `__next__` - wrapper defined files
Operations:
- application: let `W` be a Wrap, `apply W cmd args` is defined as follows:
W.__call__ apply __next__ cmd args
if `__next__` is "empty", do:
W.__call__ cmd args
#### Named Queue
Members:
- Name of the queue - Link to wrap at the top of the stack - `__top__`
Operations:
- application: simply apply the `__top__` - `push WRAP` - instantiate WRAP wrapper and push it on top of the queue - `pop` - set `__top__` to `__top__.__next__` - `tag NAME` - create new named queue `NQ` with `NAME` as its name and set `NQ.__top__` to `__top__`; then set `__top__` to `TORTILLA_QUEUE=NAME apply-queue`. Simply put this will allow splitting queue into two and pushing/popping them independently to preserve ordering.
#### Named Set
Named sets serve for "snapshotting" named queues.
Members:
- Named Queues - `queues` - Default Queue - `__main__`
Operations:
- application - apply `__main__` queue, unless another queue is requested. - `save-all NAME` - save named set under new name. - `restore-all NAME` - destroy any named queues and restore named set with given name. This must not destroy/change any named queues in any saved named set! - `save NAME QUEUE_NAME...` - save only selected named queues as new named set. - `restore NAME [QUEUE_NAME]...` - restore named queues saved in named set, but do not destroy anything. - `define-queue NAME` - create new empty named queue in the set - `set-ro` - set current named set read-only: do not allow changing any of the queues directly, must save under new name and change there.
Predefined named sets:
- `DEFAULT` - read only default for generic tasks - `default` - writable clone of `DEFAULT` - `RHTS-DEFAULT` - read only default chain for `rhts` tasks - `rhts-default` - writable clone of `RHTSr-DEFAULT` - this will likely be derived from `DEFAULT`
#### Tortilla
The main application.
Members:
- named sets
Operations:
- `define-set NAME` - create new empty named set. - application - apply selected named set/named queue.
We always need to work on a named set. Keep in mind to keep things reentrant - we may work in multiple sessions concurrently!
##### Put together:
I imagine this as a hierarchy under `/var/lib/tortilla`:
- `/var/lib/tortilla/wraps/` - directory containing all "Wraps". Wraps will use random names like this: `$(mktemp $WRAPPER.xxxxxxxx)` (or not...) - `/var/lib/tortilla/sets/$SET_NAME/` - directory containing all named sets. - `/var/lib/tortilla/sets/$SET_NAME/queues/$QUEUE_NAME/` - directory containing data for named queue. - symbolic links would be used for `__next__`, `__top__`. (:-/ What about windows???)
##### Examples
Let's define the defaults:
TORTILLA_SESSION=`tortilla-session` { tortilla-new
################################################################################ # default chain
################################################################################ tortilla-tag post __top__ tortilla-tag overrides __top__ tortilla-tag pre __top__ tortilla-tag defaults __top__ TORTILLA_ENV=/etc/profile.d/task-defaults.sh tortilla-push defaults env-wrap tortilla-push pre task-env-wrap TORTILLA_ENV=/etc/profile.d/task-overrides.sh tortilla-push overrides env-wrap tortilla-save-all --ro DEFAULT tortilla-save-all default
################################################################################ # rhts-chain
################################################################################ tortilla-push post groups-wrap TORTILLA_RUNCON=unconfined_u:unconfined_r:unconfined_t tortilla-push post runcon-wrap tortilla-save-all --ro RHTS-DEFAULT tortilla-save-all rhts-default }
# now we should have chains like this: # __top__ -> defaults -> pre -> overrides -> post tortilla-list default # default with wrappers: # __top__ -> defaults -> env-defaults -> pre -> task-env -> overrides -> env-overrides -> post tortilla-list --all default # rhts-default with wrappers: # __top__ -> defaults -> env-defaults -> pre -> task-env -> overrides -> env-overrides -> post -> runcon -> groups tortilla-list --all rhts-default
Example session to define new set based on existing one:
TORTILLA_SESSION=`tortilla-session` { tortilla-restore-all RHTS-DEFAULT tortilla-list TORTILLA_CGROUP=cpu:/ tortilla-push post cgroup-wrapper tortilla-save-all my-rhts }
Test wrapper:
# test whole chain: tortilla-apply my-rhts bash # test just single queue: tortilla-apply-queue my-rhts post bash
Test in a recipe - define param:
<param name="TORTILLA" value="my-rhts"/>
To redefine rhts-defaults:
TORTILLA_SESSION=`tortilla-session` { tortilla-restore-all rhts-default tortilla-save-all rhts-backup # modify the tortilla... tortilla-save-all rhts-default }
Now all rhts tests should use this tortilla.