We Should Refactor "prep custom"
by Tomas Radej
Hi, all.
»TL;DR available below«
As DevAssistant gains popularity, it is increasingly used by it's target
group - developers. Developers like things that just work, and hate
reading the manual. As it happens, with DevAssistant, you often need to
read the manual to understand what's going on, and I'd like to change that.
Today, I'd like to talk about the "prep custom" assistant. It is one of
the most versatile assistants out there, mainly because it can clone out
any repo you want, and even automatically fork it on GitHub if you want.
However, it is also one of the least discoverable assistants because
people don't know what "custom" means.
There are two key issues with the assistant that I have identified:
1) The assistant's name is not intuitive
2) It offers a lot of GitHub functionality, but isn't named after GitHub
Let's start with point 2: I have got multiple complaints from people who
tried to check out a GitHub repo by running 'da twk github' or the like,
because what they wanted to do is check out a GitHub repo. When talking
about DevAssistant at conferences and online, we stress that
DevAssistant is pretty damn good with GitHub, so it makes sense to have
a GitHub assistant, right? But we don't have one. I suggest we do, and
name it "prep github". It would use the USERNAME/REPONAME positional
argument, and also offer the --gh-fork functionality.
The mechanics of the github assistant should be the identical to current
"prep custom", i. e. check out the repo and try running the
.devassistant file with the user's permission. The "prep custom"
assistant would then be named "prep repo", and would only take the URL
as an argument (and path where to put the directory, of course). It
would initially support only git, but could support e. g. mercurial or
even svn in the future (the correct binary can be guessed from the URL
or by trial and error).
TL;DR: the "prep custom" assistant should be removed and replaced by
assistants that look like this:
$ da prep github USERNAME/REPONAME [-s,--ssh] [-f,--gh-fork [USERNAME]]
[-p,--path PATH]
$ da prep repo URL [-p,--path PATH]
Both assistants check out the git repo, look for /.devassistant and
execute it if the user wishes so. The "prep repo" assistant supports git
for now, but is extensible for hg or svn too.
So, what do you think?
--
Tomas Radej
8 years, 10 months
Distributing Assistants with Upstream Code
by Tomas Radej
Hello everyone.
Since DevAssistant is about to undergo some major changes (see [1]
for details), I think it would be great to factor in some additional
changes that could make working with DevAssistant even easier than
it's now. For this reason, I'd like to propose another change to the
DevAssistant workflow: storing assistants directly in upstream
repositories with the code they're supposed to work with.
Both myself and Slavek have encouraged developers to create their own
assistants and distribute them in the form of DAP archives. This
approach has caught on quite well for general purpose assistants, like
those for Gnome apps, Vala, or Android SDK [2]. However, we learnt
that developers don't like this format for distributing assistants
used with their particular projects. What I suggest is hearing out the
#1 suggestion of the users themselves: make it possible to include
assistants in the repos they're meant for.
There are a few issues with this approach (talking about those that I
identified. I'm sure you'll find more). The main one is that
DevAssistant is currently stateless, meaning it doesn't remember
anything between invocations (except for some configs stored in
~/.devassistant/config). To work with these assistants, which aren't
installed in a path recognized by DA, I believe it's necessary to
introduce a new functionality; let's call it "da open". When "opening"
a directory, DevAssistant would load assistants stored in that
directory (more about that below). These assistants would be unloaded
by running "da close". Remembering which directory is "open" would be
straightforward in the client/server architecture described in the
mail linked at [1] - the server would run continuously, remembering
which folders are open for the particular session.
In addition to storing assistants with upstream code, users of
DevAssistant would benefit from storing metadata about their project
in a standardized format, in a file that would also point to the
directory with the assistants. The reason for explicitly selecting the
directory is that if you develop another set of assistants (probably
a new DAP), loading everything in the directory would pick up even the
unfinished assistants you're working on. I believe it's better to let
the user load them only if they really mean to.
To avoid reinventing the wheel for the millionth time, DevAssistant
should use a standard metadata file that's also used by some other
platform. I have looked around, and I believe I found a suitable
solution in the form of the Nulecule specification [3], a
predominantly container-based spec that is used with Project Atomic.
This spec is not strict in its evaluation, leaving DevAssistant with
almost unlimited manœuvering space as far as the metadata format goes,
and be used for many generally useful tags, like author, homepage etc.
A sample file in the Nulecule format would look, for example, like
this:
---
specversion: 0.0.2
id: mydjango-app
metadata:
name: My Django App
appversion: 0.0.1dev
license:
name: GNU GPL v2 or later
url: https://www.gnu.org/licenses/gpl-2.0.html
authors: [ 'Tomas Radej <tradej(a)redhat.com>' ]
homepage: https://example.com/mydjangoapp
devassistant:
assistants: /assistants
minversion: 1.0a
tags: [ atomic, python, django ]
graph:
- name: mydjango-app
---
The "graph" field is there because the Nulecule spec mandates it. For
non-Atomic apps, it isn't important.
In the "metadata > devassistant" field, the location of assistants
would be stored, along with a minimal version of DevAssistant that is
supported by the project, and tags indicating what kind of assistants
can work with this project. I expect these tags to be a full
replacement for the "Project Type" field currently present in
.devassistant files, but not used anywhere. Assistants would then
feature a field called "tag" or "project type" that would be matched
against the tags in the metadata file upon running "da open", and then
presented more prominently than unrelated assistants in the GUI and/or
bash completion in the CLI. More fields can be introduced as necessary.
What do you think? Is there an issue I missed, is this good, or is it
absolute rubbish? Don't hesitate to criticise.
Thanks, Tomas
[1] DevAssistant Mailing List:
https://lists.fedoraproject.org/pipermail/devassistant/2015-April/000065....
[2] DevAssistant Package Index: https://dapi.devassistant.org/
[3] Nulecule GitHub repo: https://github.com/projectatomic/nulecule
8 years, 10 months
Going client/server
by Tomas Radej
Greetings, friends of all things DevAssistant.
For eleven minor versions, DevAssistant has been the tool to set up
your environment on your host system, on bare metal. That works very
well, but an increasing number of developers can't use it for their
use case. Why, you ask? Containers. While DevAssistant has very basic
support for containers in the 'Docker' assistant, it still does all
the magic on the host. What we need is a full-fledged containerised
app support, and DevAssistant will have to change quite fundamentally
to offer it.
Right now, DevAssistant is pretty much a monolithic application that
runs on the host machine, requires GTK+ for its GUI and makes some
assumptions about what the environment looks like. That makes it quite
hard to make it work reliably under Mac OS X or Windows, heck even
different Linux distros. To overcome this problem, I suggest
splitting DevAssistant based on the client/server architecture.
The basic idea is that the server runs in a privileged container or on
the host system, and very thin client connects to it and exchange
information about assistants and commands through an API (for example
using JSON). You may think about it as "DevAssistant as a service".
When developing this architecture, I have heavily borrowed from
Cockpit, the web-based sysadmin tool developed for Fedora, which uses
a webserver to accept commands, and then, through a bridge app, issues
calls to various system services. DevAssistant then should be
reorganised into these components:
The client
----------
While DevAssistant uses already a sort of a client-server
architecture, the clients are still quite thick, and not easily
replaceable. Our intention is to put only the bare minimum into the
client - an engine to construct a user interface from the received
data. That means that the command line interface will construct a help
message with assistants and commands based on what is available on the
server side. Similarly, the WebUI will construct a form based on
fields that the server sends to it, generated from the arguments
specified in an assistant. The client always runs under the user that
wishes to run an assistant.
I am acutely aware of the fact that by using the web interface for
running commands under the root user, I am effectively delegating the
security to the browser, but I doubt we currently have the resources
to have a new full-fledged GUI based on some desktop graphical
toolkit.
The webserver
-------------
The webserver is the front-facing part of the entire server, which
interacts with the client. What it does is spin up a "bridge" instance
(see below) for the user under whom the "client" is running, and pass
messages between the two. The webserver itself doesn't touch the
system or containers in any way, and will probably run under a unique
user, for example 'devassistant'. I expect it will take the list of
available users from /etc/passwd. I still don't know what webserver to
use, but security is definitely what we should look at first.
The bridge
----------
This is the core part of the new system, which will contain most of
today's DevAssistant package. This part runs under the user who
actually wants to execute the assistants, and is spawned by the
webserver. It loads the available assistants, runs them with
parameters supplied through the webserver, and if necessary, delegates
the execution of superuser tasks to the "service" with elevated
privileges.
The service
-----------
Since the bridge runs under a user that doesn't necessarily have all
the permissions an assistant may require, there will be a service
(DBus?) that accepts commands and executes them as the super user.
When such a command is invoked, the service will ask an authentication
agent if the user is authorised to perform such a command, and based
on the agent's reply will carry it out appropriately. If an additional
authentication is required, the bridge sends a request to the client
for proper authentication.
The agent
---------
The last part of the system is an authentication agent. This agent,
based on its policies, decides if a given user is permitted to carry
out a command passed to the "service", and return a True/False reply.
This will probably be a PolicyKit service.
This is the general plan of remodelling DevAssistant to better suit
containerised development. I would like to know your ideas about or
criticisms of this outline, as it may shape the development for months
or years to come.
Cheers, Tomas Radej
8 years, 10 months