[Fwd: Tosca widgets, only half the battle]

[John (J5) Palmieri]

On Wed, 2008-05-14 at 10:40 -0700, Toshio Kuratomi wrote:
> > 
> > After hacking away at MyFedora and producing a lot of ugly code in the
> > process I finally sat down the last two weeks to organize everything
> > into a framework make it much more extensible and have patterns for
> > people to easily create content.  Most of the technologies are
> > solidifying into my head and I have been working on hashing out an API
> > design behind the user interaction design I had started with.  The issue
> > I am running into now is the fact that Turbo Gears and related
> > technology come from a monolithic design and adhere too stringently to
> > the Model/View/Controller design pattern.  This is really an issue when
> > your models, views and controllers can come from different applications
> > or even different servers.  MyFedora is of course a mashup of different
> > tools and does not fit the, I'm grabbing data from a single database and
> > displaying it via a self contained template, mold.  What I need is a
> > complete plugin system where a person can write their own self contained
> > controllers, templates and static files which then drop in and are
> > loaded on the fly, while integrating with the global project.
> > 
> FAS2 is at a similar stage.  We want other people to be able to use 
> FAS2.  So we want to pull Fedora specific portions into a plugin 
> architecture.  We've started using setuptools entry-points since that's 
> the plugin architecture that TurboGears already uses but that only 
> solves how to load the modules, I think we got templates to work after a 
> bit of hacking but I'd have to revisit the code.  Getting the model to 
> work is the big sticking point as the model means either:
> 1) A new db and a new set of db connections
> 2) Somehow merging the plugin's model with the main apps model.
> 
> In myfedora, you might be able to use setuptools entry-points to better 
> effect since you're operating largely without a model.  If the plugins 
> don't have to save state or can save very minimal state into a generic 
> configs table you should be able to use entry-points to load up the code.

I'll have a look at this.  Most state would be cookies though I would
like the plugins to also work on the main page as a widget but the
config state there would be from the main page's controller which is a
traditional TG app.

> > Before I go further let me describe my design.
> > 
> > Vocabulary:
> > 
> > Resource - This is the starting point for MyFedora plugins.  A resource
> > is any abstract grouping such as "packages", "people" and "projects"
> > which contain tools for viewing and manipulating data within the
> > resource's context.
> > 
> > Tools - A tool is a web app for viewing or manipulating data.  For
> > example Builds would be a tool for the package resource.
> > 
> > Data Id - The data id is a pointer to a specific dataset the tools work
> > on.  For example the package resource considers each fedora package name
> > to be a data_id.
> > 
> > The way things work are Resources are placed in the resources/ directory
> > and contain the logic for routing requests to a specific tool.  They
> > also contain the master template which is a cause of path problems with
> > the current TG setup (include paths are relative to the including
> > template)
> > 
> This is how we fixed that in fas::
> 
>    <?python
>    import genshi.template.plugin
>    master = 
> genshi.template.plugin.MarkupTemplateEnginePlugin().load_template('fas.templates.master')
>    ?>
>    <xi:include href="${master.filename}" />

I think I'll do this in the root controller and inject it in as a
standard myfedora helper object where myfedora.templates.get_parent()
would do the trick. I'm trying to hide as much as the boilerplate as
possible.

> > Tools are placed in the tools/ directory and are controllers just like
> > any other TG controller.  The exception is there is a standard for
> > including the master template and the tool pulls templates and static
> > files from its own directory.  Tools can register with more than one
> > resource and must modify its behavior based on the resource calling it.
> > For instance the Build tool would be able to register with the package
> > and people resource and depending which resource is being used it would
> > display either a specific person's builds or the build history of a
> > package.  Based on the resource being used the master template is pulled
> > in by the tool's templates.
> >
> That's pretty nifty.
> 
> > Data id's are simply what the resource passes to the tool and the tool
> > needs to be able to accept when dealing with a particular resource.  For
> > instance the Packages resource would send a package name as a data id
> > and the Peoples resource would send a person's FAS username.
> > 
> An aside: It sounds like we'd want to implement some sort of global 
> cache at some point.  If we always start from the data_ids there's a lot 
> of data to be pulled in each request.  And if tools are using similar 
> pieces of information but displaying it differently (for instance, when 
> using a tool in a different resource) we'd definitely see reuse of the data.

Oh definitely but we need to be careful, as much of what the user cares
about is quick changing live data.  When we get into doing the comet
framework perhaps we can figure out a way to mark live data with data
that is unlikely to change, cache the static data locally and optimize
the live data retrieval on the server we are getting it from.

<snip />
> 
> > What do you guys think? Given my design and goals such as the ability to
> > display tools on the portal page, what is our plan of attack?  How do we
> > concoct a plugin system to make it easy for others to create integrated
> > content while really just concentrating on their bits and not the wider
> > integration infrastructure?  Are there systems/libraries out there that
> > already do this? Tosca is only part of the solution because it only
> > deals with encapsulating display and is mostly geared to generic widgets
> > like lists and not complete pages.  I would like to have a framework
> > that is simple, focused and easy to use.
> > 
> 
> I suppose part of the question is who do we want to be building these 
> plugins for MyFedora?  If it's the author of the app providing the data 
> then it should be something that their app can also use.  If it's 
> MyFedora developers, then it should be easy for them to create a new 
> controller and setup a plugin.  We might want to write a small script 
> that makes a sample plugin directory like tg-admin quickstart does for 
> the main app.

Say Dimitri with Transflex.  He can test it on his own machine and then
ask us to include the plugin on our live site.  Random widget
development for display on the front page/desktop/other portal is
another story and I suggest we look at OpenSocial for those pieces.

> Take a look at how fas does things:  fas/plugins/dummy_plugin.
> That should pretty much be a self-contained plugin using setuptools 
> entrypoints to do its dirty work.

Will do.

--
John