RFC - Delayed Launch

[Brian McLaughlin]

I'm in the process of implementing the first user story, Delayed Launch, 
per the Redmine page below [1]. I've pulled out the relevant section of 
the page for quick reference below.  I'm also adding what views I 
believe will be edited along the way.  I figure soliciting comments and 
suggestions before getting too far is a good idea...

---

User Story:
1. As a user I want to launch a deployment at a specified time.

Delayed_job can be used for this (we already have it integrated). It 
could work as follows:
1. a user sets launch time on a deployment launch page
2. after checking permissions (same as for common launching), 
deployment+instances objects are created in conductor's DB (both 
deployment and instances stay in 'NEW' state)
3. we check if match is found at this point, if not, warning is 
displayed to user, but he is allowed to continue in delayed launch
4. launch request is queued in delayed_job queue with appropriate delay 
(delay = current time - launch time set by user)
5. delayed_job invokes launch request which: checks again permissions, 
finds a match and launches the deployment. If match is not found, 
deployment is not deleted but failed status is set both for deployment 
and instances.

Deployment model in conductor would be extended with attribute 
"launch_at", this attribute would be set for delayed launches and would 
be displayed in UI.

Deployments which are waiting for launch can be deleted, in such case 
delayed_job's job should be removed from queue -> another "job_id" 
attribute would be needed for keeping this id.

Deployments which are waiting for launch can have their launch time 
edited.  One of two scenario's would occur then:
1. Update the delay of the job in the queue of delayed_job
2. Duplicate the request currently queued in delayed_job, issue a new 
request to delayed_job and delete the original request.
Both of these scenarios would also require a "job_id" attribute be 
created to keep the delayed_job id.

Views
- src/app/views/deployments/_launch_new.html.haml
- src/app/views/deployments/launch_time_params.html.haml
- src/app/views/deployments/_deployment_card_index.html.mustache
- src/app/views/deployments/show.html.haml
- src/app/views/instances/_instance_card.html.mustache
- src/app/views/deployments/_edit.html.haml

[1] 
https://redmine.aeolusproject.org/redmine/projects/aeolus/wiki/Scheduleddela...

[Mike Orazi]

On 01/14/2013 10:55 PM, Brian McLaughlin wrote:
> I'm in the process of implementing the first user story, Delayed Launch,
> per the Redmine page below [1]. I've pulled out the relevant section of
> the page for quick reference below.  I'm also adding what views I
> believe will be edited along the way.  I figure soliciting comments and
> suggestions before getting too far is a good idea...
>
> User Story:
> As a user I want to launch a deployment at a specified time.
>
> Delayed_job can be used for this (we already have it integrated). It
> could work as follows:
> a user sets launch time on a deployment launch page
> after checking permissions (same as for common launching),
>
> deployment+instances objects are created in conductor's DB (both
> deployment and instances stay in 'NEW' state)
> 3. we check if match is found at this point, if not, warning is
> displayed to user, but he is allowed to continue in delayed launch
> 4. launch request is queued in delayed_job queue with appropriate delay
> (delay = current time - launch time set by user)
> 5. delayed_job invokes launch request which: checks again permissions,
> finds a match and launches the deployment. If match is not found,
> deployment is not deleted but failed status is set both for deployment
> and instances.
>
> Deployment model in conductor would be extended with attribute
> "launch_at", this attribute would be set for delayed launches and would
> be displayed in UI.
>
> Deployments which are waiting for launch can be deleted, in such case
> delayed_job's job should be removed from queue -> another "job_id"
> attribute would be needed for keeping this id.
>
> Deployments which are waiting for launch can have their launch time
> edited.  One of two scenario's would occur then:
> Update the delay of the job in the queue of delayed_job
> Duplicate the request currently queued in delayed_job, issue a new
>
> request to delayed_job and delete the original request.
> Both of these scenarios would also require a "job_id" attribute be
> created to keep the delayed_job id.
>
> Views
> src/app/views/deployments/_launch_new.html.haml
> src/app/views/deployments/launch_time_params.html.haml
> src/app/views/deployments/_deployment_card_index.html.mustache
> src/app/views/deployments/show.html.haml
> src/app/views/instances/_instance_card.html.mustache
> src/app/views/deployments/_edit.html.haml
>
> [1]
> https://redmine.aeolusproject.org/redmine/projects/aeolus/wiki/Scheduleddela...

Without looking at the content too much, this looks like it probably 
ought to wind up on the github wiki[1].  Once the story has been 
detailed enough, it likely should wind up as at least an issue in github 
issues[2], perhaps multiple depending on how you break up the work.

You could run the raw textile through pandoc[3] to get markdown (the 
markup that folks are using on the new wiki).  Alternately, Mo might 
have a suggestion for a better toolchain to get this moved over or you 
could just do it by hand.

Let me know if you need a hand and I'll see what I can do.

Thanks in advance,
Mike

[1] https://github.com/aeolusproject/conductor/wiki
[2] https://github.com/aeolusproject/conductor/issues?state=open
[3] http://johnmacfarlane.net/pandoc/

[Martin Povolny]

On Mon, Jan 14, 2013 at 10:55:44PM -0500, Brian McLaughlin wrote:
> I'm in the process of implementing the first user story, Delayed Launch, per
> the Redmine page below [1]. I've pulled out the relevant section of the page
> for quick reference below.  I'm also adding what views I believe will be
> edited along the way.  I figure soliciting comments and suggestions before
> getting too far is a good idea...
>
> User Story:
> As a user I want to launch a deployment at a specified time.
>
> Delayed_job can be used for this (we already have it integrated). It could
> work as follows:
> a user sets launch time on a deployment launch page
> after checking permissions (same as for common launching),
>
> deployment+instances objects are created in conductor's DB (both deployment
> and instances stay in 'NEW' state)
> 3. we check if match is found at this point, if not, warning is displayed to
> user, but he is allowed to continue in delayed launch
> 4. launch request is queued in delayed_job queue with appropriate delay
> (delay = current time - launch time set by user)
> 5. delayed_job invokes launch request which: checks again permissions, finds
> a match and launches the deployment. If match is not found, deployment is
> not deleted but failed status is set both for deployment and instances.
>
> Deployment model in conductor would be extended with attribute "launch_at",
> this attribute would be set for delayed launches and would be displayed in
> UI.
>
> Deployments which are waiting for launch can be deleted, in such case
> delayed_job's job should be removed from queue -> another "job_id" attribute
> would be needed for keeping this id.
>
> Deployments which are waiting for launch can have their launch time edited.
> One of two scenario's would occur then:
> Update the delay of the job in the queue of delayed_job
> Duplicate the request currently queued in delayed_job, issue a new
>
> request to delayed_job and delete the original request.
> Both of these scenarios would also require a "job_id" attribute be created
> to keep the delayed_job id.
>
> Views
> src/app/views/deployments/_launch_new.html.haml
> src/app/views/deployments/launch_time_params.html.haml
> src/app/views/deployments/_deployment_card_index.html.mustache
> src/app/views/deployments/show.html.haml
> src/app/views/instances/_instance_card.html.mustache
> src/app/views/deployments/_edit.html.haml
>
> [1] https://redmine.aeolusproject.org/redmine/projects/aeolus/wiki/Scheduleddela...

I think that the moment when we start implementing delayed launch is the
moment to think about the architecture of conductor with regards to
long running commands.

I see here an entity, lets call it a Command. This specific command is a
DeploymentCommand. We can also have InstanceCommand ImageCommand and
maybe more.

The command comes to live when user request some non-immediate action on
such entity as Deployment. Then it runs until it succeeds or fails
changing state of other entities and notifying whoever might need to
know what is happening.

In the above scenario we have a delayed launch command, so it's a
special kind of command that has a "execute_at" or "due" attribute set
on top of the other necessary DeploymentCommand attributes such as
Deployable, Realm, user (sure it operates in the context of some user's
permissions) and more.

So I would stress out that I would not hack the launch_at into the
Deployment because it's really an attribute of the act of launch and not
of the Deployment.

The above thoughts reach out to the API because the Commands come to
life there too (among other places).

e.g. if I do 
(REST) API /{instance,deployment}/<id>/launch

(or whatever where the api guys decided to put the launch), a Command
with an id should come to live, I would think.

Next I would need to check the status of my command, I can have
something like:

(REST) API /command/list (e.g.) all based on perms

/command/<id>
    /command/<id>?type=<type>

Of course once I have the command in one entity it's easier to access them
using other transport mechanisms and it's also more straight forward to
implement call-backs for them.

Getting from there we could use the opportunity of implementing the
commands to fix the Fat Model and Fat Controller issue, that (I hope) we
all agree conductor has (CodeClimate things so too) by moving the domain
logic out of models into let's say Service Objects and instead of adding
more responsibilities to current models (Deployment in this case) actually
splitting them into classes that are closer to the single-responsibility
principle.

We could start with separating the launch logic from the Deployment and
using it from the DeploymentCommand that does the delayed launch.

So I got far from the original topic but I hope you get that I mean.

We should try to thing more in context of the overall architecture of
the project and less in the context of the individual task (here I mean
e.g. the API design and the delayed launch).

Happy hacking!

[Jan Provazník]

On 01/16/2013 11:58 AM, Martin Povolny wrote:
> On Mon, Jan 14, 2013 at 10:55:44PM -0500, Brian McLaughlin wrote:
> > I'm in the process of implementing the first user story, Delayed Launch, per
> > the Redmine page below [1]. I've pulled out the relevant section of the page
> > for quick reference below.  I'm also adding what views I believe will be
> > edited along the way.  I figure soliciting comments and suggestions before
> > getting too far is a good idea...
> >
> > User Story:
> > As a user I want to launch a deployment at a specified time.
> >
> > Delayed_job can be used for this (we already have it integrated). It could
> > work as follows:
> > a user sets launch time on a deployment launch page
> > after checking permissions (same as for common launching),
> >
> > deployment+instances objects are created in conductor's DB (both deployment
> > and instances stay in 'NEW' state)
> > 3. we check if match is found at this point, if not, warning is displayed to
> > user, but he is allowed to continue in delayed launch
> > 4. launch request is queued in delayed_job queue with appropriate delay
> > (delay = current time - launch time set by user)
> > 5. delayed_job invokes launch request which: checks again permissions, finds
> > a match and launches the deployment. If match is not found, deployment is
> > not deleted but failed status is set both for deployment and instances.
> >
> > Deployment model in conductor would be extended with attribute "launch_at",
> > this attribute would be set for delayed launches and would be displayed in
> > UI.
> >
> > Deployments which are waiting for launch can be deleted, in such case
> > delayed_job's job should be removed from queue -> another "job_id" attribute
> > would be needed for keeping this id.
> >
> > Deployments which are waiting for launch can have their launch time edited.
> > One of two scenario's would occur then:
> > Update the delay of the job in the queue of delayed_job
> > Duplicate the request currently queued in delayed_job, issue a new
> >
> > request to delayed_job and delete the original request.
> > Both of these scenarios would also require a "job_id" attribute be created
> > to keep the delayed_job id.
> >
> > Views
> > src/app/views/deployments/_launch_new.html.haml
> > src/app/views/deployments/launch_time_params.html.haml
> > src/app/views/deployments/_deployment_card_index.html.mustache
> > src/app/views/deployments/show.html.haml
> > src/app/views/instances/_instance_card.html.mustache
> > src/app/views/deployments/_edit.html.haml
> >
> > [1] https://redmine.aeolusproject.org/redmine/projects/aeolus/wiki/Scheduleddela...
>
> I think that the moment when we start implementing delayed launch is the
> moment to think about the architecture of conductor with regards to
> long running commands.
>
> I see here an entity, lets call it a Command. This specific command is a
> DeploymentCommand. We can also have InstanceCommand ImageCommand and
> maybe more.
>
> The command comes to live when user request some non-immediate action on
> such entity as Deployment. Then it runs until it succeeds or fails
> changing state of other entities and notifying whoever might need to
> know what is happening.
>
> In the above scenario we have a delayed launch command, so it's a
> special kind of command that has a "execute_at" or "due" attribute set
> on top of the other necessary DeploymentCommand attributes such as
> Deployable, Realm, user (sure it operates in the context of some user's
> permissions) and more.
>
> So I would stress out that I would not hack the launch_at into the
> Deployment because it's really an attribute of the act of launch and not
> of the Deployment.
>
> The above thoughts reach out to the API because the Commands come to
> life there too (among other places).
>
> e.g. if I do
> (REST) API /{instance,deployment}/<id>/launch
>
> (or whatever where the api guys decided to put the launch), a Command
> with an id should come to live, I would think.
>
> Next I would need to check the status of my command, I can have
> something like:
>
> (REST) API /command/list (e.g.) all based on perms
>  /command/<id>
>  /command/<id>?type=<type>
>
> Of course once I have the command in one entity it's easier to access them
> using other transport mechanisms and it's also more straight forward to
> implement call-backs for them.
>
> Getting from there we could use the opportunity of implementing the
> commands to fix the Fat Model and Fat Controller issue, that (I hope) we
> all agree conductor has (CodeClimate things so too) by moving the domain
> logic out of models into let's say Service Objects and instead of adding
> more responsibilities to current models (Deployment in this case) actually
> splitting them into classes that are closer to the single-responsibility
> principle.
>
> We could start with separating the launch logic from the Deployment and
> using it from the DeploymentCommand that does the delayed launch.
>
> So I got far from the original topic but I hope you get that I mean.
>
> We should try to thing more in context of the overall architecture of
> the project and less in the context of the individual task (here I mean
> e.g. the API design and the delayed launch).
>
> Happy hacking!

Hi,
overall I like this approach.

Jan