deltacloud-devel August 2009

deltacloud-devel@lists.fedorahosted.org

14 participants
50 discussions

[Libcloud-list] libcloud design plans going forward
by Hugh O. Brock 01 Sep '09

01 Sep '09

Now that we have a demo UI put together Scott and I wanted to start serious discussion around what to implement next for the dloud portal. Here's a list of things to discuss, please comment in-line. deltacloud-portal-design-points =============================== Date: 2009-08-25 16:47:26 EDT 1 What's missing in the models ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + Storage (and the whole boatload of issues around image management) + Custom attributes + Actions 2 Polling and updating ~~~~~~~~~~~~~~~~~~~~~~ + Lots to do here figuring out how we're going to poll the different driver/providers for updates. + Also we need to work out a parallel monitoring API -- do we need separate monitoring drivers? 3 Permissions model ~~~~~~~~~~~~~~~~~~~ + We know portal users will need and be granted permissions on pools + Do we need single-user ownership of pools as well? + Do we need to track images and realms on the cloud providers and relate them back to our users, or is the user->pool->account mapping sufficient for this? 4 Validation infrastructure, going down to the drivers ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + Names (validity, length, uniqueness) 5 Explicit mods to "Flavor" ~~~~~~~~~~~~~~~~~~~~~~~~~~~ + RAM + CPU + Storage 6 oVirt driver ~~~~~~~~~~~~~~ + dcloud pool would map to a single VM pool in oVirt 7 Quotas ~~~~~~~~ + We need to be aware of cloud-side quotas + We need to manage portal-side quotas on pools 8 Proxy vs. Portal ~~~~~~~~~~~~~~~~~~ + Is the proxy API the same as the deltacloud API, or is there more going on there 9 What additional non-REST APIs if any do we support ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + QMF? + Web services (SOAP)? 10 Monitoring/stats display ~~~~~~~~~~~~~~~~~~~~~~~~~~~ How do we fit the widget in?

7 11

[Libcloud-list] [PATCH server] First pass at some getting starting directions.
by jason.guiditta 31 Aug '09

31 Aug '09

Signed-off-by: Jason Guiditta <jason.guiditta(a)gmail.com> --- README | 44 ++++++++++++++++++++++++++++++++++---------- 1 files changed, 34 insertions(+), 10 deletions(-) diff --git a/README b/README index 95e9330..46b3f2b 100644 --- a/README +++ b/README @@ -1,10 +1,34 @@ -The oVirt Server is an open cross-platform virtualization management -system. It provides a web-based management interface that enables -users to manage hosts and storage, install and remove virtual machines -and level resources across a large group of machines. The oVirt Server -manages hosts running the oVirt Node Image. oVirt Server Suite scales -from a small group of users with little need for access control and -quota management, all the way up to hundreds of hosts with robust -control over grouping, permissions, and quotas. - -For further docs see: http://ovirt.org +Deltacloud Portal provides a web UI in front of the Deltacloud API. With Deltacloud Portal, your users can: + + * View image status and stats across clouds, all in one place + * Migrate instances from one cloud to another + * Manage images locally and provision them on any cloud + +=== Setup === +In addition to this Delatcloud Portal application, you will need to check out some other projects from git. You can 'git clone' the following (this process will be simplified soon): + +There are a few prerequisites to getting your deltacloud portal application running. + +The first is to have the correct rubygems installed. Assuming you have ruby and rails installed, you can get a basic app running pretty simply. If you try to start the application, you should get a warning from rails telling you which gems are missing. At this point, you can either install the missing gems with yum (most should be available in rpms format for fedora users), or gem install <gemname> (you may wish to do that as root, depending on your system configuration). + +The second is to have the database properly setup and configured (directions for this step can be found in config/database.yml). + +The Deltacloud Portal depends on deltacloud-client-ruby. This has a gemspec available, but currently the easiest way to include it in the app is to get check it out from its temporary home on github into vendor/plugins in the portal app like this: +[user@plugins]$ git clone git://github.com/bobmcwhirter/deltacloud-client-ruby.git + +This will allows the portal to load it up as a plugin with no additional configuration needed. + +Lastly, you will need one or more instances of the Deltacloud framework running for the portal to talk to. Directions for this can be found in the Deltacloud-framework documentation. (which can be found here: git://github.com/bobmcwhirter/deltacloud-framework.git) + +=== Getting Started === +For these steps, we will assume the mock driver is setup and running in a framework on http://localhost:3000/api, and your portal is running on http://localhost:3001 + +Taskomatic is a separate process that checks for tasks needing to be handled for portal instances. This can be run from the commandline from the taskomatic directory with 'ruby taskomatic -n'. If you are running in development mode, you can point taskomatic there by exporting 'RAILS_ENV=development'. Taskomatic needs to be running to do things like 'start an instance'. + +When you navigate to the main page of your app, you will be presented with an 'Add cloud provider' form. 'Name' can be whatever you like, types currently suppoerted are mock, ec2, and rhevm. 'URL' is the url pointing to your framework (http://localhost:3000/api for our example). + +Upon successful completion of the form, you will be brought to the main page for that provider. Here you would click 'Add a pool'. This form asks you fro the username and password for the account on the provider's service that you wish to connect to. Then just provide a name for what we will call this 'pool'. Completion of this form will attempt to connect to the provider associated with this pool and gather some information such as a list of instances that this user has access to and can be started. + +Completion of this form brings you to the main page for the newly created pool. Click 'Add a new instance' here. For some providers (especially ec2), it can sometimes take a little while to get all the data back, so if you go to the new instance for and see no images, you may need to wait a short time and then try again. + +At this point, you should have a list of available instances for your provider/pool combination, with available actions for each instance in your list. Note that in the current version, this page will need to be refreshed in order to see and changes that may occur in the 'State' column, which tells you the status of your instance. -- 1.6.2.5

1 0

[Libcloud-list] [PATCH server] Fix error handling on new instance form.
by jason.guiditta 31 Aug '09

31 Aug '09

Signed-off-by: Jason Guiditta <jason.guiditta(a)gmail.com> --- src/app/controllers/instance_controller.rb | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) diff --git a/src/app/controllers/instance_controller.rb b/src/app/controllers/instance_controller.rb index bba8857..398fdde 100644 --- a/src/app/controllers/instance_controller.rb +++ b/src/app/controllers/instance_controller.rb @@ -13,7 +13,8 @@ class InstanceController < ApplicationController end def new - _new(params[:provider],params[:id]) + @instance = Instance.new({:portal_pool_id => params[:id]}) + _new(params[:provider]) end def create @@ -29,11 +30,11 @@ class InstanceController < ApplicationController flash[:notice] = "Instance added." redirect_to :controller => "portal_pool", :action => 'show', :id => @instance.portal_pool_id else - _new(params[:provider_id],params[:instance][:portal_pool_id]) + _new(params[:provider_id]) render :action => 'new' end else - _new(params[:provider_id],params[:instance][:portal_pool_id]) + _new(params[:provider_id]) render :action => 'new' end end @@ -60,11 +61,10 @@ class InstanceController < ApplicationController private - def _new(provider, pool) + def _new(provider) @flavors = Flavor.find(:all, :conditions => {:provider_id => provider}) @images = Image.find(:all, :conditions => {:provider_id => provider}) @realms = Realm.find(:all, :conditions => {:provider_id => provider}) - @instance = Instance.new({:portal_pool_id => pool}) @provider_id = provider end -- 1.6.2.5

1 0

[Libcloud-list] [PATCH server] Log states after each command and after waiting.
by Ian Main 24 Aug '09

24 Aug '09

--- src/task-omatic/taskomatic_instance.rb | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-) diff --git a/src/task-omatic/taskomatic_instance.rb b/src/task-omatic/taskomatic_instance.rb index 2ef22fa..99161ac 100644 --- a/src/task-omatic/taskomatic_instance.rb +++ b/src/task-omatic/taskomatic_instance.rb @@ -97,10 +97,12 @@ class TaskomaticInstanceStart < TaskomaticTask @task.instance.state = dcloud_to_instance_state(dcloud_instance.state) @task.instance.save! + @logger.info("Command completed, instance now in state #{dcloud_instance.state}") while dcloud_instance.state.upcase == 'PENDING' sleep(3) dcloud_instance = client.instance((a)task.instance.external_key) end + @logger.info("After wait, instance now in state #{dcloud_instance.state}") @task.instance.state = dcloud_to_instance_state(dcloud_instance.state) @task.instance.save! @@ -124,6 +126,7 @@ class TaskomaticInstanceStop < TaskomaticTask dcloud_instance = client.instance((a)task.instance.external_key) dcloud_instance.stop! + @logger.info("Command completed, instance now in state #{dcloud_instance.state}") @task.instance.state = dcloud_to_instance_state(dcloud_instance.state) @task.instance.save! @@ -132,6 +135,7 @@ class TaskomaticInstanceStop < TaskomaticTask dcloud_instance = client.instance((a)task.instance.external_key) end + @logger.info("After wait, instance now in state #{dcloud_instance.state}") @task.instance.state = dcloud_to_instance_state(dcloud_instance.state) @task.instance.save! @@ -154,6 +158,7 @@ class TaskomaticInstanceReboot < TaskomaticTask dcloud_instance = client.instance((a)task.instance.external_key) dcloud_instance.reboot! + @logger.info("Command completed, instance now in state #{dcloud_instance.state}") @task.instance.state = dcloud_to_instance_state(dcloud_instance.state) @task.instance.save! @@ -162,6 +167,7 @@ class TaskomaticInstanceReboot < TaskomaticTask dcloud_instance = client.instance((a)task.instance.external_key) end + @logger.info("After wait, instance now in state #{dcloud_instance.state}") @task.instance.state = dcloud_to_instance_state(dcloud_instance.state) @task.instance.save! @@ -184,6 +190,7 @@ class TaskomaticInstanceDestroy < TaskomaticTask dcloud_instance = client.instance((a)task.instance.external_key) dcloud_instance.destroy! + @logger.info("Command completed, instance now in state #{dcloud_instance.state}") @task.instance.state = dcloud_to_instance_state(dcloud_instance.state) @task.instance.save! @@ -192,6 +199,7 @@ class TaskomaticInstanceDestroy < TaskomaticTask dcloud_instance = client.instance((a)task.instance.external_key) end + @logger.info("After wait, instance now in state #{dcloud_instance.state}") @task.instance.state = dcloud_to_instance_state(dcloud_instance.state) @task.instance.save! -- 1.6.2.5

1 0

[Libcloud-list] [PATCH server] Small style tweaks for provider pools page
by Jeremy Perry 24 Aug '09

24 Aug '09

Signed-off-by: Jeremy Perry <jeremy.perry(a)redhat.com> --- src/app/views/provider/show.html.erb | 2 +- src/public/stylesheets/dcloud.css | 27 ++++++++++++++++++++++++--- 2 files changed, 25 insertions(+), 4 deletions(-) diff --git a/src/app/views/provider/show.html.erb b/src/app/views/provider/show.html.erb index bfeacdb..ae5c092 100644 --- a/src/app/views/provider/show.html.erb +++ b/src/app/views/provider/show.html.erb @@ -18,7 +18,7 @@ </tbody> </table> <% end %> -<%= link_to "Add a pool", :controller => "portal_pool", :action => "new", :provider => @provider%> +<%= link_to "Add a pool", {:controller => "portal_pool", :action => "new", :provider => @provider}, :class => "actionlink" %> <% form_tag :action => 'destroy' do %> <%=hidden_field :provider, :id %> <%= submit_tag "Delete Provider", :class => "submit_link" %> diff --git a/src/public/stylesheets/dcloud.css b/src/public/stylesheets/dcloud.css index b0e8115..e2f47aa 100644 --- a/src/public/stylesheets/dcloud.css +++ b/src/public/stylesheets/dcloud.css @@ -128,16 +128,18 @@ input, select { /* Styles for table */ #content_area table { width:100%; + border-collapse: collapse; } -#content_area table th, #content table td { +#content_area table th, #content_area table td { border:1px solid #e1e2e3; border-right: 0; border-left: 0; padding:0; text-align:left; font-size: 14px; - line-height: 18px; + line-height: 24px; + padding-left: 18px; } #content_area table th { @@ -154,6 +156,26 @@ input, select { text-decoration: underline; } +.actionlink { + color: blue; + line-height: 36px; + font-size: 14px; + padding-left: 18px; +} + +.submit_link { + background: none; + border:0; + text-decoration: underline; + color: blue; + line-height: 36px; + font-size: 14px; + padding-left: 10px; +} + +.submit_link:hover {cursor:pointer;} + + /* Styles for the left navigator */ #side { @@ -162,7 +184,6 @@ input, select { } #side a.actionlink { - text-decoration: underline; color: blue; line-height: 36px; font-size: 14px; -- 1.6.2.5

1 0

[Libcloud-list] [PATCH server] Use .upcase on state so that they are consistent accross providers.
by Ian Main 24 Aug '09

24 Aug '09

--- src/task-omatic/taskomatic_instance.rb | 22 +++++++++++----------- 1 files changed, 11 insertions(+), 11 deletions(-) diff --git a/src/task-omatic/taskomatic_instance.rb b/src/task-omatic/taskomatic_instance.rb index 324f3fc..2ef22fa 100644 --- a/src/task-omatic/taskomatic_instance.rb +++ b/src/task-omatic/taskomatic_instance.rb @@ -19,7 +19,7 @@ require 'taskomatic_task' def dcloud_to_instance_state(state_str) - case state_str + case state_str.upcase when 'PENDING' return Instance::STATE_PENDING when 'RUNNING' @@ -61,8 +61,8 @@ class TaskomaticInstanceCreate < TaskomaticTask @task.instance.state = dcloud_to_instance_state(dcloud_instance.state) @task.instance.save! - while dcloud_instance.state == 'PENDING' - sleep(5) + while dcloud_instance.state.upcase == 'PENDING' + sleep(3) dcloud_instance = client.instance((a)task.instance.external_key) end @@ -97,8 +97,8 @@ class TaskomaticInstanceStart < TaskomaticTask @task.instance.state = dcloud_to_instance_state(dcloud_instance.state) @task.instance.save! - while dcloud_instance.state == 'PENDING' - sleep(5) + while dcloud_instance.state.upcase == 'PENDING' + sleep(3) dcloud_instance = client.instance((a)task.instance.external_key) end @@ -127,8 +127,8 @@ class TaskomaticInstanceStop < TaskomaticTask @task.instance.state = dcloud_to_instance_state(dcloud_instance.state) @task.instance.save! - while dcloud_instance.state == 'PENDING' or dcloud_instance.state == "SHUTTING_DOWN" - sleep(5) + while dcloud_instance.state.upcase == 'PENDING' or dcloud_instance.state.upcase == "SHUTTING_DOWN" + sleep(3) dcloud_instance = client.instance((a)task.instance.external_key) end @@ -157,8 +157,8 @@ class TaskomaticInstanceReboot < TaskomaticTask @task.instance.state = dcloud_to_instance_state(dcloud_instance.state) @task.instance.save! - while dcloud_instance.state == 'PENDING' or dcloud_instance.state == "SHUTTING_DOWN" - sleep(5) + while dcloud_instance.state.upcase == 'PENDING' or dcloud_instance.state.upcase == "SHUTTING_DOWN" + sleep(3) dcloud_instance = client.instance((a)task.instance.external_key) end @@ -187,8 +187,8 @@ class TaskomaticInstanceDestroy < TaskomaticTask @task.instance.state = dcloud_to_instance_state(dcloud_instance.state) @task.instance.save! - while dcloud_instance.state == 'PENDING' or dcloud_instance.state == "SHUTTING_DOWN" - sleep(5) + while dcloud_instance.state.upcase == 'PENDING' or dcloud_instance.state.upcase == "SHUTTING_DOWN" + sleep(3) dcloud_instance = client.instance((a)task.instance.external_key) end -- 1.6.2.5

1 0

[Libcloud-list] [PATCH server] First attempt at making actions more readable.
by jason.guiditta 24 Aug '09

24 Aug '09

Make them a pipe-delimited list for now, we can always change the css later, or add js effects. Signed-off-by: Jason Guiditta <jason.guiditta(a)gmail.com> --- src/app/views/layouts/dcloud.rhtml | 1 + src/app/views/portal_pool/show.html.erb | 5 ++++- src/public/stylesheets/dcloud.css | 14 ++++++++++++++ 3 files changed, 19 insertions(+), 1 deletions(-) diff --git a/src/app/views/layouts/dcloud.rhtml b/src/app/views/layouts/dcloud.rhtml index 74ee09c..725f630 100644 --- a/src/app/views/layouts/dcloud.rhtml +++ b/src/app/views/layouts/dcloud.rhtml @@ -56,6 +56,7 @@ error: function(xhr) {$.jGrowl(xhr.status + ' ' + xhr.statusText);} }); return false;})},function(){}); + $('ul.instance_action_list li:first-child').addClass('first'); }); </script> <%= yield :scripts -%> diff --git a/src/app/views/portal_pool/show.html.erb b/src/app/views/portal_pool/show.html.erb index 88d8e3f..1eb9ed7 100644 --- a/src/app/views/portal_pool/show.html.erb +++ b/src/app/views/portal_pool/show.html.erb @@ -15,13 +15,16 @@ <%(a)instances.each {|instance| %> <tr> <td> + <ul class="instance_action_list"> <%instance.get_action_list.each {|action|%> + <li> <%= link_to action, :controller => "instance", :action => "instance_action", :id => instance, :instance_action => action %> - <br/> + </li> <% } %> + </ul> </td> <td><%= instance.name %></td> <td><%= instance.state %></td> diff --git a/src/public/stylesheets/dcloud.css b/src/public/stylesheets/dcloud.css index 143feec..b0e8115 100644 --- a/src/public/stylesheets/dcloud.css +++ b/src/public/stylesheets/dcloud.css @@ -229,3 +229,17 @@ ul#providers li ul li a:hover { background: #f2a922 url(../images/icon_vmpool.png) no-repeat 2px 50%; color: #FFF; } + +.instance_action_list { + padding: 0; +} + +.instance_action_list li { + padding: 0 0.3em; + border-left: 1px solid #000; + display: inline; +} + +.instance_action_list li.first { + border-left: none; +} -- 1.6.2.5

1 0

[Libcloud-list] [PATCH server] Add STATE_CREATE_FAILED to instance model and use it in taskomatic.
by Ian Main 24 Aug '09

24 Aug '09

--- src/app/models/instance.rb | 3 ++- src/task-omatic/taskomatic_instance.rb | 9 +++++++-- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/src/app/models/instance.rb b/src/app/models/instance.rb index 4ce0459..582b1b8 100644 --- a/src/app/models/instance.rb +++ b/src/app/models/instance.rb @@ -24,10 +24,11 @@ class Instance < ActiveRecord::Base STATE_RUNNING = "running" STATE_SHUTTING_DOWN = "shutting_down" STATE_STOPPED = "stopped" + STATE_CREATE_FAILED = "create_failed" validates_inclusion_of :state, :in => [STATE_NEW, STATE_PENDING, STATE_RUNNING, - STATE_SHUTTING_DOWN, STATE_STOPPED] + STATE_SHUTTING_DOWN, STATE_STOPPED, STATE_CREATE_FAILED] def get_action_list(user=nil) # return empty list rather than nil diff --git a/src/task-omatic/taskomatic_instance.rb b/src/task-omatic/taskomatic_instance.rb index 216bb53..324f3fc 100644 --- a/src/task-omatic/taskomatic_instance.rb +++ b/src/task-omatic/taskomatic_instance.rb @@ -46,11 +46,16 @@ class TaskomaticInstanceCreate < TaskomaticTask client = @task.instance.portal_pool.cloud_account.connect puts "client is #{client.type}" - @logger.info "Creating instance with name #{(a)task.instance.image.external_key}, flavor #{(a)task.instance.flavor.name}, realm #{(a)task.instance.realm.name}" + @logger.info "Creating instance with name #{(a)task.instance.image.external_key}, flavor #{(a)task.instance.flavor.external_key}, realm #{(a)task.instance.realm.external_key}, name #{(a)task.instance.name}" dcloud_instance = client.create_instance((a)task.instance.image.external_key, :flavor => @task.instance.flavor.external_key, :realm => @task.instance.realm.external_key, :name => @task.instance.name) + if dcloud_instance.class == Net::HTTPInternalServerError + @task.instance.state = Instance::STATE_CREATE_FAILED + raise "Error creating dcloud instance, returned internal server error." + end + @logger.info "Instance created with key #{dcloud_instance.id} and state #{dcloud_instance.state}" @task.instance.external_key = dcloud_instance.id @task.instance.state = dcloud_to_instance_state(dcloud_instance.state) @@ -130,7 +135,7 @@ class TaskomaticInstanceStop < TaskomaticTask @task.instance.state = dcloud_to_instance_state(dcloud_instance.state) @task.instance.save! - @logger.info("Instance started.") + @logger.info("Instance stopped.") end end -- 1.6.2.5

1 0

[Libcloud-list] [PATCH server] Add :name to create_instance.
by Ian Main 24 Aug '09

24 Aug '09

--- src/task-omatic/taskomatic_instance.rb | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/src/task-omatic/taskomatic_instance.rb b/src/task-omatic/taskomatic_instance.rb index bb28d66..216bb53 100644 --- a/src/task-omatic/taskomatic_instance.rb +++ b/src/task-omatic/taskomatic_instance.rb @@ -49,7 +49,8 @@ class TaskomaticInstanceCreate < TaskomaticTask @logger.info "Creating instance with name #{(a)task.instance.image.external_key}, flavor #{(a)task.instance.flavor.name}, realm #{(a)task.instance.realm.name}" dcloud_instance = client.create_instance((a)task.instance.image.external_key, :flavor => @task.instance.flavor.external_key, - :realm => @task.instance.realm.external_key) + :realm => @task.instance.realm.external_key, + :name => @task.instance.name) @logger.info "Instance created with key #{dcloud_instance.id} and state #{dcloud_instance.state}" @task.instance.external_key = dcloud_instance.id @task.instance.state = dcloud_to_instance_state(dcloud_instance.state) -- 1.6.2.5

1 0

[Libcloud-list] [PATCH server] Use external_key values for image create.
by Ian Main 24 Aug '09

24 Aug '09

Signed-off-by: Ian Main <imain(a)redhat.com> --- src/task-omatic/taskomatic_instance.rb | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/task-omatic/taskomatic_instance.rb b/src/task-omatic/taskomatic_instance.rb index 05186e3..5159a79 100644 --- a/src/task-omatic/taskomatic_instance.rb +++ b/src/task-omatic/taskomatic_instance.rb @@ -46,8 +46,8 @@ class TaskomaticInstanceCreate < TaskomaticTask puts "client is #{client.type}" @logger.info "Creating instance with name #{(a)task.instance.image.external_key}, flavor #{(a)task.instance.flavor.name}, realm #{(a)task.instance.realm.name}" dcloud_instance = client.create_instance((a)task.instance.image.external_key) - #:flavor => @task.instance.image.external_key, - #:realm => @task.instance.image.external_key) + :flavor => @task.instance.flavor.external_key, + :realm => @task.instance.realm.external_key) @logger.info "Instance created with key #{dcloud_instance.id} and state #{dcloud_instance.state}" @task.instance.external_key = dcloud_instance.id @task.instance.state = dcloud_to_instance_state(dcloud_instance.state) -- 1.6.2.5

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

deltacloud-devel August 2009