Hi everyone,
I finnish revamp of Test::Unit to Cucumber to match testing suite used in portal.
Basic usage:
rake features
OR:
cucumber features/<feature_name>.feature -q
You don't need to start server or configure something extra. Right now, there are two failures:
1. Image with no name in Mock driver (could be easely fixed in fixtures) 2. Device name in one storage volume is empty (not sure about this)
---
One bonus thing is, that if you have 'rcov' gem installed, you will get 'code coverage' generated by this gem in './coverage' directory. This could be extra usefull for testing.
Second bonus thing is, that these test are configurable. Right now there is only configuration for 'mock' driver, but with small changes, (hopefully) we will be able to test EC2 and other providers using Cucumber as well.
Any comments/question/suggestions/acks are welcome.
- Michal
--- server/Rakefile | 9 ++ server/features/flavors.feature | 22 ++++ server/features/images.feature | 37 +++++++ server/features/instances.feature | 98 +++++++++++++++++++ server/features/realms.feature | 21 ++++ server/features/step_definitions/common_steps.rb | 101 ++++++++++++++++++++ server/features/step_definitions/flavors_steps.rb | 5 + server/features/step_definitions/images_steps.rb | 6 + .../features/step_definitions/instances_steps.rb | 87 +++++++++++++++++ server/features/step_definitions/realms_steps.rb | 4 + .../step_definitions/storage_snapshots_steps.rb | 23 +++++ .../features/step_definitions/storage_volumes.rb | 4 + server/features/storage_snapshots.feature | 28 ++++++ server/features/storage_volumes.feature | 27 +++++ server/features/support/configuration_mock.rb | 26 +++++ server/features/support/env.rb | 25 +++++ 16 files changed, 523 insertions(+), 0 deletions(-) create mode 100644 server/features/flavors.feature create mode 100644 server/features/images.feature create mode 100644 server/features/instances.feature create mode 100644 server/features/realms.feature create mode 100644 server/features/step_definitions/common_steps.rb create mode 100644 server/features/step_definitions/flavors_steps.rb create mode 100644 server/features/step_definitions/images_steps.rb create mode 100644 server/features/step_definitions/instances_steps.rb create mode 100644 server/features/step_definitions/realms_steps.rb create mode 100644 server/features/step_definitions/storage_snapshots_steps.rb create mode 100644 server/features/step_definitions/storage_volumes.rb create mode 100644 server/features/storage_snapshots.feature create mode 100644 server/features/storage_volumes.feature create mode 100644 server/features/support/configuration_mock.rb create mode 100644 server/features/support/env.rb
diff --git a/server/Rakefile b/server/Rakefile index 247acc4..34faa6a 100644 --- a/server/Rakefile +++ b/server/Rakefile @@ -19,7 +19,9 @@ # for example lib/tasks/capistrano.rake, and they will automatically be available to Rake.
require 'rake' +require 'cucumber' require 'rake/testtask' +require 'cucumber/rake/task'
desc "Run basic unit tests" Rake::TestTask.new("test") { |t| @@ -34,3 +36,10 @@ Rake::TestTask.new("test") { |t| t.verbose = false t.warning = false } + + +Cucumber::Rake::Task.new(:features) do |t| + t.cucumber_opts = "features --format pretty" + t.rcov = true +end + diff --git a/server/features/flavors.feature b/server/features/flavors.feature new file mode 100644 index 0000000..2b3e43b --- /dev/null +++ b/server/features/flavors.feature @@ -0,0 +1,22 @@ +Feature: Working with flavors + In order to work with flavors + + Background: + Given I want to get XML + + Scenario: I want to get list of all flavors + When I request index operation for flavors collection + Then I should see <FLAVOR_COUNT> flavor inside flavors + + Scenario: I want to show flavor details + When I request for '<FLAVOR_ID>' flavor + Then I should get this flavor + And flavor should have valid href parameter + And flavor should contain id parameter + And flavor should contain architecture parameter + And flavor should contain memory parameter + And flavor should contain storage parameter + + Scenario: I want filter flavors by architecture + When I want flavors with '<FLAVOR_ARCH>' architecture + Then I should get only flavors with architecture '<FLAVOR_ARCH>' diff --git a/server/features/images.feature b/server/features/images.feature new file mode 100644 index 0000000..b35f99e --- /dev/null +++ b/server/features/images.feature @@ -0,0 +1,37 @@ +Feature: Working with images + In order to work with images + + Background: + Given I want to get XML + + Scenario: I want to get list of all images + When I request index operation for images collection + Then I in order to see list of images I need to be authorized + When I enter correct username and password + And I request index operation for images collection + Then I should see <IMAGE_COUNT> image inside images + + Scenario: I want to show image details + Given I am authorized to show image '<IMAGE_ID>' + When I request for '<IMAGE_ID>' image + Then I should get this image + And image should have valid href parameter + And image should contain id parameter + And image should contain name parameter + And image should contain owner_id parameter + And image should contain description parameter + And image should contain architecture parameter + + Scenario: I want filter images by owner_id + When I want images with '<IMAGE_OWNER>' owner_id + Then I should get only images with owner_id '<IMAGE_OWNER>' + + Scenario: I want filter images by architecture + When I want images with '<IMAGE_ARCH>' architecture + Then I should get only images with architecture '<IMAGE_ARCH>' + + Scenario: I want filter images by architecture + When I want images with '<IMAGE_ARCH>' architecture + And images with '<IMAGE_OWNER>' owner_id + Then I should get only images with architecture '<IMAGE_ARCH>' + And this images should also have owner_id '<IMAGE_OWNER>' diff --git a/server/features/instances.feature b/server/features/instances.feature new file mode 100644 index 0000000..6d47c40 --- /dev/null +++ b/server/features/instances.feature @@ -0,0 +1,98 @@ +Feature: Managing instances + In order to manage instances + + Background: + Given I want to get XML + + Scenario: I want to get list of all instances + Given I am authorized to list instances + When I request index operation for instances collection + # FIXME: Add starting number here + Then I should see some instance inside instances + + Scenario: I want to create a new instance + Given I am authorized to create instance + When I request create new instance with: + | name | <INSTANCE_1_NAME> | + | image_id | <INSTANCE_IMAGE_ID> | + Then I should request this instance + And this instance should be 'RUNNING' + And this instance should have name '<INSTANCE_1_NAME>' + And this instance should be image '<INSTANCE_IMAGE_ID>' + + Scenario: I want to create a new instance using realm + Given I am authorized to create instance + When I request create new instance with: + | name | <INSTANCE_2_NAME> | + | image_id | <INSTANCE_IMAGE_ID> | + | realm | <INSTANCE_REALM> | + Then I should request this instance + And this instance should be 'RUNNING' + And this instance should have name '<INSTANCE_2_NAME>' + And this instance should be image '<INSTANCE_IMAGE_ID>' + And this instance should have realm '<INSTANCE_REALM>' + + Scenario: I want to show instance details + Given I am authorized to show instance '<INSTANCE_1_ID>' + When I request for '<INSTANCE_1_ID>' instance + Then I should get this instance + And instance should contain id parameter + And instance should contain name parameter + And instance should contain owner_id parameter + And instance should contain state parameter + And instance state should be RUNNING + When instance state is RUNNING + Then instance should have one public address + And instance should have one private address + When instance state is RUNNING + Then instance should include link to 'reboot' action + And instance should include link to 'stop' action + + Scenario: I want to get instance image + Given I am authorized to show instance '<INSTANCE_1_ID>' + Given I request for '<INSTANCE_1_ID>' instance + When I want to get details about instance image + Then I could follow image href attribute + And this attribute should point me to valid image + + Scenario: I want to get instance flavor + Given I am authorized to show instance '<INSTANCE_1_ID>' + Given I request for '<INSTANCE_1_ID>' instance + When I want to get details about instance flavor + Then I could follow flavor href attribute + And this attribute should point me to valid flavor + + Scenario: I want to get instance realm + Given I am authorized to show instance '<INSTANCE_1_ID>' + Given I request for '<INSTANCE_1_ID>' instance + When I want to get details about instance realm + Then I could follow realm href attribute + And this attribute should point me to valid realm + + Scenario: I want to stop instance + Given I am authorized to show instance '<INSTANCE_1_ID>' + Given I request for '<INSTANCE_1_ID>' instance + When I want to stop this instance + Then I could follow stop action in actions + And this instance state should be 'STOPPED' + + Scenario: I want to start instance + Given I am authorized to show instance '<INSTANCE_1_ID>' + Given I request for '<INSTANCE_1_ID>' instance + When I want to stop this instance + Then I could follow start action in actions + And this instance state should be 'RUNNING' + + Scenario: I want to reboot instance + Given I am authorized to show instance '<INSTANCE_1_ID>' + Given I request for '<INSTANCE_1_ID>' instance + When I want to reboot this instance + Then I could follow reboot action in actions + And this instance state should be 'RUNNING' + + Scenario: I want to destroy instance + Given I am authorized to show instance '<INSTANCE_1_ID>' + Given I request for '<INSTANCE_1_ID>' instance + When I want to stop this instance + Then I could follow stop action in actions + And this instance state should be 'STOPPED' diff --git a/server/features/realms.feature b/server/features/realms.feature new file mode 100644 index 0000000..15d0184 --- /dev/null +++ b/server/features/realms.feature @@ -0,0 +1,21 @@ +Feature: Working with realms + In order to work with realms + + Background: + Given I want to get XML + + Scenario: I want to get list of all realms + When I request index operation for realms collection + Then I should see <REALM_COUNT> realm inside realms + + Scenario: I want to show realm details + When I request for '<REALM_ID>' realm + Then I should get this realm + And realm should have valid href parameter + And realm should contain id parameter + And realm should contain name parameter + And realm should contain state parameter + + Scenario: I want filter realms by state + When I want realms with '<REALM_STATE>' state + Then I should get only realms with state '<REALM_STATE>' diff --git a/server/features/step_definitions/common_steps.rb b/server/features/step_definitions/common_steps.rb new file mode 100644 index 0000000..0f99944 --- /dev/null +++ b/server/features/step_definitions/common_steps.rb @@ -0,0 +1,101 @@ +require 'nokogiri' + +When /^I want to get (HTML|XML)$/ do |format| + case format.downcase + when 'xml': + header 'Accept', 'application/xml' + end +end + +When /^I request (.+) operation for (.+) collection$/ do |operation, collection| + operation = '/'+operation + operation = operation.eql?('/index') ? '' : operation + collection.tr!(' ', '_') + get '/api/'+collection+operation, {} +end + +Then /^I should see ([\w<>_-]+) (.+) inside (.+)$/ do |count, model, collection| + collection.tr!(' ', '-') + model.tr!(' ', '-') + if count.eql?('some') + Nokogiri::XML(last_response.body).xpath("/#{collection}/#{model}").size.should_not == 0 + else + count = replace_variables(count) + Nokogiri::XML(last_response.body).xpath("/#{collection}/#{model}").size.should == count.to_i + end +end + +When /^I request for '(.+)' (.+)$/ do |id, model| + model.tr!(' ', '_') + get '/api/'+model+'s'+'/'+replace_variables(id), {} +end + +Then /^I should get this (.+)$/ do |model| + model.tr!(' ', '-') + Nokogiri::XML(last_response.body).xpath("/#{model}").size.should == 1 +end + +When /^I want (.+) with '(.+)' (.+)$/ do |collection, value, parameter| + @params = [] + @params << [parameter, replace_variables(value)] +end + +When /^images with '(.+)' (.+)$/ do |value, parameter| + @params << [parameter, replace_variables(value)] +end + +Then /^I should get only (.+) with (.+) '(.+)'$/ do |collection, parameter, value| + params = {} + value = replace_variables(value) + @params.collect { |param| params[:"#{param[0]}"] = param[1] } + get '/api/'+collection, params, {} + p = [] + Nokogiri::XML(last_response.body).xpath("/#{collection}/#{collection.gsub(/s$/, '')}").each do |m| + p << m.xpath("#{parameter}").text + end + p.uniq! + p.size.should == 1 + p.first.should == value +end + +Then /^this (.+) should also have (.+) '(.+)'$/ do |collection, parameter, value| + params = {} + value = replace_variables(value) + @params.collect { |param| params[:"#{param[0]}"] = param[1] } + get '/api/'+collection, params, {} + p = [] + Nokogiri::XML(last_response.body).xpath("/#{collection}/#{collection.gsub(/s$/, '')}").each do |m| + p << m.xpath("#{parameter}").text + end + p.uniq! + p.size.should == 1 + p.first.should == value +end + +Then /^I in order to see (list of|this) (.+) I need to be authorized$/ do |t, collection| + last_response.body.strip.should == 'Not authorized' +end + +When /^I enter correct username and password$/ do + authorize 'mockuser', 'mockpassword' +end + +Given /^I am authorized to show (.+) '(.+)'$/ do |model, id| + model.tr!(' ', '_') + authorize 'mockuser', 'mockpassword' + get '/api/'+model+'s/'+model+'/'+replace_variables(id), {} + last_response.body.strip.should_not == 'Not authorized' +end + +Then /^(.+) should contain (.+) parameter$/ do |model, parameter| + model.tr!(' ', '-') + Nokogiri::XML(last_response.body).xpath("/#{model}/#{parameter}").first.should_not == nil + Nokogiri::XML(last_response.body).xpath("/#{model}/#{parameter}").first.text.should_not == '' +end + +Given /^I am authorized to (list) (.+)$/ do |operation, collection| + authorize 'mockuser', 'mockpassword' + collection.tr!(' ', '_') + get '/api/'+collection, {} + last_response.body.strip.should_not == 'Not authorized' +end diff --git a/server/features/step_definitions/flavors_steps.rb b/server/features/step_definitions/flavors_steps.rb new file mode 100644 index 0000000..05830a9 --- /dev/null +++ b/server/features/step_definitions/flavors_steps.rb @@ -0,0 +1,5 @@ +Then /^flavor should have valid href parameter$/ do + href=Nokogiri::XML(last_response.body).xpath('/flavor').first[:href] + href.should == "http://example.org/api/flavors/#%7BCONFIG%5B$DRIVER%5D%5B:flavor_id%5D%7D" +end + diff --git a/server/features/step_definitions/images_steps.rb b/server/features/step_definitions/images_steps.rb new file mode 100644 index 0000000..972d3f1 --- /dev/null +++ b/server/features/step_definitions/images_steps.rb @@ -0,0 +1,6 @@ + +Then /^image should have valid href parameter$/ do + href=Nokogiri::XML(last_response.body).xpath('/image').first[:href] + href.should == "http://example.org/api/images/#%7BCONFIG%5B$DRIVER%5D%5B:image_id%5D%7D" +end + diff --git a/server/features/step_definitions/instances_steps.rb b/server/features/step_definitions/instances_steps.rb new file mode 100644 index 0000000..ec2ad97 --- /dev/null +++ b/server/features/step_definitions/instances_steps.rb @@ -0,0 +1,87 @@ +Then /^instance state should be (.+)$/ do |state| + Nokogiri::XML(last_response.body).xpath("/instance/state").first.text.should == state +end + +When /^instance state is (.+)$/ do |state| + Nokogiri::XML(last_response.body).xpath("/instance/state").first.text.should == state +end + +Then /^instance should have one (public|private) address$/ do |type| + adr = Nokogiri::XML(last_response.body).xpath("/instance/#{type}-addresses/address").first + adr.text.to_s.should_not == nil + adr.text.to_s.should_not == '' +end + +Then /^instance should include link to '(.+)' action$/ do |action| + links = Nokogiri::XML(last_response.body).xpath("/instance/actions/link") + actions = [] + links.each do |link| + actions << link[:rel] + end + actions.include?(action).should == true +end + +When /^I want to get details about instance (.+)$/ do |model| +end + +Then /^I could follow (image|realm|flavor) href attribute$/ do |model| + m = Nokogiri::XML(last_response.body).xpath("/instance/#{model}").first + model_url = URI.parse(m[:href]).path + get model_url, {} +end + +Then /^this attribute should point me to valid (image|realm|flavor)$/ do |model| + Nokogiri::XML(last_response.body).xpath("/#{model}").first.name.should == model +end + +When /^I want to (.+) this instance$/ do |action| +end + +Given /^I am authorized to create instance$/ do + last_response.body.strip.should_not == 'Not authorized' +end + +Then /^I could follow (.+) action in actions$/ do |action| + link = Nokogiri::XML(last_response.body).xpath("/instance/actions/link[@rel='#{action}']").first + post URI.parse(link[:href]).path, {} +end + +Then /^this instance state should be '(.+)'$/ do |state| + Nokogiri::XML(last_response.body).xpath("/instance/state").first.text.should == state +end + +When /^I request create new instance with:$/ do |table| + params = {} + table.raw.map { |a,b| params[:"#{a}"] = replace_variables(b) } + post '/api/instances.xml', params + @instance_id = Nokogiri::XML(last_response.body).xpath("/instance/id").first.text + @instance_id.should_not == nil + CONFIG[$DRIVER][:instance_1_id] = @instance_id unless CONFIG[$DRIVER][:instance_1_id] +end + +Then /^I should request this instance$/ do + get URI.encode('/api/instances/'+@instance_id), {} + Nokogiri::XML(last_response.body).xpath("/instance").first.should_not == nil +end + +Then /^this instance should be '(.+)'$/ do |state| + get URI.encode('/api/instances/'+@instance_id), {} + Nokogiri::XML(last_response.body).xpath("/instance/state").first.text.should == state +end + +Then /^this instance should have name '(.+)'$/ do |name| + get URI.encode('/api/instances/'+@instance_id), {} + Nokogiri::XML(last_response.body).xpath("/instance/name").first.text.should == replace_variables(name) +end + +Then /^this instance should be image '(.+)'$/ do |image| + get URI.encode('/api/instances/'+@instance_id), {} + get URI.parse(Nokogiri::XML(last_response.body).xpath("/instance/image").first[:href]).path, {} + Nokogiri::XML(last_response.body).xpath("/image/id").first.text.should == replace_variables(image) +end + +Then /^this instance should have realm '(.+)'$/ do |realm| + get URI.encode('/api/instances/'+@instance_id), {} + get URI.parse(Nokogiri::XML(last_response.body).xpath("/instance/realm").first[:href]).path, {} + Nokogiri::XML(last_response.body).xpath("/realm/id").first.text.should == replace_variables(realm) +end diff --git a/server/features/step_definitions/realms_steps.rb b/server/features/step_definitions/realms_steps.rb new file mode 100644 index 0000000..7ee408b --- /dev/null +++ b/server/features/step_definitions/realms_steps.rb @@ -0,0 +1,4 @@ +Then /^realm should have valid href parameter$/ do + href=Nokogiri::XML(last_response.body).xpath('/realm').first[:href] + href.should == "http://example.org/api/realms/#%7BCONFIG%5B$DRIVER%5D%5B:realm_id%5D%7D" +end diff --git a/server/features/step_definitions/storage_snapshots_steps.rb b/server/features/step_definitions/storage_snapshots_steps.rb new file mode 100644 index 0000000..777c69c --- /dev/null +++ b/server/features/step_definitions/storage_snapshots_steps.rb @@ -0,0 +1,23 @@ +Then /^storage snapshot should have valid href parameter$/ do + href=Nokogiri::XML(last_response.body).xpath('/storage-snapshot').first[:href] + href.should == "http://example.org/api/storage_snapshots/snap2" +end + +Then /^storage snapshot should have storage-volume with valid href attribute$/ do + href=Nokogiri::XML(last_response.body).xpath('/storage-snapshot/storage-volume').first[:href] + href.should == "http://example.org/api/storage_volumes/vol2" +end + +When /^I want to get details about storage volume$/ do + @storage_volume_url=Nokogiri::XML(last_response.body).xpath('/storage-snapshot/storage-volume').first[:href] +end + +Then /^I could follow storage volume href attribute$/ do + url=URI.parse(@storage_volume_url) + get url.path, {} + last_response.should_not == nil +end + +Then /^this attribute should point me to valid storage volume$/ do + Nokogiri::XML(last_response.body).xpath("/storage-volume").size.should == 1 +end diff --git a/server/features/step_definitions/storage_volumes.rb b/server/features/step_definitions/storage_volumes.rb new file mode 100644 index 0000000..91d1110 --- /dev/null +++ b/server/features/step_definitions/storage_volumes.rb @@ -0,0 +1,4 @@ +Then /^storage volume should have valid href parameter$/ do + href=Nokogiri::XML(last_response.body).xpath('/storage-volume').first[:href] + href.should == "http://example.org/api/storage_volumes/vol2" +end diff --git a/server/features/storage_snapshots.feature b/server/features/storage_snapshots.feature new file mode 100644 index 0000000..67146f8 --- /dev/null +++ b/server/features/storage_snapshots.feature @@ -0,0 +1,28 @@ +Feature: Working with storage snapshots + In order to work with storage snapshots + + Background: + Given I want to get XML + + Scenario: I want to get list of all storage snapshots + Given I am authorized to list storage snapshots + When I request index operation for storage snapshots collection + Then I should see <STORAGE_SNAPSHOT_COUNT> storage snapshot inside storage snapshots + + Scenario: I want to show storage snapshot details + Given I am authorized to show storage snapshot '<STORAGE_SNAPSHOT_ID>' + When I request for '<STORAGE_SNAPSHOT_ID>' storage snapshot + Then I should get this storage snapshot + And storage snapshot should have valid href parameter + And storage snapshot should contain id parameter + And storage snapshot should contain created parameter + And storage snapshot should contain state parameter + And storage snapshot should have storage-volume with valid href attribute + When I want to get details about storage volume + Then I could follow storage volume href attribute + And this attribute should point me to valid storage volume + + Scenario: I want filter storage snapshots by state + Given I am authorized to list storage snapshots + When I want storage snapshots with '<STORAGE_SNAPSHOT_STATE>' state + Then I should get only realms with state '<STORAGE_SNAPSHOT_STATE>' diff --git a/server/features/storage_volumes.feature b/server/features/storage_volumes.feature new file mode 100644 index 0000000..2cff225 --- /dev/null +++ b/server/features/storage_volumes.feature @@ -0,0 +1,27 @@ +Feature: Working with storage volumes + In order to work with storage volumes + + Background: + Given I want to get XML + + Scenario: I want to get list of all storage volumes + Given I am authorized to list storage volumes + When I request index operation for storage volumes collection + Then I should see <STORAGE_VOLUME_COUNT> storage volume inside storage volumes + + Scenario: I want to show storage volume details + Given I am authorized to show storage volume '<STORAGE_VOLUME_ID>' + When I request for '<STORAGE_VOLUME_ID>' storage volume + Then I should get this storage volume + And storage volume should have valid href parameter + And storage volume should contain id parameter + And storage volume should contain created parameter + And storage volume should contain state parameter + And storage volume should contain capacity parameter + And storage volume should contain device parameter + And storage volume should contain instance parameter + + Scenario: I want filter storage volumes by state + Given I am authorized to list storage volumes + When I want storage volumes with '<STORAGE_VOLUME_STATE>' state + Then I should get only realms with state '<STORAGE_VOLUME_STATE>' diff --git a/server/features/support/configuration_mock.rb b/server/features/support/configuration_mock.rb new file mode 100644 index 0000000..adbfc5c --- /dev/null +++ b/server/features/support/configuration_mock.rb @@ -0,0 +1,26 @@ +CONFIG[:mock] = { + :driver_name => 'mock', + :realm_id => 'us', + :realm_state => 'AVAILABLE', + :realm_count => 2, + :flavor_id => 'm1-small', + :flavor_count => 5, + :flavor_arch => 'x86_64', + :image_owner => 'fedoraproject', + :image_arch => 'i386', + :image_id => 'img2', + :image_count => 3, + :storage_snapshot_id => 'snap2', + :storage_snapshot_state => 'AVAILABLE', + :storage_snapshot_count => '2', + :storage_volume_id => 'vol2', + :storage_volume_state => 'AVAILABLE', + :storage_volume_count => 2, + :instances_count => 180, + :instance_1_name => "#{Time.now.to_i} testing instance", + :instance_2_name => "#{Time.now.to_i+1} testing instance", + :instance_image_id => 'img2', + :instance_realm => 'us' +} + +$DRIVER = :mock diff --git a/server/features/support/env.rb b/server/features/support/env.rb new file mode 100644 index 0000000..5539ade --- /dev/null +++ b/server/features/support/env.rb @@ -0,0 +1,25 @@ +require 'sinatra' +require 'server' +require 'rack/test' +require 'base64' + +Sinatra::Application.set :environment, :test + +CONFIG = {} +load 'features/support/configuration_mock.rb' + +World do + def app + @app = Rack::Builder.new do + run Sinatra::Application + end + end + + def replace_variables(str) + CONFIG[$DRIVER].each_key.collect { |k| str.gsub!(/<#{k.to_s.upcase}>/, "#{CONFIG[$DRIVER][k]}") } + return str + end + + include Rack::Test::Methods +end +
On Mon, 2010-03-15 at 14:17 +0100, mfojtik@redhat.com wrote:
I finnish revamp of Test::Unit to Cucumber to match testing suite used in portal.
Cool.
Basic usage:
rake features
OR:
cucumber features/<feature_name>.feature -q
You don't need to start server or configure something extra. Right now, there are two failures:
- Image with no name in Mock driver (could be easely fixed in fixtures)
- Device name in one storage volume is empty (not sure about this)
I get a long list of failures after applying your patch to the HEAD of master. I've attached the output I get from 'rake features'.
Out of curiosity, is there a way to generate pretty HTML output ? Slogging through the dump on the console seems like a drag.
One bonus thing is, that if you have 'rcov' gem installed, you will get 'code coverage' generated by this gem in './coverage' directory. This could be extra usefull for testing.
For me, it only worked if I had rcov installed - I agree that it's useful, but since generating the rcov report takes quite a bit of time, it should only be done optionally.
Second bonus thing is, that these test are configurable. Right now there is only configuration for 'mock' driver, but with small changes, (hopefully) we will be able to test EC2 and other providers using Cucumber as well.
Any idea how hard it would be to completely stub out the interaction with the backend cloud for the tests ? For an integration/release test, we'd definitely want to run them against the actual clouds, but for individual developers, that takes too long and costs money.
David
On 16/03/10 01:22 +0000, David Lutterkort wrote:
On Mon, 2010-03-15 at 14:17 +0100, mfojtik@redhat.com wrote:
I finnish revamp of Test::Unit to Cucumber to match testing suite used in portal.
Cool.
Basic usage:
rake features
OR:
cucumber features/<feature_name>.feature -q
You don't need to start server or configure something extra. Right now, there are two failures:
- Image with no name in Mock driver (could be easely fixed in fixtures)
- Device name in one storage volume is empty (not sure about this)
I get a long list of failures after applying your patch to the HEAD of master. I've attached the output I get from 'rake features'.
Hmm interesting. Which version of Ruby you are using ? I applied same patch against HEAD and my results seems OK.
I used a small method for replacing variables in features with variables from configuration file. Seems like it's something with this method.
Out of curiosity, is there a way to generate pretty HTML output ? Slogging through the dump on the console seems like a drag.
Sure, you could edit Rakefile and replace --format pretty with --format html --out result.html.
Or you can do 'cucumber --format html --out result.html' in 'server' directory.
HTML output looks much nicer [1] ;-)
One bonus thing is, that if you have 'rcov' gem installed, you will get 'code coverage' generated by this gem in './coverage' directory. This could be extra usefull for testing.
For me, it only worked if I had rcov installed - I agree that it's useful, but since generating the rcov report takes quite a bit of time, it should only be done optionally.
Good idea. I'll split this into two tasks 'rake features' and 'rake rcow'
Second bonus thing is, that these test are configurable. Right now there is only configuration for 'mock' driver, but with small changes, (hopefully) we will be able to test EC2 and other providers using Cucumber as well.
Any idea how hard it would be to completely stub out the interaction with the backend cloud for the tests ? For an integration/release test, we'd definitely want to run them against the actual clouds, but for individual developers, that takes too long and costs money.
Sure, we could talk about creating a mock EC2 driver. But this doesn't change basic concept, which is that you could configure these tests to run against this driver. (or against real EC2, if it's needed)
- Michal
[1] http://mifo.sk/cucumber.html
On Tue, 2010-03-16 at 13:17 +0100, Michal Fojtik wrote:
Hmm interesting. Which version of Ruby you are using ? I applied same patch against HEAD and my results seems OK.
Latest from Fedora 12: ruby-1.8.6.383-6.fc12.x86_64
Sure, we could talk about creating a mock EC2 driver. But this doesn't change basic concept, which is that you could configure these tests to run against this driver. (or against real EC2, if it's needed)
Agreed.
David
deltacloud-devel@lists.fedorahosted.org