[cloud] #167: Decide a process about failed tests for Autocloud

[Fedora Cloud Trac Tickets]

#167: Decide a process about failed tests for Autocloud
--------------------+---------------------
 Reporter:  kushal  |      Owner:
     Type:  task    |     Status:  new
 Priority:  normal  |  Milestone:  Future
Component:  ---     |   Keywords:  meeting
--------------------+---------------------
 If we have a test failing the compose (right test, catching a real bug),
 should we stop the release till the time the bug gets resolved?

My personal opinion is yes, first fix, and then only go ahead.

For example [https://apps.fedoraproject.org/autocloud/jobs/353/output#283
 this test] fails time to time on the vagrant virtualbox image.