On Thu, Aug 26, 2010 at 10:24 AM, Justin Harris <jharris(a)redhat.com> wrote:
On Thu, 26 Aug 2010, bkearney(a)redhat.com wrote:
> R we seeing errors because of this?
Not at the moment, but I was noticing this in some of our rspecs. It is
not an immediate problem but I started to have flashbacks of previous
projects I have been on where it was next to impossible to get all tests
in the suite to pass because there were random values generated for all
kinds of stuff, and different tests would pass/fail on subsequent runs for
no real reason except that these values were different. A big value that
I see is that we can be very explicit about inputs for the test, and
outputs we expect. Often times when calculating inputs, you basically
have to recompute what the expected output is, mirroring what is happening
in the actual method under test. This IMO is kind of defeating the
purpose, because most likely the same person is writing the test that
wrote the feature and is baking in whatever assumptions they have about
the implementation into the test as well.
For name generation - what if you get a name collision on a 'register'
call which results in an error. A rerun would probably not, and the test
runs fine. This IMO is not a good thing, because you get two runs of the same
test with different outputs. It is true that a static name can also have
a collision, but it is consistent and forces us to solve that issue in a
more comprehensive way.
Static strings are more likely to cause those collisions, and a
redeploy will often wipe the db and your error would disappear, it
seems like the same scenario to me, two runs, different results.
I agree to a point, certainly common sense needs to be used, we
shouldn't use a random integer for the quantity of a sub/pool for
instance. But for object names (consumers/products/owners), I
genuinely cannot think of how it would be a problem aside from a
collision, which is less likely than with statics. I would also
definitely agree we should never use a random value directly related
to something we're testing, like what characters are suitable for an
owner name. :)
Static strings get copied everywhere (which is just bad), and coming
up with a unique owner name every time we create one is kind of
irritating. My pref would be to let the test util methods stub out
random names for products/consumers/owners.
That said, day to day, this issue is a far cry from a big one for me,
so I'm cool with giving everything an explicit name if that's the
route we wanna go.
Cheers,
Devan
--
Devan Goodwin <dgoodwin(a)rm-rf.ca>
http://rm-rf.ca