That I think covers it all. Basically I think the tasks would be

1) Get a set of keys
2) Setup test architecture.
3) Work out initial issues of how to make/destroy and deal with
potential problems.
4) Begin to architect how it would roll out.
 a) Work with yubikey and Fedora security experts on how it would be
best built for our needs.
 b) Write up procedural issues for who keys are made for, how they are
made, how they are destroyed, etc.
 c) Get political/social buyin/acceptance on procedural issues.
 d) Determine what systems would be in test environment.
5) Build a test environment with architecture.
 a) Work on breaking it
 b) Work out how much of the breakage we can accept and what we would
do when it happens.
6) Go/no-go
7) If Go, start rolling out further.

I'd be happy to get an initial key and help test it all out. I might be able to help setup the architecture, I've worked with two factor before.
-Anand