Fedora 23 Cloud Atomic Developer Mode Preview

[Jonathan Lebon]

Hi all,

As some of you may already know, I've been working on adding
a new feature to the Fedora 23 Cloud Atomic image called
"Developer Mode" (I'm not sure yet if this is the correct
name for it). The Trello card is available at [1].

The high-level goal is to make Atomic more accessible by
providing a new GRUB 2 menu item labeled e.g. "Fedora 23
(Twenty Three) Developer Mode". This mode is an attempt to
provide a painless experience for folks who want to try out
Atomic, but (1) do not want to bother setting up a cloud-
init datasource, or (2) do not know anything about cloud-
init, or even (3) do not have much experience with
Linux overall.

Since the functionality is completely integrated into the
image, there are no requirements on the host system, other
than its ability to boot VMs.

When booted in Developer Mode, the following happens:
    - cloud-init uses a local built-in datasource
    - a new root password is generated
    - the root user is automatically logged in on tty1
    - the cockpit/ws image is downloaded and started
    - a tmux session is started on tty1 to provide all the
      relevant information (root password, IP address,
      Cockpit console address)

The first Developer Mode image is available at [2]. I invite
you all to try it out and let me know what you think! I've
enabled the plymouth splash screen, which for now uses the
Fedora theme, but might later be switched over to an Atomic
theme.

I'd still like to give users a more helpful welcome message
after login. Maybe we could link to a new page on
projectatomic.io which describes Developer Mode in more
details.

As Matthew Miller pointed out, one of the drawbacks of this
approach is that the GRUB 2 menu timeout will probably have
to be slightly increased to give users more time to make
their selection (at the expense of also increasing boot
times in contexts that don't care about Developer Mode).

In my experience, increasing it by only 1s (for a total of
2s) was enough, but we should probably discuss it more. We
can also minimize this by shipping with a grub.cfg that uses
a 2s timeout, but with the default of 1s, so that at the
next grub2-mkconfig (e.g. on an upgrade/rebase), it will go
back to 1s.

Code is available at [3]. Some modifications to the
kickstart file were also necessary.

Cheers,

Jonathan

---

TL;DR: I added a new boot menu item to the Atomic image so
that you don't have to set up cloud-init. You can download
it from [2].

[1] https://trello.com/c/eK54YRTp
[2] https://jlebon.fedorapeople.org/atomic-devmode/latest/
[3] https://github.com/jlebon/atomic-devmode

[Colin Walters]

On Wed, Dec 23, 2015, at 11:50 AM, Jonathan Lebon wrote:
>  Hi all,
>  
>  As some of you may already know, I've been working on adding
>  a new feature to the Fedora 23 Cloud Atomic image called
>  "Developer Mode" (I'm not sure yet if this is the correct
>  name for it). The Trello card is available at [1].
>  
>      - a tmux session is started on tty1 to provide all the
>        relevant information (root password, IP address,
>        Cockpit console address) 
The whole thing is definitely slick, particularly this part.  I have
a small concern that people unfamiliar with tmux might
be thrown by this, but that can likely be ameliorated by
docs.

Anyone else have a chance to try this?

At a very high level, we need to figure out a better flow/explanation
from the websites to deliverables.  The recently updated and
revamped ADB should also be part of this I'd say:

https://lists.centos.org/pipermail/centos-devel/2015-December/014153.html

(Currently it's only for CentOS, but if we made a Fedora version
 as well it would have the negative of expanding our test matrix,
 but the positive of consistency where the distro variant is stability
 and not format)

[Adam Miller]

On Mon, Jan 4, 2016 at 11:35 AM, Colin Walters <walters(a)verbum.org&gt; wrote:
>  On Wed, Dec 23, 2015, at 11:50 AM, Jonathan Lebon wrote:
> > Hi all,
> >
> > As some of you may already know, I've been working on adding
> > a new feature to the Fedora 23 Cloud Atomic image called
> > "Developer Mode" (I'm not sure yet if this is the correct
> > name for it). The Trello card is available at [1].
> >
> >     - a tmux session is started on tty1 to provide all the
> >       relevant information (root password, IP address,
> >       Cockpit console address)
>
>  The whole thing is definitely slick, particularly this part.  I have
>  a small concern that people unfamiliar with tmux might
>  be thrown by this, but that can likely be ameliorated by
>  docs. 
Maybe we could do something similar to how the anaconda team is using
tmux in the text based installer and provide a config file that
displays usage information on screen and maps the navigation keys to
something a little less foreign to users unfamiliar to tmux. Thoughts?

>
> Anyone else have a chance to try this?
>
> At a very high level, we need to figure out a better flow/explanation
> from the websites to deliverables.  The recently updated and
> revamped ADB should also be part of this I'd say:
>
> https://lists.centos.org/pipermail/centos-devel/2015-December/014153.html
>
> (Currently it's only for CentOS, but if we made a Fedora version
>  as well it would have the negative of expanding our test matrix,
>  but the positive of consistency where the distro variant is stability
>  and not format)
> _______________________________________________
> cloud mailing list
> cloud(a)lists.fedoraproject.org
> http://lists.fedoraproject.org/admin/lists/[e-mail]

[Jonathan Lebon]

----- Original Message -----
>  On Mon, Jan 4, 2016 at 11:35 AM, Colin Walters
> <walters(a)verbum.org&gt; wrote:
>  > On Wed, Dec 23, 2015, at 11:50 AM, Jonathan Lebon wrote:
>  >> Hi all,
>  >>
>  >> As some of you may already know, I've been working on adding
>  >> a new feature to the Fedora 23 Cloud Atomic image called
>  >> "Developer Mode" (I'm not sure yet if this is the correct
>  >> name for it). The Trello card is available at [1].
>  >>
>  >>     - a tmux session is started on tty1 to provide all the
>  >>       relevant information (root password, IP address,
>  >>       Cockpit console address)
>  >
>  > The whole thing is definitely slick, particularly this part.  I have
>  > a small concern that people unfamiliar with tmux might
>  > be thrown by this, but that can likely be ameliorated by
>  > docs.
>  
>  Maybe we could do something similar to how the anaconda team is using
>  tmux in the text based installer and provide a config file that
>  displays usage information on screen and maps the navigation keys to
>  something a little less foreign to users unfamiliar to tmux. Thoughts? 
Hmmm, that's a good idea. I'll look into this. At least so that they
can change from the top and bottom panels easily, as well as maybe
spawn new windows if they'd like.

[Colin Walters]

On Mon, Jan 4, 2016, at 12:35 PM, Colin Walters wrote:

>  At a very high level, we need to figure out a better
> flow/explanation
>  from the websites to deliverables.  The recently updated and
>  revamped ADB should also be part of this I'd say: 
On this topic, what if we just added this to the Vagrant box, which
is already a separate image?

It would make a lot of sense I think, as the Vagrant box is already
intentionally similar, and this would make it possible to use even
if you don't have Vagrant.

That way we're not adding boot timeouts into everyone's EC2/OpenStack
instances etc.

[Colin Walters]

One other idea:

We simply do this by default for all cloud images, without a timeout - if no cloud-init
metadata is provided, you can log in to the hypervisor console and see an autogenerated
root password.

I'd say we should also ensure that *remote* ssh access is disabled in this scenario -
if you want to log in over ssh, you'd need to change the password and enable remote
ssh password auth (cloud-init disables it by default).

It'd be a notable policy change, but in practice I think quite secure - if you have
access to the hypervisor console you tend to have total control over a system anyways. 
And we're only talking about cloud images, not bare metal.

[Jonathan Lebon]

----- Original Message -----
>  One other idea:
>  
>  We simply do this by default for all cloud images, without a timeout - if no
>  cloud-init metadata is provided, you can log in to the hypervisor console
>  and see an autogenerated root password. 
I like that. So just to make sure I understand what you mean here:
  - No new boot menu item
  - Whenever cloud-init fails to find a source, we generate an
    expired root pw, disable ssh pwauth, and autologin
  - Atomic Developer Mode is started by a user command (e.g.
    a /usr/bin/atomic-devmode)

Is that correct?

One issue here is that cloud-init takes some time before realizing
that it has no source. It tries to connect to the default metadata
server address. It uses a 50s timeout it seems. We would probably
have to reduce this to something more sensible (5s? 10s?).

[Colin Walters]

On Wed, Jan 6, 2016, at 12:08 PM, Jonathan Lebon wrote:

>  One issue here is that cloud-init takes some time before realizing
>  that it has no source. It tries to connect to the default metadata
>  server address. It uses a 50s timeout it seems. We would probably
>  have to reduce this to something more sensible (5s? 10s?). 
Hm...that means we're a bit more reliant on metadata server availability.

Realistically, it *should* be very reliable for a cloud provider - it's just
hosting a tiny amount of data.  The cloud-init timeout here does seem
extremely conservative.

But I'm hesitant to speak confidently about *all* cloud providers.

That said, the failure scenario here is just that we autogenerate a
local root password only accessible on the physical console.

(In fact in this scenario, why generate a password at all?  Just
 auto-login the first physical console, then the user can set a root
 password if they want)

[Jonathan Lebon]

I looked into this a bit more. Cloud-init provides an easy way to
give a fallback datasource when all other sources fail. It tries
multiple networked datasources before that.

For each of them, there is a 'timeout' and a 'max_wait' field. The
former is the timeout per connection attempt, and the latter is the
total amount of time before completely giving up retrying.

These are the URLs it tries to connect to:
- http://169.254.169.254/openstack (OpenStack, 10s timeout, no wait)
- http://169.254.169.254/2009-04-04/meta-data/instance-id (EC2, 50s timeout, 120s
max_wait)
- http://192.168.122.1/latest/meta-data/instance-id (CloudStack, 50s timeout, 120s
max_wait)

So, with the default behaviour, it takes 10+120+120 = 250s before
cloud-init finally decides to use the fallback source. I'm not sure
what the rationale is for making the EC2 and CloudStack values so
high (esp. in contrast to OpenStack).

If do go this way, I think we'd have to lower them to more
reasonable values so that the total time is at least under a
minute.

[Micah Abbott]

I tried devmode this afternoon and it worked very well.  I discussed a few nits with
jlebon on #atomic, but thought I would mention them here.

- The generated root password can include similar characters l (letter L), 1 (number 1), O
(upper-case letter O), 0 (number zero), etc.  I got bit by this when I tried to login to
the cockpit interface.  Perhaps we could eventually use something like a series of
dictionary words separated by a certain character?  (I found a discussion on StackOverflow
[1] about this potential problem.)

- I think we should include a string under the cockpit URL that says something like
"Login to the cockpit console with the username "root" and the generated
password above".  If an intended audience is people who have never used Linux, we
should be fairly explicit with instructions right out of the box.


[1]
http://stackoverflow.com/questions/55556/characters-to-avoid-in-automatic...

----- Original Message -----
> From: "Jonathan Lebon" <jlebon(a)redhat.com&gt;
> To: cloud(a)lists.fedoraproject.org, atomic-devel(a)projectatomic.io
> Sent: Wednesday, December 23, 2015 11:50:14 AM
> Subject: Fedora 23 Cloud Atomic Developer Mode Preview
> 
> Hi all,
> 
> As some of you may already know, I've been working on adding
> a new feature to the Fedora 23 Cloud Atomic image called
> "Developer Mode" (I'm not sure yet if this is the correct
> name for it). The Trello card is available at [1].
> 
> The high-level goal is to make Atomic more accessible by
> providing a new GRUB 2 menu item labeled e.g. "Fedora 23
> (Twenty Three) Developer Mode". This mode is an attempt to
> provide a painless experience for folks who want to try out
> Atomic, but (1) do not want to bother setting up a cloud-
> init datasource, or (2) do not know anything about cloud-
> init, or even (3) do not have much experience with
> Linux overall.
> 
> Since the functionality is completely integrated into the
> image, there are no requirements on the host system, other
> than its ability to boot VMs.
> 
> When booted in Developer Mode, the following happens:
>     - cloud-init uses a local built-in datasource
>     - a new root password is generated
>     - the root user is automatically logged in on tty1
>     - the cockpit/ws image is downloaded and started
>     - a tmux session is started on tty1 to provide all the
>       relevant information (root password, IP address,
>       Cockpit console address)
> 
> The first Developer Mode image is available at [2]. I invite
> you all to try it out and let me know what you think! I've
> enabled the plymouth splash screen, which for now uses the
> Fedora theme, but might later be switched over to an Atomic
> theme.
> 
> I'd still like to give users a more helpful welcome message
> after login. Maybe we could link to a new page on
> projectatomic.io which describes Developer Mode in more
> details.
> 
> As Matthew Miller pointed out, one of the drawbacks of this
> approach is that the GRUB 2 menu timeout will probably have
> to be slightly increased to give users more time to make
> their selection (at the expense of also increasing boot
> times in contexts that don't care about Developer Mode).
> 
> In my experience, increasing it by only 1s (for a total of
> 2s) was enough, but we should probably discuss it more. We
> can also minimize this by shipping with a grub.cfg that uses
> a 2s timeout, but with the default of 1s, so that at the
> next grub2-mkconfig (e.g. on an upgrade/rebase), it will go
> back to 1s.
> 
> Code is available at [3]. Some modifications to the
> kickstart file were also necessary.
> 
> Cheers,
> 
> Jonathan
> 
> ---
> 
> TL;DR: I added a new boot menu item to the Atomic image so
> that you don't have to set up cloud-init. You can download
> it from [2].
> 
> [1] https://trello.com/c/eK54YRTp
> [2] https://jlebon.fedorapeople.org/atomic-devmode/latest/
> [3] https://github.com/jlebon/atomic-devmode
> _______________________________________________
> cloud mailing list
> cloud(a)lists.fedoraproject.org
> http://lists.fedoraproject.org/admin/lists/[e-mail]
>

[Jonathan Lebon]

----- Original Message -----
>  I tried devmode this afternoon and it worked very well.  I discussed
> a few
>  nits with jlebon on #atomic, but thought I would mention them here.
>  
>  - The generated root password can include similar characters l (letter L), 1
>  (number 1), O (upper-case letter O), 0 (number zero), etc.  I got bit by
>  this when I tried to login to the cockpit interface.  Perhaps we could
>  eventually use something like a series of dictionary words separated by a
>  certain character?  (I found a discussion on StackOverflow [1] about this
>  potential problem.) 
I'm thinking of just rerunning pwmake until it satisfies our criteria,
rather than either crafting passwords ourselves, or messing with the string
it outputs and reducing the entropy.

>  - I think we should include a string under the cockpit URL that says
>  something like "Login to the cockpit console with the username "root" and
>  the generated password above".  If an intended audience is people who have
>  never used Linux, we should be fairly explicit with instructions right out
>  of the box. 
Agreed, good idea.

[Jonathan Lebon]

Just uploaded a newer image (available at [1]) which
addresses some of the feedback received:
    - The generated passwords should now be easier to type,
      with no easily misidentified letters.
    - Helper message to explicitly say how to log into
      Cockpit.
    - New tmux terminals and easy shortcuts for switching
      between them.

Cheers!

Jonathan

[1] https://jlebon.fedorapeople.org/atomic-devmode/latest/

F24 Change? Re: Fedora 23 Cloud Atomic Developer Mode Preview

[Joe Brockmeier]

Hi Jonathan,

On 01/11/2016 12:01 PM, Jonathan Lebon wrote:
>  Just uploaded a newer image (available at [1]) which
>  addresses some of the feedback received:
>      - The generated passwords should now be easier to type,
>        with no easily misidentified letters.
>      - Helper message to explicitly say how to log into
>        Cockpit.
>      - New tmux terminals and easy shortcuts for switching
>        between them. 
Are you considering putting this in as a change for Fedora 24? The
deadline is tomorrow. You can find a lot of change examples here:

https://fedoraproject.org/wiki/Category:SelfContainedChange

Best,

jzb
-- 
Joe Brockmeier | Community Team, OSAS
jzb(a)redhat.com | http://community.redhat.com/
Twitter: @jzb  | http://dissociatedpress.net/

F24 Change? Re: Fedora 23 Cloud Atomic Developer Mode Preview

[Jonathan Lebon]

Hi Joe,

Just wrote the change proposal for this[1]. I haven't yet tagged it
in the relevant categories. Since it's my first change proposal, I
wanted to run it by you (and others) to see if it's all in order.

Thanks,

Jonathan

[1] https://fedoraproject.org/wiki/Changes/Atomic_Developer_Mode