auto unlock encrypted disks using clevis/tang works for ext4 but not btrfs?

[Barry Scott]

I have setup a tang server to offer up the unlock key for
by fedora systems that uses encrypted disks.

This works great with my file server that uses LVM and ext4.

But my desktop system that uses the btrfs does not unlock the
disk automatically. I see the logs on the tang server that show
that there are transactions to ask for the key but it does not work.

I'm jumping to the difference being btrfs, but admit that I'm far
from having evidence to show that is the problem.

I used the exact same setup steps for both systems so I'm reasonably
confident that the config is good.

Anyone else see this issue?

Barry

[Chris Murphy]

On Sun, Jun 5, 2022 at 2:28 PM Barry Scott  wrote:
> I have setup a tang server to offer up the unlock key for
> by fedora systems that uses encrypted disks.
>
> This works great with my file server that uses LVM and ext4.
>
> But my desktop system that uses the btrfs does not unlock the
> disk automatically. I see the logs on the tang server that show
> that there are transactions to ask for the key but it does not work.
>
> I'm jumping to the difference being btrfs, but admit that I'm far
> from having evidence to show that is the problem.
>
> I used the exact same setup steps for both systems so I'm reasonably
> confident that the config is good.
>
> Anyone else see this issue?

Need logs. And it might help to have the exact same binaries available
in A vs B config, i.e. LUKS LVM ext4 vs LUKS Btrfs, but both are
Fedora 36.

I don't know that much about tang or clevis, but my understanding is
the central aspect is `clevis luks unlock` and once the LUKS volume is
unlocked, then libblkid should see the btrfs volume on it, and then
possible to mount it.

While the LUKS volume is locked, the Btrfs volume in effect is
invisible (all ciphertext) so the fact it's btrfs is obscured and
can't be a factor until the LUKS volume is unlocked. So I'm thinking
unlock problems are unrelated to the fs selection, and it's some other
factor (package versions, network latency, race condition).

[Barry Scott]

> On 10 Jun 2022, at 04:05, Chris Murphy  wrote:
>
> Need logs. And it might help to have the exact same binaries available
> in A vs B config, i.e. LUKS LVM ext4 vs LUKS Btrfs, but both are
> Fedora 36.

Both systems are running the exact same binaries of all packages.
They are both running F36 and are updated within a couple of minutes of each other.

What logs do I need to collect?

I'm wondering if I need to turn on some kernel debug or verbose options to see
mot details of what is going on.

dmesg did not give me any clues, but I'm not sure what I need to look for.
Is there a log that shows that a key has been accepted and the luks volume unlocked?

Barry

[Gordon Messmer]

On 6/10/22 11:29, Barry Scott wrote:
> What logs do I need to collect?

I'd start by looking at "journalctl -b0"

You might be able to find clevis information in there.

Another interesting item might be to run "ls -l /boot/initramfs-$(uname 
-r).img > initramfs-$(hostname)" and to diff the two resulting files.

[Barry]

> On 10 Jun 2022, at 23:09, Gordon Messmer  wrote:
>
> On 6/10/22 11:29, Barry Scott wrote:
> > What logs do I need to collect?
>
> I'd start by looking at "journalctl -b0"
>
> You might be able to find clevis information in there.

Already did that and there is nothing that stands out.
> Another interesting item might be to run "ls -l /boot/initramfs-$(uname -r).img > initramfs-$(hostname)" and to diff the two resulting files.

Will do might give a clue. But I see both systems cause the tang server to log key lookups, so I think the init is setup.

I am planning the read the source of the Clovis pieces to see what they will log.

Barry
> users mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedoraproject.org/archives/list/[e-mail]
> Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure

[Barry Scott]

> On 11 Jun 2022, at 18:36, Barry  wrote:
> > On 10 Jun 2022, at 23:09, Gordon Messmer  wrote:
> >
> > On 6/10/22 11:29, Barry Scott wrote:
> > > What logs do I need to collect?
> >
> > I'd start by looking at "journalctl -b0"
> >
> > You might be able to find clevis information in there.
>
> Already did that and there is nothing that stands out.

I have been slowly reproducing the issue and checking logs.
I found an error from clevis - no network!

The initramfs does not contain any code to bring up the network.

On the system that works I use systemd-networkd and that is in
the initramfs.

But on the system that does not work its was using NetworkManager
and the initramfs has not network modules init.

I changed the system to use systemd-networkd and stopped using NetworkManager.
But dracut does not add systemd-networkd to the initramfs.

I spent some time trying to figure out why by adding echo statements to the dracut
code, but I'm no closer to understanding why systemd-networkd is not added.

What I did do is:

$ dracut -f --add systemd-networkd

and that makes clevis unlocking work.

I'll add a dracut conf file to force this on.

I think this is worth raising as a bugzilla and will do that.

Barry
> > Another interesting item might be to run "ls -l /boot/initramfs-$(uname -r).img > initramfs-$(hostname)" and to diff the two resulting files.
>
> Will do might give a clue. But I see both systems cause the tang server to log key lookups, so I think the init is setup.
> I am planning the read the source of the Clovis pieces to see what they will log.
>
> Barry
> > users mailing list -- [e-mail]
> > To unsubscribe send an email to [e-mail]
> > Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives: https://lists.fedoraproject.org/archives/list/[e-mail]
> > Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
>
> users mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedoraproject.org/archives/list/[e-mail]
> Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure