fwupd broke my wifi on new laptop

[Neal Becker]

Brand new lenovo x1 carbon.

Today the "discover" sw update app suggest fw updates.  I went ahead with
them.  Now wifi isn't working properly.  At boot, I get a hang on
Job NetworkManager-wait-online.service/start running

Then it will finish boot after some time, but only 2.4G wifi works.  It
won't connect to my 5GHz wifi, although it is detected.

I tried an older kernel but got same results.

I just received this laptop yesterday, I suppose I could try to return it.
But is there maybe someway to revert the firmware updates?  Or any other
ideas?

Thanks,
Neal

[Jeffrey Walton]

On Tue, Dec 20, 2022 at 10:18 AM Neal Becker <ndbecker2(a)gmail.com&gt; wrote:
>  Brand new lenovo x1 carbon.
>
>  Today the "discover" sw update app suggest fw updates.  I went ahead with them.
>  Now wifi isn't working properly.  At boot, I get a hang on
>  Job NetworkManager-wait-online.service/start running 
Sorry to hear that. I usually find the opposite is true... Newer
machines need the UEFI updates to get wifi working correctly.

>  Then it will finish boot after some time, but only 2.4G wifi works. 
> It won't connect to my 5GHz wifi, although it is detected.
>
>  I tried an older kernel but got same results.
>
>  I just received this laptop yesterday, I suppose I could try to return it.  But is there
> maybe someway to revert the firmware updates?  Or any other ideas?
>  
I believe you can use fwupdmgr to remove the firmware update. I think
you want `fwupdmgr downgrade [DEVICE]`. I believe you can show the
devices with `fwupdmgr get-devices`. But for the downgrade, you may
get a numbered list, and only need to specify a number.

fwupdmgr is not well documented. The man pages suck, and there's not
much info out there on the web. The best questions/answers seem to
happen in the fwupdmgr bug reporter in GitHub. The next best set of
docs is from the --help option.

$ fwupdmgr --help
Usage:
  fwupdmgr [OPTION…]

  activate [DEVICE-ID|GUID]         Activate devices
  block-firmware [CHECKSUM]         Blocks a specific firmware from
being installed
  clear-results DEVICE-ID|GUID      Clears the results from the last update
  device-test [FILENAME1] [FILENAME2]
                                    Test a device using a JSON manifest
  disable-remote REMOTE-ID          Disables a given remote
  downgrade [DEVICE-ID|GUID]        Downgrades the firmware on a device
  download LOCATION                 Download a file
  enable-remote REMOTE-ID           Enables a given remote
  get-approved-firmware             Gets the list of approved firmware
  get-blocked-firmware              Gets the list of blocked firmware
  get-details FILE                  Gets details about a firmware file
  get-devices                       Get all devices that support
firmware updates
  get-history                       Show history of firmware updates
  get-plugins                       Get all enabled plugins registered
with the system
  get-releases [DEVICE-ID|GUID]     Gets the releases for a device
  get-remotes                       Gets the configured remotes
  get-results DEVICE-ID|GUID        Gets the results from the last update
  get-topology                      Alias to get-devices
  get-updates [DEVICE-ID|GUID]      Gets the list of updates for
connected hardware
  get-upgrades [DEVICE-ID|GUID]     Alias to get-updates
  install FILE [DEVICE-ID|GUID]     Install a firmware file on this hardware
  modify-config KEY,VALUE           Modifies a daemon configuration value
  modify-remote REMOTE-ID KEY VALUE
                                    Modifies a given remote
  refresh [FILE FILE_SIG REMOTE-ID]
                                    Refresh metadata from remote server
  reinstall [DEVICE-ID|GUID]        Reinstall current firmware on the device
  report-history                    Share firmware history with the developers
  security                          Gets the host security attributes
  set-approved-firmware FILENAME|CHECKSUM1[,CHECKSUM2][,CHECKSUM3]
                                    Sets the list of approved firmware
  switch-branch [DEVICE-ID|GUID] [BRANCH]
                                    Switch the firmware branch on the device
  sync-bkc                          Sync firmware versions to the host
best known configuration
  unblock-firmware [CHECKSUM]       Unblocks a specific firmware from
being installed
  unlock DEVICE-ID|GUID             Unlocks the device for firmware access
  update [DEVICE-ID|GUID]           Updates all specified devices to
latest firmware version, or all devices if unspecified
  upgrade [DEVICE-ID|GUID]          Alias to update
  verify [DEVICE-ID|GUID]           Checks cryptographic hash matches firmware
  verify-update [DEVICE-ID|GUID]    Update the stored cryptographic
hash with current ROM contents

Help Options:
  -h, --help                Show help options

Application Options:
  -v, --verbose             Show extra debugging information
  --version                 Show client and daemon versions
  --offline                 Schedule installation for next reboot when possible
  --allow-reinstall         Allow reinstalling existing firmware versions
  --allow-older             Allow downgrading firmware versions
  --allow-branch-switch     Allow switching firmware branch
  --force                   Force the action by relaxing some runtime checks
  -y, --assume-yes          Answer yes to all questions
  --sign                    Sign the uploaded data with the client certificate
  --no-unreported-check     Do not check for unreported history
  --no-metadata-check       Do not check for old metadata
  --no-remote-check         Do not check if download remotes should be enabled
  --no-reboot-check         Do not check or prompt for reboot after update
  --no-safety-check         Do not perform device safety checks
  --no-device-prompt        Do not prompt for devices
  --no-history              Do not write to the history database
  --show-all                Show all results
  --disable-ssl-strict      Ignore SSL strict checks when downloading files
  --ipfs                    Only use IPFS when downloading files
  --filter                  Filter with a set of device flags using a
~ prefix to exclude, e.g. 'internal,~needs-reboot'
  --json                    Output in JSON format

This tool allows an administrator to query and control the fwupd
daemon, allowing them to perform actions such as installing or
downgrading firmware.

[Neal Becker]

Well I performed 2 downgrades.  On the 2nd one it totally hung, saying 100%
done and will reboot.  It won't.  Even holding the power button does
nothing.

On Tue, Dec 20, 2022 at 10:27 AM Jeffrey Walton <noloader(a)gmail.com&gt; wrote:

>  On Tue, Dec 20, 2022 at 10:18 AM Neal Becker
> <ndbecker2(a)gmail.com&gt; wrote:
>  >
>  > Brand new lenovo x1 carbon.
>  >
>  > Today the "discover" sw update app suggest fw updates.  I went ahead
>  with them.  Now wifi isn't working properly.  At boot, I get a hang on
>  > Job NetworkManager-wait-online.service/start running
>
>  Sorry to hear that. I usually find the opposite is true... Newer
>  machines need the UEFI updates to get wifi working correctly.
>
>  > Then it will finish boot after some time, but only 2.4G wifi works.  It
>  won't connect to my 5GHz wifi, although it is detected.
>  >
>  > I tried an older kernel but got same results.
>  >
>  > I just received this laptop yesterday, I suppose I could try to return
>  it.  But is there maybe someway to revert the firmware updates?  Or any
>  other ideas?
>  >
>
>  I believe you can use fwupdmgr to remove the firmware update. I think
>  you want `fwupdmgr downgrade [DEVICE]`. I believe you can show the
>  devices with `fwupdmgr get-devices`. But for the downgrade, you may
>  get a numbered list, and only need to specify a number.
>
>  fwupdmgr is not well documented. The man pages suck, and there's not
>  much info out there on the web. The best questions/answers seem to
>  happen in the fwupdmgr bug reporter in GitHub. The next best set of
>  docs is from the --help option.
>
>  $ fwupdmgr --help
>  Usage:
>    fwupdmgr [OPTION…]
>
>    activate [DEVICE-ID|GUID]         Activate devices
>    block-firmware [CHECKSUM]         Blocks a specific firmware from
>  being installed
>    clear-results DEVICE-ID|GUID      Clears the results from the last update
>    device-test [FILENAME1] [FILENAME2]
>                                      Test a device using a JSON manifest
>    disable-remote REMOTE-ID          Disables a given remote
>    downgrade [DEVICE-ID|GUID]        Downgrades the firmware on a device
>    download LOCATION                 Download a file
>    enable-remote REMOTE-ID           Enables a given remote
>    get-approved-firmware             Gets the list of approved firmware
>    get-blocked-firmware              Gets the list of blocked firmware
>    get-details FILE                  Gets details about a firmware file
>    get-devices                       Get all devices that support
>  firmware updates
>    get-history                       Show history of firmware updates
>    get-plugins                       Get all enabled plugins registered
>  with the system
>    get-releases [DEVICE-ID|GUID]     Gets the releases for a device
>    get-remotes                       Gets the configured remotes
>    get-results DEVICE-ID|GUID        Gets the results from the last update
>    get-topology                      Alias to get-devices
>    get-updates [DEVICE-ID|GUID]      Gets the list of updates for
>  connected hardware
>    get-upgrades [DEVICE-ID|GUID]     Alias to get-updates
>    install FILE [DEVICE-ID|GUID]     Install a firmware file on this
>  hardware
>    modify-config KEY,VALUE           Modifies a daemon configuration value
>    modify-remote REMOTE-ID KEY VALUE
>                                      Modifies a given remote
>    refresh [FILE FILE_SIG REMOTE-ID]
>                                      Refresh metadata from remote server
>    reinstall [DEVICE-ID|GUID]        Reinstall current firmware on the
>  device
>    report-history                    Share firmware history with the
>  developers
>    security                          Gets the host security attributes
>    set-approved-firmware FILENAME|CHECKSUM1[,CHECKSUM2][,CHECKSUM3]
>                                      Sets the list of approved firmware
>    switch-branch [DEVICE-ID|GUID] [BRANCH]
>                                      Switch the firmware branch on the
>  device
>    sync-bkc                          Sync firmware versions to the host
>  best known configuration
>    unblock-firmware [CHECKSUM]       Unblocks a specific firmware from
>  being installed
>    unlock DEVICE-ID|GUID             Unlocks the device for firmware access
>    update [DEVICE-ID|GUID]           Updates all specified devices to
>  latest firmware version, or all devices if unspecified
>    upgrade [DEVICE-ID|GUID]          Alias to update
>    verify [DEVICE-ID|GUID]           Checks cryptographic hash matches
>  firmware
>    verify-update [DEVICE-ID|GUID]    Update the stored cryptographic
>  hash with current ROM contents
>
>  Help Options:
>    -h, --help                Show help options
>
>  Application Options:
>    -v, --verbose             Show extra debugging information
>    --version                 Show client and daemon versions
>    --offline                 Schedule installation for next reboot when
>  possible
>    --allow-reinstall         Allow reinstalling existing firmware versions
>    --allow-older             Allow downgrading firmware versions
>    --allow-branch-switch     Allow switching firmware branch
>    --force                   Force the action by relaxing some runtime
>  checks
>    -y, --assume-yes          Answer yes to all questions
>    --sign                    Sign the uploaded data with the client
>  certificate
>    --no-unreported-check     Do not check for unreported history
>    --no-metadata-check       Do not check for old metadata
>    --no-remote-check         Do not check if download remotes should be
>  enabled
>    --no-reboot-check         Do not check or prompt for reboot after update
>    --no-safety-check         Do not perform device safety checks
>    --no-device-prompt        Do not prompt for devices
>    --no-history              Do not write to the history database
>    --show-all                Show all results
>    --disable-ssl-strict      Ignore SSL strict checks when downloading files
>    --ipfs                    Only use IPFS when downloading files
>    --filter                  Filter with a set of device flags using a
>  ~ prefix to exclude, e.g. 'internal,~needs-reboot'
>    --json                    Output in JSON format
>
>  This tool allows an administrator to query and control the fwupd
>  daemon, allowing them to perform actions such as installing or
>  downgrading firmware.
>  _______________________________________________
>  users mailing list -- users(a)lists.fedoraproject.org
>  To unsubscribe send an email to users-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>  Do not reply to spam, report it:
>  https://pagure.io/fedora-infrastructure/new_issue
>  

-- 
*Those who don't understand recursion are doomed to repeat it*

[Richard Hughes]

On Tue, 20 Dec 2022 at 15:27, Jeffrey Walton <noloader(a)gmail.com&gt; wrote:
>  fwupdmgr is not well documented. The man pages suck 
Remember to be awesome. This is an open source project and the number
of people writing documentation is less than one.

Richard

[Neal Becker]

On Tue, Dec 20, 2022 at 12:25 PM Richard Hughes <hughsient(a)gmail.com&gt; wrote:

>  On Tue, 20 Dec 2022 at 15:27, Jeffrey Walton
> <noloader(a)gmail.com&gt; wrote:
>  > fwupdmgr is not well documented. The man pages suck
>
>  Remember to be awesome. This is an open source project and the number
>  of people writing documentation is less than one.
>
>  Richard
>  
I'm not familiar with pinhole reset.  I waited for battery to drain.  Then
powered up and booted fine!  And the offending firmware is reverted, and my
wifi is working again.

I don't know where to report the problem with the firmware update.

[Richard Hughes]

Can you file it here please https://github.com/fwupd/firmware-lenovo/issues
-- and then I'll tag the right people. Thanks R

On Wed, 21 Dec 2022, 12:12 Neal Becker, <ndbecker2(a)gmail.com&gt; wrote:

>  On Tue, Dec 20, 2022 at 12:25 PM Richard Hughes <hughsient(a)gmail.com&gt;
>  wrote:
>
> > On Tue, 20 Dec 2022 at 15:27, Jeffrey Walton <noloader(a)gmail.com&gt; wrote:
> > > fwupdmgr is not well documented. The man pages suck
> >
> > Remember to be awesome. This is an open source project and the number
> > of people writing documentation is less than one.
> >
> > Richard
> >
>  I'm not familiar with pinhole reset.  I waited for battery to drain.  Then
>  powered up and booted fine!  And the offending firmware is reverted, and my
>  wifi is working again.
>
>  I don't know where to report the problem with the firmware update.
>
>  _______________________________________________
>  users mailing list -- users(a)lists.fedoraproject.org
>  To unsubscribe send an email to users-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>  Do not reply to spam, report it:
>  https://pagure.io/fedora-infrastructure/new_issue
>

[Tim]

On Wed, 2022-12-21 at 07:11 -0500, Neal Becker wrote:
>  I'm not familiar with pinhole reset. 
Many devices have a tiny hole leading to a hidden reset button, that
you have to poke something through to reset them.  I wouldn't recommend
actually using a pin, though.  They can cause damage.

I have a packet of wooden kebab sticks that get used for all sorts of
things that have nothing to do with their original purpose, likewise
with toothpicks and paperclips.

>  I waited for battery to drain. 
Yet another reason why I don't like batteries sealed in units.

>  Then powered up and booted fine!  And the offending firmware is
>  reverted, and my wifi is working again. 
You're lucky that *that* firmware update is just a patch applied each
runtime, then.  If it had been and update that actually reflashed the
device, it'd be a permanent change.
 
-- 
 
uname -rsvp
Linux 3.10.0-1160.80.1.el7.x86_64 #1 SMP Tue Nov 8 15:48:59 UTC 2022 x86_64
 
Boilerplate:  All unexpected mail to my mailbox is automatically deleted.
I will only get to see the messages that are posted to the mailing list.

[Neal Becker]

>  After some more investigation I'm suspecting that this may have been 
unrelated
to the fw update. I've seen 2 times since where my phone has
failed trying to connect to the 5G until I rebooted the router. I did not
think the boot delay and message was caused by this because I did not think
the system connected to wifi at boot time, but now I think I was mistaken.
I see on the login screen that the wifi is already connected.

I will say that attempting fw downgrade is probably dangerous.  That's
probably true of all software as upgrades are usually tested, but not
downgrades.

Thanks for all your help!

[Tim]

On Thu, 2022-12-22 at 07:29 -0500, Neal Becker wrote:
>  After some more investigation I'm suspecting that this may have
> been
>  unrelated to the fw update. I've seen 2 times since where my phone
>  has failed trying to connect to the 5G until I rebooted the router. I
>  did not think the boot delay and message was caused by this because I
>  did not think the system connected to wifi at boot time, but now I
>  think I was mistaken. I see on the login screen that the wifi is
>  already connected. 
Router's do stuff up, so it's well worth rebooting a router when
networking goes haywire.  I dare say it's possible for a malfunctioning
client to do something that triggers some kind of stuff-up.  Even one
without bugs can get upset by power glitches.  They also have firmware
updates, and it's worth considering installing them.  

They are the thing between you and the internet, and many have security
shortcomings.  If it's an older one, it's also well worth considering
upgrading it and you find out it has security flaws and there's no
updated firmware for it.


>  I will say that attempting fw downgrade is probably dangerous. 
>  That's probably true of all software as upgrades are usually tested,
>  but not downgrades. 
Some firmware updates are just a patch that's run at every boot.  Don't
run the patch, and it's still running how it used to before hand.

Other firmware updates replace the firmware on the device.  Switching
back may not be possible.  You'd need a copy of the older firmware, and
you'd need the device to accept installing an older version.  Some
won't.

You could be lucky and have a device with enough RAM that it won't
actually install a replacement, but loads it as a second firmware.  But
I've only heard of that on motherboards.

-- 
 
uname -rsvp
Linux 3.10.0-1160.80.1.el7.x86_64 #1 SMP Tue Nov 8 15:48:59 UTC 2022 x86_64
 
Boilerplate:  All unexpected mail to my mailbox is automatically deleted.
I will only get to see the messages that are posted to the mailing list.

[Richard Hughes]

On Tue, 20 Dec 2022 at 15:18, Neal Becker <ndbecker2(a)gmail.com&gt; wrote:
>  But is there maybe someway to revert the firmware updates?  Or any
> other ideas? 
"fwupdmgr get-history" would be helpful in knowing what you updated,
and hopefully the older update is also there -- so "fwupdmgr
downgrade" would restore the old version. Either way, filing an issue
https://github.com/fwupd/firmware-lenovo would allow me to ping the
right person.

>  Even holding the power button does nothing. 
Hmm, I'm erring on on a hardware issue. If you do the pinhole reset
does that get things working again?

Richard