Speaking of nvidia and akmods, I just had an issue this morning. On my server I'm using nvidia proprietary driver from rpmfusion. Apparently it was updated (automatically) last night and when I tried to run experiments on the gpu this morning I got the dreaded version mismatch between the kernel driver and cuda library (or something).
I tried rebooting but that didn't fix the issue.
I tried running akmods manually but it didn't find anything to build. The man page for akmods was unhelpful, and looking in /lib/modules wasn't useful because you can't see the version numbers on the modules to see that everything is updated.
Finally I tried rebooting a 2nd time on the theory that the akmod got run and the driver built on the first reboot, but too late - the old driver had already loaded.
Surprise, that worked. So what's going on? It looks like akmod is run too late and 2 reboots are needed to get the new driver built and loaded?
I checked sudo systemctl status akmods ● akmods.service - Builds and install new kmods from akmod packages Loaded: loaded (/usr/lib/systemd/system/akmods.service; enabled; vendor preset: enabled) Active: active (exited) since Wed 2022-07-06 07:32:31 EDT; 28min ago Process: 1166 ExecStart=/usr/sbin/akmods --from-init (code=exited, status=0/SUCCESS) Main PID: 1166 (code=exited, status=0/SUCCESS) CPU: 215ms
Jul 06 07:32:29 nbecker8 systemd[1]: Starting Builds and install new kmods from akmod packages... Jul 06 07:32:31 nbecker8 akmods[1166]: Checking kmods exist for 5.18.9-100.fc35.x86_64[ OK ] Jul 06 07:32:31 nbecker8 systemd[1]: Finished Builds and install new kmods from akmod packages.
Any ideas?
On 06/07/2022 13:07, Neal Becker wrote:
Speaking of nvidia and akmods, I just had an issue this morning. On my server I'm using nvidia proprietary driver from rpmfusion. Apparently it was updated (automatically) last night and when I tried to run experiments on the gpu this morning I got the dreaded version mismatch between the kernel driver and cuda library (or something).
I tried rebooting but that didn't fix the issue.
I tried running akmods manually but it didn't find anything to build. The man page for akmods was unhelpful, and looking in /lib/modules wasn't useful because you can't see the version numbers on the modules to see that everything is updated.
Finally I tried rebooting a 2nd time on the theory that the akmod got run and the driver built on the first reboot, but too late - the old driver had already loaded.
Surprise, that worked. So what's going on? It looks like akmod is run too late and 2 reboots are needed to get the new driver built and loaded?
I checked sudo systemctl status akmods ● akmods.service - Builds and install new kmods from akmod packages Loaded: loaded (/usr/lib/systemd/system/akmods.service; enabled; vendor preset: enabled) Active: active (exited) since Wed 2022-07-06 07:32:31 EDT; 28min ago Process: 1166 ExecStart=/usr/sbin/akmods --from-init (code=exited, status=0/SUCCESS) Main PID: 1166 (code=exited, status=0/SUCCESS) CPU: 215ms
Jul 06 07:32:29 nbecker8 systemd[1]: Starting Builds and install new kmods from akmod packages... Jul 06 07:32:31 nbecker8 akmods[1166]: Checking kmods exist for 5.18.9-100.fc35.x86_64[ OK ] Jul 06 07:32:31 nbecker8 systemd[1]: Finished Builds and install new kmods from akmod packages.
Any ideas?
Well, I'm still on Fedora 35 and the 470xx nvidia driver from rpmfusion. Rebooting after a kernel upgrade is still giving nouveau and lower screen resolution than expected. sudo depmod -ae, followed by sudo systemctl reboot, has worked for me.
https://rpmfusion.org/CommonBugs
John P
From what I can tell the boot sequence on an nvidia and/or kernel update is this:
Machine boots up. Akmods begins building new driver and/or driver for new kernel. Available old driver loads (if new kernel the driver will be nouveau, if new nvidia this will be older nvidia) and Graphics start up. Akmods finishes and installs the nvidia module.
On the machine I have that uses the rpmfusion nvidia, I login and manually start up graphics, and I have been checking dmesg and waiting the 3-5 minutes it takes for akmods to finish building the module.. If I do not wait and immediately login(text console) and start graphics then I will get nouveau and lower screen resolutions.
Note modinfo <drivername> will show you the internal version number that is installed.
On Wed, Jul 6, 2022 at 7:27 AM Tom Horsley horsley1953@gmail.com wrote:
On Wed, 6 Jul 2022 08:07:10 -0400 Neal Becker wrote:
Any ideas?
You can examine /var/log/akmods/akmods.log to see the progress of the driver build and make sure the build finished before rebooting. _______________________________________________ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
On 7/6/2022 10:27 AM, Roger Heflin wrote:
From what I can tell the boot sequence on an nvidia and/or kernel update is this:
Machine boots up. Akmods begins building new driver and/or driver for new kernel. Available old driver loads (if new kernel the driver will be nouveau, if new nvidia this will be older nvidia) and Graphics start up. Akmods finishes and installs the nvidia module.
On the machine I have that uses the rpmfusion nvidia, I login and manually start up graphics, and I have been checking dmesg and waiting the 3-5 minutes it takes for akmods to finish building the module.. If I do not wait and immediately login(text console) and start graphics then I will get nouveau and lower screen resolutions.
Note modinfo <drivername> will show you the internal version number that is installed.
On Wed, Jul 6, 2022 at 7:27 AM Tom Horsley horsley1953@gmail.com wrote:
On Wed, 6 Jul 2022 08:07:10 -0400 Neal Becker wrote:
Any ideas?
You can examine /var/log/akmods/akmods.log to see the progress of the driver build and make sure the build finished before rebooting.
When I install a new kernel, a new nvidia is pulled in as a dependency, and akmod is rebuilt before I reboot. I need to wait long enough for this to finish. I am currently still running Fedora 35. Is this not what will happen with Fedora 36?
On Wed, 6 Jul 2022 11:04:01 -0400 Lester Petrie wrote:
I need to wait long enough for this to finish. I am currently still running Fedora 35. Is this not what will happen with Fedora 36?
That's what happens for me. The only difference is the log file moved, so if you "tail -f" to see when it is done, you have to find the log in a different place now (/var/log/akmods/akmods.log, used to be in /var/cache). If you reboot before it finishes, then it will start building it again when the system boots (but I never make it have to do that).
I manually do the update and reboot, and likely do not wait for all of the kernel module rebuilds to get done.
Likely others are doing similar since they are having similar issues.
On Wed, Jul 6, 2022 at 10:23 AM Tom Horsley horsley1953@gmail.com wrote:
On Wed, 6 Jul 2022 11:04:01 -0400 Lester Petrie wrote:
I need to wait long enough for this to finish. I am currently still running Fedora 35. Is this not what will happen with Fedora 36?
That's what happens for me. The only difference is the log file moved, so if you "tail -f" to see when it is done, you have to find the log in a different place now (/var/log/akmods/akmods.log, used to be in /var/cache). If you reboot before it finishes, then it will start building it again when the system boots (but I never make it have to do that). _______________________________________________ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Hi.
On Wed, 06 Jul 2022 11:05:54 -0500 Roger Heflin wrote:
I manually do the update and reboot, and likely do not wait for all of the kernel module rebuilds to get done.
To prevent that, you can enable the akmods-shutdown.service: it will build and install all the kmods for all the installed kernel during the shutdown.
This service does;
ExecStart=/bin/true ExecStop=-/usr/sbin/akmods-shutdown
On Wed, Jul 6, 2022, at 11:12 PM, Francis.Montagnac@inria.fr wrote:
Hi.
On Wed, 06 Jul 2022 11:05:54 -0500 Roger Heflin wrote:
I manually do the update and reboot, and likely do not wait for all of the kernel module rebuilds to get done.
To prevent that, you can enable the akmods-shutdown.service: it will build and install all the kmods for all the installed kernel during the shutdown.
This service does;
ExecStart=/bin/true ExecStop=-/usr/sbin/akmods-shutdown
If dnf has a kernel update I check for the modules with my 'kerncheck' alias:
kerncheck: aliased to ls -ltr /lib/modules/*/extra;echo;ls -ltr /lib/modules;echo;echo -n "Currently booted to: ";uname -r;echo
That format is for csh/tcsh.