I've seen several ISOs lately that after they were written to a thumb drive using media writer they wouldn't boot. I won't be recounting the details I sent in prior motes to @test, but here is a little more information.
The last one I tried was Workstation Live 0207.n.0. It failed to boot initially, but I rewrote the same downloaded image to the same thumb drive with Media Writer again. After that it would boot. This might raise the possibility that Media Writer is involved with the boot problems. I guess I'll just keep track of this from now on.
I have been using the same thumb drive plugged into the same USB port all along. Today just for grins I ran badblocks on the thumb drive and no bad blocks were found. "badblocks -w -s -o Thumberror.log /dev/sdb)"
Side note: With 0207 I once again encountered the white screen sad face. when I rebooted after the initial install to do the complete the install tasks. The complete the install windows popped up on top of the sad face and I was able to coomplete the install. I did a restart after completing the install. The restart ran normally and I have seen no problems. Though I havent tested 0207 extensively. I'll get started again when Branched F34 is available.
Have a Great Day!
Pat (tablepc)
To just maybe add to this, I have experienced the write/failure/rewrite cycle at time with media writer as well.
On Tue, 2021-02-09 at 12:23 -0500, pmkellly@frontier.com wrote:
I've seen several ISOs lately that after they were written to a thumb drive using media writer they wouldn't boot. I won't be recounting the details I sent in prior motes to @test, but here is a little more information.
The last one I tried was Workstation Live 0207.n.0. It failed to boot initially, but I rewrote the same downloaded image to the same thumb drive with Media Writer again. After that it would boot. This might raise the possibility that Media Writer is involved with the boot problems. I guess I'll just keep track of this from now on.
I have been using the same thumb drive plugged into the same USB port all along. Today just for grins I ran badblocks on the thumb drive and no bad blocks were found. "badblocks -w -s -o Thumberror.log /dev/sdb)"
Side note: With 0207 I once again encountered the white screen sad face. when I rebooted after the initial install to do the complete the install tasks. The complete the install windows popped up on top of the sad face and I was able to coomplete the install. I did a restart after completing the install. The restart ran normally and I have seen no problems. Though I havent tested 0207 extensively. I'll get started again when Branched F34 is available.
Have a Great Day!
Pat (tablepc) _______________________________________________ test mailing list -- test@lists.fedoraproject.org To unsubscribe send an email to test-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
On Tue, Feb 9, 2021 at 11:05 AM Stephen Snow s40w5s@gmail.com wrote:
To just maybe add to this, I have experienced the write/failure/rewrite cycle at time with media writer as well.
Curious because it does verify by re-reading the whole stick.
I haven't had the issue recently, but I also rotated out of use one of my older thumb drives because I suspected it was failing. I do remember tha last time I had it happen, after failure I deleted the image I had downloaded and went with the easy selection of workstation and everything was fine.
On Tue, 2021-02-09 at 11:20 -0700, Chris Murphy wrote:
On Tue, Feb 9, 2021 at 11:05 AM Stephen Snow s40w5s@gmail.com wrote:
To just maybe add to this, I have experienced the write/failure/rewrite cycle at time with media writer as well.
Curious because it does verify by re-reading the whole stick.
-- Chris Murphy _______________________________________________ test mailing list -- test@lists.fedoraproject.org To unsubscribe send an email to test-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
On Tue, Feb 9, 2021 at 10:23 AM pmkellly@frontier.com pmkellly@frontier.com wrote:
I've seen several ISOs lately that after they were written to a thumb drive using media writer they wouldn't boot. I won't be recounting the details I sent in prior motes to @test, but here is a little more information.
The last one I tried was Workstation Live 0207.n.0. It failed to boot initially, but I rewrote the same downloaded image to the same thumb drive with Media Writer again. After that it would boot. This might raise the possibility that Media Writer is involved with the boot problems. I guess I'll just keep track of this from now on.
I have been using the same thumb drive plugged into the same USB port all along. Today just for grins I ran badblocks on the thumb drive and no bad blocks were found. "badblocks -w -s -o Thumberror.log /dev/sdb)"
I would use f3. The gist is format the USB stick (file system doesn't matter) and mount it. Then
sudo f3write /mnt sudo f3read /mnt
But the thing is, transient errors from USB sticks is a real thing. Also, I once had a USB stick that would transiently corrupt on writes if the dd bs size was too high. I forget the value. But somehow it'd either lose writes or reorder them, and I'd get a completely bootable USB stick but it'd spew piles of file system errors during the installation.
https://github.com/AltraMayor/f3 https://fight-flash-fraud.readthedocs.io/en/latest/
Side note: With 0207 I once again encountered the white screen sad face. when I rebooted after the initial install to do the complete the install tasks. The complete the install windows popped up on top of the sad face and I was able to coomplete the install. I did a restart after completing the install. The restart ran normally and I have seen no problems. Though I havent tested 0207 extensively. I'll get started again when Branched F34 is available.
This could also be transient corruptions on read. USB sticks notoriously do not report discrete read errors, they just return bad data.
On 2/9/21 13:19, Chris Murphy wrote:
On Tue, Feb 9, 2021 at 10:23 AM pmkellly@frontier.com pmkellly@frontier.com wrote:
I've seen several ISOs lately that after they were written to a thumb drive using media writer they wouldn't boot. I won't be recounting the details I sent in prior motes to @test, but here is a little more information.
The last one I tried was Workstation Live 0207.n.0. It failed to boot initially, but I rewrote the same downloaded image to the same thumb drive with Media Writer again. After that it would boot. This might raise the possibility that Media Writer is involved with the boot problems. I guess I'll just keep track of this from now on.
I have been using the same thumb drive plugged into the same USB port all along. Today just for grins I ran badblocks on the thumb drive and no bad blocks were found. "badblocks -w -s -o Thumberror.log /dev/sdb)"
I would use f3. The gist is format the USB stick (file system doesn't matter) and mount it. Then
sudo f3write /mnt sudo f3read /mnt
Actually f3 is part of the Fedora repos now. I decided to use badblocks because I understand it's tests. It's the same basic tests I run on a new board prototype when there is ram or flash on the board. Besides testing the memory, it allows me to check the address and data buses too. Though this does not apply when testing a USB Flash decvice.
As I understand f3 it just writes files to the flash and reads them back checking for errors. I'll go ahead and try f3 on the same thumb drive.
I see that the file size is constant at 1.1GB except for the last one where it just uses up the left over space. There are 14 files of 1.1GB and 1 file of 5xxMB. so the size checked out. I'm wondering if they pick the file size based on the flash block size. That can make a difference testing flash. I checked out github to see if there is more info there. From what I read f3 seems to only verify size and basic functionality. I think badblocks with it's pattern testing is more likely to find problems. In any case f3read has finish running and pronounced my thumb drive I use for installs to be free of problems. The same result I got from badblocks.
But the thing is, transient errors from USB sticks is a real thing. Also, I once had a USB stick that would transiently corrupt on writes if the dd bs size was too high. I forget the value. But somehow it'd either lose writes or reorder them, and I'd get a completely bootable USB stick but it'd spew piles of file system errors during the installation.
https://github.com/AltraMayor/f3 https://fight-flash-fraud.readthedocs.io/en/latest/
Well I only use Flash from reputable manufacturers and though I never trust any single flash drive with anything important they have provided good results for me. Also, it's not clear that the flash drive is the problem in this case.
Way beyond flash drive fraud. Flash drives, especially the NOR types which seem to be the most common have a few error modes. Though some manufacturers advertise they have new designs good for millions of write cycles. Most of those available now only do about 100K cycles. Following my de-rating rules I never implement flash in any case where more than 50K cycles will be required. An install ISO puts a lot of data on the thumb drive. Then there are the wear leveling strategies to take into account to work out how all the spare blocks get used etc. Estimating the number of Live install cycles a thumb drive should be relied on will be hard and mostly guess work. Even so I would think they should be good for a thousand installs. This thumb drive only has about a hundred on it.
Another failure mode is that reads in adjacent cells can flip bits in other cells. They can also be sensitive to x-ray, and em fields depending on flux density.
We could go back to using DVDs for testing or perhaps use net install. I think I'd pass on both of those.
If it turned out that thumb drives were the issue. I guess we'd just have to change to a new thumb drive more frequently, but right now with the good test results I got on my thumb drive I don't think that will be necessary. I guess the two suspects for now are the ISO image's boot stuff, and Media Writer.
Side note: With 0207 I once again encountered the white screen sad face. when I rebooted after the initial install to do the complete the install tasks. The complete the install windows popped up on top of the sad face and I was able to coomplete the install. I did a restart after completing the install. The restart ran normally and I have seen no problems. Though I havent tested 0207 extensively. I'll get started again when Branched F34 is available.
This could also be transient corruptions on read. USB sticks notoriously do not report discrete read errors, they just return bad data.
Well I don't have my thumb drive plugged into my test machine when power up or down. Restarts do not change the power supply state. At least that's the case on my Lenovo machines. My work area is fully static protected. I really doubt the +5VDC supply in my test machine is producing any transients. There would be other highly observable effects. Though with some trouble I could get a scope and monitor it if someone believes that might be the trouble.
Have a Great Day!
Pat (tablepc)
On Tue, Feb 9, 2021 at 3:29 PM pmkellly@frontier.com pmkellly@frontier.com wrote:
On 2/9/21 13:19, Chris Murphy wrote:
On Tue, Feb 9, 2021 at 10:23 AM pmkellly@frontier.com pmkellly@frontier.com wrote:
I've seen several ISOs lately that after they were written to a thumb drive using media writer they wouldn't boot. I won't be recounting the details I sent in prior motes to @test, but here is a little more information.
The last one I tried was Workstation Live 0207.n.0. It failed to boot initially, but I rewrote the same downloaded image to the same thumb drive with Media Writer again. After that it would boot. This might raise the possibility that Media Writer is involved with the boot problems. I guess I'll just keep track of this from now on.
I have been using the same thumb drive plugged into the same USB port all along. Today just for grins I ran badblocks on the thumb drive and no bad blocks were found. "badblocks -w -s -o Thumberror.log /dev/sdb)"
I would use f3. The gist is format the USB stick (file system doesn't matter) and mount it. Then
sudo f3write /mnt sudo f3read /mnt
Actually f3 is part of the Fedora repos now. I decided to use badblocks because I understand it's tests. It's the same basic tests I run on a new board prototype when there is ram or flash on the board. Besides testing the memory, it allows me to check the address and data buses too. Though this does not apply when testing a USB Flash decvice.
As I understand f3 it just writes files to the flash and reads them back checking for errors. I'll go ahead and try f3 on the same thumb drive.
It can distinguish between different kinds of problems.
Well I only use Flash from reputable manufacturers and though I never trust any single flash drive with anything important they have provided good results for me. Also, it's not clear that the flash drive is the problem in this case.
The fake flash folks have gotten very good at faking seemingly legit hardware down to the packaging. It's not so much reputable brand as it is reputable reseller.
Way beyond flash drive fraud. Flash drives, especially the NOR types which seem to be the most common have a few error modes. Though some manufacturers advertise they have new designs good for millions of write cycles. Most of those available now only do about 100K cycles. Following my de-rating rules I never implement flash in any case where more than 50K cycles will be required. An install ISO puts a lot of data on the thumb drive. Then there are the wear leveling strategies to take into account to work out how all the spare blocks get used etc. Estimating the number of Live install cycles a thumb drive should be relied on will be hard and mostly guess work. Even so I would think they should be good for a thousand installs. This thumb drive only has about a hundred on it.
Well it's a bit needle in a haystack trying to track down transient boot problems.
Side note: With 0207 I once again encountered the white screen sad face. when I rebooted after the initial install to do the complete the install tasks. The complete the install windows popped up on top of the sad face and I was able to coomplete the install. I did a restart after completing the install. The restart ran normally and I have seen no problems. Though I havent tested 0207 extensively. I'll get started again when Branched F34 is available.
This could also be transient corruptions on read. USB sticks notoriously do not report discrete read errors, they just return bad data.
Well I don't have my thumb drive plugged into my test machine when power up or down. Restarts do not change the power supply state. At least that's the case on my Lenovo machines. My work area is fully static protected. I really doubt the +5VDC supply in my test machine is producing any transients. There would be other highly observable effects. Though with some trouble I could get a scope and monitor it if someone believes that might be the trouble.
Oh I thought you were booting from that same stick, but it had been imaged with a different ISO. If the media is failing, the manifestation of the failure can be different between uses.
On 2/9/21 18:11, Chris Murphy wrote:
On Tue, Feb 9, 2021 at 3:29 PM pmkellly@frontier.com pmkellly@frontier.com wrote:
As I understand f3 it just writes files to the flash and reads them back checking for errors. I'll go ahead and try f3 on the same thumb drive.
It can distinguish between different kinds of problems.
I have no doubt, but the granularity of detecting faults on a formatted and mounted storage device by writing files must be less than a pattern test that does not use any preexisting disk data structure and the disk is even dismounted when the test is run.
Well I only use Flash from reputable manufacturers and though I never trust any single flash drive with anything important they have provided good results for me. Also, it's not clear that the flash drive is the problem in this case.
The fake flash folks have gotten very good at faking seemingly legit hardware down to the packaging. It's not so much reputable brand as it is reputable reseller.
Yes, I fully understand and I always buy from the same reputable retailer. The good news is that the thumb drive I use for installing images for test passed both the f3 tests and the badblocks test.
Way beyond flash drive fraud. Flash drives, especially the NOR types which seem to be the most common have a few error modes. Though some manufacturers advertise they have new designs good for millions of write cycles. Most of those available now only do about 100K cycles. Following my de-rating rules I never implement flash in any case where more than 50K cycles will be required. An install ISO puts a lot of data on the thumb drive. Then there are the wear leveling strategies to take into account to work out how all the spare blocks get used etc. Estimating the number of Live install cycles a thumb drive should be relied on will be hard and mostly guess work. Even so I would think they should be good for a thousand installs. This thumb drive only has about a hundred on it.
Well it's a bit needle in a haystack trying to track down transient boot problems.
Well I certainly understand the problems with flash. However I don't think they are quite as bad as that.
Side note: With 0207 I once again encountered the white screen sad face. when I rebooted after the initial install to do the complete the install tasks. The complete the install windows popped up on top of the sad face and I was able to coomplete the install. I did a restart after completing the install. The restart ran normally and I have seen no problems. Though I havent tested 0207 extensively. I'll get started again when Branched F34 is available.
This could also be transient corruptions on read. USB sticks notoriously do not report discrete read errors, they just return bad data.
Well I don't have my thumb drive plugged into my test machine when power up or down. Restarts do not change the power supply state. At least that's the case on my Lenovo machines. My work area is fully static protected. I really doubt the +5VDC supply in my test machine is producing any transients. There would be other highly observable effects. Though with some trouble I could get a scope and monitor it if someone believes that might be the trouble.
Oh I thought you were booting from that same stick, but it had been imaged with a different ISO. If the media is failing, the manifestation of the failure can be different between uses.
I have one machine I use for testing (Lenovo M83 with i5-4570). I always use that machine and I have been using the same thumb drive (Sandisk 16GB USB 3). As I said before I would guess that it has about 100 Fedora Workstation Live install cycles on it. That's all that thumb drive gets used for. There are no power cycles involved during an install. I do a restart to get from the currently installed version on the system to running from the new version on the thumb drive. When the Live part of the install is complete, I use a restart to get from running Live to running the newly installed version on the hard drive to complete the install. I just pull out the thumb drive after Live is done and the machine is getting ready to start BIOS/UEFI for the boot. My clue is the monitor indicates that BIOS/UEFI has found the monitor but has not found the disk drives yet. This is the procedure I have used since we started using thumb drives with no issues. Come to think of it I used the same procedure back when we used DVDs and again with no problems.
I have new thumb drives of the same make and model on hand. If I find time, I will retrain myself to use "dd" in the CLI to write the ISO to the thumb drive. I seem to recall it can be used for that. As long as I see these failures to boot I will keep working on it to see if I can come to a conclusion as if I need to change thumb drives more often, report a bug against Media Writer, or report a bug against the ISO images.
A couple questions if I may. I've never taken the opportunity to look into Media Writer. Is it just a GUI for dd or is it independent of dd? If it's just a GUI for dd I guess their wouldn't be much benefit from using dd directly. Second though I really doubt this would be the case, but is Media Writer actively involved in telling any of the start up, boot loader, etc lies you mentioned earlier, or are they just contained in the ISO and Media Writer is not actively involved in them
Have a Hreat Day!
Pat (tablepc)
On Tue, Feb 9, 2021 at 7:34 PM pmkellly@frontier.com pmkellly@frontier.com wrote:
On 2/9/21 18:11, Chris Murphy wrote:
On Tue, Feb 9, 2021 at 3:29 PM pmkellly@frontier.com pmkellly@frontier.com wrote:
As I understand f3 it just writes files to the flash and reads them back checking for errors. I'll go ahead and try f3 on the same thumb drive.
It can distinguish between different kinds of problems.
I have no doubt, but the granularity of detecting faults on a formatted and mounted storage device by writing files must be less than a pattern test that does not use any preexisting disk data structure and the disk is even dismounted when the test is run.
Probably true most of the time. But it depends on the defect.
It is true that there's some small metadata space that isn't tested because it's taken up by the file system itself but if that corrupts, chances are you'll see weird problems during the read. Or you can just use Btrfs which will unambiguously tell you of an error in metadata locations that aren't covered by the test.
Well it's a bit needle in a haystack trying to track down transient boot problems.
Well I certainly understand the problems with flash. However I don't think they are quite as bad as that.
This is about as many boot failures as I've seen on the test list in quite a while though.
On 2/9/21 2:29 PM, pmkellly@frontier.com wrote:
I see that the file size is constant at 1.1GB except for the last one where it just uses up the left over space. There are 14 files of 1.1GB and 1 file of 5xxMB. so the size checked out. I'm wondering if they pick the file size based on the flash block size. That can make a difference
Depending on what you're using to see the size, it's most likely a difference between GB and GiB.
$ dd if=/dev/zero of=x.dat bs=1M count=1K 1024+0 records in 1024+0 records out 1073741824 bytes (1.1 GB, 1.0 GiB) copied
$ ls -l x.dat -rw-rw-r--. 1 samuel samuel 1073741824 Feb 9 21:27 x.dat $ ls -lh x.dat -rw-rw-r--. 1 samuel samuel 1.0G Feb 9 21:27 x.dat $ ls -lh --si x.dat -rw-rw-r--. 1 samuel samuel 1.1G Feb 9 21:27 x.dat
-
Chris Murphy -
pmkellly＠frontier.com -
Samuel Sieb -
Stephen Snow