Hi,
My dnf seems broken. Below is what I get with dnf upgrade. I did dnf clean all, dnf clean metadata, I tested to add zchunk=False, I set back fastestmirror to false but nothing changes. I was able to upgrade a lot of packages and while it works for them, I get a lot of messages like: /sbin/ldconfig: File /lib64/libvisio-0.1.so.1.0.7 is empty, not checked.
Error: Transaction check error: file /usr/share/doc/vulkan-loader/CONTRIBUTING.md from install of vulkan-loader-1.1.114.0-1.fc30.i686 conflicts with file from package vulkan-loader-1.1.97.0-0.fc30.x86_64 file /usr/share/man/man1/pango-view.1.gz from install of pango-1.43.0-4.fc30.i686 conflicts with file from package pango-1.43.0-3.fc30.x86_64 file /usr/share/doc/libxcrypt/NEWS from install of libxcrypt-4.4.7-1.fc30.i686 conflicts with file from package libxcrypt-4.4.6-2.fc30.x86_64 file /usr/share/licenses/libxcrypt/LICENSING from install of libxcrypt-4.4.7-1.fc30.i686 conflicts with file from package libxcrypt-4.4.6-2.fc30.x86_64 file /usr/share/doc/libgomp/ChangeLog.bz2 from install of libgomp-9.2.1-1.fc30.i686 conflicts with file from package libgomp-9.1.1-1.fc30.x86_64 file /usr/share/locale/de/LC_MESSAGES/elfutils.mo from install of elfutils-libelf-0.177-1.fc30.i686 conflicts with file from package elfutils-libelf-0.176-3.fc30.x86_64 file /usr/share/locale/en@boldquot/LC_MESSAGES/elfutils.mo from install of elfutils-libelf-0.177-1.fc30.i686 conflicts with file from package elfutils-libelf-0.176-3.fc30.x86_64 file /usr/share/locale/en@quot/LC_MESSAGES/elfutils.mo from install of elfutils-libelf-0.177-1.fc30.i686 conflicts with file from package elfutils-libelf-0.176-3.fc30.x86_64 file /usr/share/locale/es/LC_MESSAGES/elfutils.mo from install of elfutils-libelf-0.177-1.fc30.i686 conflicts with file from package elfutils-libelf-0.176-3.fc30.x86_64 file /usr/share/locale/ja/LC_MESSAGES/elfutils.mo from install of elfutils-libelf-0.177-1.fc30.i686 conflicts with file from package elfutils-libelf-0.176-3.fc30.x86_64 file /usr/share/locale/pl/LC_MESSAGES/elfutils.mo from install of elfutils-libelf-0.177-1.fc30.i686 conflicts with file from package elfutils-libelf-0.176-3.fc30.x86_64 file /usr/share/locale/uk/LC_MESSAGES/elfutils.mo from install of elfutils-libelf-0.177-1.fc30.i686 conflicts with file from package elfutils-libelf-0.176-3.fc30.x86_64
Error Summary -------------
On 9/3/19 10:46 PM, Frédéric wrote:
My dnf seems broken. Below is what I get with dnf upgrade. I did dnf clean all, dnf clean metadata, I tested to add zchunk=False, I set back fastestmirror to false but nothing changes. I was able to upgrade a lot of packages and while it works for them, I get a lot of messages like: /sbin/ldconfig: File /lib64/libvisio-0.1.so.1.0.7 is empty, not checked.
This is a corrupt file. Try running "rpm -qV libvisio". If you have a lot of them, that sounds like you have a problem with your filesystem.
Error: Transaction check error: file /usr/share/doc/vulkan-loader/CONTRIBUTING.md from install of vulkan-loader-1.1.114.0-1.fc30.i686 conflicts with file from package vulkan-loader-1.1.97.0-0.fc30.x86_64
These are all multilib issues. For some reason you're getting a newer version of the 32-bit package, but not the 64-bit one. I have the .97 version installed and I see the .114 version available in updates for both.
Hi Samuel,
/sbin/ldconfig: File /lib64/libvisio-0.1.so.1.0.7 is empty, not checked.
This is a corrupt file. Try running "rpm -qV libvisio". If you have a lot of them, that sounds like you have a problem with your filesystem.
You were right. My SSD hard drive was corrupted. I bought a new one and reinstalled Fedora 30 on it.
Thanks,
F
On Sun, 2019-09-22 at 06:13 +0200, Fr??d??ric wrote:
My SSD hard drive was corrupted. I bought a new one and reinstalled Fedora 30 on it.
Check that the drive actually is faulty before discarding it. Filesystem corruptions can happen without it being the drive's fault.
Below I've just pasted a bit of an email:
On Sun, 2019-09-22 at 06:13 +0200, Fr??d??ric wrote:
It's from my reply to a message. That mangled name is supposed to be "Frederic" with acute accents above both letter "e"s. It was when I wrote it, and my mail client *CORRECTLY* sent it as UTF-8, but somewhere between then and my receiving it back, the message content has been scrambled by some broken email software that stuffed the character encoding up.
On 9/22/19 12:13 PM, Frédéric wrote:
Not answering the question.
Checking for Tim how my email is processed.
On 9/22/19 7:17 PM, Tim via users wrote:
Below I've just pasted a bit of an email:
On Sun, 2019-09-22 at 06:13 +0200, Fr??d??ric wrote:
It's from my reply to a message. That mangled name is supposed to be "Frederic" with acute accents above both letter "e"s. It was when I wrote it, and my mail client *CORRECTLY* sent it as UTF-8, but somewhere between then and my receiving it back, the message content has been scrambled by some broken email software that stuffed the character encoding up.
My reply showed up in the archives with Frédéric correctly shown.
So, I would blame yahoo. :-) :-)
On Sun, 2019-09-22 at 19:29 +0800, Ed Greshko wrote:
So, I would blame yahoo. :-) :-)
Ordinarily, I'd agree. But if I post directly through Yahoo (using their SMTP servers), and receive from a Yahoo address, international characters come through unmangled.
I've seen this kind of thing before, when I've posted here using Thunderbird on a Mac.
On 9/22/19 10:00 PM, Tim via users wrote:
On Sun, 2019-09-22 at 19:29 +0800, Ed Greshko wrote:
So, I would blame yahoo. :-) :-)
Ordinarily, I'd agree. But if I post directly through Yahoo (using their SMTP servers), and receive from a Yahoo address, international characters come through unmangled.
I've seen this kind of thing before, when I've posted here using Thunderbird on a Mac.
I see. I'm curious though. If you respond to this and keep the contents will Frédéric survive?
On Sun, 2019-09-22 at 22:09 +0800, Ed Greshko wrote:
On 9/22/19 10:00 PM, Tim via users wrote:
On Sun, 2019-09-22 at 19:29 +0800, Ed Greshko wrote:
So, I would blame yahoo. :-) :-)
Ordinarily, I'd agree. But if I post directly through Yahoo (using their SMTP servers), and receive from a Yahoo address, international characters come through unmangled.
I've seen this kind of thing before, when I've posted here using Thunderbird on a Mac.
I see. I'm curious though. If you respond to this and keep the contents will Fr??d??ric survive?
Very punny. ;-) Here's the reply.
On 9/22/19 10:41 PM, Tim via users wrote:
I see. I'm curious though. If you respond to this and keep the contents will Fr??d??ric survive?
Very punny.;-) Here's the reply.
And it did not.
I think you've said a yahoo to yahoo message is fine. Do you have a non-yahoo account you can send to directly to check?
On Sun, 2019-09-22 at 22:09 +0800, Ed Greshko wrote:
On 9/22/19 10:00 PM, Tim via users wrote:
On Sun, 2019-09-22 at 19:29 +0800, Ed Greshko wrote:
So, I would blame yahoo. :-) :-)
Ordinarily, I'd agree. But if I post directly through Yahoo (using their SMTP servers), and receive from a Yahoo address, international characters come through unmangled.
I've seen this kind of thing before, when I've posted here using Thunderbird on a Mac.
I see. I'm curious though. If you respond to this and keep the contents will Frédéric survive?
Just because I want to eliminate evolution as the culprit
On Sun, 2019-09-22 at 22:52 +0800, Ed Greshko wrote:
And it did not.
I noticed the same. On the messages that get corrupted, I've received them as base64 encoded (they weren't sent that way), though your messages with Frederic correctly accented was also base64 encodede. Most other messages coming from the list are just text. Do you get those messages as text or base64?
I think you've said a yahoo to yahoo message is fine. Do you have a non-yahoo account you can send to directly to check?
Yes, that works fine. It's only messages going through the list that I notice getting mangled.
On 9/23/19 12:04 AM, Tim via users wrote:
On Sun, 2019-09-22 at 22:52 +0800, Ed Greshko wrote:
And it did not.
I noticed the same. On the messages that get corrupted, I've received them as base64 encoded (they weren't sent that way), though your messages with Frederic correctly accented was also base64 encodede. Most other messages coming from the list are just text. Do you get those messages as text or base64?
Yes. For some reason I can't fathom it seems the mailing list software turns every Content-Transfer-Encoding to base64
When sending the initial message is sent from here as
Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit
I think you've said a yahoo to yahoo message is fine. Do you have a non-yahoo account you can send to directly to check?
Yes, that works fine. It's only messages going through the list that I notice getting mangled.
Weird. If it were something on the mailing list side you'd think that everyone would see the same issue.
On Mon, Sep 23, 2019 at 07:51:45AM +0800, Ed Greshko wrote:
On 9/23/19 12:04 AM, Tim via users wrote:
On Sun, 2019-09-22 at 22:52 +0800, Ed Greshko wrote:
And it did not.
I noticed the same. On the messages that get corrupted, I've received them as base64 encoded (they weren't sent that way), though your messages with Frederic correctly accented was also base64 encodede. Most other messages coming from the list are just text. Do you get those messages as text or base64?
Yes. For some reason I can't fathom it seems the mailing list software turns every Content-Transfer-Encoding to base64
When sending the initial message is sent from here as
Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit
I think you've said a yahoo to yahoo message is fine. Do you have a non-yahoo account you can send to directly to check?
Yes, that works fine. It's only messages going through the list that I notice getting mangled.
Weird. If it were something on the mailing list side you'd think that everyone would see the same issue.
FWIW, I find these headers in your posting on my system:
Content-Type: text/plain; charset="utf-8"; format="flowed" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by fcshome.stoneham.ma.us id x8MNrkId016512
On Sun, 22 Sep 2019 20:16:17 -0400 Fred Smith wrote:
FWIW, I find these headers in your posting on my system:
For what it is worth, I glanced at this thread over on the fedora mailing list archives (which isn't an easy thread to find over there because someone changed the subject line of an existing thread :-).
Some of the messages show the international characters, and some do not. It would be interesting to examine the detailed headers of the message where the characters disappeared, but that seems to be impossible with hyperkitty.
On 9/23/19 8:16 AM, Fred Smith wrote:
FWIW, I find these headers in your posting on my system:
Content-Type: text/plain; charset="utf-8"; format="flowed" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by fcshome.stoneham.ma.us id x8MNrkId016512
Your side "autoconverted" it back to what I find more reasonable.
I suppose there may still be a few non 8-bit capable MTA's out in the world. I've not not encountered one recently.
On Mon, 2019-09-23 at 07:51 +0800, Ed Greshko wrote:
For some reason I can't fathom it seems the mailing list software turns every Content-Transfer-Encoding to base64
Old fashioned stupidity? Thinking 7-bit email is still necessary?
It should still work as base64, but the encoding is doing it wrong. I can't force an encoding scheme when viewing the message, it's mangled to begin with, and just gets worse.
It's only messages going through the list that I notice getting mangled.
Weird. If it were something on the mailing list side you'd think that everyone would see the same issue.
Chances are that most people just ignore things like it.
On 9/23/19 1:36 PM, Tim via users wrote:
On Mon, 2019-09-23 at 07:51 +0800, Ed Greshko wrote:
For some reason I can't fathom it seems the mailing list software turns every Content-Transfer-Encoding to base64
Old fashioned stupidity? Thinking 7-bit email is still necessary?
It should still work as base64, but the encoding is doing it wrong. I can't force an encoding scheme when viewing the message, it's mangled to begin with, and just gets worse.
It's only messages going through the list that I notice getting mangled.
Weird. If it were something on the mailing list side you'd think that everyone would see the same issue.
Chances are that most people just ignore things like it.
Well, that doesn't explain why I can write 楊秀茵 or Frédéric and it will show up correctly and you cannot.
I wonder what happens if you did include 楊秀茵 in one of your messages.
Ed Greshko writes:
Yes. For some reason I can't fathom it seems the mailing list software turns every Content-Transfer-Encoding to base64
It's just "old fashioned stupidity" far beyond the ken of today's users inexperienced with the details of the global mail system. Let me explain.
Mailman needs to decode text/* parts of the message body to check for administrivia directed to the list, to decorate it (mostly to add a footer), and to manipulate MIME structure (quarantining .exes and the like). When it forwards to the MTA, it normally chooses the Content- Transfer-Encoding from the RFC-mandated ASCII and BASE64 otherwise. I am not familiar with the rules used by Mailman 3, offhand.
Mailman is not an MTA to negotiate SMTP options like 8BIT and UTF8, and doesn't know what any of the MTAs are, not even its own. It needs to be compatible with them all, and there may be thousands of instances of dozens of programs involved. So Mailman uses the least- common-denominator standard that can handle all messages, which is RFC 5322 and RFCs 2045-2049. (Fortunately it doesn't mess with addresses at all, so it doesn't need to know about IDNA and friends.) If the MTA wants to be smarter than that fine by us, but we have to handle everything the world spews out, so we reduce to the most compatible protocols we know.
There remains mail software in active use, such as fetchmail, which is still not RFC 6531-conforming.
Weird. If it were something on the mailing list side you'd think that everyone would see the same issue.
Agreed 100%. If messages were personalized (eg, a personal link to Postorius in the footer), I'd only be 99% confident (it's possible there are weird interactions between the charset of personalized material and that of the main text). But users@ doesn't personalize, and the footer is pure ASCII AFAICS, so that isn't it. Another possibility would be if the mail had text/html content type: Mailman delegates conversion to plain text to external software such as Lynx. But all the example messages were text/plain, so that's not it.
I'm 99% sure that this is not a Mailman issue, but rather a problem with MUAs that don't format the message correctly or don't interpret it correctly, or MTAs that translate inaccurately when converting (this might include spam or virus checkers, especially the latter which frequently modify message bodies). Nevertheless, "when the impossible is eliminated, what's left, however improbable, must be the truth." Feel free to contact mailman-developers@mail.python.org if it begins to seem more likely that Mailman is causing the problem somehow, maybe somebody's heard of something like this.
Steve
Tom Horsley writes:
Some of the messages show the international characters,
In this thread, those are Ed's.
and some do not.
Tim's.
It would be interesting to examine the detailed headers of the message where the characters disappeared,
I looked in my local folder, and there's nothing interesting the headers. Not surprising; the mail header is not very expressive. Archiving happens *after* message transformations, so I'm seeing the same headers you would in HyperKitty. We'd probably need to see the pre-Mailman header to identify any problems with Mailman via the header, but Mailman has never kept those. Perhaps if Tim sent a mail both to the list and to himself, comparing those headers might tell us something, but his messages are simple text/plain; charset=UTF-8, so I doubt it. I'm prety sure it has something to do with the Unicode encoding itself.
Ed sent his message in Unicode with NFC normalization (the é is pre-composed) and UTF-8. Tim's message contains two ?, indicating a pair of unknown characters. One possibility is that Tim's MUA (Evolution) converts that to NFD normalization and something in between chokes on that, and produces the doubled ?? instead of a single é. Renormalization is perfectly conformant to both Unicode and mail standards, but lots of software has issues with NFD. Mailman should not, since it doesn't need to interpret anything other than ASCII text, and passes anything else along (or deletes/quarantines whole MIME parts), and I've not heard of such problems (but Mailman 3 is a completely new code base, so it's possible a new issues has been introduced).
It's also possible that Tim's MUA double-UTF-8-encodes the é, which results in an illegal code point sequence which might also be represented as ??. Of course double-encoding is a bug, and if Mailman receives such email, it's quite likely that it would replace the broken text with ??. This seems highly unlikely, as Tim would be seeing issues all over the place, including in mail directly to himself, which he has tried without problem.
So most likely something between Tim's MUA and Mailman (the list manager, not HyperKitty) is mishandling the text, in one of the ways described above. I can't exclude either endpoint, but in both cases somebody should be seeing a lot of similar mojibake. Tim reports some, but not in direct to self, and I've never seen Mailman cause anything like this. More objectively, the fact that the ??s are in HyperKitty rather than some 8-bit mojibake strongly suggests that even if Mailman is directly responsible for the ??s, it was replacing existing mojibake with ??.
but that seems to be impossible with hyperkitty.
I believe there's work being done on more detailed archiving at GNU Mailman that might help diagnosing this kind of issue (not done yet though). I don't know if lists.fedoraproject is tracking us, though.
On 9/23/19 3:39 PM, Stephen J. Turnbull wrote:
Ed sent his message in Unicode with NFC normalization (the é is pre-composed) and UTF-8. Tim's message contains two ?, indicating a pair of unknown characters. One possibility is that Tim's MUA (Evolution) converts that to NFD normalization and something in between chokes on that, and produces the doubled ?? instead of a single é.
FWIW, I sent a message on Sun, 22 Sep 2019 23:22:44 +0800 using Evolution 3.34.0 (3.34.0-1.fc31) as I only have a F31 Beta GNOME VM
In that message Frédéric survived.
Check that the drive actually is faulty before discarding it. Filesystem corruptions can happen without it being the drive's fault.
I did force fsck but it did not seem to do anything or it is so quick that I did not see anything. Anyway to perform a deep check?
I put the disk in a USB3 box and bought a new one. I know that SSD disks loose memory with time. I wonder how this works and I thought that maybe if I reformat the disk, I will be able to use it again.
F
On 9/25/19 11:56 AM, Frédéric wrote:
Check that the drive actually is faulty before discarding it. Filesystem corruptions can happen without it being the drive's fault.
I did force fsck but it did not seem to do anything or it is so quick that I did not see anything. Anyway to perform a deep check?
I put the disk in a USB3 box and bought a new one. I know that SSD disks loose memory with time. I wonder how this works and I thought that maybe if I reformat the disk, I will be able to use it again.
I don't recall if you mentioned the model of SSD you have. I wonder if it has SMART capability.
What is the output of
sudo smartctl --all /dev/sdX
where X is the device identifier.
I put the disk in a USB3 box and bought a new one. I know that SSD disks loose memory with time. I wonder how this works and I thought that maybe if I reformat the disk, I will be able to use it again.
I don't recall if you mentioned the model of SSD you have. I wonder if it has SMART capability.
What is the output of
sudo smartctl --all /dev/sdX
where X is the device identifier
I put the disk in a USB3.1 box. It says: smartctl 7.0 2019-03-31 r4903 [x86_64-linux-5.2.13-200.fc30.x86_64] (local build) Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org /dev/sda: USB to NVMe bridge [please try '-d sntjmicron' and report result to: smartmontools-support@listi.jpberlin.de] Please specify device type with the -d option. Use smartctl -h to get a usage summary
The other new one gives much more information. I should probably try when plugged directly to the mother card.
F
On 9/29/19 12:33 PM, Frédéric wrote:
I put the disk in a USB3.1 box. It says: smartctl 7.0 2019-03-31 r4903 [x86_64-linux-5.2.13-200.fc30.x86_64] (local build) Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org /dev/sda: USB to NVMe bridge [please try '-d sntjmicron' and report result to: smartmontools-support@listi.jpberlin.de] Please specify device type with the -d option. Use smartctl -h to get a usage summary
The other new one gives much more information. I should probably try when plugged directly to the mother card.
smartctl doesn't usually work well when the drive is attached using USB.
On Sun, 29 Sep 2019 at 21:22, Samuel Sieb samuel@sieb.net wrote:
On 9/29/19 12:33 PM, Frédéric wrote:
I put the disk in a USB3.1 box. It says: smartctl 7.0 2019-03-31 r4903 [x86_64-linux-5.2.13-200.fc30.x86_64] (local build) Copyright (C) 2002-18, Bruce Allen, Christian Franke,
www.smartmontools.org
/dev/sda: USB to NVMe bridge [please try '-d sntjmicron' and report result to: smartmontools-support@listi.jpberlin.de] Please specify device type with the -d option. Use smartctl -h to get a usage summary
The other new one gives much more information. I should probably try when plugged directly to the mother card.
smartctl doesn't usually work well when the drive is attached using USB.
"Usually doesn't work" is a bit strong. With current hardware I'd say it usually does work with USB. See: https://www.smartmontools.org/wiki/USB for details. The big problem is that low-end vendors don't tell you what chipset they use, and sometimes the same model USB case is sold with a variety of chipsets, so you can't count on smartmon support even if it works with the last case you bought.
-
Ed Greshko -
Fred Smith -
Frédéric -
George N. White III -
Samuel Sieb -
Stephen J. Turnbull -
Tim -
Tom Horsley