Dear folks,
My problem is that new file created at /usr won't be compressed.
I accenticly that none of my files under /usr is compressed[1], so I tried to run `sudo btrfs fi def -czstd -v -r /usr` to compress them, that seems to work. And `sudo compsize /usr` now gives Processed 431312 files, 224528 regular extents (230971 refs), 253758 inline. Type Perc Disk Usage Uncompressed Referenced TOTAL 56% 7.2G 12G 13G none 100% 3.5G 3.5G 3.5G zstd 39% 3.6G 9.2G 9.7G
This seems pretty good, but when I am testing dding to dump a file to /usr to test compress for new file, problem happens:
[root@yan-desktop /]# dd if=/dev/zero of=/usr/1 bs=10240 count=10000 记录了10000+0 的读入 记录了10000+0 的写出 102400000字节（102 MB，98 MiB）已复制，0.0426441 s，2.4 GB/s [root@yan-desktop /]# dd if=/dev/zero of=/etc/1 bs=10240 count=10000 记录了10000+0 的读入 记录了10000+0 的写出 102400000字节（102 MB，98 MiB）已复制，0.0585055 s，1.8 GB/s [root@yan-desktop /]# compsize /usr/1 Processed 1 file, 1 regular extents (1 refs), 0 inline. Type Perc Disk Usage Uncompressed Referenced TOTAL 100% 97M 97M 97M none 100% 97M 97M 97M [root@yan-desktop /]# compsize /etc/1 Processed 1 file, 782 regular extents (782 refs), 0 inline. Type Perc Disk Usage Uncompressed Referenced TOTAL 3% 3.0M 97M 97M zstd 3% 3.0M 97M 97M
Just ignore those Chinese output by dd, it doesn't important... But as you can see: file created at /usr/1 is not compressed at all while file created at /etc/1 gets compressed. There should not be any difficult to compress all-zero contents for zstd.
There should not be so much difference between /usr and /etc, they are in same subvolume named "root" mounted at /.
[root@yan-desktop /]# btrfs su li / ID 256 gen 1530662 top level 5 path home ID 257 gen 1530662 top level 5 path root ID 266 gen 1530614 top level 257 path var/lib/machines ID 292 gen 1530585 top level 257 path var/lib/mock ...[some container storage subvolumes]
and mount options for / is rw,noatime,seclabel,compress=zstd:3,ssd,space_cache=v2,subvolid=257,sub vol=/root (subvol=root,noatime,compress=zstd in fstab)
lsattr /|grep -E "var|etc" gives -------------------- /etc -------------------- /var
`btrfs property get /usr` or `btrfs property get /etc` gives empty output.
How do I troubleshoot this problem?
[1] That is also a problem since I setup compress after upgrading to fedora 34, and have done many package updates since then. So new extents written to /usr should be compressed while compsize reports none of them.
在 2021-05-11星期二的 22:57 +0800，Qiyu Yan写道：
Dear folks,
My problem is that new file created at /usr won't be compressed.
I accenticly that none of my files under /usr is compressed[1], so I tried to run `sudo btrfs fi def -czstd -v -r /usr` to compress them, that seems to work. And `sudo compsize /usr` now gives Processed 431312 files, 224528 regular extents (230971 refs), 253758 inline. Type Perc Disk Usage Uncompressed Referenced TOTAL 56% 7.2G 12G 13G none 100% 3.5G 3.5G 3.5G zstd 39% 3.6G 9.2G 9.7G
This seems pretty good, but when I am testing dding to dump a file to /usr to test compress for new file, problem happens:
[root@yan-desktop /]# dd if=/dev/zero of=/usr/1 bs=10240 count=10000 记录了10000+0 的读入 记录了10000+0 的写出 102400000字节（102 MB，98 MiB）已复制，0.0426441 s，2.4 GB/s [root@yan-desktop /]# dd if=/dev/zero of=/etc/1 bs=10240 count=10000 记录了10000+0 的读入 记录了10000+0 的写出 102400000字节（102 MB，98 MiB）已复制，0.0585055 s，1.8 GB/s [root@yan-desktop /]# compsize /usr/1 Processed 1 file, 1 regular extents (1 refs), 0 inline. Type Perc Disk Usage Uncompressed Referenced TOTAL 100% 97M 97M 97M none 100% 97M 97M 97M [root@yan-desktop /]# compsize /etc/1 Processed 1 file, 782 regular extents (782 refs), 0 inline. Type Perc Disk Usage Uncompressed Referenced TOTAL 3% 3.0M 97M 97M zstd 3% 3.0M 97M 97M
Just ignore those Chinese output by dd, it doesn't important... But as you can see: file created at /usr/1 is not compressed at all while file created at /etc/1 gets compressed. There should not be any difficult to compress all-zero contents for zstd.
There should not be so much difference between /usr and /etc, they are in same subvolume named "root" mounted at /.
[root@yan-desktop /]# btrfs su li / ID 256 gen 1530662 top level 5 path home ID 257 gen 1530662 top level 5 path root ID 266 gen 1530614 top level 257 path var/lib/machines ID 292 gen 1530585 top level 257 path var/lib/mock ...[some container storage subvolumes]
and mount options for / is rw,noatime,seclabel,compress=zstd:3,ssd,space_cache=v2,subvolid=257,s ub vol=/root (subvol=root,noatime,compress=zstd in fstab)
lsattr /|grep -E "var|etc" gives -------------------- /etc -------------------- /var
`btrfs property get /usr` or `btrfs property get /etc` gives empty output.
How do I troubleshoot this problem?
[1] That is also a problem since I setup compress after upgrading to fedora 34, and have done many package updates since then. So new extents written to /usr should be compressed while compsize reports none of them.
And by setting compress-force=zstd, new files in /usr will get compressed as expected. I suspect there to be something wrong with the compressible check process builtin in btrfs?
On Tue, May 11, 2021 at 8:58 AM Qiyu Yan yanqiyu@fedoraproject.org wrote:
Dear folks,
My problem is that new file created at /usr won't be compressed.
I accenticly that none of my files under /usr is compressed[1], so I tried to run `sudo btrfs fi def -czstd -v -r /usr` to compress them, that seems to work. And `sudo compsize /usr` now gives Processed 431312 files, 224528 regular extents (230971 refs), 253758 inline. Type Perc Disk Usage Uncompressed Referenced TOTAL 56% 7.2G 12G 13G none 100% 3.5G 3.5G 3.5G zstd 39% 3.6G 9.2G 9.7G
This seems pretty good, but when I am testing dding to dump a file to /usr to test compress for new file, problem happens:
[root@yan-desktop /]# dd if=/dev/zero of=/usr/1 bs=10240 count=10000 记录了10000+0 的读入 记录了10000+0 的写出 102400000字节（102 MB，98 MiB）已复制，0.0426441 s，2.4 GB/s [root@yan-desktop /]# dd if=/dev/zero of=/etc/1 bs=10240 count=10000 记录了10000+0 的读入 记录了10000+0 的写出 102400000字节（102 MB，98 MiB）已复制，0.0585055 s，1.8 GB/s [root@yan-desktop /]# compsize /usr/1 Processed 1 file, 1 regular extents (1 refs), 0 inline. Type Perc Disk Usage Uncompressed Referenced TOTAL 100% 97M 97M 97M none 100% 97M 97M 97M [root@yan-desktop /]# compsize /etc/1 Processed 1 file, 782 regular extents (782 refs), 0 inline. Type Perc Disk Usage Uncompressed Referenced TOTAL 3% 3.0M 97M 97M zstd 3% 3.0M 97M 97M
I'm unable to reproduce this.
Can you do:
ls -li
to list the inode number of a file in /usr that should be compress but isn't and then plug that inode number into
btrfs insp dump-t -t 257 /dev/xyz | grep -C 20 $INUM
This may expose file names for other files. Doesn't matter to me if you include the whole output of the above comment or trim to just the cluster of items referencing that inode number.
-
Chris Murphy -
Qiyu Yan