[zerofree] New upstream version 1.0.3.
Richard W.M. Jones
rjones at fedoraproject.org
Mon Nov 18 16:39:39 UTC 2013
commit 8f17ae2ce01567269615e748be9bfe71e05ff377
Author: Richard W.M. Jones <rjones at redhat.com>
Date: Mon Nov 18 16:36:03 2013 +0000
New upstream version 1.0.3.
- New upstream version of sparsify.c.
- Fix the license.
- Modernize the spec file.
- Remove sparsify patch (equivalent patch has gone upstream).
.gitignore | 2 +-
index.html | 43 ++++--
sources | 2 +-
sparsify-opaque-group-desc.patch | 14 --
sparsify.c | 271 ++++++++++++++++++++++++-------------
zerofree.spec | 30 ++---
6 files changed, 217 insertions(+), 145 deletions(-)
---
diff --git a/.gitignore b/.gitignore
index 021b6dd..dda2bde 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1 +1 @@
-zerofree-1.0.1.tgz
+/zerofree-*.tgz
diff --git a/index.html b/index.html
index 92e5b0d..c779213 100644
--- a/index.html
+++ b/index.html
@@ -56,10 +56,18 @@ zeroes.
As an alternative approach, and as practice in mucking about with ext2
filesystems, I've written a utility which scans the free blocks in an
ext2 filesystem and fills any non-zero blocks with zeroes.
-(The source, <a href="zerofree-1.0.1.tgz">zerofree-1.0.1.tgz</a>, is
-available for download.) The <code>zerofree</code> utility is faster
-than <code>dd</code>, especially when the filesystem is already partly sparse.
-The filesystem to be processed should be unmounted or mounted read-only.
+The source, <a href="zerofree-1.0.3.tgz">zerofree-1.0.3.tgz</a>, is
+available for download. It's also available in a git repository:
+<code>git clone http://intgat.tigress.co.uk/rmy/git/zerofree.git</code>
+<p>
+<ul>
+<li>A cautious user would run fsck on the filesytem both before and after
+running zerofree.
+<li>The filesystem to be processed should be unmounted or mounted read-only.
+<li>The utility also works on ext3 or ext4 filesystems.
+<li>Binary packages are available in the standard repositories for Debian and Fedora.
+<li>The <a href="http://partedmagic.com">Parted Magic</a> live distribution includes zerofree.
+</ul>
<p>
Better than either of these would be to have the guest kernel keep the free
blocks empty. My original inspiration was the
@@ -81,15 +89,14 @@ with zeroes.
Remember, this extra work will hurt disk performance.
Note that the ext3 patch doesn't support data journalling
mode, so deleted metadata isn't zeroed. It also hasn't been tested
-as thoroughly as the patch for ext2.
+as thoroughly as the patch for ext2. And neither has been maintained for a
+very long time.
<p>
However, the above techniques are only half the story: the empty free
blocks still consume space in the underlying filesystem, so something
-must to be done to reclaim that space. One approach would be to
-implement a system call, like the legendary
-<a href="http://www.uwsg.iu.edu/hypermail/linux/kernel/0106.3/1180.html">
-sys_punch</a>, which could be used to write a utility to make any
-suitable file sparse.
+must to be done to reclaim that space. For filesystems and kernels that
+support it the fallocate call with <code>FALLOC_FL_PUNCH_HOLE</code> can
+be used to deallocate space in a file.
<p>
An existing alternative is to use the sparse file handling capabilities
of the GNU <code>cp</code> command to take a copy of the filesystem image with
@@ -103,10 +110,16 @@ additional disk space to work its magic, but it does require that the
filesystem containing the filesystem image is unmounted, which is just a
different sort of inconvenience.
<p>
-(The usual disclaimers apply: this worked for me
-when I tested it but it might destroy your data. Only use it on
-disposable filesystems, or have a full backup available. <code>e2fsck</code>
-is your friend.)
+<ul>
+<li>The usual disclaimers apply: this worked for me when I tested it but it
+might destroy your data.</li>
+<li>Versions of libext2fs prior to 1.41 do not support ext4;
+later versions should work with all filesystem types.</li>
+<li>In 1.41 the progress monitor may exceed 100% as the iterator visits some
+blocks more than once.</li>
+<li>The code doesn't support huge (>2TB) files. It will issue a warning and
+refuse to process them.</li>
+</ul>
<p>
As an example, suppose we have an unmounted filesystem
image, <code>fs.image</code>, in the directory <code>/data</code>, which is the
@@ -123,7 +136,7 @@ blocks and make it sparse like so:
<hr>
<address>
<a href="mailto:rmy at tigress.co.uk">Ron Yorston</a><br>
-29th July 2008
+9th August 2012
</address>
</body>
</html>
diff --git a/sources b/sources
index 2c5121c..38a03b9 100644
--- a/sources
+++ b/sources
@@ -1 +1 @@
-a8c772fdd134448f25ab4e7e12004595 zerofree-1.0.1.tgz
+7fffca9639a2acc9c889c49b3f94a0c6 zerofree-1.0.3.tgz
diff --git a/sparsify.c b/sparsify.c
index fbcd71f..4c52e00 100644
--- a/sparsify.c
+++ b/sparsify.c
@@ -1,10 +1,10 @@
/*
* sparsify - a tool to make files on an ext2 filesystem sparse
*
- * Copyright (C) 2004 R M Yorston
+ * Copyright (C) 2004-2012 R M Yorston
*
* This file may be redistributed under the terms of the GNU General Public
- * License.
+ * License, version 2.
*/
#include <ext2fs/ext2fs.h>
#include <stdio.h>
@@ -13,181 +13,262 @@
#define USAGE "usage: %s [-n] [-v] filesystem filename ...\n"
+/* initially assume pre-ext4 API version */
+#define API 140
+
+#if defined(BLOCK_FLAG_READ_ONLY)
+#undef API
+#define API 141
+#endif
+
+#if defined(EXT2_FLAG_64BITS)
+#undef API
+#define API 142
+#endif
+
struct process_data {
- struct ext2_inode *inode ;
unsigned char *buf;
- int dryrun ;
- int changed ;
-} ;
+ int verbose;
+ int dryrun;
+ blk_t count;
+ blk_t blocks;
+ blk_t total_blocks;
+ int old_percent;
+};
static int process(ext2_filsys fs, blk_t *blocknr, e2_blkcnt_t blockcnt,
blk_t ref_block, int ref_offset, void *priv)
{
- struct process_data *p ;
- errcode_t ret ;
- int i, group ;
+ struct process_data *p;
+ errcode_t errcode;
+ int i, group;
+ int ret = 0;
- p = (struct process_data *)priv ;
+ p = (struct process_data *)priv;
+ p->blocks++;
if ( blockcnt >= 0 ) {
- ret = io_channel_read_blk(fs->io, *blocknr, 1, p->buf);
- if ( ret ) {
- return BLOCK_ABORT ;
+ errcode = io_channel_read_blk(fs->io, *blocknr, 1, p->buf);
+ if ( errcode ) {
+ return BLOCK_ABORT;
}
for ( i=0; i < fs->blocksize; ++i ) {
if ( p->buf[i] ) {
- break ;
+ break;
}
}
- if ( i == fs->blocksize && !p->dryrun ) {
- ext2fs_unmark_block_bitmap(fs->block_map, *blocknr) ;
- group = ext2fs_group_of_blk(fs, *blocknr);
- fs->group_desc[group].bg_free_blocks_count++;
- fs->super->s_free_blocks_count++ ;
- /* the inode counts blocks of 512 bytes */
- p->inode->i_blocks -= fs->blocksize / 512 ;
- *blocknr = 0 ;
- /* direct blocks need to be zeroed in the inode */
- if ( blockcnt < EXT2_NDIR_BLOCKS ) {
- p->inode->i_block[blockcnt] = 0 ;
+ if ( i == fs->blocksize ) {
+ p->count++;
+
+ if ( !p->dryrun ) {
+ ext2fs_unmark_block_bitmap(fs->block_map, *blocknr);
+ group = ext2fs_group_of_blk(fs, *blocknr);
+#if API >= 142
+ ext2fs_bg_free_blocks_count_set(fs, group,
+ ext2fs_bg_free_blocks_count(fs, group)+1);
+ ext2fs_free_blocks_count_add(fs->super, (blk64_t)1);
+#else
+ fs->group_desc[group].bg_free_blocks_count++;
+ fs->super->s_free_blocks_count++;
+#endif
+#if API >= 141
+ ext2fs_group_desc_csum_set(fs, group);
+#endif
+ *blocknr = 0;
+ ret = BLOCK_CHANGED;
+ }
+ }
+
+ if ( p->verbose ) {
+ double percent;
+
+ percent = 100.0 * (double)p->blocks/(double)p->total_blocks;
+
+ if ( (int)(percent*10) != p->old_percent ) {
+ fprintf(stderr, "\r%4.1f%%", percent);
+ p->old_percent = (int)(percent*10);
}
- p->changed = 1 ;
- return BLOCK_CHANGED ;
}
}
- return 0 ;
+ return ret;
}
int main(int argc, char **argv)
{
- int verbose = 0 ;
- int dryrun = 0 ;
- errcode_t ret ;
- int flags ;
- int superblock = 0 ;
- int open_flags = EXT2_FLAG_RW ;
- int blocksize = 0 ;
- ext2_filsys current_fs = NULL;
- struct ext2_inode inode ;
- ext2_ino_t root, cwd, inum ;
- int i, c ;
- struct process_data pdata ;
+ int verbose = 0;
+ int dryrun = 0;
+ errcode_t ret;
+ int flags;
+ int superblock = 0;
+ int open_flags = EXT2_FLAG_RW;
+ int iter_flags = 0;
+ int blocksize = 0;
+ ext2_filsys fs = NULL;
+ struct ext2_inode inode;
+ ext2_ino_t root, cwd, inum;
+ int i, c;
+ struct process_data pdata;
while ( (c=getopt(argc, argv, "nv")) != -1 ) {
switch (c) {
case 'n' :
- dryrun = 1 ;
- break ;
+ dryrun = 1;
+#if defined(BLOCK_FLAG_READ_ONLY)
+ iter_flags |= BLOCK_FLAG_READ_ONLY;
+#endif
+ break;
case 'v' :
- verbose = 1 ;
- break ;
+ verbose = 1;
+ break;
default :
- fprintf(stderr, USAGE, argv[0]) ;
- return 1 ;
+ fprintf(stderr, USAGE, argv[0]);
+ return 1;
}
}
if ( argc < optind+2 ) {
- fprintf(stderr, USAGE, argv[0]) ;
- return 1 ;
+ fprintf(stderr, USAGE, argv[0]);
+ return 1;
}
- ret = ext2fs_check_if_mounted(argv[optind], &flags) ;
+ ret = ext2fs_check_if_mounted(argv[optind], &flags);
if ( ret ) {
fprintf(stderr, "%s: failed to determine filesystem mount state %s\n",
- argv[0], argv[optind]) ;
- return 1 ;
+ argv[0], argv[optind]);
+ return 1;
}
if ( flags & EXT2_MF_MOUNTED ) {
fprintf(stderr, "%s: filesystem %s is mounted\n",
- argv[0], argv[optind]) ;
- return 1 ;
+ argv[0], argv[optind]);
+ return 1;
}
ret = ext2fs_open(argv[optind], open_flags, superblock, blocksize,
- unix_io_manager, ¤t_fs);
+ unix_io_manager, &fs);
if ( ret ) {
fprintf(stderr, "%s: failed to open filesystem %s\n",
- argv[0], argv[optind]) ;
- return 1 ;
+ argv[0], argv[optind]);
+ return 1;
}
- pdata.buf = (unsigned char *)malloc(current_fs->blocksize) ;
+ pdata.buf = (unsigned char *)malloc(fs->blocksize);
if ( pdata.buf == NULL ) {
- fprintf(stderr, "%s: out of memory (surely not?)\n", argv[0]) ;
- return 1 ;
+ fprintf(stderr, "%s: out of memory (surely not?)\n", argv[0]);
+ return 1;
}
- ret = ext2fs_read_inode_bitmap(current_fs);
+ ret = ext2fs_read_inode_bitmap(fs);
if ( ret ) {
fprintf(stderr, "%s: error while reading inode bitmap\n", argv[0]);
- return 1 ;
+ return 1;
}
- ret = ext2fs_read_block_bitmap(current_fs);
+ ret = ext2fs_read_block_bitmap(fs);
if ( ret ) {
fprintf(stderr, "%s: error while reading block bitmap\n", argv[0]);
- return 1 ;
+ return 1;
}
- root = cwd = EXT2_ROOT_INO ;
+ root = cwd = EXT2_ROOT_INO;
for ( i=optind+1; i<argc; ++i ) {
- if ( verbose ) {
- printf("processing %s\n", argv[i]) ;
- }
-
- ret = ext2fs_namei(current_fs, root, cwd, argv[i], &inum) ;
+ ret = ext2fs_namei(fs, root, cwd, argv[i], &inum);
if ( ret ) {
- fprintf(stderr, "%s: failed to find file %s\n", argv[0], argv[i]) ;
- continue ;
+ fprintf(stderr, "%s: failed to find file %s\n", argv[0], argv[i]);
+ continue;
}
- ret = ext2fs_read_inode(current_fs, inum, &inode) ;
+ ret = ext2fs_read_inode(fs, inum, &inode);
if ( ret ) {
- fprintf(stderr, "%s: failed to open inode %d\n", argv[0], inum) ;
- continue ;
+ fprintf(stderr, "%s: failed to open inode %d\n", argv[0], inum);
+ continue;
}
if ( !ext2fs_inode_has_valid_blocks(&inode) ) {
- fprintf(stderr, "%s: file %s has no valid inodes\n", argv[0],
- argv[i]) ;
- continue ;
+ fprintf(stderr, "%s: file %s has no valid blocks\n", argv[0],
+ argv[i]);
+ continue;
+ }
+
+#if defined(EXT4_EXTENTS_FL) && API < 141
+ if ( inode.i_flags & EXT4_EXTENTS_FL ) {
+ fprintf(stderr, "%s: unable to process %s, it uses extents\n",
+ argv[0], argv[i]);
+ continue;
}
+#endif
- pdata.inode = &inode ;
- pdata.dryrun = dryrun ;
- pdata.changed = 0 ;
- ret = ext2fs_block_iterate2(current_fs, inum, BLOCK_FLAG_DATA_ONLY,
- NULL, process, &pdata) ;
+#if defined(EXT4_FEATURE_RO_COMPAT_HUGE_FILE) && defined(EXT4_HUGE_FILE_FL)
+ if ( (fs->super->s_feature_ro_compat & EXT4_FEATURE_RO_COMPAT_HUGE_FILE)
+ && (inode.i_flags & EXT4_HUGE_FILE_FL) ) {
+ fprintf(stderr, "%s: unable to process %s, it's huge\n",
+ argv[0], argv[i]);
+ continue;
+ }
+#endif
+
+ if ( verbose ) {
+ printf("processing %s\n", argv[i]);
+ }
+
+ pdata.verbose = verbose;
+ pdata.dryrun = dryrun;
+ pdata.count = pdata.blocks = 0;
+ pdata.total_blocks = inode.i_blocks/(fs->blocksize >> 9);
+ pdata.old_percent = 1000;
+ ret = ext2fs_block_iterate2(fs, inum, iter_flags, NULL,
+ process, &pdata);
if ( ret ) {
fprintf(stderr, "%s: failed to process file %s\n", argv[0],
- argv[i]) ;
- continue ;
+ argv[i]);
+ continue;
}
- if ( pdata.changed ) {
- ret = ext2fs_write_inode(current_fs, inum, &inode) ;
+ if ( pdata.count && !dryrun ) {
+ ext2fs_mark_bb_dirty(fs);
+ ext2fs_mark_super_dirty(fs);
+
+ ret = ext2fs_read_inode(fs, inum, &inode);
+ if ( ret ) {
+ fprintf(stderr, "%s: failed to open inode (%s)\n", argv[0],
+ argv[i]);
+ continue;
+ }
+
+#if API >= 141
+ ret = ext2fs_iblk_sub_blocks(fs, &inode, (blk64_t)pdata.count);
if ( ret ) {
- fprintf(stderr, "%s: failed to write inode data %s\n", argv[0],
- argv[i]) ;
- continue ;
+ fprintf(stderr, "%s: failed to update block count (%s)\n",
+ argv[0], argv[i]);
+ continue;
}
+#else
+ inode.i_blocks -= pdata.count * (fs->blocksize >> 9);
+#endif
- ext2fs_mark_bb_dirty(current_fs) ;
- ext2fs_mark_super_dirty(current_fs) ;
+ ret = ext2fs_write_inode(fs, inum, &inode);
+ if ( ret ) {
+ fprintf(stderr, "%s: failed to write inode (%s)\n",
+ argv[0], argv[i]);
+ continue;
+ }
+ }
+
+ if ( verbose ) {
+ printf("\r%d/%d/%d %s\n", pdata.count, pdata.blocks,
+ pdata.total_blocks, argv[i]);
}
}
- ret = ext2fs_close(current_fs) ;
+ ret = ext2fs_close(fs);
if ( ret ) {
- fprintf(stderr, "%s: error while closing filesystem\n", argv[0]) ;
- return 1 ;
+ fprintf(stderr, "%s: error while closing filesystem\n", argv[0]);
+ return 1;
}
- return 0 ;
+ return 0;
}
diff --git a/zerofree.spec b/zerofree.spec
index f7624a8..41a67b4 100644
--- a/zerofree.spec
+++ b/zerofree.spec
@@ -1,9 +1,10 @@
Summary: Utility to force unused ext2 inodes and blocks to zero
Name: zerofree
-Version: 1.0.1
-Release: 13%{?dist}
+Version: 1.0.3
+Release: 1%{?dist}
License: GPL+
-Group: System Environment/Libraries
+
+URL: http://intgat.tigress.co.uk/rmy/uml/
Source0: http://intgat.tigress.co.uk/rmy/uml/%{name}-%{version}.tgz
Source1: http://intgat.tigress.co.uk/rmy/uml/sparsify.c
@@ -16,16 +17,8 @@ Source2: http://intgat.tigress.co.uk/rmy/uml/index.html
Source3: zerofree.sgml
Source4: zerofree.8
-# e2fsprogs >= 1.42 prevents you from accessing the block group
-# descriptor struct directly.
-Patch1: sparsify-opaque-group-desc.patch
-
-URL: http://intgat.tigress.co.uk/rmy/uml/
-
BuildRequires: e2fsprogs-devel
-BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)
-
%description
zerofree is a utility to set unused filesystem inodes and blocks of an
@@ -44,7 +37,6 @@ should be careful.
%setup -q
cp -p %{SOURCE1} .
cp -p %{SOURCE2} .
-%patch1 -p1
%build
@@ -53,26 +45,26 @@ gcc $RPM_OPT_FLAGS sparsify.c -o sparsify -lext2fs
%install
-rm -rf $RPM_BUILD_ROOT
-
install -D -p -m 755 zerofree $RPM_BUILD_ROOT%{_sbindir}/zerofree
install -D -p -m 755 sparsify $RPM_BUILD_ROOT%{_sbindir}/sparsify
install -D -p -m 644 %{SOURCE4} $RPM_BUILD_ROOT%{_mandir}/man8/zerofree.8
%files
-%defattr(-,root,root,-)
%doc COPYING index.html
%{_sbindir}/zerofree
%{_sbindir}/sparsify
%{_mandir}/man8/zerofree.8*
-%clean
-rm -rf $RPM_BUILD_ROOT
-
-
%changelog
+* Mon Nov 18 2013 Richard W.M. Jones <rjones at redhat.com> - 1.0.3-1
+- New upstream version 1.0.3.
+- New upstream version of sparsify.c.
+- Fix the license.
+- Modernize the spec file.
+- Remove sparsify patch (equivalent patch has gone upstream).
+
* Sun Aug 04 2013 Fedora Release Engineering <rel-eng at lists.fedoraproject.org> - 1.0.1-13
- Rebuilt for https://fedoraproject.org/wiki/Fedora_20_Mass_Rebuild
More information about the scm-commits
mailing list