[RFC/PATCH] cp: Add option to pre-allocate space for files

[Mark]

Hi,

Here's a patch for cp which adds a new --preallocate option. When
specified, cp allocates disk space for the destination file before writing
data. It uses fallocate() with FALLOC_FL_KEEP_SIZE on Linux, falling back
to posix_fallocate() if that fails.

Benefits of preallocation:
- Disk fragmentation can be greatly reduced. That means faster file
access and less filesystem overhead (fewer extents).
- Recovering data after filesystem corruption should be more successful,
since files are more likely to be contiguous.
- If you're e.g. copying a virtual machine disk image file, the
destination should be (almost) contiguous, meaning that running a disk
optimiser/defragmenter in the guest OS would work as it should (i.e.
improve performance).

This is a very preliminary patch for testing. Hopefully someone will find
it useful. And hopefully someone who (a) has a clue when it comes to C
programming, and (b) is familiar with the coreutils source (I'm neither)
can work from this to produce something which could be included in a
future release.

Note that posix_fallocate() sets the destination file size. If your system
doesn't support fallocate() with FALLOC_FL_KEEP_SIZE, you can't e.g. do
"ls -l destfilename" to monitor the progress of a large file copy; the
length shown will always be the final length.

Pre-allocating space can defeat the object of --sparse=always (or the
default sparse-checking heuristic). If copying files with large holes you
probably won't want to use --preallocate. If you do, regions in the
destination corresponding to holes in the source will be allocated but
unwritten. You'll lose the disk-space-saving benefit, but keep the
fast-reading-of-holes benefit. On the other hand, that feature could be
useful sometimes.

In the general case of copying non-sparse files, it should be beneficial
to use --preallocate. However on some systems, when the destination
filesystem does not support pre-allocation (e.g. FAT32), the
implementation of posix_fallocate() might try to fill the region to be
pre-allocated with zeros. That would double copy time for no benefit.

To-do list:
- Add --preallocate option to mv as well
- Should the option name be changed to --pre-allocate?
- Maybe have an option to tell cp to pre-allocate space for all
destination files in one go, rather than pre-allocating space for each
individual file before copying?
- Check the error code that fallocate() returns. If it says the
filesystem does not support fallocate(), don't call it again for every
other file being copied.
- Better handling of sparse files, e.g. don't call fallocate() if source
file is sparse and --sparse=always is given.
- If pre-allocation fails due to insufficient disk space, cp prints a
message and continues. So typically it will fill up the disk then abort
with an out-of-disk-space error. It would be nice to be able to tell cp
to abort when a pre-allocation fails, so it can exit without wasting
time.

The attached patch is based on coreutils 8.17.


-- Mark

[Pádraig Brady]

Post by Mark
Hi,
Here's a patch for cp which adds a new --preallocate option. When
specified, cp allocates disk space for the destination file before writing
data. It uses fallocate() with FALLOC_FL_KEEP_SIZE on Linux, falling back
to posix_fallocate() if that fails.

Thanks for taking the time to do this.
This feature is already under consideration.
See the comments at: http://bugs.gnu.org/9500

Post by Mark
- Disk fragmentation can be greatly reduced. That means faster file
access and less filesystem overhead (fewer extents).
- Recovering data after filesystem corruption should be more successful,
since files are more likely to be contiguous.
- If you're e.g. copying a virtual machine disk image file, the
destination should be (almost) contiguous, meaning that running a disk
optimiser/defragmenter in the guest OS would work as it should (i.e.
improve performance).
This is a very preliminary patch for testing. Hopefully someone will find
it useful. And hopefully someone who (a) has a clue when it comes to C
programming, and (b) is familiar with the coreutils source (I'm neither)
can work from this to produce something which could be included in a
future release.
Note that posix_fallocate() sets the destination file size. If your system
doesn't support fallocate() with FALLOC_FL_KEEP_SIZE, you can't e.g. do
"ls -l destfilename" to monitor the progress of a large file copy; the
length shown will always be the final length.
Pre-allocating space can defeat the object of --sparse=always (or the
default sparse-checking heuristic). If copying files with large holes you
probably won't want to use --preallocate. If you do, regions in the
destination corresponding to holes in the source will be allocated but
unwritten. You'll lose the disk-space-saving benefit, but keep the
fast-reading-of-holes benefit. On the other hand, that feature could be
useful sometimes.
In the general case of copying non-sparse files, it should be beneficial
to use --preallocate. However on some systems, when the destination
filesystem does not support pre-allocation (e.g. FAT32), the
implementation of posix_fallocate() might try to fill the region to be
pre-allocated with zeros. That would double copy time for no benefit.
- Add --preallocate option to mv as well
- Should the option name be changed to --pre-allocate?
- Maybe have an option to tell cp to pre-allocate space for all
destination files in one go, rather than pre-allocating space for each
individual file before copying?

I don't think there should be an option at all.
cp should have enough info to do the right thing.
Why would you even not want to preallocate?
In saying that, using fallocate with XFS triggers
alignment behavior that causes fragmentation.
But this might change, and the user can't be expected to know this.
BTW I'm thinking of adding a new FALLOC_FL_ALIGN flag
to the kernel, that XFS can use in its tools to enable that
separate functionality.

Post by Mark
- Check the error code that fallocate() returns. If it says the
filesystem does not support fallocate(), don't call it again for every
other file being copied.
- Better handling of sparse files, e.g. don't call fallocate() if source
file is sparse and --sparse=always is given.

That's an important consideration.

Post by Mark
- If pre-allocation fails due to insufficient disk space, cp prints a
message and continues. So typically it will fill up the disk then abort
with an out-of-disk-space error. It would be nice to be able to tell cp
to abort when a pre-allocation fails, so it can exit without wasting
time.

Yes it should exit immediately on ENOSPC

cheers,
Pádraig.

[Mark]

Hi,

Post by Pádraig Brady

Post by Mark
Here's a patch for cp which adds a new --preallocate option. When
specified, cp allocates disk space for the destination file before
...
- Add --preallocate option to mv as well
- Should the option name be changed to --pre-allocate?
- Maybe have an option to tell cp to pre-allocate space for all
destination files in one go, rather than pre-allocating space for each
individual file before copying?

I don't think there should be an option at all.
cp should have enough info to do the right thing.
Why would you even not want to preallocate?

Apart from the case I mentioned where posix_fallocate() might write out
zeros on some systems, I can't think of many cases where pre-allocating
wouldn't be a good idea. So perhaps at some point in the future it would
be the default, with a --no-preallocate option to disable.

Post by Pádraig Brady

Post by Mark
- Better handling of sparse files, e.g. don't call fallocate() if
source file is sparse and --sparse=always is given.

That's an important consideration.

Even for sparse files, if cp can determine where the holes are beforehand
(using SEEK_HOLE/SEEK_DATA or fiemap), it could pre-allocate the non-hole
regions. So with more work, pre-allocation could be used with sparse
files. Implementing that is beyond my abilities though...

It would still be helpful to allow pre-allocation to be disabled. (E.g.
source file has many written/allocated all-zero regions, which the user
wants to turn into holes in the destination file.)

Still another option: the ability to use FALLOC_FL_PUNCH_HOLE to punch
holes in (pre-allocated) destination files. Then any all-zero non-hole
regions in the source could be turned into holes in the destination in
conjunction with --sparse=always. By pre-allocating, cp would at least
guarantee the copy won't fail due to insufficient disk space, which may be
useful in some cases.

Post by Pádraig Brady

Post by Mark
- If pre-allocation fails due to insufficient disk space, cp prints a
message and continues. So typically it will fill up the disk then abort
with an out-of-disk-space error. It would be nice to be able to tell cp
to abort when a pre-allocation fails, so it can exit without wasting
time.

Yes it should exit immediately on ENOSPC

Aborting by default would make sense. Ideally that would be configurable.
In some cases the user might prefer to get a warning, so they can Ctrl-Z
suspend cp and delete/move other files to make space, instead of having cp
abort.


An interesting aside: I tried using cp to pre-allocate space for a very
large file on an ext4 partition, much larger than the amount of free
space. IMHO it would be best for the filesystem to fail immediately in
that case. ext4 does a lot of work (there was a lot of disk activity and
it took a long time to fail). ext4 pre-allocates as much of the requested
region as possible, rather than succeeding or failing all-or-nothing. So
you get a disk-full condition. (Of course that's no worse than what
happens when you run cp normally. But it would happen much more quickly
with pre-allocation.)

You can try that by doing something like this on an ext4 partition:
truncate -s 9999999999999 testfile # (create a sparse file larger than
free space)
cp --preallocate testfile testfile.copy


-- Mark

[Philipp Thomas]

Post by Pádraig Brady
This feature is already under consideration.
See the comments at: http://bugs.gnu.org/9500

FWIW, SUSE patches coreutils (patch attached) to use fallocate when
sparse=never or non-sparse files with sparse=auto (!make_holes in
copy.c(copy_reg). But as Eric wrote in the above thread, it would need a
gnulib module for a correct implementation.

BTW, biggest win is for ocfs2 as it must take cluster wide locks on
fragmentation, which was the reason we added the patch.

Philipp