[Fedora-livecd-list] Write cacheing on a live USB stick (was: Preventing access to /mnt/live (inter alia))

[James Heather]

Time to update you on my little experiments with write cacheing on a
live USB stick.

It does look as though I'd misunderstood what the settings
in /proc/sys/vm/ are for. I'd taken it that these just controlled paging
of virtual memory (hence the 'vm'), i.e., writing to swap. But it seems
that they also control the filesystem write cache. So I have stuck with
ext4, and I now have in my kickstart:


        %post
        cat >> /etc/rc.d/rc.local << EOF
        #more aggressive page writing in case of crash or power cut
        echo 500 > /proc/sys/vm/dirty_expire_centisecs
        EOF
        %end
        

which means that nothing is allowed to remain in the cache for more than
5 sec. (Strictly speaking, it is marked as dirty after 5 sec, and then
gets written at the next wakeup, which happens every 5 sec by default;
so in fact it can stay in the cache for up to 10 sec.)

Experiments suggest that this works nicely. I just wrote something like


        for ((;;)); do date; sleep 1; done | tee logfile
        

and then yanked the USB stick out and rebooted it to see how much had
been written; it wasn't ever more than a few seconds behind the terminal
window. The LED also flashed briefly every 5-10 sec, which is what you'd
hope.

This does seem more efficient than forcibly syncing every few seconds,
which will work, but will also block disk reads during the sync process.
It also means that the benefits of delayed allocation don't get seen.

Let's just hope nothing crashes during the real exam and it's all
irrelevant!

James

On Mon, 2010-03-08 at 08:07 +0000, James Heather wrote:

> On Sat, 2010-03-06 at 19:14 +0000, Mads Kiilerich wrote: 
> 
> > J.Heather at surrey.ac.uk wrote, On 03/06/2010 10:42 AM:
> > > The one thing left to sort is the write cache. One of the machines
> > > crashed, and one of them lost quite a lot of her work because it hadn't
> > > hit the usb stick by then.
> > >    
> > 
> > Launching something like
> >      (while sleep 1; do sync; done) &
> > somewhere might help.
> 
> 
> Well, I did wonder about that, or a cron job that runs once a minute.
> It's something of a kludge, though, and it might make the machine
> unresponsive.
> 
> There must surely be the equivalent of the dirty_writeback setting for
> vm paging? (That sets how long something can remain in the paging
> write cache before being written to disk.) It's an obvious thing to
> have.
> 
> James


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.fedoraproject.org/pipermail/livecd/attachments/20100312/95a478b3/attachment.html