On 05/05/2011 08:21 AM, James Laska wrote:
On Thu, 2011-05-05 at 07:34 -0600, Tim Flink wrote:
On 05/05/2011 07:24 AM, James Laska wrote:
Hey gang,
Instead of resolving this issue manually by removing older test results, I thought others would might have some better ideas for resolving the immediate issue of no free disk space.
I'm checking in with infrastructure to see if there are any free disks that can be assigned to our guest to alleviate matters. In the meantime, any recommendations to resolve this issue in the short-term (today/tomorrow) so we can get jobs processing again?
Do bad ideas count? What about setting up an NFS share on the staging hardware that we haven't set up yet for an additional ~130G of space?
I'll include that one when I get a hold of someone from infrastructure.
Otherwise, we could gzip large logs with a script that runs every couple of hours. A qnd hack and bit of a PITA for our users but better than no logs at all.
What is our long term solution for this. I remember hearing a couple of different things but wasn't sure what we were going forward with.
Fix the tests ... imo, we shouldn't have any tests with result directories larger than 100M. Heck ... 20M still seems high.
Agreed. Not sure if this would be done today or tomorrow, though. Will start looking into it.
Propose a patch for autotest (or leverage a callback if available) to cleanup logs when the job completes.
That's an interesting idea, hadn't thought of anything like that. A patch for autotest might be a better idea than a cron script for the medium term.
Get more disk for our production server.
I was more wondering what our plans were for getting more disk space. We'll probably end up using any extra space that we do get, so a log retention policy will be needed at some point but that's a bit farther down the road since we don't seem to be able to store more than a day's worth of logs right now.
Tim