Hi all,
I have a cloned GCC git repository, on PC with 1.5GB of RAM and 3GB swap.
When I run "$git gc --aggressive", I will get after few hours an error:
$ git gc --aggressive Counting objects: 1332887, done. Delta compression using up to 2 threads. fatal: Out of memory, malloc failed (tried to allocate 4838335 bytes) error: failed to run repack $
4.5GB of memory is not enough, what is that?
I wonder what git is internally doing, it seems to me as a pretty non-optimal implementation. It is the first application I have which has been killed by OOM killer.
The size of .git directory is >900MB.
BR
Vaclav M.
On 03/30/2012 08:43 AM, Vaclav Mocek wrote:
Hi all,
I have a cloned GCC git repository, on PC with 1.5GB of RAM and 3GB swap.
When I run "$git gc --aggressive", I will get after few hours an error:
$ git gc --aggressive Counting objects: 1332887, done. Delta compression using up to 2 threads. fatal: Out of memory, malloc failed (tried to allocate 4838335 bytes) error: failed to run repack $
4.5GB of memory is not enough, what is that?
File a bug report
Rahul
Rahul Sundaram wrote:
On 03/30/2012 08:43 AM, Vaclav Mocek wrote:
Hi all,
I have a cloned GCC git repository, on PC with 1.5GB of RAM and 3GB swap.
When I run "$git gc --aggressive", I will get after few hours an error:
I'm curious why you're using git gc --aggressive on a freshly cloned repository. Any particular reason?
$ git gc --aggressive Counting objects: 1332887, done. Delta compression using up to 2 threads. fatal: Out of memory, malloc failed (tried to allocate 4838335 bytes) error: failed to run repack $
4.5GB of memory is not enough, what is that?
It's just a fact of the way --aggressive works by default. You can tune this a bit using the gc.aggressiveWindow config option. Generally, git gc --aggressive isn't something you want to run. Linus has said it...
is mostly dumb. It's really only useful for the case of "I know I have a *really* bad pack, and I want to throw away all the bad packing decisions I have done".¹
File a bug report
If you do, please use the git mailing list to report this. As a Fedora git maintainer, I can assure you that we won't be patching git gc --aggressive ourselves. And trying to ferry information back and forth to the git list for a bug like this would be painful and inefficient for everyone involved.
Ideally, you should probably experiment with git repack if you have a need to trim the size of the repository. Some discussion of this is available at the same URL referenced earlier with Linus' commends on git gc --aggressive not being all that useful.
¹ http://gcc.gnu.org/ml/gcc/2007-12/msg00165.html
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 03/30/2012 04:13 AM, Vaclav Mocek wrote:
Hi all,
I have a cloned GCC git repository, on PC with 1.5GB of RAM and 3GB swap.
When I run "$git gc --aggressive", I will get after few hours an error:
$ git gc --aggressive Counting objects: 1332887, done. Delta compression using up to 2 threads. fatal: Out of memory, malloc failed (tried to allocate 4838335 bytes) error: failed to run repack $
4.5GB of memory is not enough, what is that?
I wonder what git is internally doing, it seems to me as a pretty non-optimal implementation. It is the first application I have which has been killed by OOM killer.
Are you sure it got oom-killed? It appears to have just received a malloc failure and quit. You'd expect abnormal termination via a signal for an oomkill.
It would also be interesting to see the /proc/<pid>/{s,}maps output or even a top snapshot of the git process before this happens - the failing allocation was only for a little over 4MiB.
You might also get better results trying git-repack or git-fsck on your tree first (git-repack on a cluttered repo can speed things up greatly although I don't know if you may run into similar memory consumption problems there).
With that said I'm not sure the behaviour is that out of line considering the description of --aggressive:
--aggressive Usually git gc runs very quickly while providing good disk space utilization and performance. This option will cause git gc to more aggressively optimize the repository at the expense of taking much more time. The effects of this optimization are persistent, so this option only needs to be used occasionally; every few hundred changesets or so.
Regards, Bryn.
-
Bryn M. Reeves -
Rahul Sundaram -
Todd Zullinger -
Vaclav Mocek