hi, i've to ask it again what i already asked a few times. is there any way to speed up mock build? here is a simple example when i build gstreamer-plugins-bad-free on rhel-6.2: ----------------------- 2012-05-30 00:45:24,367 - Mock Version: 1.1.22 2012-05-30 00:45:24,377 - State Changed: unpacking root cache 2012-05-30 00:45:27,610 - State Changed: cleaning yum metadata 2012-05-30 00:45:27,639 - State Changed: running yum 2012-05-30 00:45:30,914 - State Changed: unlock buildroot 2012-05-30 00:45:31,026 - State Changed: setup 2012-05-30 00:50:09,980 - State Changed: build 2012-05-30 00:52:34,094 - State Changed: end ----------------------- as you can see it takes about 5 minutes to setup the buildroot environment, while it takes 2.5 minutes to build it. so it's still 2 times more the extra stuff compared to the build. even in a case of a 'long' build package (some other package build less then a minutes).
disk space and cache size is not a problem.
any kind of tipp would be welcome. thanks in advance. regards.
On Qua, 2012-05-30 at 01:03 +0200, Farkas Levente wrote:
hi, i've to ask it again what i already asked a few times. is there any way to speed up mock build? here is a simple example when i build gstreamer-plugins-bad-free on rhel-6.2:
2012-05-30 00:45:24,367 - Mock Version: 1.1.22 2012-05-30 00:45:24,377 - State Changed: unpacking root cache 2012-05-30 00:45:27,610 - State Changed: cleaning yum metadata 2012-05-30 00:45:27,639 - State Changed: running yum 2012-05-30 00:45:30,914 - State Changed: unlock buildroot 2012-05-30 00:45:31,026 - State Changed: setup 2012-05-30 00:50:09,980 - State Changed: build 2012-05-30 00:52:34,094 - State Changed: end
as you can see it takes about 5 minutes to setup the buildroot environment, while it takes 2.5 minutes to build it. so it's still 2 times more the extra stuff compared to the build. even in a case of a 'long' build package (some other package build less then a minutes).
disk space and cache size is not a problem.
any kind of tipp would be welcome. thanks in advance. regards.
mock -r fedora-16-x86_64 --rebuild (...).src.rpm --no-clean
On 05/30/2012 12:03 AM, Farkas Levente wrote:
hi, i've to ask it again what i already asked a few times. is there any way to speed up mock build? here is a simple example when i build gstreamer-plugins-bad-free on rhel-6.2:
2012-05-30 00:45:24,367 - Mock Version: 1.1.22 2012-05-30 00:45:24,377 - State Changed: unpacking root cache 2012-05-30 00:45:27,610 - State Changed: cleaning yum metadata 2012-05-30 00:45:27,639 - State Changed: running yum 2012-05-30 00:45:30,914 - State Changed: unlock buildroot 2012-05-30 00:45:31,026 - State Changed: setup 2012-05-30 00:50:09,980 - State Changed: build 2012-05-30 00:52:34,094 - State Changed: end
as you can see it takes about 5 minutes to setup the buildroot environment, while it takes 2.5 minutes to build it. so it's still 2 times more the extra stuff compared to the build. even in a case of a 'long' build package (some other package build less then a minutes).
disk space and cache size is not a problem.
any kind of tipp would be welcome.
Are you using the tmpfs plugin? That improves things significantly if you can spare the RAM. Here's the mock config options I use:
config_opts['plugin_conf']['tmpfs_enable'] = True config_opts['plugin_conf']['tmpfs_opts']['required_ram_mb'] = 4096 config_opts['plugin_conf']['tmpfs_opts']['max_fs_size'] = '2048m'
Paul.
On 05/30/2012 10:48 AM, Paul Howarth wrote:
On 05/30/2012 12:03 AM, Farkas Levente wrote:
hi, i've to ask it again what i already asked a few times. is there any way to speed up mock build? here is a simple example when i build gstreamer-plugins-bad-free on rhel-6.2:
2012-05-30 00:45:24,367 - Mock Version: 1.1.22 2012-05-30 00:45:24,377 - State Changed: unpacking root cache 2012-05-30 00:45:27,610 - State Changed: cleaning yum metadata 2012-05-30 00:45:27,639 - State Changed: running yum 2012-05-30 00:45:30,914 - State Changed: unlock buildroot 2012-05-30 00:45:31,026 - State Changed: setup 2012-05-30 00:50:09,980 - State Changed: build 2012-05-30 00:52:34,094 - State Changed: end
as you can see it takes about 5 minutes to setup the buildroot environment, while it takes 2.5 minutes to build it. so it's still 2 times more the extra stuff compared to the build. even in a case of a 'long' build package (some other package build less then a minutes).
disk space and cache size is not a problem.
any kind of tipp would be welcome.
Are you using the tmpfs plugin? That improves things significantly if you can spare the RAM. Here's the mock config options I use:
config_opts['plugin_conf']['tmpfs_enable'] = True config_opts['plugin_conf']['tmpfs_opts']['required_ram_mb'] = 4096 config_opts['plugin_conf']['tmpfs_opts']['max_fs_size'] = '2048m'
we just plug 4GB more ram and test the above. unfortunately in this case eg thunderbird build will fail with "No space left on device". it seems mock not use this space very clever way. and it's rather hard to always check the build results and modify the conf...
On 05/30/2012 10:48 AM, Paul Howarth wrote:
Are you using the tmpfs plugin? That improves things significantly if you can spare the RAM. Here's the mock config options I use:
config_opts['plugin_conf']['tmpfs_enable'] = True config_opts['plugin_conf']['tmpfs_opts']['required_ram_mb'] = 4096 config_opts['plugin_conf']['tmpfs_opts']['max_fs_size'] = '2048m'
could you or someone explain me what does the: - required_ram_mb - max_fs_size means?
and what these value mean if i run two mock build with different configuration file? ie the above values are mock global or mock instant local values?
thanks in advance. regards.
On 06/14/2012 03:26 PM, Farkas Levente wrote:
On 05/30/2012 10:48 AM, Paul Howarth wrote:
Are you using the tmpfs plugin? That improves things significantly if you can spare the RAM. Here's the mock config options I use:
config_opts['plugin_conf']['tmpfs_enable'] = True config_opts['plugin_conf']['tmpfs_opts']['required_ram_mb'] = 4096 config_opts['plugin_conf']['tmpfs_opts']['max_fs_size'] = '2048m'
could you or someone explain me what does the:
- required_ram_mb
The build machine must have at least that much RAM to use the tmpfs plugin.
- max_fs_size
Limit for the size of the filesystem used for that instance's chroot. So if your buildroot for a given package exceeds that size, the build will fail. The default is half the system RAM I believe.
means?
and what these value mean if i run two mock build with different configuration file? ie the above values are mock global or mock instant local values?
If you have configurations with different 'required_ram_mb' values, one of those will run on a host with less memory than the other.
If you have multiple mock instances running together, each one would have its own tmpfs instance and you could end up using as much RAM as the sum of the 'max_fs_size' value for each concurrently-running instance.
That's my understanding anyway.
Paul.
On 06/14/2012 05:34 PM, Paul Howarth wrote:
On 06/14/2012 03:26 PM, Farkas Levente wrote:
On 05/30/2012 10:48 AM, Paul Howarth wrote:
Are you using the tmpfs plugin? That improves things significantly if you can spare the RAM. Here's the mock config options I use:
config_opts['plugin_conf']['tmpfs_enable'] = True config_opts['plugin_conf']['tmpfs_opts']['required_ram_mb'] = 4096 config_opts['plugin_conf']['tmpfs_opts']['max_fs_size'] = '2048m'
could you or someone explain me what does the:
- required_ram_mb
The build machine must have at least that much RAM to use the tmpfs plugin.
- max_fs_size
Limit for the size of the filesystem used for that instance's chroot. So if your buildroot for a given package exceeds that size, the build will fail. The default is half the system RAM I believe.
means?
and what these value mean if i run two mock build with different configuration file? ie the above values are mock global or mock instant local values?
If you have configurations with different 'required_ram_mb' values, one of those will run on a host with less memory than the other.
If you have multiple mock instances running together, each one would have its own tmpfs instance and you could end up using as much RAM as the sum of the 'max_fs_size' value for each concurrently-running instance.
That's my understanding anyway.
thanks.
to translate into to my setup. if i've 24gb ram in the compile machine, than - i've to set required_ram_mb=24576 or somewhere around ~24000 (probably in the site-defaults.cfg, since there is no reason to put into the configuration files). - and the sum of each concurrent running running mock instance's max_fs_size must be less then required_ram_mb?
On Thu, 14 Jun 2012 17:57:01 +0200 Farkas Levente lfarkas@lfarkas.org wrote:
On 06/14/2012 05:34 PM, Paul Howarth wrote:
On 06/14/2012 03:26 PM, Farkas Levente wrote:
On 05/30/2012 10:48 AM, Paul Howarth wrote:
Are you using the tmpfs plugin? That improves things significantly if you can spare the RAM. Here's the mock config options I use:
config_opts['plugin_conf']['tmpfs_enable'] = True config_opts['plugin_conf']['tmpfs_opts']['required_ram_mb'] = 4096 config_opts['plugin_conf']['tmpfs_opts']['max_fs_size'] = '2048m'
could you or someone explain me what does the:
- required_ram_mb
The build machine must have at least that much RAM to use the tmpfs plugin.
- max_fs_size
Limit for the size of the filesystem used for that instance's chroot. So if your buildroot for a given package exceeds that size, the build will fail. The default is half the system RAM I believe.
means?
and what these value mean if i run two mock build with different configuration file? ie the above values are mock global or mock instant local values?
If you have configurations with different 'required_ram_mb' values, one of those will run on a host with less memory than the other.
If you have multiple mock instances running together, each one would have its own tmpfs instance and you could end up using as much RAM as the sum of the 'max_fs_size' value for each concurrently-running instance.
That's my understanding anyway.
thanks.
to translate into to my setup. if i've 24gb ram in the compile machine, than
- i've to set required_ram_mb=24576 or somewhere around ~24000 (probably
in the site-defaults.cfg, since there is no reason to put into the configuration files).
- and the sum of each concurrent running running mock instance's
max_fs_size must be less then required_ram_mb?
I believe Paul got it right and I think you summarized correctly.
The thing to remember is that mock is using the tmpfs filesystem, which pulls from one pool of memory. You can't add up all the max_fs_size parameters and have them equal the amount of available memory, because you'll either swap yourself into the ground or start triggering the OOM killer (or both). You have to leave some pad of memory available for Linux and the already running processes. I don't have any concrete numbers for you but I'd say that you would want to use no more than two-thirds or three-quarters of available RAM for your tmpfs'es.
Clark
buildsys@lists.fedoraproject.org
-
Clark Williams -
Farkas Levente -
Paul Howarth -
Sérgio Basto