Multi-Threaded make

[Gabriel VLASIU]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, 28 Nov 2012, JD wrote:

> Date: Wed, 28 Nov 2012 15:37:54 -0700
> From: JD <jd1008 at gmail.com>
> To: Community support for Fedora users <users at lists.fedoraproject.org>
> User-Agent: Mozilla/5.0 (X11; Linux i686;
>     rv:15.0) Gecko/20120911 Thunderbird/15.0.1
> X-Spam-Status: No, score=-4.7 required=5.0 tests=AWL,BAYES_00,
>     DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,FREEMAIL_FROM,RCVD_IN_DNSWL_MED,
>     RP_MATCHES_RCVD,SPF_PASS,T_DKIM_INVALID autolearn=ham version=3.3.1
> Subject: Multi-Threaded make
> 
> While building the kernel from source RPM, I run
> the command ps -ef | grep make in another terminal,
> and I do see that 4 make processes are running, with
> each one building objects for a different kernel source
> directory.
> 
> However, something else seems to be single threaded
> which is causing the huge delay in finishing kernel build.
> 
> I found that when I run top to see the upper-most
> cpu consumers, I see only one cc1 process at the top.
> 
> So I ran ps again to see how many cc1 processes there
> are.
> 
> Always - just one cc1 process.
> 
> I would expect to see anywhere from 1 to 4 cc1 processes
> at a time. But that is not the case.
> 
> What is forcing only a single instance of cc1 at a time?
> 
> I hope the devs can have a look at either the compiler
> tools or the make util.

make is invoked from kernel.spec file like this:

make -s ARCH=$Arch V=1 %{?_smp_mflags} $MakeTarget %{?sparse_mflags}
make -s ARCH=$Arch V=1 %{?_smp_mflags} modules %{?sparse_mflags} || exit 1

where %{?_smp_mflags} is:
1 core : -j 1
2 cores: -j 2
4 cores: -j 4
...

You can see multiple make proccesses. But only 2 (in my case) do the real 
work:
user1  22073 12810  0 13:13 pts/0    00:00:00 make -f scripts/Makefile.build obj=fs
user1  24474 22073  0 13:15 pts/0    00:00:00 make -f scripts/Makefile.build obj=fs/cifs
user1  24530 22073  0 13:15 pts/0    00:00:00 make -f scripts/Makefile.build obj=fs/coda

Make proccess with pid 22073 only invoke two addititonal make proccesses 
which call cc1:
user1  3617  3616  0 13:21 pts/0    00:00:00 /usr/libexec/gcc/x86_64-redhat-linux/4.7.2/cc1 -quiet -nostdinc ....
user1  3627  3626  0 13:21 pts/0    00:00:00 /usr/libexec/gcc/x86_64-redhat-linux/4.7.2/cc1 -quiet -nostdinc ...

One cc1 -> one core or you called make with -j 1.


Gabriel

- -- 

// Gabriel VLASIU
//
// OpenGPG-KeyID      : 44952F15
// OpenGPG-Fingerprint: 4AC5 7C26 2FE9 02DA 4906  24B2 D32B 7ED7 4495 2F15
// OpenGPG-URL        : http://www.vlasiu.net/public.key


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQIcBAEBAgAGBQJQt0XlAAoJENMrftdElS8VipoQAJ5/rHhGqhIlR7n42EYX7Pqm
D3TjolBfLwX7fIoG/7gwr8J3bTW31sdSZiNjlyYSy1k8z7uRRZhm9vficczFE9Po
l9uuvCQOmLOettrXVRw2dA/SFqCdb463upl2dD7b3/uc6sBx46mNyAePm3H8pY94
SgtByV3z7MqVlpdcTCGOtcIIbt7fwYNkL7JOkifMECaACZOCDTzyZW/oaf3f+JdO
kp6BZSW5sw8/mWq5p+nK949GSi35Ksf++Ibmjxa7K/OhG3cfmP0y1KpM/qBAxaAh
xe0GLSi4WgdvuBSo+DPGveeTPW4LZBsfRWI4NQo3mTzXeZkEf0DgvRc5tqA3U1hO
rpC65YMLGnts80MKiWqOlkb3vSxYUdmZ6Vs2+wNK4nmHEFvOxGhK9Ts5KeYGwLmG
22b+NMdSL8VlTMXi7JUn6JmAl4921f8twC52n7nemV9XLBiJud4m6Vj5B6vNyj41
mHVGW8k5sEkDI++IFBQnSEnDsxeYsYzlF9qXE8tRTDzU45DnNjlG25MkCh/kQp2G
BjkJchkPqZ40lhIOIt29OAV41hTu/7fPyLfyRKuEEJtbzaoJV5eRnCuhZFmZ1zDg
agWRongU/wIO47v0PP54WhqVHKkhqsEircuFhodbUfDDxoU3n24sDLVXiu84XmON
Tj3VakoDyGwoDfu0064U
=gExC
-----END PGP SIGNATURE-----