Presumed memory error on koji

[Antonio Trande]

Hi all.

I'm obtaining some (presumed) memory errors on some scratch builds of
Trilinos
(https://github.com/trilinos/Trilinos/issues/751#issuecomment-257428660),
the work-around looks be use a simple "make" command instead of
"parallel make":

{standard input}: Assembler messages:
{standard input}:1002829: Warning: end of file not at end of a line;
newline inserted
{standard input}: Error: .size expression for
_ZN6Galeri6Xpetra5Utils26CreateCartesianCoordinatesIdiiN6Xpetra3MapIiiN6Kokkos6Compat23KokkosDeviceWrapperNodeINS5_6OpenMPENS5_9HostSpaceEEEEENS3_11MultiVectorIdiiSA_EEEEN7Teuchos3RCPIT3_EERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKNSF_IKT2_EERNSE_13ParameterListE
does not evaluate to a constant
c++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report


Upstream developers are asking me how much memory is available on koji,
how much is it?

Regards.
-- 
---
Antonio Trande
mailto: sagitter 'at' fedoraproject 'dot' org
http://fedoraos.wordpress.com/
https://fedoraproject.org/wiki/User:Sagitter
GPG Key: 0x6CE6D08A
Check on https://keys.fedoraproject.org/

[Dan Horák]

On Thu, 3 Nov 2016 14:32:05 +0100
Antonio Trande <anto.trande(a)gmail.com&gt; wrote:

>  Hi all.
>  
>  I'm obtaining some (presumed) memory errors on some scratch builds of
>  Trilinos
>  (https://github.com/trilinos/Trilinos/issues/751#issuecomment-257428660),
>  the work-around looks be use a simple "make" command instead of
>  "parallel make":
>  
>  {standard input}: Assembler messages:
>  {standard input}:1002829: Warning: end of file not at end of a line;
>  newline inserted
>  {standard input}: Error: .size expression for
>
> _ZN6Galeri6Xpetra5Utils26CreateCartesianCoordinatesIdiiN6Xpetra3MapIiiN6Kokkos6Compat23KokkosDeviceWrapperNodeINS5_6OpenMPENS5_9HostSpaceEEEEENS3_11MultiVectorIdiiSA_EEEEN7Teuchos3RCPIT3_EERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKNSF_IKT2_EERNSE_13ParameterListE
>  does not evaluate to a constant
>  c++: internal compiler error: Killed (program cc1plus)
>  Please submit a full bug report 
yes, this is out of memory situation on the builder
 
>  
>  Upstream developers are asking me how much memory is available on
>  koji, how much is it? 
what arch is it?


		Dan

[Antonio Trande]

On 11/03/2016 02:44 PM, Dan Horák wrote:
>  On Thu, 3 Nov 2016 14:32:05 +0100
>  Antonio Trande <anto.trande(a)gmail.com&gt; wrote:
>  
> > Hi all.
> >
> > I'm obtaining some (presumed) memory errors on some scratch builds of
> > Trilinos
> > (https://github.com/trilinos/Trilinos/issues/751#issuecomment-257428660),
> > the work-around looks be use a simple "make" command instead of
> > "parallel make":
> >
> > {standard input}: Assembler messages:
> > {standard input}:1002829: Warning: end of file not at end of a line;
> > newline inserted
> > {standard input}: Error: .size expression for
> >
> _ZN6Galeri6Xpetra5Utils26CreateCartesianCoordinatesIdiiN6Xpetra3MapIiiN6Kokkos6Compat23KokkosDeviceWrapperNodeINS5_6OpenMPENS5_9HostSpaceEEEEENS3_11MultiVectorIdiiSA_EEEEN7Teuchos3RCPIT3_EERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKNSF_IKT2_EERNSE_13ParameterListE
> > does not evaluate to a constant
> > c++: internal compiler error: Killed (program cc1plus)
> > Please submit a full bug report
>  
>  yes, this is out of memory situation on the builder
>   
> >
> > Upstream developers are asking me how much memory is available on
> > koji, how much is it?
>  
>  what arch is it? 
x86_64

-- 
---
Antonio Trande
mailto: sagitter 'at' fedoraproject 'dot' org
http://fedoraos.wordpress.com/
https://fedoraproject.org/wiki/User:Sagitter
GPG Key: 0x6CE6D08A
Check on https://keys.fedoraproject.org/

[Dennis Gilmore]

On jueves, 3 de noviembre de 2016 2:32:05 PM CDT Antonio Trande wrote:
>  Hi all.
>  
>  I'm obtaining some (presumed) memory errors on some scratch builds of
>  Trilinos
>  (https://github.com/trilinos/Trilinos/issues/751#issuecomment-257428660),
>  the work-around looks be use a simple "make" command instead of
>  "parallel make":
>  
>  {standard input}: Assembler messages:
>  {standard input}:1002829: Warning: end of file not at end of a line;
>  newline inserted
>  {standard input}: Error: .size expression for
>  _ZN6Galeri6Xpetra5Utils26CreateCartesianCoordinatesIdiiN6Xpetra3MapIiiN6Kokk
>  os6Compat23KokkosDeviceWrapperNodeINS5_6OpenMPENS5_9HostSpaceEEEEENS3_11Mult
>  iVectorIdiiSA_EEEEN7Teuchos3RCPIT3_EERKNSt7__cxx1112basic_stringIcSt11char_t
>  raitsIcESaIcEEERKNSF_IKT2_EERNSE_13ParameterListE does not evaluate to a
>  constant
>  c++: internal compiler error: Killed (program cc1plus)
>  Please submit a full bug report
>  
>  
>  Upstream developers are asking me how much memory is available on koji,
>  how much is it?
>  
>  Regards. 
What is the koji task?

Dennis

[Kevin Fenzi]

On Thu, 3 Nov 2016 14:46:28 +0100
Antonio Trande <anto.trande(a)gmail.com&gt; wrote:

>  On 11/03/2016 02:44 PM, Dan Horák wrote:
>  > On Thu, 3 Nov 2016 14:32:05 +0100
>  > Antonio Trande <anto.trande(a)gmail.com&gt; wrote: 
...snip...
>  >> Upstream developers are asking me how much memory is
> available on
>  >> koji, how much is it?  
>  > 
>  > what arch is it?  
>  
>  x86_64 
buildvm's all have 10GB memory. 
buildhw's all have 20GB memory.

kevin

[Antonio Trande]

On 11/03/2016 03:51 PM, Dennis Gilmore wrote:
>  On jueves, 3 de noviembre de 2016 2:32:05 PM CDT Antonio Trande
> wrote:
> > Hi all.
> >
> > I'm obtaining some (presumed) memory errors on some scratch builds of
> > Trilinos
> > (https://github.com/trilinos/Trilinos/issues/751#issuecomment-257428660),
> > the work-around looks be use a simple "make" command instead of
> > "parallel make":
> >
> > {standard input}: Assembler messages:
> > {standard input}:1002829: Warning: end of file not at end of a line;
> > newline inserted
> > {standard input}: Error: .size expression for
> > _ZN6Galeri6Xpetra5Utils26CreateCartesianCoordinatesIdiiN6Xpetra3MapIiiN6Kokk
> > os6Compat23KokkosDeviceWrapperNodeINS5_6OpenMPENS5_9HostSpaceEEEEENS3_11Mult
> > iVectorIdiiSA_EEEEN7Teuchos3RCPIT3_EERKNSt7__cxx1112basic_stringIcSt11char_t
> > raitsIcESaIcEEERKNSF_IKT2_EERNSE_13ParameterListE does not evaluate to a
> > constant
> > c++: internal compiler error: Killed (program cc1plus)
> > Please submit a full bug report
> >
> >
> > Upstream developers are asking me how much memory is available on koji,
> > how much is it?
> >
> > Regards.
>  What is the koji task?
>  
>  Dennis
>   
http://koji.fedoraproject.org/koji/taskinfo?taskID=16280163


>  buildvm's all have 10GB memory.
>  buildhw's all have 20GB memory.
>
>  kevin 
Okay; how is chose that used? Automatically?

-- 
---
Antonio Trande
mailto: sagitter 'at' fedoraproject 'dot' org
http://fedoraos.wordpress.com/
https://fedoraproject.org/wiki/User:Sagitter
GPG Key: 0x6CE6D08A
Check on https://keys.fedoraproject.org/

[Dennis Gilmore]

On jueves, 3 de noviembre de 2016 5:21:38 PM CDT Antonio Trande wrote:
>  On 11/03/2016 03:51 PM, Dennis Gilmore wrote:
>  > On jueves, 3 de noviembre de 2016 2:32:05 PM CDT Antonio Trande wrote:
>  >> Hi all.
>  >> 
>  >> I'm obtaining some (presumed) memory errors on some scratch builds of
>  >> Trilinos
>  >> (https://github.com/trilinos/Trilinos/issues/751#issuecomment-257428660),
>  >> the work-around looks be use a simple "make" command instead of
>  >> "parallel make":
>  >> 
>  >> {standard input}: Assembler messages:
>  >> {standard input}:1002829: Warning: end of file not at end of a line;
>  >> newline inserted
>  >> {standard input}: Error: .size expression for
>  >> _ZN6Galeri6Xpetra5Utils26CreateCartesianCoordinatesIdiiN6Xpetra3MapIiiN6K
>  >> okk
>  >> os6Compat23KokkosDeviceWrapperNodeINS5_6OpenMPENS5_9HostSpaceEEEEENS3_11
>  >> Mult
>  >> iVectorIdiiSA_EEEEN7Teuchos3RCPIT3_EERKNSt7__cxx1112basic_stringIcSt11ch
>  >> ar_t raitsIcESaIcEEERKNSF_IKT2_EERNSE_13ParameterListE does not evaluate
>  >> to a constant
>  >> c++: internal compiler error: Killed (program cc1plus)
>  >> Please submit a full bug report
>  >> 
>  >> 
>  >> Upstream developers are asking me how much memory is available on koji,
>  >> how much is it?
>  >> 
>  >> Regards.
>  > 
>  > What is the koji task?
>  > 
>  > Dennis
>  
>  http://koji.fedoraproject.org/koji/taskinfo?taskID=16280163 
okay thanks

>  > buildvm's all have 10GB memory.
>  > buildhw's all have 20GB memory.
>  > 
>  > kevin
>  
>  Okay; how is chose that used? Automatically? 
the first builder to check in with capacaity takes the task. builds do not get 
dedicated builders, so there is also the possibility that there was 2 or more 
tasks on the same builder, resulting in less memory available.  This is the 
first time I have seen c++ oom on x86_64, I have seen it a few times on 32 bit 
platforms. which ended up being that it was trying to allocate more than $GiB 
ram in a single process. and could not allocate more ram

Dennis