Some Depcheck runs taking long time

[Josef Skladanka]

Hello,

yesterday, Kamil spotted some depcheck run, which took quite a lot of time.
It was caused by the depcheck_pretty() routines, which took some time to parse those 70k lines of text. Sadly, this is not-a-bug, but a feature. The positive thing is, that logs this long are not common and mostly sign a error in dependencies.

J.

[James Laska]

On Thu, 2011-06-23 at 07:04 -0400, Josef Skladanka wrote:
> Hello,
>
> yesterday, Kamil spotted some depcheck run, which took quite a lot of
> time.
> It was caused by the depcheck_pretty() routines, which took some time
> to parse those 70k lines of text. Sadly, this is not-a-bug, but a
> feature. The positive thing is, that logs this long are not common and
> mostly sign a error in dependencies.

Thanks for the heads up Josef.  What sort of delays are you seeing?  Is
this minutes or hours?

Thanks,
James

[Kamil Paral]

> On Thu, 2011-06-23 at 07:04 -0400, Josef Skladanka wrote:
> > Hello,
> >
> > yesterday, Kamil spotted some depcheck run, which took quite a lot
> > of
> > time.
> > It was caused by the depcheck_pretty() routines, which took some
> > time
> > to parse those 70k lines of text. Sadly, this is not-a-bug, but a
> > feature. The positive thing is, that logs this long are not common
> > and
> > mostly sign a error in dependencies.
>
> Thanks for the heads up Josef. What sort of delays are you seeing? Is
> this minutes or hours?

Yesterday I ran depcheck and it checked lots of perl packages (hundreds of dependencies). Depcheck/yum spit out thousands of lines (the whole log was about 6 MB IIRC). Depcheck_pretty found 70 000 lines related to depsolving problems. All lines are checked with regular expressions. Several thousand of lines were highlighted, causing pretty_log also to take some time to process. Overall it added around 5 minutes to the test length.

It's not convenient, but I don't consider it a problem at the moment. Usually the logs are much shorter and there is no noticeable delay.

[Tim Flink]

On 06/23/2011 07:11 AM, Kamil Paral wrote:
> > On Thu, 2011-06-23 at 07:04 -0400, Josef Skladanka wrote:
> > > Hello,
> > >
> > > yesterday, Kamil spotted some depcheck run, which took quite a
> > > lot of time. It was caused by the depcheck_pretty() routines,
> > > which took some time to parse those 70k lines of text. Sadly,
> > > this is not-a-bug, but a feature. The positive thing is, that
> > > logs this long are not common and mostly sign a error in
> > > dependencies.
> >
> > Thanks for the heads up Josef. What sort of delays are you seeing?
> > Is this minutes or hours?
>
> Yesterday I ran depcheck and it checked lots of perl packages
> (hundreds of dependencies). Depcheck/yum spit out thousands of lines
> (the whole log was about 6 MB IIRC). Depcheck_pretty found 70 000
> lines related to depsolving problems. All lines are checked with
> regular expressions. Several thousand of lines were highlighted,
> causing pretty_log also to take some time to process. Overall it
> added around 5 minutes to the test length.
>
> It's not convenient, but I don't consider it a problem at the moment.
> Usually the logs are much shorter and there is no noticeable delay.

Ah, I suppose that would make sense. One wonders what would have
happened with the multi-gigabyte log files that we were seeing for a while.

Are there any other issues that would keep us from doing another
pre-release build to run on the test instance and assuming that we don't
run into any other issues - release and deploy on Monday?

Tim

[Kamil Paral]

> Are there any other issues that would keep us from doing another
> pre-release build to run on the test instance

I don't know about any. Let's do it.
> and assuming that we
> don't
> run into any other issues - release and deploy on Monday?

+1

[James Laska]

On Thu, 2011-06-23 at 10:27 -0400, Kamil Paral wrote:
> > Are there any other issues that would keep us from doing another
> > pre-release build to run on the test instance
>
> I don't know about any. Let's do it.

Okay, I'll commit a autoqa.spec change, build and push out
autoqa-0.5.0-0.3.pre to the -testing repos today.

Thanks,
James