Bringing Hadoop to Fedora

[Tim St Clair]

Thanks Alejandro. 

comments inline below

----- Original Message -----
> From: "Alejandro Abdelnur" <tucu at cloudera.com>
> To: common-dev at hadoop.apache.org
> Sent: Monday, July 8, 2013 4:06:37 PM
> Subject: Re: Bringing Hadoop to Fedora
> 
> [moving bigtop to bcc]
> 
> Tim,
> 
> Except from HADOOP-9680 which has significant code changes and some false
> changes (which I did not go thru), all other changes seem OK.
> 
> * Have you had a change to run ALL Hadoop testcases with them applied to
> make sure there are not regression?
> 

Yes for everything, except (https://issues.apache.org/jira/browse/HADOOP-9680 / 9623).

> * Have you look at the output of running 'mvn dependency:tree' without/with
> the patches to verify unwanted dependencies are not sneaking in?
> 

We've been tracking the dep-graph with a pretty close eye, but it would always be good to have another set of eyes gander at the delta.

> On HADOOP-9650 (not included in the above list):

Done on purpose at this point because the shift in Fedora 19 was just recently absorbed, and we are running through 
tests and use cases to verify we didn't break the world ;-).

> 
> Moving from Jetty6 to Jetty9 should be OK if there are not regressions. I
> was chatting with ToddL about the issues we had before in and according to
> Jetty guys they've been fixed in newer Jetty versions. Also, we are not
> using Jetty for shuffle anymore (we use Netty), so Jetty is not 'stress'
> that much anymore as it is used for the web UIs and for NN-SNN
> checkpointing.
> 
> From the patch some code changes are required, once that is take care we
> should repeat the bullet item above for this patch.
> 
> In addition, we have to see how this will play with other projects like
> HBase that are using HttpServer from hadoop-common. I'll forward this part
> of the email to their dev@ so they can watch/jump-in if necessary in the
> JIRA.

Agreed, which is part of the reason why we wanted to include BIGTOP on this thread, as they would likely 
know more about the stack dependencies then we would.  


> 
> Thanks.
> 
> 
> On Mon, Jul 8, 2013 at 11:28 AM, Tim St Clair <tstclair at redhat.com> wrote:
> 
> > Arun,
> >
> > I realize you're probably pretty busy (or on vacation), but I figured I
> > would re-ping this thread to inquire about the status of the patch set
> > listed below.?.?
> >
> > https://issues.apache.org/jira/browse/HADOOP-9594
> > https://issues.apache.org/jira/browse/HADOOP-9610
> > https://issues.apache.org/jira/browse/HADOOP-9611
> > https://issues.apache.org/jira/browse/HADOOP-9613
> > https://issues.apache.org/jira/browse/HADOOP-9680 / 9623
> >
> > Cheers,
> > Tim
> >
> > ----- Original Message -----
> > > From: "Arun C Murthy" <acm at hortonworks.com>
> > > To: yarn-dev at hadoop.apache.org
> > > Cc: common-dev at hadoop.apache.org, mapreduce-dev at hadoop.apache.org,
> > hdfs-dev at hadoop.apache.org, dev at bigtop.apache.org
> > > Sent: Tuesday, June 11, 2013 12:54:11 PM
> > > Subject: Re: Bringing Hadoop to Fedora
> > >
> > > Tim,
> > >
> > >  This is great! I'll take a look at some of these patches, welcome!
> > >
> > > Arun
> > >
> > > On Jun 11, 2013, at 8:54 AM, Tim St Clair wrote:
> > >
> > > > Greetings Hadoop Development Community,
> > > >
> > > > As some of you are aware, we've been looking to bring the upstream 2.X
> > > > series into the Fedora channels.  We believe that there are several
> > > > benefits that this can bring to the community:
> > > >
> > > > - Updated & streamlined rpm packaging
> > > > - Deeper level of system integration
> > > > - System managed dependencies (security + defect tracking)
> > > > - Proving ground for OpenJDK7
> > > > - Greater exposure for the community (both adoption and defect
> > tracking)
> > > > ...
> > > >
> > > > During our evaluation (https://fedoraproject.org/wiki/Features/Hadoop
> > ),
> > > > we've come across some build issues as they relate to Fedora packaging
> > > > guidelines (http://fedoraproject.org/wiki/Packaging:Java).  Most of
> > these
> > > > issues (listed below) are very minor dependency changes, and we were
> > > > wondering if there are folks who would be willing to review.
> > > >
> > > > fedora-patch-math: https://issues.apache.org/jira/browse/HADOOP-9594
> > > > fedora-patch-collections:
> > https://issues.apache.org/jira/browse/HADOOP-9610
> > > > fedora-patch-cglib: https://issues.apache.org/jira/browse/HADOOP-9611
> > > > fedora-patch-jersey: https://issues.apache.org/jira/browse/HADOOP-9613
> > > > fedora-patch-jets3t: https://issues.apache.org/jira/browse/HADOOP-9623
> > > > (uber deprecated)
> > > >
> > > > There are other patches we have in the queue but they still need to be
> > > > flushed out, and in an effort to follow the KISS principles we figure
> > this
> > > > would be a good start.
> > > >
> > > > Cheers,
> > > > Tim
> > >
> > > --
> > > Arun C. Murthy
> > > Hortonworks Inc.
> > > http://hortonworks.com/
> > >
> > >
> > >
> >
> 
> 
> 
> --
> Alejandro
>