FYI,
The Fedora Hadoop 2.x initiative began development several months ago based on the stability of the released F18 distro. Now that F19 is released we have rebased the packaging work on the various newer dependencies offered by F19. One significant change is that Jetty 8.1.x is no longer available and has made way for the latest version (9.0.x). This has introduced several API changes, particularly in the SSL API. It also has some benefit in that Jetty 9 incorporates a JSP compilation maven plugin which is required by the Hadoop build. The upstream version of Hadoop relies on a combination of Tomcat 5.5, Jetty 6.1.x, and the Codehaus jspc plugin for this step. However, F19 provides Tomcat 7 which is incompatible for this scenario with the Jetty container. The Jetty 9 version is Glassfish-based.
An important consideration for this change is that Jetty 9 is _completely incompatible with Java 6_ and only supported on Java 7, obviously the only version available in the current Fedora package repositories. Thus, the forthcoming Hadoop packaged jars for Fedora will also only be incompatible with the now EOL Oracle JRE 6 that Apache upstream uses for its current baseline.
The Hadoop packaging team feels that the reasonable path forward is to adapt the Hadoop build for Jetty 9 and eschew any and all Java 6 support for those bits. However, it does carry the caveat that there may be corner cases of behavioral or JVM-related differences in certain tests and use cases. The packaging team continually examines the Fedora Hadoop builds for "correctness" using the source tree JUnit test suite as well as basic smoke testing. Test-related problems have been identified and compensated for already once they are comprehended.
Fedora Software Collections https://fedoraproject.org/wiki/SoftwareCollections are of interest to the Hadoop packaging team but their suitability for this project and the current FPG policies regarding their official usage require further investigation.
Please let us know if you have any questions or concerns regarding this proposed direction.
\Pete https://fedoraproject.org/wiki/Changes/Hadoop
Given that the other patches will likely make their way upstream (sans Jetty), does it make sense to carry that patch and apply separately allowing for (EP)/EL %wrappers to enable/disable based on distribution?
Are any other patches that may cause an issue? Also does the previous jasper patch roll in now with the Jetty9 patch?
Cheers, Tim
----- Original Message -----
From: "Peter MacKinnon" pmackinn@redhat.com To: bigdata@lists.fedoraproject.org Sent: Friday, July 19, 2013 1:08:49 PM Subject: Jetty 9 implications for Hadoop 2.x in Fedora
FYI,
The Fedora Hadoop 2.x initiative began development several months ago based on the stability of the released F18 distro. Now that F19 is released we have rebased the packaging work on the various newer dependencies offered by F19. One significant change is that Jetty 8.1.x is no longer available and has made way for the latest version (9.0.x). This has introduced several API changes, particularly in the SSL API. It also has some benefit in that Jetty 9 incorporates a JSP compilation maven plugin which is required by the Hadoop build. The upstream version of Hadoop relies on a combination of Tomcat 5.5, Jetty 6.1.x, and the Codehaus jspc plugin for this step. However, F19 provides Tomcat 7 which is incompatible for this scenario with the Jetty container. The Jetty 9 version is Glassfish-based.
An important consideration for this change is that Jetty 9 is completely incompatible with Java 6 and only supported on Java 7, obviously the only version available in the current Fedora package repositories. Thus, the forthcoming Hadoop packaged jars for Fedora will also only be incompatible with the now EOL Oracle JRE 6 that Apache upstream uses for its current baseline.
The Hadoop packaging team feels that the reasonable path forward is to adapt the Hadoop build for Jetty 9 and eschew any and all Java 6 support for those bits. However, it does carry the caveat that there may be corner cases of behavioral or JVM-related differences in certain tests and use cases. The packaging team continually examines the Fedora Hadoop builds for "correctness" using the source tree JUnit test suite as well as basic smoke testing. Test-related problems have been identified and compensated for already once they are comprehended.
Fedora Software Collections are of interest to the Hadoop packaging team but their suitability for this project and the current FPG policies regarding their official usage require further investigation.
Please let us know if you have any questions or concerns regarding this proposed direction.
-- Peter MacKinnon MRG Grid/Big Data Red Hat Inc. Raleigh, NC
bigdata mailing list bigdata@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/bigdata
On 07/19/2013 02:54 PM, Tim St Clair wrote:
Given that the other patches will likely make their way upstream (sans Jetty), does it make sense to carry that patch and apply separately allowing for (EP)/EL %wrappers to enable/disable based on distribution?
I'll defer to Rob on this.
Are any other patches that may cause an issue? Also does the previous jasper patch roll in now with the Jetty9 patch?
Potentially. The current Jetty 9 patch may expand further beyond the current jspc-specific touchpoints. However, the TC Jasper runtime is declared in other parts of the code (e.g., hadoop-client).
Cheers, Tim
*From: *"Peter MacKinnon" <pmackinn@redhat.com> *To: *bigdata@lists.fedoraproject.org *Sent: *Friday, July 19, 2013 1:08:49 PM *Subject: *Jetty 9 implications for Hadoop 2.x in Fedora FYI, The Fedora Hadoop 2.x initiative began development several months ago based on the stability of the released F18 distro. Now that F19 is released we have rebased the packaging work on the various newer dependencies offered by F19. One significant change is that Jetty 8.1.x is no longer available and has made way for the latest version (9.0.x). This has introduced several API changes, particularly in the SSL API. It also has some benefit in that Jetty 9 incorporates a JSP compilation maven plugin which is required by the Hadoop build. The upstream version of Hadoop relies on a combination of Tomcat 5.5, Jetty 6.1.x, and the Codehaus jspc plugin for this step. However, F19 provides Tomcat 7 which is incompatible for this scenario with the Jetty container. The Jetty 9 version is Glassfish-based. An important consideration for this change is that Jetty 9 is completely incompatible with Java 6 and only supported on Java 7, obviously the only version available in the current Fedora package repositories. Thus, the forthcoming Hadoop packaged jars for Fedora will also only be incompatible with the now EOL Oracle JRE 6 that Apache upstream uses for its current baseline. The Hadoop packaging team feels that the reasonable path forward is to adapt the Hadoop build for Jetty 9 and eschew any and all Java 6 support for those bits. However, it does carry the caveat that there may be corner cases of behavioral or JVM-related differences in certain tests and use cases. The packaging team continually examines the Fedora Hadoop builds for "correctness" using the source tree JUnit test suite as well as basic smoke testing. Test-related problems have been identified and compensated for already once they are comprehended. Fedora Software Collections <https://fedoraproject.org/wiki/SoftwareCollections> are of interest to the Hadoop packaging team but their suitability for this project and the current FPG policies regarding their official usage require further investigation. Please let us know if you have any questions or concerns regarding this proposed direction. \Pete https://fedoraproject.org/wiki/Changes/Hadoop -- Peter MacKinnon MRG Grid/Big Data Red Hat Inc. Raleigh, NC _______________________________________________ bigdata mailing list bigdata@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/bigdata
bigdata mailing list bigdata@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/bigdata
bigdata@lists.fedoraproject.org
-
Peter MacKinnon -
Tim St. Clair