bigdata September 2013

bigdata@lists.fedoraproject.org

17 participants
20 discussions

greetings with a proposal
by Amir Sanjar 20 Sep '13

20 Sep '13

Hi all, For the last two years my team at IBM has been involve in maintaining Apache hadoop and related projects ( i.e. HBase, HIVE, PIG,..). We have extensive knowledge in area of porting Apache hadoop projects to other platforms (i.e. IBM JAVA, POWER, SystemZ,..). We would like to bring this knowledge to aid expanding fedora's hadoop ecosystems. To start, I would like to propose adding support for following platforms (Sorted by importance) to fedora hadoop: 1) IBM POWER 2) IBM JAVA 7 (post October release) - This might be belong to Fedora JAVA SIG 3) IBM SystemZ Best Regards Amir Sanjar IBM PowerLinux Hadoop projects Architect and lead IBM Senior Software Engineer Phone# 512-286-8393 Fax# 512-838-8858

4 4

Hive packaging wiki
by Peter MacKinnon 18 Sep '13

18 Sep '13

Hello friends, I have assembled a Hive packaging wiki <https://fedoraproject.org/wiki/Hive_packaging>. Due to the nature of the Hive ivy build and the way it can compile to several versions of Hadoop, it is still a little open with regards to missing dependencies. However, the next certain one will be the datanucleus deps which I will be working on. This dep will need some hacking to eliminate some source level dependencies. That project is set up to build all supported RDBMS adapters, but it includes an Oracle JDBC dep which won't make it into Fedora obviously. A repo <https://github.com/fedora-bigdata> will be set up for Hive itself soon once the datanucleus package has sufficiently progressed. Let me know if you have any questions. \Pete -- Peter MacKinnon MRG Grid/Big Data Red Hat Inc. Raleigh, NC

2 2

MapReduce Bootstrapping on F20
by Tim St. Clair 17 Sep '13

17 Sep '13

http://timothysc.github.io/blog/2013/09/14/hadoop-mapreduce/ -- Cheers, Tim

2 1

New hadoop package (2.0.5-9)
by Robert Rati 13 Sep '13

13 Sep '13

There is a new hadoop package that has been pushed to F20 testing. This new package includes new client and test subpackages that help help resolve hadoop related dependencies for parts of the ecosystem. It also fixes a few issues with the lichdfs package and a file conflict with qt-devel. Rob

1 0

Fwd: [PROPOSAL] Aurora for Apache Incubation
by Tim St. Clair 05 Sep '13

05 Sep '13

I'm game, but my bandwidth will be limited. https://wiki.apache.org/incubator/AuroraProposal I was playing in this space in condor for a while, some of the isolation work (auto-fuse-mounting,chroots,cgroups,namespaces) will likely translate. Cheers, Tim ----- Forwarded Message ----- > From: "Dave Lester" <dave(a)ischool.berkeley.edu> > To: dev(a)mesos.apache.org > Sent: Wednesday, September 4, 2013 8:00:13 PM > Subject: Re: [PROPOSAL] Aurora for Apache Incubation > > The good news is that we seem to be just weeks away from being able to > initially share the code publicly. The purpose of sharing the proposal to > the incubator was to gain interest in the project, and recruit Apache > Incubator mentors. > > If you have a specific need for Aurora right away and wish to become an > initial Apache contributor, there is a possibility that we could reach an > agreement to provide an early release. (Primary concern here is IP as we > continue to scrub the code) If you're interested, contact me directly. For > the vast majority of folks on this list, I'd suggest you hang in there and > we'll make sure to announce any releases on the Mesos dev and user lists. > > Best, > Dave > > On Sat, Aug 31, 2013 at 2:17 PM, vonPuh fonPuhendorf < > vonpuhfonpuhendorf(a)gmail.com> wrote: > > > What means by request? I would like to download it. > > На 27.08.2013 20:58 "Dave Lester" <dave(a)ischool.berkeley.edu> написа: > > > > > Hi Jim, > > > > > > The code has not yet been released, but may be made available upon > > request > > > as we work toward open sourcing it. > > > > > > Best, > > > Dave > > > > > > On Tue, Aug 27, 2013 at 8:55 AM, Jim Morrison <jmorrison(a)quantcast.com > > > >wrote: > > > > > > > Hi Dave, > > > > > > > > Is the code released anywhere? > > > > > > > > Jim Morrison > > > > jmorrison(a)quantcast.com > > > > > > > > > > > > > > > > On Aug 26, 2013, at 3:39 PM, Dave Lester <dave(a)ischool.berkeley.edu> > > > > wrote: > > > > > > > > > Hi All, > > > > > > > > > > We're pleased to share a draft ASF incubation proposal for Aurora, a > > > > > service scheduler used to schedule jobs onto Apache Mesos that we've > > > > > developed at Twitter. Aurora provides all of the primitives necessary > > > to > > > > > quickly deploy and scale stateless and fault tolerant services in a > > > > > datacenter. The complete proposal can be found online: > > > > > https://wiki.apache.org/incubator/AuroraProposal > > > > > > > > > > In particular, we'd love to add additional mentors to the project. > > Your > > > > > feedback is appreciated, which you can make on the > > > > > general(a)incubator.apache.org mailing list. > > > > > > > > > > Thanks! > > > > > > > > > > Dave > > > > > > > > > > -- Cheers, Tim

3 2

Fwd: [Test-Announce] Call for Fedora 20 Test Days
by Matthew Farrellee 05 Sep '13

05 Sep '13

Is anyone interested in doing one of these? The Hadoop package seems like a good candidate. Best, matt -------- Original Message -------- Subject: [Test-Announce] Call for Fedora 20 Test Days Date: Mon, 26 Aug 2013 22:16:13 -0600 From: Mike R <roshi(a)fedoraproject.org> Reply-To: Development discussions related to Fedora <devel(a)lists.fedoraproject.org> To: devel(a)lists.fedoraproject.org, test(a)lists.fedoraproject.org Hey all, it's a bit late but it's time to consider putting in your requests for Test Days for Fedora 20. As a recap, a Test Day is an online event aimed at testing a specific feature of an upcoming Fedora Release. By utilizing IRC for organization and coordination and a Wiki page for instructions and results Test Days are easy to organize. Anyone can request to host a Test Day or request that the QA team help you out with the organization of the test day. A test day can be ran for any feature or area of a distribution that focused testing would be useful for. More information on test days can be found here: https://fedoraproject.org/wiki/QA/Test_Days . To propose a Test Day, file a ticket on the QA Trac. A full explanation can be found here: https://fedoraproject.org/wiki/QA/Test_Days/Create . The SOP for hosting a test day is here: https://fedoraproject.org/wiki/QA/SOP_Test_Day_management . Typically Test Days are on a Thursday, or for a series of Test Days over a Tues-Thursday period (this is how we usually run the X Test Week, for instance). There are many days available starting today 2013-08-26 all the way through 2013-11-26. Take into consideration the release schedule here: http://fedoraproject.org/wiki/Releases/20/Schedule . It might be a good idea to schedule your test day before the Final Change Deadline. And finally, if you really want to run a Test Day in a specific time-frame due to the development schedule, but the Thursday slot for that week is full, we can add a slot on another day. We're flexible! Just put in your ticket the date or time-frame you'd like, and we'll figure it out from there. If you have any questions about Test Days or the process, please don't hesitate to contact me or any other QA Team member in #fedora-qa on Freenode. Thanks and happy testing! Mike Ruckman Fedora QA IRC: roshi -- devel mailing list devel(a)lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

3 6

Re: [distcomp] Need hadoop-common packaged for ecosystem
by Peter MacKinnon 04 Sep '13

04 Sep '13

Nevermind, I see the problem. I need to "enhance" my local Ivy resolver to check /usr/lib also. \Pete On 09/04/2013 04:10 PM, Peter MacKinnon wrote: > Hey Rob, I think we're going to need to package the hadoop-common jar > and frags in the next spin. Hive (and likely others) will need it. > I can take it on if you like. > > \Pete > > [ivy:resolve] > [ivy:resolve] :: problems summary :: > [ivy:resolve] :::: WARNINGS > [ivy:resolve] module not found: > hadoop#hadoop-common;2.0.5-alpha > [ivy:resolve] ==== fedora1: tried > [ivy:resolve] -- artifact > hadoop#hadoop-common;2.0.5-alpha!hadoop-common.jar: > [ivy:resolve] /usr/share/java/hadoop-common.jar > [ivy:resolve] ==== fedora2: tried > [ivy:resolve] -- artifact > hadoop#hadoop-common;2.0.5-alpha!hadoop-common.jar: > [ivy:resolve] /usr/share/java/hadoop/hadoop-common.jar > [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: > [ivy:resolve] :: UNRESOLVED DEPENDENCIES :: > [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: > [ivy:resolve] :: hadoop#hadoop-common;2.0.5-alpha: not found > [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: > > -- Peter MacKinnon MRG Grid/Big Data Red Hat Inc. Raleigh, NC

1 0

Packaging ambari for Fedora
by Robert Rati 03 Sep '13

03 Sep '13

I've created a wiki detailing the issues that need to be addressed in order to get ambari packaged in Fedora. Comments and discussions about these issues are welcome. https://fedoraproject.org/wiki/Ambari_packaging Rob

2 1

Fwd: Broken dependencies: glusterfs-hadoop
by Matthew Farrellee 03 Sep '13

03 Sep '13

What was the fix for this? -------- Original Message -------- Subject: Broken dependencies: glusterfs-hadoop Date: Mon, 2 Sep 2013 10:56:29 +0000 (UTC) From: buildsys(a)fedoraproject.org To: glusterfs-hadoop-owner(a)fedoraproject.org glusterfs-hadoop has broken dependencies in the F-20 tree: On armhfp: glusterfs-hadoop-2.1.2-1.fc20.noarch requires mvn(org.apache.hadoop:hadoop-common) Please resolve this as soon as possible.

2 2

GlusterFS Hadoop - GlusterFS Hadoop Compatible File System (HCFS) plugin
by Matthew Farrellee 02 Sep '13

02 Sep '13

Thanks to gil for the review! The glusterfs-hadoop package is now available in F20+. It provides an implementation of Hadoop's distributed filesystem 1.x and 2.x interfaces (http://wiki.apache.org/hadoop/HCFS) that work with GlusterFS. The feature provided is the ability to run any computational workloads (MapReduce, HBase, Hive, etc etc) on top of data stored in Gluster. The project's homepage is https://forge.gluster.org/hadoop Best, matt

1 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

bigdata September 2013