Packaging GlusterFS Hadoop
by Matthew Farrellee
I'm starting to package glusterfs-hadoop, a shim on top of GlusterFS
that allows applications written for HDFS to run on GlusterFS.
The SRPM and spec are -
http://matt.fedorapeople.org/pkg/0/glusterfs-hadoop-2.1.2-1.src.rpm
http://matt.fedorapeople.org/pkg/0/glusterfs-hadoop.spec
I'm trying to get it to build in an F20 mock environment before
submitting a review request. However, I'm running into a problem...
[ERROR] Failed to execute goal on project glusterfs: Could not resolve
dependencies for project
org.apache.hadoop.fs.glusterfs:glusterfs:jar:2.0-SNAPSHOT: Could not
find artifact com.sun:tools:jar:1.6 at specified path
/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.25-2.3.12.3.fc20.x86_64/jre/../lib/tools.jar
-> [Help 1]
com.sun:tools is coming in from...
[DEBUG] Loading depmap file:
/usr/share/maven-fragments/javapackages-depmap.xml
[DEBUG] Added mapping [com.sun:tools] => [JAVA_HOME:../lib/tools]
...
[DEBUG] org.apache.hadoop:hadoop-annotations:jar:2.0.5-alpha:compile
[DEBUG] com.sun:tools:jar:1.6:system
The javapackages tools is 2.0.1-1.fc20 and xmvn is 0.5.1-4.fc20.
Has anyone seen this before, or have suggestions on getting past it?
Best,
matt
10 years, 7 months
Fwd: Broken dependencies: stout
by Tim St. Clair
Is there any reason to get this into the epel channels? I know some folks had expressed interest, but currently I have no interest in maintaining for those channels.
-Tim
----- Forwarded Message -----
> From: buildsys(a)fedoraproject.org
> To: stout-owner(a)fedoraproject.org
> Sent: Saturday, August 17, 2013 11:32:29 PM
> Subject: Broken dependencies: stout
>
>
>
> stout has broken dependencies in the epel-5 tree:
> On ppc:
> stout-devel-0.1.2-1.099483f.el5.ppc requires gmock-devel
> stout-devel-0.1.2-1.099483f.el5.ppc requires glog-devel
> On x86_64:
> stout-devel-0.1.2-1.099483f.el5.x86_64 requires gmock-devel
> stout-devel-0.1.2-1.099483f.el5.x86_64 requires glog-devel
> On i386:
> stout-devel-0.1.2-1.099483f.el5.i386 requires gmock-devel
> stout-devel-0.1.2-1.099483f.el5.i386 requires glog-devel
> Please resolve this as soon as possible.
>
>
>
--
Cheers,
Tim
10 years, 7 months
Fwd: [VOTE] Release Apache Hadoop 2.1.0-beta
by Tim St. Clair
FYI. It may make sense just to track and update once the release comes out.
----- Forwarded Message -----
> From: "Arun C Murthy" <acm(a)hortonworks.com>
> To: common-dev(a)hadoop.apache.org, hdfs-dev(a)hadoop.apache.org, mapreduce-dev(a)hadoop.apache.org,
> yarn-dev(a)hadoop.apache.org
> Sent: Sunday, August 25, 2013 2:38:50 PM
> Subject: Re: [VOTE] Release Apache Hadoop 2.1.0-beta
>
> With 9 +1s including mine (6 binding) and no -1s, the vote passes. Thanks to
> all who voted.
>
> Arun
>
> On Aug 15, 2013, at 2:15 PM, Arun C Murthy <acm(a)hortonworks.com> wrote:
>
> > Folks,
> >
> > I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would
> > like to get released - this fixes the bugs we saw since the last go-around
> > (rc1).
> >
> > The RC is available at:
> > http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/
> > The RC tag in svn is here:
> > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2
> >
> > The maven artifacts are available via repository.apache.org.
> >
> > Please try the release and vote; the vote will run for the usual 7 days.
> >
> > thanks,
> > Arun
> >
> > --
> > Arun C. Murthy
> > Hortonworks Inc.
> > http://hortonworks.com/
> >
> >
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
--
Cheers,
Tim
10 years, 7 months
Big Data packaging wiki
by Matthew Farrellee
Hey fellow big data fans,
https://fedoraproject.org/wiki/Big_data_packaging
I just updated the packaging wiki page to include some of the things
that have been done, a few that are in progress, and a bit of a wishlist
I've heard over the past year.
Feel free to edit the page at will. Especially if some existing packages
aren't listed, or if you are working on something that isn't listed.
This entire ecosystem is quite unruly when it comes to massaging into
Fedora-acceptable packages, and it turns out there is often overlap
under the covers (e.g. thrift for spark and hive). So, if you're looking
to start a new package, let's us know on the mailing list.
Best,
matt
10 years, 7 months
Savanna - Hadoop on OpenStack
by Matthew Farrellee
Hello Big Data SIG folks,
If you aren't familiar, Savanna is an OpenStack project that provides
Hadoop cluster and workload management. Cluster - construct and manage
the lifecycle of Hadoop clusters. Workload - workflow for big data
processing with Hadoop (similar to AWS EMR).
The project home page is https://launchpad.net/savanna
Savanna is made up of 4 sub-projects -
. savanna, the main services
. savannadashboard, web UI integration with OpenStack Horizon
. python-savannaclient, python bindings for the REST API
. savanna-extra - diskimage-builder elements for...image building
As of today all those are available in F19, F20, and EL6*
openstack-savanna - https://bugzilla.redhat.com/show_bug.cgi?id=986615
python-django-savanna - https://bugzilla.redhat.com/show_bug.cgi?id=998123
python-savannaclient - https://bugzilla.redhat.com/show_bug.cgi?id=998701
savanna-image-elements - https://bugzilla.redhat.com/show_bug.cgi?id=998702
Thanks for all the community help getting Savanna into Fedora,
especially the #rdo folks.
With any luck the project will have Fedora based cloud images with its
next release. Right now all the images are Ubuntu based.
Best,
matt
* openstack-savanna (package for savanna sub-project) has a dep on
pycrypto and isn't available on EL6 yet, savanna-image-elements depends
on diskimage-builder which isn't included in EL6 yet
-
Install -
# yum --enablerepo=updates-testing install openstack-savanna
python-django-savanna (make sure you get python-django-savanna-0.2-2)
Setup & start savanna-api service -
# sed -i "s/^#os_admin_password=/os_admin_password=$OS_PASSWORD/"
/etc/savanna/savanna.conf
# systemctl start openstack-savanna-api
Setup and load Dashboard plugin -
# echo "SAVANNA_URL = 'http://localhost:8386/v1.0'" >>
/etc/openstack-dashboard/local_settings
Edit /usr/share/openstack-dashboard/openstack_dashboard/settings.py -
. HORIZON_CONFIG = {
'dashboards': ('project', 'admin', 'settings', 'savanna'),
. INSTALLED_APPS = (
'savannadashboard',
'openstack_dashboard',
# systemctl reload httpd
10 years, 7 months
Hadoop is in Fedora
by Robert Rati
The hadoop package has passed review and has been built for Fedora 20.
Rob
10 years, 8 months
