Big Data Cloud Collaboration

[Julien Eid]

Hey guys,

I'm helping out the Cloud image guys who are looking to collaborate with
this SIG to make Big Data images for people to use.  We were wondering what
packages and features you would like to see in the image that we could
include and if there were specific things you guys were looking for. We're
currently thinking of making a Hadoop ecosystem image geared for people
wanting to get interested in that kind of thing and maybe some day down the
line make a Mesos ecosystem one. We are interested in having the image be
usable in different topologies (single node or multi node). Haïkel Guémar (
[e-mail]) and I will be acting as a liaison with the Cloud
SIG and both SIG's would be the co-owners of this image. I have a proposal
here that talks about the general plan but I would love to get feedback to
be more specific in what gets made.
https://fedoraproject.org/wiki/Changes/Big_Data_Cloud_Image

Thanks!

[Erik Erlandson]

We would have some interest in images that include Spark capabilities.   That might fall into the "hadoop ecosystem" category, although Spark can stand up independently as well.

----- Original Message -----
> Hey guys,
>
> I'm helping out the Cloud image guys who are looking to collaborate with this
> SIG to make Big Data images for people to use. We were wondering what
> packages and features you would like to see in the image that we could
> include and if there were specific things you guys were looking for. We're
> currently thinking of making a Hadoop ecosystem image geared for people
> wanting to get interested in that kind of thing and maybe some day down the
> line make a Mesos ecosystem one. We are interested in having the image be
> usable in different topologies (single node or multi node). Haïkel Guémar (
> [e-mail] ) and I will be acting as a liaison with the Cloud
> SIG and both SIG's would be the co-owners of this image. I have a proposal
> here that talks about the general plan but I would love to get feedback to
> be more specific in what gets mad e.
> https://fedoraproject.org/wiki/Changes/Big_Data_Cloud_Image
>
> Thanks!
>
> bigdata mailing list
> [e-mail]
> https://admin.fedoraproject.org/mailman/listinfo/bigdata

[Matthew Miller]

On Fri, Jun 13, 2014 at 03:35:02PM -0400, Erik Erlandson wrote:
> We would have some interest in images that include Spark capabilities.
> That might fall into the "hadoop ecosystem" category, although Spark can
> stand up independently as well.

We'll have a generic image which people can use to build anything they want
(via cloud-init and yum install). I'd like to see the big data image be as
useful as possible "out of the box" for a specific use of some sort. That
way, we can make Fedora as easy to use and deploy as possible for a specific
niche. If that's Spark, awesome. And once that's successful, we can expand
more into other niches.

For the future for more complicated things, we might want to look at using
the same mechanism Fedora Server is using for "server roles" (but still to
make images). For now, though, I think the approach is "whatever works". :)