Meeting minutes from Env-and-Stacks WG meeting (2013-12-17)

Tue Dec 17 17:13:47 UTC 2013

============================================
#fedora-meeting: env and stacks (2013-12-17)
============================================

Meeting started by mmaslano at 16:01:35 UTC. The full logs are available
at
http://meetbot.fedoraproject.org/fedora-meeting/2013-12-17/env_and_stacks.2013-12-17-16.01.log.html
.

Meeting summary
---------------
* init process  (mmaslano, 16:06:46)
   * Big Data specific case: right now, if you want to use Scala for real
     work on Fedora, your option is basically to install a bunch of
     binaries maintained by upstreams.  It would be great if we were able
     to better support "new" language ecosystems wholly in Fedora.
     (mmaslano, 16:31:08)
   * ACTION: tjanez will sum up Big Data SIG needs  (mmaslano, 16:58:29)
   * ACTION: everyone will look at
     https://fedoraproject.org/wiki/User:Mmaslano/Draft:Env_and_Stack_PRD
     and think/work on tasks.  (mmaslano, 17:05:11)

Meeting ended at 17:10:52 UTC.

Action Items
------------
* tjanez will sum up Big Data SIG needs
* everyone will look at
   https://fedoraproject.org/wiki/User:Mmaslano/Draft:Env_and_Stack_PRD
   and think/work on tasks.

Action Items, by person
-----------------------
* tjanez
   * tjanez will sum up Big Data SIG needs
* **UNASSIGNED**
   * everyone will look at
     https://fedoraproject.org/wiki/User:Mmaslano/Draft:Env_and_Stack_PRD
     and think/work on tasks.

People Present (lines said)
---------------------------
* mmaslano (70)
* tjanez (51)
* mattf (30)
* hhorak (18)
* willb (14)
* sochotni (10)
* zodbot (7)
* Subfusc (5)
* pingou (3)
* pkovar1 (1)
* drieden (1)
* samkottler (1)
* bkabrda (0)
* abadger1999 (0)
* juhp (0)
* handsome_pirate (0)
* pkovar (0)

Generated by `MeetBot`_ 0.1.4

.. _`MeetBot`: http://wiki.debian.org/MeetBot

16:01:35 <mmaslano> #startmeeting env and stacks (2013-12-17)
16:01:35 <zodbot> Meeting started Tue Dec 17 16:01:35 2013 UTC.  The 
chair is mmaslano. Information about MeetBot at 
http://wiki.debian.org/MeetBot.
16:01:35 <zodbot> Useful Commands: #action #agreed #halp #info #idea 
#link #topic.
16:01:45 <mmaslano> #meetingname env and stacks
16:01:45 <zodbot> The meeting name has been set to 'env_and_stacks'
16:01:59 <mmaslano> #chair abadger1999 pkovar tjanez samkottler bkabrda 
handsome_pirate hhorak juhp
16:01:59 <zodbot> Current chairs: abadger1999 bkabrda handsome_pirate 
hhorak juhp mmaslano pkovar samkottler tjanez
16:02:30 * samkottler waves & will have to leave in about 15 minutes for 
a doctor's appointment
16:03:01 * pkovar1 is here
16:03:05 <mmaslano> who else is here
16:03:30 <mattf> o/
16:04:05 <willb> I'm here
16:05:28 * tjanez is being late
16:06:17 <drieden> I'm here too
16:06:46 <mmaslano> #topic init process
16:06:55 <mmaslano> great, let's start
16:07:04 <mmaslano> we don't have much time for something like PRD
16:07:17 <hhorak> Hi, sorry for being late..
16:07:38 <mmaslano> so I created something inspired by Cloud PRD 
https://fedoraproject.org/wiki/User:Mmaslano/Draft:Env_and_Stack_PRD
16:07:53 * pingou around (and late, sorry)
16:08:26 <mmaslano> the real tasks are still little fuzzy, but I plan to 
work on them more if you agree with it
16:08:53 <mmaslano> mattf: great you are here. You can tell us what you need
16:09:07 <mmaslano> mattf is from Big data SIG
16:09:17 <mattf> willb, would you start?
16:10:25 <mattf> mmaslano, willb has been feeling most of the pain 
dealing w/ non-primary (c/c++,python,perl) language stacks in fedora
16:10:37 <willb> I think a lot of our headaches in Big Data relate to 
language packaging ecosystems vs Fedora
16:10:46 <willb> in particular, I have been working on Scala lately
16:12:29 <willb> in short, the main issues are widespread Ivy for 
dependency management (mizdebsk has been working on this) and very 
brittle versioning (there is the expectation that you'll be able to have 
particular versions of the runtime, compiler, and libraries to run any 
nontrivial application)
16:14:22 <willb> like a lot of language stacks (e.g. rubygems, maven, 
eggs), there is the assumption that any developer would just be using 
the language stack instead of downstream dependency management, so I've 
had to work around a lot of those sorts of issues
16:15:15 <mattf> all those places where we have to work around things 
the language community does are hurdles to eventually automating / 
reducing maintenance package work
16:16:28 <mmaslano> we were already speaking about automation of 
packaging (not ready) and hosting such packages in separated repositories
16:16:29 <mattf> we also ran into this around node.js, which has its 
package/dep management tool available in fedora, but very little of the 
language ecosystem / base libraries available
16:17:04 <mmaslano> what would be good solution for you?
16:17:18 <mmaslano> SCL, or latest versions from upstream?
16:17:22 <mattf> for us the application is where we want to focus our 
time, but we find ourselves in the toolchain management and not getting 
to the application
16:18:07 <tjanez> willb, mattf, I'm glad to hear your perspective w.r.t. 
Stacks. Problems like you describe are very aligned with the scope of our WG
16:18:30 <mattf> the one dep i've been harping on lately is jetty. the 
latest from upstream doesn't support the java version that our app 
upstream wants to use. so having the latest in fedora is a problem.
16:18:59 <willb> jetty seems to touch a staggering number of projects in 
our SIG, too
16:19:54 <mattf> mmaslano, something that would have helped (and 
probably will in the future) is a way to provide language ecosystems in 
fedora in a way that aligns w/ how the language itself is used
16:20:27 <mattf> all the modern languages have their own dep & ver 
management tooling
16:20:37 <mattf> and most of it doesn't align well w/ fedora
16:21:34 <mattf> mmaslano, as for SCL, they could very likely help. 
however, they represent an add-on to fedora
16:22:04 <mattf> almost like rpm fusion, or all the extra repos from 
early fedora version
16:22:09 <sochotni> so what was that about jetty?
16:22:15 <willb> in the scala case, Fedora would allow us to provide 
scala 2.9 and 2.10 packages in the same release, but to really work with 
scala, we'd need both of those resolvable via Ivy, and not just 
"scalac29" and "scalac210" binaries
16:22:30 <mmaslano> mattf: yes, on last fesco meeting was decided, that 
additional repositories are acceptable, but content has to be reviewed 
anyway
16:23:53 <mattf> sochotni, rsquared is the best person to talk about the 
jetty situation. the exec summary is the version of jetty in fedora does 
not work w/ java 6 and the hadoop upstream isn't ready to abandon java 
6. so we end up maintaining a patch to hadoop for jetty 9 that will live 
purely in fedora for quite a while to come
16:25:02 <mattf> sochotni, i believe there was some chatter about compat 
packages for jetty libs too. that might be a workaround for f21/22 or 
so, but ultimately there's a mismatch in expectations between fedora and 
languages like java / scala / js that should be resolved
16:25:26 <mattf> (i should note, there's been a lot of good work on 
figuring it out for java)
16:25:29 <sochotni> mattf: we don't jave JDK 6 in Fedora 18+
16:26:08 <mattf> sochotni, i believe the compat lib discussion hinged on 
fedora policy and the compat library working w/ java 7
16:26:26 <sochotni> mattf: compat lib of what package?
16:26:27 <mattf> of course that workaround might stop working if the 
only jetty code doesn't work w/ say java 8 or java 9
16:26:58 <mattf> sochotni, we should chat w/ rsquared on the specifics
16:27:03 <tjanez> So, the question is, is WG willing to work towards 
integrating upstream language packaging systems
16:27:31 <tjanez> This could complement the existing RPM-based packaging
16:28:17 <mmaslano> tjanez: good point
16:28:30 <tjanez> It would certainly benefit the needs of the Big Data SIG
16:28:43 <sochotni> this is probably too specific task
16:28:44 <tjanez> The Big Data SIG use case could also be put in the PRD
16:28:55 <sochotni> might be a good test case but that's probably it
16:29:03 <mattf> i'd argue that steps in that direction will benefit the 
needs of groups that want to bring applications written in "new" 
languages to fedora
16:29:52 <mattf> (where "new" == actually really old languages)
16:29:55 <tjanez> mmaslano, do you feel upstream packaging is in our 
scope or not?
16:30:13 <sochotni> tjanez: that's not something you will solve in a WG
16:30:14 <mmaslano> tjanez: I thought we agreed it could be done 
automatically
16:30:22 <sochotni> you need someone who will *actually* work on the code
16:30:24 <mmaslano> tjanez: at least for some languages
16:30:41 <willb> again with my specific case:  right now, if you want to 
use Scala for real work on Fedora, your option is basically to install a 
bunch of binaries maintained by upstreams.  It would be great if we were 
able to better support "new" language ecosystems wholly in Fedora.
16:30:43 <sochotni> all you can agree is "yes we want to do that"
16:31:08 <mmaslano> #info Big Data specific case: right now, if you want 
to use Scala for real work on Fedora, your option is basically to 
install a bunch of binaries maintained by upstreams.  It would be great 
if we were able to better support "new" language ecosystems wholly in 
Fedora.
16:31:26 <mattf> nice
16:31:31 <tjanez> sochotni: well, I hope our WG will start working on 
things we put in the PRD
16:31:31 <tjanez> and by working I mean coding
16:31:51 <mmaslano> still someone has to write automation for "new" 
language and maintain it
16:32:07 <mmaslano> also I don't think packages without review can get 
into Fedora
16:32:12 <mmaslano> legal issues..
16:32:47 <hhorak> mmaslano: reviews could be done the first time and 
every-time something big changes in the package, but simple rebase could 
be done outomatically
16:33:07 <tjanez> sochotni: First I would like to see if we at least 
agree that complementing RPM packaging with upstream packaging is a 
viable way for the future
16:33:13 <mattf> that's an interesting point. there's always going to be 
a human component to packaging. however, we could probably write down 
that small list and try to automate the rest.
16:33:25 <mmaslano> tjanez: I agree
16:33:44 <tjanez> mmaslano: well, we would start with one language
16:33:52 <hhorak> so the packaging tool should be able to say "this 
rebase is suspiciously serious, we need a manual review"
16:34:13 <tjanez> mmaslano: but the common thing with all languages 
would be the concept of co-existance with RPM packages
16:34:28 <mmaslano> now I'm confused
16:34:39 <mmaslano> mattf: do you want to package upstream into rpm or not?
16:34:49 <sochotni> what I *could* imagine is some wrapper around 
pip/rubygems or alternative packaging systems
16:35:11 <hhorak> mmaslano: I think the tool should produce rpms in the end
16:35:18 <tjanez> mmaslano: +1 good question
16:35:25 <sochotni> but policy of updates...my head starts to hurt just 
thinking about it
16:35:37 <mattf> mmaslano, i want to package upstream into something 
that fedora will happily include directly in its ecosystem
16:35:59 <mmaslano> mattf: I'd like to see something like that too
16:36:20 <mattf> in other words, the form (rpm, deb, npm, other) isn't 
as important to me
16:36:41 <hhorak> sochotni: even now many people do big updates that 
introduce new issues according to guidelines and only if someone notices 
it it gets resolved..
16:36:48 <mmaslano> #proposal add automatic packaging of upstreams into 
WG goals
16:36:59 <mattf> hhorak, too true!
16:37:47 <tjanez> mattf, well from the consumer (aka big data user) 
point-of-view, the form is not important, from the distribution 
point-of-view it is
16:38:14 <mattf> hhorak, there's little infrastructure that i see for CI 
around fedora. you're a lib that has 12+ users, well when you update you 
should maybe rebuild your deps to find breaks.
16:38:24 <mattf> hhorak, fedora has a very reactive model atm
16:38:26 <hhorak> what I can't imagine is including a really new package 
without a review.. I mean at least initial review would have to be done 
manually, but with something like fedora-review it shouldn't be too hard
16:38:36 <mattf> tjanez, that's fair
16:40:03 <mmaslano> #proposal  add automatic packaging of upstreams into 
WG goals. Initial review of packages will be neded.
16:40:03 <tjanez> mmaslano, I'm ok with that, I would just like to 
clarify a bit more what the final form of that packaging would look like
16:40:03 <tjanez> would it be Fedora-approved subset of upstream 
packaging repositories
16:40:03 <tjanez> and the users would use tools like pip
16:40:04 <tjanez> or would it be an rpm repository automatically 
generated with *2rpm tools
16:40:05 <mmaslano> any votes?
16:40:07 <tjanez> and the users use dnf (distro tools)
16:40:18 <hhorak> mmaslano: +1
16:40:29 <mmaslano> tjanez: I would stay with rpm, we alredy know how it 
works
16:40:33 <mmaslano> (most of the time)
16:41:01 <willb> tjanez, to some extent, the existing processes are 
important for consumers as well as for the distro.  For any reasonable 
list of advantages we can list for using Fedora, most of them come from 
being well-integrated with the Fedora ecosystem and from having packages 
that follow Fedora processes.
16:42:25 <hhorak> tjanez: I don't think we should provide any bits 
outside rpms, it would require to solve all the issues that rpm solves 
today again and again for every language stack
16:42:36 <willb> hhorak, +1
16:43:07 <tjanez> Ok, you convinced me
16:43:22 <tjanez> I just wanted to put it out so we discuss it
16:43:23 <hhorak> tjanez: what I can imagine is creating something that 
would work like pip, but would actually work with rpms in the 
background.. (not familiar to pip, so maybe it is not easy)
16:44:01 <willb> IMHO RPM/rubygems integration and xmvn are good 
examples of language-specific packaging systems working well in Fedora
16:44:15 <mmaslano> hhorak: why? we have dnf/yum already. it's not 
obvious to me, why create another tool for installation
16:44:18 <tjanez> hhorak: +1
16:44:49 <tjanez> mmaslano: it would be just a conveniece wrapper for 
dnf/yum
16:45:10 * pingou doesn't see the point of wrapping wrapper
16:45:18 <mmaslano> tjanez: I wouldn't say just in case of dnf
16:46:37 <mmaslano> it seems to me Fedora users need some way how 
automatically generate packages from upstream. Implementation details 
can be solved later
16:47:02 <tjanez> well, the use case I can see is to attract developers 
with different backgrounds
16:47:03 <tjanez> they maybe very familiar with pip/rubygems/...
16:47:19 <tjanez> ok, fair enough
16:48:33 <mmaslano> tjanez: yeah, but which tool would you pick? 
pip/rubygems? you would probably have to patch all these tools
16:49:12 <tjanez> mmaslano, I would change your proposal to something 
like: automatic packaging of upstream language repositories into 
rpm-based repositories. Initial review of distributed packages would 
still be needed
16:49:17 <hhorak> mmaslano: I understand that as users just don't want 
to use dnf, because the style of work is way different from their 
language-specific tools.. that's why they would like to use what they 
use in other distributions (language specific ways, pip for example).. 
but command name doesn't matter, it's more about style of work, so if 
dnf can be as easy to use as these language-specific tools, then we can 
stay with pure dnf.. Maybe I didn't ca
16:50:14 <mmaslano> hhorak: we are missing end of sentence, not all irc 
clients can display so long messages ;-)
16:50:18 <mmaslano> tjanez: ok +1
16:50:24 <tjanez> mmaslano: yes, you would have to patch all of them. Or 
rather, replace them with commands that have the same CLI
16:51:02 <tjanez> mmaslano: but yea, this is an implementation detail 
that can be added or not later
16:51:08 <mmaslano> tjanez: at this moment it seems to me as far away 
future, because we don't even have the automatic packaging :)
16:51:10 <hhorak> mmaslano: sorry ;) my point was to try to understand 
why people don't like dnf/rpm/yum and prefer language specific tools
16:51:32 <Subfusc> the advantage of distribution tools for language 
specific purposes is not that they can install $random_language_pack but 
that they work in isolated environments (like with python-virtualenv)
16:51:50 <Subfusc> atleast that is how i see it
16:52:00 <pingou> mmaslano: we do have a number of *spec tools :)
16:52:01 <mmaslano> Subfusc: I guess in near future everything will be 
in container. So that's solved
16:52:40 <tjanez> #proposal Add to tasks/goals of our WG: Automatic 
packaging of upstream language repositories into rpm-based repositories. 
Initial review of distributed packages would still be needed
16:53:46 <tjanez> My proposal would actually just clarify the "The 
automatic packaging" task in mmaslano's PRD draft
16:53:49 <mmaslano> tjanez: still +1
16:54:46 <hhorak> tjanez: +1
16:55:37 <tjanez> It would be great if we also add a short summary of 
what mattf and willb said earlier
16:55:39 <hhorak> Subfusc: thanks for that point. After quick reading it 
seems like virtualenv is kind of a python version of software 
collections, or do I understood it wrong?
16:56:17 <tjanez> Maybe as a "problem case" the WG is solving with that 
particular task
16:56:19 <mmaslano> tjanez: do you want to sum it up?
16:56:40 <tjanez> Yes, I can and then put it in the wiki
16:57:11 <willb> tjanez, if you want some more detail, I wrote up some 
of the Scala-specific issues in August 
(http://chapeau.freevariable.com/2013/08/making-fedora-a-better-place-for-scala.html) 
and have been posting updates to the wiki here: 
https://fedoraproject.org/wiki/SIGs/bigdata/packaging/Scala
16:57:23 <tjanez> Maybe better if I do it after the meeting
16:57:42 <tjanez> willb, thanks, I will look at it
16:57:49 <willb> thanks
16:58:29 <mmaslano> #action tjanez will sum up Big Data SIG needs
16:58:40 <Subfusc> hhorak: I'm not that familiar with software 
collections, but it works in complete isolation with every virtualenv 
having its own libraries and it lives on $HOME so that non-admins can 
administer it. Software collections does not seem to completely envelop 
this concept
16:59:20 <Subfusc> the install in home folder is the primary feature 
that makes virtualenv easy to use for developers
16:59:33 <tjanez> mmaslano: should "problem cases" be a new section or 
just a sub-bullet under automated packaging task
16:59:41 * mmaslano will need to leave in few minutes
16:59:56 <mmaslano> tjanez: the whole task section is open for editing
17:00:04 <hhorak> Subfusc: right, user-specific installation is missing..
17:00:06 <mmaslano> I guess the rest is okay
17:00:22 <mmaslano> tjanez: I copied topic which were discussed in last 
few weeks
17:00:36 <tjanez> Ok, are there any takers for writing up other tasks in 
the PRD?
17:00:59 <hhorak> if the task list gets longer in the future, tasks 
could be separated into two groups as discussed two weaks back, to 
primary and secondary (less priority) tasks
17:01:18 <Subfusc> hhorak: its also easy to install and edit libraries 
without affecting anything else (which a system wide installation could 
not emulate)
17:01:20 <tjanez> We have to speed things up a bit
17:01:38 <tjanez> Jan 14th is coming very quickly
17:01:40 <mmaslano> tjanez: I'll try to specify some. But it should be 
short definition up to 6 sentences
17:01:55 <mmaslano> tjanez: yeah, that's the reason why I've started 
with prd without discussion
17:01:58 <tjanez> When will we have our next meeting?
17:02:06 <hhorak> tjanez: you mean what we discussed so far or some new 
tasks?
17:02:24 <tjanez> hhorak: the things from our discussions
17:02:30 <mmaslano> tjanez: imho January 7
17:02:54 <mmaslano> not much time, let's add as many tasks as possible now
17:03:01 <mmaslano> and we can review it by email
17:03:04 <tjanez> I think we have to look through the IRC logs and 
extract the things already said
17:03:41 <mmaslano> I tried to do it on my blog, but only from last 3 
meetings
17:04:11 <tjanez> Ok, I'll look at your blog post
17:04:43 <mmaslano> for the record http://mmaslano.livejournal.com/
17:04:51 <tjanez> I'll certainly help, since I'll have more free time 
around the holidays
17:05:11 <mmaslano> #action everyone will look at 
https://fedoraproject.org/wiki/User:Mmaslano/Draft:Env_and_Stack_PRD and 
think/work on tasks.
17:05:32 <mmaslano> #info deadline for PRD is January 17th
17:05:44 <hhorak> mmaslano: ^date -d '2014-01-07' +"%V" says 02 so it 
should be at 13:00UTC
17:05:45 <mmaslano> or 14th?
17:05:47 <tjanez> I hope others will join to create a better PrD
17:06:02 <mmaslano> tjanez: hopefully
17:07:02 <tjanez> mmaslano: well it's actually Jan 13 (see: 
https://fedorahosted.org/fesco/ticket/1196)
17:07:10 <mmaslano> #undo
17:07:10 <zodbot> Removing item from minutes: <MeetBot.items.Info object 
at 0x11d6e7d0>
17:07:16 <mmaslano> #info deadline for PRD is January 14th
17:07:43 <mmaslano> #info next meeting will be on 13:00UTC January 7th
17:07:50 <mmaslano> tjanez: thanks
17:08:01 <mmaslano> if you don't have any other topics, I would like to 
close the meeting
17:08:09 <mattf> mmaslano, thanks for having us
17:08:24 <tjanez> mmaslano, you still put in the wrong date?
17:08:44 <tjanez> or has it been changed from Jan 13 to Jan 14?
17:09:15 <willb> thanks all
17:09:42 <mmaslano> aaa
17:09:49 <mmaslano> #undo
17:09:49 <zodbot> Removing item from minutes: <MeetBot.items.Info object 
at 0xf892910>
17:09:55 <hhorak> thank you all and enjoy Chrismas!
17:10:02 <mmaslano> #undo
17:10:02 <zodbot> Removing item from minutes: <MeetBot.items.Info object 
at 0xf892dd0>
17:10:10 <mmaslano> #info deadline for PRD is January 13th
17:10:17 <mmaslano> #info next meeting will be on 13:00UTC January 7th
17:10:20 <tjanez> mmaslano: sorry for the trouble
17:10:35 <mmaslano> I hope meeting log will be nice :)
17:10:42 <mmaslano> let's go home
17:10:48 <mattf> ciao
17:10:49 <tjanez> Thank you for a good meeting
17:10:52 <mmaslano> #endmeeting