Bundled/embedded data in a package

[Ben Rosser]

So I've had a review ticket open on bugzilla for a while now for this
package:
https://bugzilla.redhat.com/show_bug.cgi?id=823679

The software, pdfminer, bundles Adobe cmap data to process Chinese,
Japanese, and Korean languages in a pdf file. However, the question is:
does this violate the general prohibition against bundling libraries? I'm
not exactly sure, because I think it's more like bundled data than bundled
code- but I don't know if that makes a difference to the packaging
guidelines.

If it does violate that prohibition, the cmap stuff can easily be removed
and not built into the library (it's an optional component).

Ben

[Tomas Radej]

=== DISCLAIMER ===
I am simply a licensing enthusiast and no expert on law or Packaging 
Guidelines, please understand this message solely as my personal opinion.
==================

Hi,

On 09/13/2012 05:02 AM, Ben Rosser wrote:
>  The software, pdfminer, bundles Adobe cmap data to process Chinese,
>  Japanese, and Korean languages in a pdf file. However, the question is:
>  does this violate the general prohibition against bundling libraries? 
The CMap
files you are talking about are a part of a separate project, 
which is apparently subject to change. Since there are other projects in 
the wild that use Adobe CMap, I would interpret the guidelines to say 
that packaging the Adobe files in a separate package would be in order.

However, given that Fedora tries to be as close to upstream, using a 
different version of these files might lead to different behaviour of 
the pdfminer application from upstream. Security concerns are out of the 
question as these files are plain text, so I suggest you keep the files, 
make a note about it in the spec file, make sure the licence of the 
package reflects the inclusion of these files, and that's it.

If somebody thinks it's a bad idea, please do correct me.

>  If it does violate that prohibition, the cmap stuff can easily be
> removed
>  and not built into the library (it's an optional component). 
I wouldn't do
that as that would make the package different from upstream.

In any case, you can make the Legal department have a look at it by 
blocking FE-LEGAL.

Regards, Tomas Radej

--
Tomas Radej
FAS, IRC nick tradej

[Nicolas Mailhot]

Le Jeu 13 septembre 2012 05:02, Ben Rosser a écrit :
>  So I've had a review ticket open on bugzilla for a while now for
> this
>  package:
>  https://bugzilla.redhat.com/show_bug.cgi?id=823679
>
>  The software, pdfminer, bundles Adobe cmap data to process Chinese,
>  Japanese, and Korean languages in a pdf file. However, the question is:
>  does this violate the general prohibition against bundling libraries? I'm
>  not exactly sure, because I think it's more like bundled data than bundled
>  code- but I don't know if that makes a difference to the packaging
>  guidelines.
>
>  If it does violate that prohibition, the cmap stuff can easily be removed
>  and not built into the library (it's an optional component). 
Adobe CMAP is used by lots of font, ps and pdf related apps and IIRC
packaging it is currently on the wishlist.

You should unbundle it and create a separate package.

I don't think CMAP has security implications, but projects that bundle old
versions of the Adobe datafiles will behave in non-optimal ways when they
hit a font/text combination that requires a mapping Adobe introduced in a
later version.

-- 
Nicolas Mailhot

[Ben Rosser]

On Fri, Sep 14, 2012 at 9:46 AM, Nicolas Mailhot <
nicolas.mailhot(a)laposte.net&gt; wrote:

>  Adobe CMAP is used by lots of font, ps and pdf related apps and IIRC
>  packaging it is currently on the wishlist.
>
>  You should unbundle it and create a separate package.
>
>  I don't think CMAP has security implications, but projects that bundle old
>  versions of the Adobe datafiles will behave in non-optimal ways when they
>  hit a font/text combination that requires a mapping Adobe introduced in a
>  later version.
>  
That sounds like a good idea.

How should it work though? I can grab all the .tar.z files from
http://sourceforge.net/adobe/cmap/home/Home/, extract them, and throw them
somewhere into /usr/share (/usr/share/cmap/)... then packages can grab
up-to-date copies of the cmap resources from there?

Ben

[Rex Dieter]

Ben Rosser wrote:

>  On Fri, Sep 14, 2012 at 9:46 AM, Nicolas Mailhot <
>  nicolas.mailhot(a)laposte.net&gt; wrote:
>  
> >
> > Adobe CMAP is used by lots of font, ps and pdf related apps and IIRC
> > packaging it is currently on the wishlist.
> >
> > You should unbundle it and create a separate package.
> >
> > I don't think CMAP has security implications, but projects that bundle
> > old versions of the Adobe datafiles will behave in non-optimal ways when
> > they hit a font/text combination that requires a mapping Adobe introduced
> > in a later version.
> >
>  
>  That sounds like a good idea.
>  
>  How should it work though? I can grab all the .tar.z files from
>  http://sourceforge.net/adobe/cmap/home/Home/, extract them, and throw them
>  somewhere into /usr/share (/usr/share/cmap/)... then packages can grab
>  up-to-date copies of the cmap resources from there? 
that content should (loosely at least) match the content within the poppler-
data package.

-- rex

[Nicolas Mailhot]

Le Sam 15 septembre 2012 20:53, Rex Dieter a écrit :
>  Ben Rosser wrote:
>
> > On Fri, Sep 14, 2012 at 9:46 AM, Nicolas Mailhot <
> > nicolas.mailhot(a)laposte.net&gt; wrote:
> >
> >>
> >> Adobe CMAP is used by lots of font, ps and pdf related apps and IIRC
> >> packaging it is currently on the wishlist.
> >>
> >> You should unbundle it and create a separate package.
> >>
> >> I don't think CMAP has security implications, but projects that bundle
> >> old versions of the Adobe datafiles will behave in non-optimal ways
> >> when
> >> they hit a font/text combination that requires a mapping Adobe
> >> introduced
> >> in a later version.
> >>
> >
> > That sounds like a good idea.
> >
> > How should it work though? I can grab all the .tar.z files from
> > http://sourceforge.net/adobe/cmap/home/Home/, extract them, and throw
> > them
> > somewhere into /usr/share (/usr/share/cmap/)... then packages can grab
> > up-to-date copies of the cmap resources from there?
>
>  that content should (loosely at least) match the content within the
>  poppler-
>  data package. 
Yes, poppler is clearly one of the things that embedded cmap and needs
unbundling (I *think* at least fontforge does the same, it's been a long
time since I checked, but cmap embedding was widespread, and the legal
aspects dubious before Adobe released a separate cmap project after some
Debian prodding IIRC)

-- 
Nicolas Mailhot

[Adam Williamson]

On Fri, 2012-09-14 at 15:00 -0400, Ben Rosser wrote:
>  On Fri, Sep 14, 2012 at 9:46 AM, Nicolas Mailhot
>  <nicolas.mailhot(a)laposte.net&gt; wrote:
>          
>          
>          Adobe CMAP is used by lots of font, ps and pdf related apps
>          and IIRC
>          packaging it is currently on the wishlist.
>          
>          You should unbundle it and create a separate package.
>          
>          I don't think CMAP has security implications, but projects
>          that bundle old
>          versions of the Adobe datafiles will behave in non-optimal
>          ways when they
>          hit a font/text combination that requires a mapping Adobe
>          introduced in a
>          later version.
>  
>  
>  That sounds like a good idea.
>  
>  
>  How should it work though? I can grab all the .tar.z files
>  from http://sourceforge.net/adobe/cmap/home/Home/, extract them, and
>  throw them somewhere into /usr/share (/usr/share/cmap/)... then
>  packages can grab up-to-date copies of the cmap resources from there? 
Don't know the specifics of this case, but in general when you're trying
to extract something that multiple packages have embedded like this, you
need to create a package of the shared resource and then ensure that
each package which is currently embedding it is adjusted to use the
system-wide version. Sometimes the build process is already capable of
doing this and it's just a case of doing whatever you have to do
(BuildRequires, configure parameters...) to tell it to use it, sometimes
you'll have to patch it and send the patch upstream.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net

[Ben Rosser]

Here's a preliminary cmap rpm review request:
https://bugzilla.redhat.com/show_bug.cgi?id=859246

So far, the only package I've worked on making this compatible with is my
own, pdfminer: https://bugzilla.redhat.com/show_bug.cgi?id=823679. So it
will almost certainly require modifications to work nicely with
poppler-data and whatever other bundled packages there are- that's a work
in progress.

Any idea what the simplest way would be to get a list of all packages
bundling cmap data?

Ben

[Nicolas Mailhot]

Le Jeu 20 septembre 2012 23:38, Ben Rosser a écrit :
>  Here's a preliminary cmap rpm review request:
>  https://bugzilla.redhat.com/show_bug.cgi?id=859246
>
>  So far, the only package I've worked on making this compatible with is my
>  own, pdfminer: https://bugzilla.redhat.com/show_bug.cgi?id=823679. So it
>  will almost certainly require modifications to work nicely with
>  poppler-data and whatever other bundled packages there are- that's a work
>  in progress.
>
>  Any idea what the simplest way would be to get a list of all packages
>  bundling cmap data? 
A first order approximation would be a repoquery on significant file name
patterns

If that does not work, check everything involved in creating pdfs of
maniputaling fonts.

Another good idea would be to ask debian-side as they had their own cmap
unbundling to do

Best regards,

-- 
Nicolas Mailhot

[Nicolas Mailhot]

Le Ven 21 septembre 2012 13:36, Nicolas Mailhot a écrit :
>  Le Jeu 20 septembre 2012 23:38, Ben Rosser a écrit :
> > Here's a preliminary cmap rpm review request:
> > https://bugzilla.redhat.com/show_bug.cgi?id=859246
> >
> > So far, the only package I've worked on making this compatible with is
> > my
> > own, pdfminer: https://bugzilla.redhat.com/show_bug.cgi?id=823679. So it
> > will almost certainly require modifications to work nicely with
> > poppler-data and whatever other bundled packages there are- that's a
> > work
> > in progress.
> >
> > Any idea what the simplest way would be to get a list of all packages
> > bundling cmap data?
>
>  A first order approximation would be a repoquery on significant file name
>  patterns
>
>  If that does not work, check everything involved in creating pdfs of
>  maniputaling fonts.
>
>  Another good idea would be to ask debian-side as they had their own cmap
>  unbundling to do 
BTW, I havent looked at your app, but if it bundles cmap it is highly
probable it also bundles unicode.org codepoint tables (and those need
unbundling everywhere, starting from perl IIRC).

-- 
Nicolas Mailhot