More python 2.7 fun: deprecation of PyCObject API

[Toshio Kuratomi]

On Mon, Aug 16, 2010 at 02:03:51PM -0400, David Malcolm wrote:
> On Fri, 2010-08-13 at 21:57 -0400, Toshio Kuratomi wrote:
> > > > On Fri, Aug 13, 2010 at 02:20:51PM -0400, David Malcolm wrote:
> > If this is okay, then I'd modify your point (a) to be this plan:
> > 
> > When code that turns all warnings into errors is encountered, have it
> > instead cause PendingDeprecationWarning to print to stderr via
> > warnings.simplefilter('default', PendingDeprecationWarning)
> 
> I don't like this approach: it's a divergence from upstream, and it
> seems like magic: it seems to be second-guessing the API call.
> 
> It seems simpler and clearer to ask that code that enables errors on
> warnings need to turn it off for PendingDeprecationWarning to be able to
> use modules that are known to break in the presence of the
> PendingDeprecationWarning-treated-as-exceptions.
> 

As long as you're saying turn off 'error' for all
PendingDeprecationWarnings, not just ones that are known to break,
I think we're saying the same thing in different words.

Here's how I see it working.

python-foo:  Does not touch the warnings filters.  No need to modify
anything.

pyhton-bar: Sets warnings.simplefilter('default').  No need to modify
anything

pyhon-baz: Sets warnings.simplefilter('errors').  Patch the code to set
warnings.simplefilter('default', PendingDeprecationWarning) and send a note
to upstream that 'errors' can cause segfaults in pieces of code unrelated to
python-baz.

python-qux: Contains code using the old PyCObject API and doesn't catch the
exception.  No need to modify.

The key here, and only place where I'm not sure if we're saying the same
thing is that setting simplefilter('errors') can break unrelated code.  So
python-baz might not use any code that breaks but python-corge might use
both python-baz and python-qux which breaks in this combination.  So we
never want that to go through without adding a second filter somewhere.  The
easiest place to track the adding of the second filter is in python-baz,
right after the filter that sets up 'errors'.


> > > > > One issue here is that this API expresses a binary interface between
> > > > > different Python modules, and that we don't yet have a way to express
> > > > > this at the rpm metadata level.  I think we should, to make it easier to
> > > > > track these issues in the future.  I don't think it's possible to track
> > > > > these automatically, but we could do it manually.
> > > > > 
> > > > Tracking this manualy is no good unless you can explain to people how to
> > > > detect it.  Once you can explain how to manually detect it, it might be
> > > > possible to automatically detect it....
> > > 
> > > You have to scrape through for ABI calls to PyCObject: the presence of
> > > the calls are visible in the ELF metadata, but not the exact strings.
> > > Actually, it _might_ be possible to figure them out via disassembly of
> > > the machine code, but this seems fragile.
> > > 
> > This already sounds like something that is too involved for maintainers and
> > package reviewers to do.  I think this might be something that doesn't leave
> > the drawing board without tooling to at least do part of the detective work.
> 
> Fundamentally the review process for this involves grepping through the
> source code; any usage of a PyCObject_* function will return NULL if
> PendingDeprectationWarning is set to "error".  It needs a little ability
> to read C code, but (I hope) not much.  The issue with macros in the
> header files is a nasty gotcha, though :-(
> 
What I'm commenting on here is the desire to stick all uses of the PyCapsule
API into an RPM virtual Provide and Require.

It doesn't seem like something that we can require of packagers and
reviewers without raising the barrier of entry for their skillset quite
a bit.

> 
> I had a go at writing some release notes for this issue here:
> https://fedoraproject.org/wiki/Features/Python_2.7#Caveat:_PyCObject_and_warnings
> 
> How does this look?
> 
Looks good but change: 
- warnings.simplefilter('ignore', PendingDeprecationWarning)
+ warnings.simplefilter('default', PendingDeprecationWarning)

If the coder is going through the trouble of turning warnings into 'error',
they likely want to be at least warned about PendingDeprecationWarning as
well.

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
Url : http://lists.fedoraproject.org/pipermail/devel/attachments/20100816/2aa1ba3a/attachment.bin