May 2002 Progress Report
Contents
- Introduction
- New Features
- Shared Library Support for AIX
-
- Class Enhancements
-
- Operators
-
- Iterators
-
- Properties
-
- setattr
-
- __module__ Attribute
-
- back_reference
-
- Documentation
- Miscellaneous
- Converters
- Checkins Mailing List
- Shared Libraries
- What's Next
Aside from library development, work on Boost.Python in May was
focused on reducing the support burden. In recent weeks, responding to
requests for support, espcially surrounding building the library, had
begun to impede progress on development. There was a major push to
release a stable 1.28.0 of Boost, including documentation of Boost.Build and specific
instructions for building Boost.Python
v1. The documentation for Boost.Python v2 was also updated as
described here.
The Kull group required the ability to build and test Boost.Python
extensions on AIX, a platform with "creatively designed"
shared library semantics. Making this work was a multi-pronged
effort, involving changes to Boost.Build and some great research by
Martin Casado which uncovered the key mechanism required to allow
shared libraries to use functions from the Python executable. The
current solution used in Boost.Build relies on a Python
Script as part of the build process. This is not a problem for
Boost.Python, as Python will be available. However, the commands
issued by the script are so simple that a 100%-pure-Boost.Jam
solution is surely possible. Linking on AIX is sufficiently
interesting to have skewed the Boost.Python development schedule a
bit.
Support for exposing C++ operators and functions as the corresponding
Python special methods was added. Thinking that the Boost.Python
v1 interface was a little too esoteric (especially the use of
left_operand<...>/right_operand<...>
for
asymmetric operands), I introduced a simple form of expression
templates which allow users to simply write the expressions that
should be wrapped, as in this example.
Python iterator support as required by the Kull project resulted in a
highly flexible interface allowing:
- Direct exposure of a class'
begin()
and
end()
functions:
...
.def("__iter__", iterator<list_int>())
-
- Creation of iterators from member functions...
...
.def("__iter__"
, range(&my_class::x_begin, &my_class::x_end))
)
-
- ...and member data:
...
.def("__iter__"
, range(&std::pair<char*,char*>::first, &std::pair<char*,char*>::second))
)
-
- The ability to specify CallPolicies, e.g. to prevent copying of
heavyweight values:
...
.def("__iter__",
, range<return_value_policy<copy_non_const_reference> >(
&my_sequence<heavy>::begin
, &my_sequence<heavy>::end))
-
The Kull iteration interfaces also required the ability to iterate
over a sequence specified by an instance's attribute:
>>> f = field()
>>> for e in f.elements:
... print e,
This forced the exposure of the property
interface used internally to implement the data member exposure
facility described in March. Properties are an
incredibly useful idiom, so it's good to be able to provide them
at little new development cost.
class_<>
acquired a setattr
member
function which allows users to easily add new Python objects as class
attributes.
Ralf Grosse-Kunstleve has been working on pickling support for v2. To
make it work correctly, he had to make sure that a class'
__module__
attribute was set correctly.
The new back_reference<T>
template can be used as a
function parameter when the user needs access to both a T
argument and to the Python object which manages it. The function will
only match in the overload resolution process if it would match the
same function signature with T
substituted for
back_reference<T>
. This feature is not yet
documented.
In a major effort to prepare Boost.Python v2 to replace v1, many pages
of new reference documentation were added:
- CallPolicies.html
-
- Dereferenceable.html
-
- Extractor.html
-
- HolderGenerator.html
-
- ResultConverter.html
-
- call_method.html
-
- callbacks.html
-
- data_members.html
-
- has_back_reference.html
-
- implicit.html
-
- instance_holder.html
-
- operators.html
-
- ptr.html
-
- type_id.html
-
- with_custodian_and_ward.html
-
Major updates were made to the following pages:
- call.html
-
- updated
-
- class.html
-
- reference.html
-
As usual, careful documentation forces one to consider the
interface again, and there were many interface changes
associated with this effort, including the elevation of the
following components from implementation detail to
first-class library citizen:
- type_id.hpp
-
- pointee.hpp
-
- lvalue_from_pytype.hpp
It appears that the world of C++ <==> Python conversion rules is
an endlessly-rich area of exploration. Completing the conversions for
char
and char const*
types, as described at
the end of April's report,
uncovered some interesting new shades to the problem. It turns out to
be worth distinguishing mutable and immutable lvalue conversions,
because despite the fact that Python doesn't understand
const
, it does understand immutability (c.f. Python
strings, which expose an immutable char
pointer). It is
also worth recognizing types which represent lvalue sequences,
to prevent Python "foobar"
from being silently
truncated to C++ 'f'
. More details on this insight can be
found in the mailing list
archive. I don't plan to do anything about this immediately, but I
do think it's the right direction to go in the long run.
In order to better coordinate changes made by multiple developers, I
enabled syncmail
for the Boost.Python CVS trees, and established an associated mailing
list. Subscribe to this list to receive notices of each new
checkin.
Beyond the vagaries of dynamic linking on AIX, I have been
participating in a more-general discussion of dynamic linking for
C++. Needless to say, C++ dynamic linking is of critical importance to
Boost.Python: all extension modules are normally built as shared
libraries, and Boost.Python extension modules share a common library
as well.
In fact, there are at least two separate conversations. One
in the C++ standard extensions mailing list concerns what can be
standardized for C++ and shared libraries; the other, mostly on the gcc mailing list, concerns the
behavior of GCC on Posix/ELF platforms.
Some of the GCC threads are here:
http://gcc.gnu.org/ml/gcc/2002-05/msg02002.html
http://gcc.gnu.org/ml/gcc/2002-05/msg02945.html
http://gcc.gnu.org/ml/gcc/2002-05/msg01758.html
Development is focused on what's needed to be able to retire
Boost.Python v1. At the moment, that means deciding the user-friendly
interfaces for to_/from_python conversion, and formally exposing the
Python object smart pointers and object wrapper classes. Quite a few
questions have also been showing up recently about how to embed Python
with Boost.Python, and how to link with it statically; the solutions
to these issues will probably have to be formalized before long.
Revised
13 November, 2002
© Copyright Dave Abrahams
2002.