[Chimera-users] Numeric/numpy revisited

[Thomas Goddard]

Hi Randy,

  I have spent the past several days switching Chimera from Numeric to
NumPy.  I observe the same problem that you do trying to import both
packages on Mac OS 10.4.8 using Chimera version 1.2328.

  I think the issue is a namespace clash between

	site-packages/Numeric/multiarray.so
	site-packages/numpy/core/multiarray.so

and I think it only happens with the Chimera Python because we build
our Python with a compilation flag -DUSE_DYLD_GLOBAL_NAMESPACE.
That is in Chimera source code chimera/foreign/Python/GNUmakefile.
Thanks to Greg Couch for pointing that out.  Because of that flag
only one of the two initmultiarray() routines from the Numeric and
numpy libraries is being called -- probably just the first one loaded.

Here's the comment about USE_DYLD_GLOBAL_NAMEPSPACE from Python source
code file Python-2.4.3/Python/dynload_next.c

/*
** Python modules are Mach-O MH_BUNDLE files. The best way to load these
** is each in a private namespace, so you can load, say, a module bar and a
** module foo.bar. If we load everything in the global namespace the two
** initbar() symbols will conflict.
** However, it seems some extension packages depend upon being able to access
** each others' global symbols. There seems to be no way to eat our cake and
** have it, so the USE_DYLD_GLOBAL_NAMESPACE define determines which behaviour
** you get.
*/

  Why do we use -DUSE_DYLD_GLOBAL_NAMESPACE?  I couldn't remember so
I compiled Python without -DUSE_DYLD_GLOBAL_NAMESPACE and found that
Chimera will not start using it.  It fails to find _init_chimera(),
the python initialization function fo the _chimera.so module.  Why is
that?  There is an _chimera.dylib (dynamic library) and a _chimera.so
(mach bundle).  The latter is for importing using Python, and the former
is for other Chimera modules (eg _surface.so) that need to link against
it.  The linking and Python loading could not be done with the same
object a year ago when we investigated this.  The _chimera.so just
links against the _chimera.dylib.  And for some reason the Python import
(a dlopen() system call) then sees no symbols in the _chimera.so unless
the USE_DYLD_GLOBAL_NAMESPACE is used.  We have considered complicated
fixes for that but have not pursued them.

  So the only two options I see are to wait a bit for our numpy Chimera,
or wait for us to work out a way to drop USE_DYLD_GLOBAL_NAMESPACE.
Both are not so good, involve waiting.  I think the conversion to numpy
will happen faster. That conversion may take some weeks or a month.
We are having problems compiling numpy on IRIX and Tru64 platforms and
about 1/3 of the Chimera Numeric code remains to be converted to numpy.
The other 2/3 which is the volume data code has been converted and is
working fine under numpy on Linux and Mac.  The total effort for the
conversion is probably just another 3 days but we are busy with many
things and that work will be spread out over a longer time.

	Tom