I was working on replacing some mockup code for testing an internal library with python. Basically, the C code is incredibly big and I would rather mock the 3rd party API we are using in python. I wrote a generator to create wrapping code that forwards the C API calls to my python module.

Now that most of the work is finished, I get the following error message from my python mock:

What’s going on here? This issue reminds me of a bug of my ancient Debian times which affected loading of GTK theme engines from python-gtk. The bug (#38138) is so old, it’s not even in the BTS archive anymore…

The problem is illustrated by the following example program (consisting of three files):

demo.c

1 2 3 4 5 6 7

#include <Python.h> void test_python() {     Py_Initialize();     PyRun_SimpleString("import threading\n"); }

main.c

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

#include <stdio.h> #include <dlfcn.h> int main(void) {     const char *error;     void *handle = dlopen("./demo.so", RTLD_LAZY | EXTRA_RTLD_FLAGS);     void (*test_python)();     *(void**) &test_python = dlsym(handle, "test_python");     if ((error = dlerror())) {         fprintf(stderr, "%s\n", error);         return 1;     }     printf("Calling test_python @ %p in shared object @ %p.\n", test_python, handle);     (*test_python)();     dlclose(handle);     printf("Feels fine, finishing.\n");     return 0; }

run_it.sh

1 2 3 4 5 6 7 8 9 10 11 12 13

#! /bin/sh gcc -shared `python-config --includes` -o demo.so demo.c `python-config --ldflags` echo "Running example with default RTLD flags:" gcc -DEXTRA_RTLD_FLAGS=0 -o main main.c -ldl ./main echo echo "Passing RTLD_GLOBAL in addition:" gcc -DEXTRA_RTLD_FLAGS=RTLD_GLOBAL -o main main.c -ldl ./main echo

On my system, this results in the following output:

torsten@pulsar:~/sh_bug$ ./run_it.sh
Running example with default RTLD flags:
Calling test_python @ 0xb77084bc in shared object @ 0x96bc018.
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python2.5/threading.py", line 11, in 
    from time import time as _time, sleep as _sleep
ImportError: /usr/lib/python2.5/lib-dynload/time.so: undefined symbol: PyExc_ValueError
Feels fine, finishing.

Passing RTLD_GLOBAL in addition:
Calling test_python @ 0xb783f4bc in shared object @ 0x95e1018.
Feels fine, finishing.

Sucky. So embedding Python into an application is easy but if you want to use the interpreter and its modules from a plugin, you are out of luck. Somebody knows a way around this problem? The only solution I can think of is to link libpython.so into each of the python plugins.

Update: Work around

Small update: For my current problem, the work around is to run the application with the python library preloaded: LD_PRELOAD=/usr/lib/libpython2.5.so.1.0 app. Back to adding functionality…