Bluehorn's Blog » dynamic linker http://www.landschoff.net/blog Ramblings of Torsten Landschoff Thu, 08 Jul 2010 10:43:01 +0000 http://wordpress.org/?v=2.9.2 en hourly 1 Hitting the dynamic linker wall… http://www.landschoff.net/blog/2009/12/hitting-the-dynamic-linker-wall/ http://www.landschoff.net/blog/2009/12/hitting-the-dynamic-linker-wall/#comments Wed, 23 Dec 2009 17:01:27 +0000 admin http://www.landschoff.net/blog/?p=127 I was working on replacing some mockup code for testing an internal library with python. Basically, the C code is incredibly big and I would rather mock the 3rd party API we are using in python. I wrote a generator to create wrapping code that forwards the C API calls to my python module.

Now that most of the work is finished, I get the following error message from my python mock:

1
2
3
4
5
6
Traceback (most recent call last):
  File "simple_mockup.py", line 2, in <module>
    import threading
  File "/usr/lib/python2.5/threading.py", line 11, in <module>
    from time import time as _time, sleep as _sleep
ImportError: /usr/lib/python2.5/lib-dynload/time.so: undefined symbol: PyExc_ValueError

What’s going on here? This issue reminds me of a bug of my ancient Debian times which affected loading of GTK theme engines from python-gtk. The bug (#38138) is so old, it’s not even in the BTS archive anymore…

The problem is illustrated by the following example program (consisting of three files):

demo.c
1
2
3
4
5
6
7
#include <Python.h>

void test_python()
{
    Py_Initialize();
    PyRun_SimpleString("import threading\n");
}
main.c
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
#include <stdio.h>
#include <dlfcn.h>

int main(void)
{
    const char *error;

    void *handle = dlopen("./demo.so", RTLD_LAZY | EXTRA_RTLD_FLAGS);
    void (*test_python)();
    *(void**) &test_python = dlsym(handle, "test_python");
    if ((error = dlerror())) {
        fprintf(stderr, "%s\n", error);
        return 1;
    }

    printf("Calling test_python @ %p in shared object @ %p.\n", test_python, handle);
    (*test_python)();
    dlclose(handle);
    printf("Feels fine, finishing.\n");
    return 0;
}
run_it.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
#! /bin/sh

gcc -shared `python-config --includes` -o demo.so demo.c `python-config --ldflags`

echo "Running example with default RTLD flags:"
gcc -DEXTRA_RTLD_FLAGS=0 -o main main.c -ldl
./main
echo

echo "Passing RTLD_GLOBAL in addition:"
gcc -DEXTRA_RTLD_FLAGS=RTLD_GLOBAL -o main main.c -ldl
./main
echo

On my system, this results in the following output:

torsten@pulsar:~/sh_bug$ ./run_it.sh
Running example with default RTLD flags:
Calling test_python @ 0xb77084bc in shared object @ 0x96bc018.
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python2.5/threading.py", line 11, in 
    from time import time as _time, sleep as _sleep
ImportError: /usr/lib/python2.5/lib-dynload/time.so: undefined symbol: PyExc_ValueError
Feels fine, finishing.

Passing RTLD_GLOBAL in addition:
Calling test_python @ 0xb783f4bc in shared object @ 0x95e1018.
Feels fine, finishing.

Sucky. So embedding Python into an application is easy but if you want to use the interpreter and its modules from a plugin, you are out of luck. Somebody knows a way around this problem? The only solution I can think of is to link libpython.so into each of the python plugins.

Update: Work around

Small update: For my current problem, the work around is to run the application with the python library preloaded: LD_PRELOAD=/usr/lib/libpython2.5.so.1.0 app. Back to adding functionality…

]]> http://www.landschoff.net/blog/2009/12/hitting-the-dynamic-linker-wall/feed/ 0