Opened 5 years ago

Closed 5 years ago

#3748 closed defect (fixed)

Can't find split-off atomic lib on Linux

Reported by: goddard@… Owned by: pett
Priority: critical Milestone: 1.2
Component: Build System Version:
Keywords: Cc: chimera-programmers
Blocked By: Blocking:
Notify when closed: Platform: all
Project: ChimeraX

Description (last modified by Tom Goddard) 

The following bug report has been submitted:
Platform:        Linux-5.4.0-47-generic-x86_64-with-debian-buster-sid
ChimeraX Version: 1.2.dev202009220514 (2020-09-22 05:14:36 UTC)
Description
ChimeraX starts with errors, cannot open libatomstruct.so

Log:
Startup Errors  
---  
errors | Bundle 'ChimeraX-DistMonitor' custom initialization failed  
Bundle 'ChimeraX-Atomic' custom initialization failed  
Bundle 'ChimeraX-SwapRes' custom initialization failed  
Bundle 'ChimeraX-ChemGroup' custom initialization failed  
Bundle 'ChimeraX-Markers' custom initialization failed  
warnings | Traceback (most recent call last):  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/toolshed/info.py", line 452, in get_module  
m = importlib.import_module(self.package_name)  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/importlib/__init__.py", line
127, in import_module  
return _bootstrap._gcd_import(name[level:], package, level)  
File "<frozen importlib._bootstrap>", line 1006, in _gcd_import  
File "<frozen importlib._bootstrap>", line 983, in _find_and_load  
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked  
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked  
File "<frozen importlib._bootstrap_external>", line 728, in exec_module  
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/dist_monitor/__init__.py", line 15, in <module>  
from .cmd import SimpleMeasurable, ComplexMeasurable  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/dist_monitor/cmd.py", line 27, in <module>  
from chimerax.atomic import Atom  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/atomic/__init__.py", line 15, in <module>  
import chimerax.atomic_lib  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/atomic_lib/__init__.py", line 34, in <module>  
from .lib import _load_libs  
ImportError: libatomstruct.so: cannot open shared object file: No such file or
directory  
  
During handling of the above exception, another exception occurred:  
  
Traceback (most recent call last):  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/toolshed/info.py", line 341, in initialize  
api = self._get_api(session.logger)  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/toolshed/info.py", line 478, in _get_api  
m = self.get_module()  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/toolshed/info.py", line 454, in get_module  
raise ToolshedError("Error importing bundle %s's module: %s" % (self.name,
str(e)))  
chimerax.core.toolshed.ToolshedError: Error importing bundle ChimeraX-
DistMonitor's module: libatomstruct.so: cannot open shared object file: No
such file or directory  
  
Traceback (most recent call last):  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/toolshed/info.py", line 452, in get_module  
m = importlib.import_module(self.package_name)  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/importlib/__init__.py", line
127, in import_module  
return _bootstrap._gcd_import(name[level:], package, level)  
File "<frozen importlib._bootstrap>", line 1006, in _gcd_import  
File "<frozen importlib._bootstrap>", line 983, in _find_and_load  
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked  
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked  
File "<frozen importlib._bootstrap_external>", line 728, in exec_module  
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/atomic/__init__.py", line 15, in <module>  
import chimerax.atomic_lib  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/atomic_lib/__init__.py", line 34, in <module>  
from .lib import _load_libs  
ImportError: libatomstruct.so: cannot open shared object file: No such file or
directory  
  
During handling of the above exception, another exception occurred:  
  
Traceback (most recent call last):  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/toolshed/info.py", line 341, in initialize  
api = self._get_api(session.logger)  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/toolshed/info.py", line 478, in _get_api  
m = self.get_module()  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/toolshed/info.py", line 454, in get_module  
raise ToolshedError("Error importing bundle %s's module: %s" % (self.name,
str(e)))  
chimerax.core.toolshed.ToolshedError: Error importing bundle ChimeraX-Atomic's
module: libatomstruct.so: cannot open shared object file: No such file or
directory  
  
Traceback (most recent call last):  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/toolshed/info.py", line 452, in get_module  
m = importlib.import_module(self.package_name)  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/importlib/__init__.py", line
127, in import_module  
return _bootstrap._gcd_import(name[level:], package, level)  
File "<frozen importlib._bootstrap>", line 1006, in _gcd_import  
File "<frozen importlib._bootstrap>", line 983, in _find_and_load  
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked  
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked  
File "<frozen importlib._bootstrap_external>", line 728, in exec_module  
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/swap_res/__init__.py", line 14, in <module>  
from .swap_res import swap_aa, get_rotamers  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/swap_res/swap_res.py", line 17, in <module>  
from chimerax.atomic import AtomicStructure  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/atomic/__init__.py", line 15, in <module>  
import chimerax.atomic_lib  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/atomic_lib/__init__.py", line 34, in <module>  
from .lib import _load_libs  
ImportError: libatomstruct.so: cannot open shared object file: No such file or
directory  
  
During handling of the above exception, another exception occurred:  
  
Traceback (most recent call last):  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/toolshed/info.py", line 341, in initialize  
api = self._get_api(session.logger)  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/toolshed/info.py", line 478, in _get_api  
m = self.get_module()  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/toolshed/info.py", line 454, in get_module  
raise ToolshedError("Error importing bundle %s's module: %s" % (self.name,
str(e)))  
chimerax.core.toolshed.ToolshedError: Error importing bundle ChimeraX-
SwapRes's module: libatomstruct.so: cannot open shared object file: No such
file or directory  
  
Traceback (most recent call last):  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/toolshed/info.py", line 452, in get_module  
m = importlib.import_module(self.package_name)  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/importlib/__init__.py", line
127, in import_module  
return _bootstrap._gcd_import(name[level:], package, level)  
File "<frozen importlib._bootstrap>", line 1006, in _gcd_import  
File "<frozen importlib._bootstrap>", line 983, in _find_and_load  
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked  
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked  
File "<frozen importlib._bootstrap_external>", line 728, in exec_module  
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/chem_group/__init__.py", line 14, in <module>  
from .chem_group import find_group  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/chem_group/chem_group.py", line 18, in <module>  
from chimerax.atomic.idatm import type_info, tetrahedral, planar, linear,
single  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/atomic/__init__.py", line 15, in <module>  
import chimerax.atomic_lib  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/atomic_lib/__init__.py", line 34, in <module>  
from .lib import _load_libs  
ImportError: libatomstruct.so: cannot open shared object file: No such file or
directory  
  
During handling of the above exception, another exception occurred:  
  
Traceback (most recent call last):  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/toolshed/info.py", line 341, in initialize  
api = self._get_api(session.logger)  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/toolshed/info.py", line 478, in _get_api  
m = self.get_module()  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/toolshed/info.py", line 454, in get_module  
raise ToolshedError("Error importing bundle %s's module: %s" % (self.name,
str(e)))  
chimerax.core.toolshed.ToolshedError: Error importing bundle ChimeraX-
ChemGroup's module: libatomstruct.so: cannot open shared object file: No such
file or directory  
  
Traceback (most recent call last):  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/toolshed/info.py", line 452, in get_module  
m = importlib.import_module(self.package_name)  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/importlib/__init__.py", line
127, in import_module  
return _bootstrap._gcd_import(name[level:], package, level)  
File "<frozen importlib._bootstrap>", line 1006, in _gcd_import  
File "<frozen importlib._bootstrap>", line 983, in _find_and_load  
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked  
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked  
File "<frozen importlib._bootstrap_external>", line 728, in exec_module  
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/markers/__init__.py", line 67, in <module>  
from .markers import MarkerSet, create_link, selected_markers, selected_links  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/markers/markers.py", line 14, in <module>  
from chimerax.atomic import Structure  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/atomic/__init__.py", line 15, in <module>  
import chimerax.atomic_lib  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/atomic_lib/__init__.py", line 34, in <module>  
from .lib import _load_libs  
ImportError: libatomstruct.so: cannot open shared object file: No such file or
directory  
  
During handling of the above exception, another exception occurred:  
  
Traceback (most recent call last):  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/toolshed/info.py", line 341, in initialize  
api = self._get_api(session.logger)  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/toolshed/info.py", line 478, in _get_api  
m = self.get_module()  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/toolshed/info.py", line 454, in get_module  
raise ToolshedError("Error importing bundle %s's module: %s" % (self.name,
str(e)))  
chimerax.core.toolshed.ToolshedError: Error importing bundle ChimeraX-
Markers's module: libatomstruct.so: cannot open shared object file: No such
file or directory  
  
Traceback (most recent call last):  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/toolshed/info.py", line 531, in start_tool  
ti = api._api_caller.start_tool(api, session, self, tool_info)  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/toolshed/__init__.py", line 1251, in start_tool  
return cls._get_func(api, "start_tool")(session, ti.name)  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/model_panel/__init__.py", line 21, in start_tool  
return model_panel(session, tool_name)  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/model_panel/tool.py", line 350, in model_panel  
_mp = ModelPanel(session, tool_name)  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/model_panel/tool.py", line 100, in __init__  
from chimerax import atomic  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/atomic/__init__.py", line 15, in <module>  
import chimerax.atomic_lib  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/atomic_lib/__init__.py", line 34, in <module>  
from .lib import _load_libs  
ImportError: libatomstruct.so: cannot open shared object file: No such file or
directory  
  
During handling of the above exception, another exception occurred:  
  
Traceback (most recent call last):  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/tools.py", line 379, in start_tools  
bi.start_tool(session, tool_name)  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/toolshed/info.py", line 537, in start_tool  
"start_tool() failed for tool %s in bundle %s:\n%s" % (tool_name, self.name,
str(e)))  
chimerax.core.toolshed.ToolshedError: start_tool() failed for tool Model Panel
in bundle ChimeraX-ModelPanel:  
libatomstruct.so: cannot open shared object file: No such file or directory  
  
Tool "Model Panel" failed to start:  
chimerax.core.toolshed.ToolshedError: start_tool() failed for tool Model Panel
in bundle ChimeraX-ModelPanel:  
libatomstruct.so: cannot open shared object file: No such file or directory  
  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/toolshed/info.py", line 537, in start_tool  
"start_tool() failed for tool %s in bundle %s:\n%s" % (tool_name, self.name,
str(e)))  
  
See log for complete Python traceback.  
  
UCSF ChimeraX version: 1.2.dev202009220514 (2020-09-22)  
© 2016-2020 Regents of the University of California. All rights reserved.  
How to cite UCSF ChimeraX  
Traceback (most recent call last):  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/updateloop.py", line 138, in _redraw_timer_callback  
drew = self.draw_new_frame()  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/updateloop.py", line 56, in draw_new_frame  
from chimerax import atomic  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/atomic/__init__.py", line 15, in <module>  
import chimerax.atomic_lib  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/atomic_lib/__init__.py", line 34, in <module>  
from .lib import _load_libs  
ImportError: libatomstruct.so: cannot open shared object file: No such file or
directory  
  
ImportError: libatomstruct.so: cannot open shared object file: No such file or
directory  
  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/atomic_lib/__init__.py", line 34, in  
from .lib import _load_libs  
  
See log for complete Python traceback.  
  
Traceback (most recent call last):  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/updateloop.py", line 138, in _redraw_timer_callback  
drew = self.draw_new_frame()  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/updateloop.py", line 56, in draw_new_frame  
from chimerax import atomic  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/atomic/__init__.py", line 15, in <module>  
import chimerax.atomic_lib  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/atomic_lib/__init__.py", line 34, in <module>  
from .lib import _load_libs  
ImportError: libatomstruct.so: cannot open shared object file: No such file or
directory  
  
ImportError: libatomstruct.so: cannot open shared object file: No such file or
directory  
  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/atomic_lib/__init__.py", line 34, in  
from .lib import _load_libs  
  
See log for complete Python traceback.  
  
Traceback (most recent call last):  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/updateloop.py", line 138, in _redraw_timer_callback  
drew = self.draw_new_frame()  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/updateloop.py", line 56, in draw_new_frame  
from chimerax import atomic  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/atomic/__init__.py", line 15, in <module>  
import chimerax.atomic_lib  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/atomic_lib/__init__.py", line 34, in <module>  
from .lib import _load_libs  
ImportError: libatomstruct.so: cannot open shared object file: No such file or
directory  
  
ImportError: libatomstruct.so: cannot open shared object file: No such file or
directory  
  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/atomic_lib/__init__.py", line 34, in  
from .lib import _load_libs  
  
See log for complete Python traceback.  
  
Traceback (most recent call last):  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/updateloop.py", line 138, in _redraw_timer_callback  
drew = self.draw_new_frame()  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/core/updateloop.py", line 56, in draw_new_frame  
from chimerax import atomic  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/atomic/__init__.py", line 15, in <module>  
import chimerax.atomic_lib  
File "/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-
packages/chimerax/atomic_lib/__init__.py", line 34, in <module>  
from .lib import _load_libs  
ImportError: libatomstruct.so: cannot open shared object file: No such file or
directory  
  
ImportError: libatomstruct.so: cannot open shared object file: No such file or
directory

Change History (24)

comment:1 by Tom Goddard, 5 years ago

Description: modified (diff)
Platform: → all
Project: → ChimeraX

Here are the libraries in atomic_lib/lib

/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-packages/chimerax/atomic_lib/lib:

total used in directory 2004 available 197759388
drwxr-xr-x 3 root root 4096 Sep 22 18:13 .
drwxr-xr-x 6 root root 4096 Sep 22 18:13 ..
-rw-r--r-- 1 root root 0 Sep 22 02:50 init.py
-rwxr-xr-x 1 root root 1946512 Sep 22 02:50 libatomstruct.so
-rwxr-xr-x 1 root root 63792 Sep 22 02:50 libelement.so
-rwxr-xr-x 1 root root 9344 Sep 22 02:50 libpyinstance.so
-rwxr-xr-x 1 root root 10400 Sep 22 02:50 _load_libs.so
drwxr-xr-x 2 root root 4096 Sep 22 18:13 pycache

comment:2 by Tom Goddard, 5 years ago

Tried ldd on _load_libs.so and libatomstruct.so to see what they are trying to link to.

goddard@beetracker:/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-packages/chimerax/atomic_lib/lib$ ls
init.py libelement.so* _load_libs.so*
libatomstruct.so* libpyinstance.so* pycache/
goddard@beetracker:/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-packages/chimerax/atomic_lib/lib$ ldd _load_libs.so

linux-vdso.so.1 (0x00007ffd8551f000)
libatomstruct.so => not found
libelement.so => not found
libpyinstance.so => not found
libarrays.so => not found
liblogger.so => not found
libappdirs.so => not found
libpython3.7m.so.1.0 => not found
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fc996b0f000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fc996771000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fc996559000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fc996168000)
/lib64/ld-linux-x86-64.so.2 (0x00007fc996e98000)

goddard@beetracker:/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-packages/chimerax/atomic_lib/lib$ ldd libatomstruct.so

linux-vdso.so.1 (0x00007fff98386000)
libelement.so => not found
libpyinstance.so => not found
libappdirs.so => not found
liblogger.so => not found
libarrays.so => not found
libpython3.7m.so.1.0 => not found
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f23b9f78000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f23b9bda000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f23b99c2000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f23b95d1000)
/lib64/ld-linux-x86-64.so.2 (0x00007f23ba301000)

goddard@beetracker:/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-packages/chimerax/atomic_lib/lib$

comment:3 by Tom Goddard, 5 years ago

I see that even though libatomstruct.so and _load_libs.so are in the same directory, ldd _load_libs.so does not find libatomstruct.so. So maybe runtime linking does not check the same directory.

comment:4 by pett, 5 years ago

Cc: chimera-programmers added
Component: Unassigned → Build System
Milestone: → 1.2
Owner: set to pett
Priority: normal → critical
Status: new → accepted
Summary: ChimeraX bug report submission → Can't find split-off atomic lib on Linux

comment:5 by Tom Goddard, 5 years ago

Starting with atomic_lib/lib added to the loader path and ChimeraX starts without errors, loads a PDB, swapaa mouse mode works..., all seems working.

$ LD_LIBRARY_PATH=/usr/lib/ucsf-chimerax-daily/lib/python3.7/site-packages/chimerax/atomic_lib/lib chimerax-daily

comment:6 by Tom Goddard, 5 years ago

I tried adding a RUNPATH = . to _load_libs.so with chrpath but it currently does not have an RPATH or RUNPATH and the chrpath program cannot add one it can only change an existing one. But it seems like the needed fix is that _load_libs.so needs a RUNPATH = . allowing it to find dependent libraries in the current directory.

$ which chrpath
$ chrpath _load_libs.so
_load_libs.so: no rpath or runpath tag found.
$ chrpath libatomstruct.so
libatomstruct.so: no rpath or runpath tag found.
$ cp -p _load_libs.so _load_libs.so.orig
$ sudo cp -p _load_libs.so _load_libs.so.orig
$ sudo chrpath -r . _load_libs.so
_load_libs.so: no rpath or runpath tag found.

comment:7 by pett, 5 years ago

Switched to having that library built by bundle builder, which does some rpath magic, instead of building "by hand". Guessing it will work since the other split libs build their loaders that way. I guess we'll see tomorrow.

comment:8 by Tom Goddard, 5 years ago

Still does not work -- I compiled on Ubuntu 18.04 and now the startup error look like below, trouble finding libelement.so.

$ cx
WARNING: Traceback (most recent call last):

File "/home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/site-packages/chimerax/core/toolshed/info.py", line 452, in get_module

m = importlib.import_module(self.package_name)

File "/home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/importlib/init.py", line 127, in import_module

return _bootstrap._gcd_import(name[level:], package, level)

File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/site-packages/chimerax/swap_res/init.py", line 14, in <module>

from .swap_res import swap_aa, get_rotamers

File "/home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/site-packages/chimerax/swap_res/swap_res.py", line 17, in <module>

from chimerax.atomic import AtomicStructure

File "/home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/site-packages/chimerax/atomic/init.py", line 15, in <module>

import chimerax.atomic_lib

File "/home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/site-packages/chimerax/atomic_lib/init.py", line 34, in <module>

from . import _load_libs

ImportError: libelement.so: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "/home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/site-packages/chimerax/core/toolshed/info.py", line 341, in initialize

api = self._get_api(session.logger)

File "/home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/site-packages/chimerax/core/toolshed/info.py", line 478, in _get_api

m = self.get_module()

File "/home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/site-packages/chimerax/core/toolshed/info.py", line 454, in get_module

raise ToolshedError("Error importing bundle %s's module: %s" % (self.name, str(e)))

chimerax.core.toolshed.ToolshedError: Error importing bundle ChimeraX-SwapRes's module: libelement.so: cannot open shared object file: No such file or directory

Now atomic_lib contains _load_libs*.so and the other libraries are in the lib subdirectory.

/home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/site-packages/chimerax/atomic_lib:
total used in directory 176 available 193095636
drwxrwxr-x 6 goddard goddard 4096 Sep 22 23:02 .
drwxrwxr-x 140 goddard goddard 4096 Sep 22 22:10 ..
drwxrwxr-x 3 goddard goddard 4096 Sep 22 22:10 data
drwxrwxr-x 5 goddard goddard 4096 Sep 22 22:10 include
-rw-rw-r-- 1 goddard goddard 1060 Sep 22 22:10 init.py
drwxrwxr-x 2 goddard goddard 4096 Sep 22 22:10 lib
-rwxrwxr-x 1 goddard goddard 151224 Sep 22 22:10 _load_libs.cpython-37m-x86_64-linux-gnu.so
drwxrwxr-x 2 goddard goddard 4096 Sep 22 22:22 pycache

/home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/site-packages/chimerax/atomic_lib/lib:
total used in directory 1896 available 193095636
drwxrwxr-x 2 goddard goddard 4096 Sep 22 22:10 .
drwxrwxr-x 6 goddard goddard 4096 Sep 22 23:02 ..
-rwxrwxr-x 1 goddard goddard 1856936 Sep 22 22:10 libatomstruct.so
-rwxrwxr-x 1 goddard goddard 60832 Sep 22 22:10 libelement.so
-rwxrwxr-x 1 goddard goddard 8424 Sep 22 22:10 libpyinstance.so

And _load_libs*.so has a RUNPATH:

$ chrpath _load_libs.cpython-37m-x86_64-linux-gnu.so
_load_libs.cpython-37m-x86_64-linux-gnu.so: RUNPATH=$ORIGIN/lib

And it is able to find libatomstruct.so using that RUNPATH:

$ ldd _load_libs.cpython-37m-x86_64-linux-gnu.so

linux-vdso.so.1 (0x00007ffe1551e000)
libatomstruct.so => /home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/site-packages/chimerax/atomic_lib/./lib/libatomstruct.so (0x00007f4de324a000)
libpython3.7m.so.1.0 => not found
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f4de2c24000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f4de2a0c000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f4de261b000)
libelement.so => not found
libpyinstance.so => not found
libappdirs.so => not found
liblogger.so => not found
libarrays.so => not found
libpython3.7m.so.1.0 => not found
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f4de227d000)
/lib64/ld-linux-x86-64.so.2 (0x00007f4de31b0000)

But libatomstruct.so links to libelement.so and libpyinstance.so (_load_libs*.so does not link directly to those) and libatomstruct.so cannot find those because the RUNPATH from _load_libs*.so only is used for the directly linked libraries of _load_libs*.so on Ubuntu 18 from what I read online. Now it seems libatomstruct.so needs a RUNPATH = . but it has no RUNPATH:

$ chrpath libatomstruct.so
libatomstruct.so: no rpath or runpath tag found.

comment:9 by Tom Goddard, 5 years ago

With the latest build it all works and opens PDB files if LD_LIBRARY_PATH is set to find the missing libraries:

$ LD_LIBRARY_PATH=~/ucsf/chimerax/ChimeraX.app/lib/python3.7/site-packages/chimerax/atomic_lib/lib cx

Here "cx" is a shell script shorthand for running chimerax.

comment:10 by Tom Goddard, 5 years ago

Here are the lines from the make output that create libatomstruct.so and _load_libs*.so. Seems the libatomstruct.so compile needs a "-Wl,-rpath,." option to g++ but I don't know how to achieve that.

g++ -pipe -fPIC -fdebug-prefix-map=/home/goddard/ucsf/chimerax/build=. -fstack-protector-strong -std=c++11 -fvisibility-ms-compat -fuse-ld=gold -Wl,--no-allow-shlib-undefined,--as-needed -O3 -Wall -Wextra -D_FORTIFY=2 -shared -o libatomstruct.so Structure.o Point.o destruct.o Sequence.o PBGroup.o ChangeTracker.o CoordSet.o PBManager.o MolResId.o RingCalc.o AtomicStructure.o Residue.o Atom.o Pseudobond.o AtomTypes.o seq_assoc.o StructureSeq.o Bond.o CompSS.o Ring.o Chain.o -L/home/goddard/ucsf/chimerax/build/lib tmpl/*.o -L../lib -lelement -lpyinstance -lioutil -lchutil -lappdirs -llogger -L/home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/site-packages/chimerax/arrays/lib -larrays -L/home/goddard/ucsf/chimerax/build/lib -lpython3.7m

c++ -pipe -fPIC -std=gnu99 -fdebug-prefix-map=/home/goddard/ucsf/chimerax/build=. -fstack-protector-strong -I/home/goddard/ucsf/chimerax/build/include -L/home/goddard/ucsf/chimerax/build/lib -DUSE_DYLD_GLOBAL_NAMESPACE -pthread -shared -O3 -Wall -Wextra -D_FORTIFY=2 build/temp.linux-x86_64-3.7/atomic_cpp/load_libs.o -L/home/goddard/ucsf/chimerax/ChimeraX.app/lib -Lsrc/lib -L/home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/site-packages/chimerax/arrays/lib -L/home/goddard/ucsf/chimerax/build/lib -latomstruct -lpython3.7m -o build/lib.linux-x86_64-3.7/chimerax/atomic_lib/_load_libs.cpython-37m-x86_64-linux-gnu.so -Wl,-rpath,$ORIGIN/lib

comment:11 by pett, 5 years ago

Well, I was going to convert atomic_lib to completely use bundle builder at some point because "it would be better that way". Now it seems that I need to do it to get things to actually work! Will be working on the conversion today.

comment:12 by pett, 5 years ago

Status: accepted → feedback

Well, it was touch and go for awhile, with me thinking multiple times that I was going to have to split atomic_lib into a bundle per library, but I did eventually manage to convert everything over to Bundle Builder. The changes are pushed. Works on the Mac. Remains to be seen about Linux/Windows.

comment:13 by Tom Goddard, 5 years ago

With git source pulled 5 minutes ago and make install done in atomic_lib ChimeraX starts up with this error:

$ cx
WARNING: Traceback (most recent call last):

File "/home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/site-packages/chimerax/core/toolshed/info.py", line 452, in get_module

m = importlib.import_module(self.package_name)

File "/home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/importlib/init.py", line 127, in import_module

return _bootstrap._gcd_import(name[level:], package, level)

File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/site-packages/chimerax/atomic/init.py", line 15, in <module>

import chimerax.atomic_lib

File "/home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/site-packages/chimerax/atomic_lib/init.py", line 34, in <module>

from . import _load_libs

ImportError: libpyinstance.so: cannot open shared object file: No such file or directory

_load_libs*.so is in the installed atomic_lib

/home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/site-packages/chimerax/atomic_lib:
total used in directory 176 available 193289336
drwxrwxr-x 6 goddard goddard 4096 Sep 23 15:54 .
drwxrwxr-x 140 goddard goddard 4096 Sep 23 15:54 ..
drwxrwxr-x 3 goddard goddard 4096 Sep 23 15:54 data
drwxrwxr-x 5 goddard goddard 4096 Sep 23 15:54 include
-rw-rw-r-- 1 goddard goddard 1060 Sep 23 15:54 init.py
drwxrwxr-x 2 goddard goddard 4096 Sep 23 15:54 lib
-rwxrwxr-x 1 goddard goddard 151224 Sep 23 15:54 _load_libs.cpython-37m-x86_64-linux-gnu.so
drwxrwxr-x 2 goddard goddard 4096 Sep 23 15:54 pycache

And the libpyinstance.so is in atomic_lib/lib

/home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/site-packages/chimerax/atomic_lib/lib:
total used in directory 41988 available 193289336
drwxrwxr-x 2 goddard goddard 4096 Sep 23 15:54 .
drwxrwxr-x 6 goddard goddard 4096 Sep 23 15:54 ..
-rwxrwxr-x 1 goddard goddard 27693648 Sep 23 15:54 libatomstruct.so
-rw-rw-r-- 1 goddard goddard 14673176 Sep 23 15:54 libatomstruct_tmpl.a
-rwxrwxr-x 1 goddard goddard 491544 Sep 23 15:54 libelement.so
-rwxrwxr-x 1 goddard goddard 118304 Sep 23 15:54 libpyinstance.so

_load_libs.so has RUNPATH pointing to lib director and has only a direct dependency on libatomstruct.so

$ chrpath _load_libs.cpython-37m-x86_64-linux-gnu.so
_load_libs.cpython-37m-x86_64-linux-gnu.so: RUNPATH=$ORIGIN/lib

libatomstruct.so depends on libpyinstance.so and has an incorrect RUNPATH looking in a non-existent lib subdirectory

$ chrpath libatomstruct.so
libatomstruct.so: RUNPATH=$ORIGIN/lib

Same with libelement.so

$ chrpath libelement.so
libelement.so: RUNPATH=$ORIGIN/lib

If I change the RUNPATH of both libatomstruct.so and libelement.so to . then both libraries find libpyinstance.so:

$ ldd libatomstruct.so

linux-vdso.so.1 (0x00007ffc6e797000)
libpyinstance.so => ./libpyinstance.so (0x00007f0c88a6e000)
libelement.so => ./libelement.so (0x00007f0c88862000)
libappdirs.so => not found
liblogger.so => not found
libarrays.so => not found
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f0c884d9000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f0c8813b000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f0c87f23000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f0c87b32000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f0c87913000)
/lib64/ld-linux-x86-64.so.2 (0x00007f0c8900a000)

$ ldd libelement.so

linux-vdso.so.1 (0x00007ffff3eb2000)
libpyinstance.so => ./libpyinstance.so (0x00007ff7c86a0000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007ff7c8317000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007ff7c80ff000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ff7c7d0e000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007ff7c7aef000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007ff7c7751000)
/lib64/ld-linux-x86-64.so.2 (0x00007ff7c8aaf000)

But still _load_libs*.so does not resolve libpyinstance.so

$ ldd _load_libs.cpython-37m-x86_64-linux-gnu.so

linux-vdso.so.1 (0x00007ffecebfc000)
libatomstruct.so => /home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/site-packages/chimerax/atomic_lib/./lib/libatomstruct.so (0x00007fee1f100000)
libpython3.7m.so.1.0 => not found
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fee1ed77000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fee1eb5f000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fee1e76e000)
libpyinstance.so => not found
libelement.so => not found
libappdirs.so => not found
liblogger.so => not found
libarrays.so => not found
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fee1e3d0000)
/lib64/ld-linux-x86-64.so.2 (0x00007fee1f69c000)

I don't get it.

If I move all the libraries into atomic_lib and change the RUNPATHS of _load_libs*.so, libatomstruct.so, libelement.so all to "." then ldd on _load_libs*.so says it finds the libraries

$ ldd _load_libs.cpython-37m-x86_64-linux-gnu.so

linux-vdso.so.1 (0x00007ffcdcb60000)
libatomstruct.so => ./libatomstruct.so (0x00007fd0aeb9c000)
libpython3.7m.so.1.0 => not found
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fd0ae813000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fd0ae5fb000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd0ae20a000)
libpyinstance.so => ./libpyinstance.so (0x00007fd0ae007000)
libelement.so => ./libelement.so (0x00007fd0addfb000)
libappdirs.so => not found
liblogger.so => not found
libarrays.so => not found
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fd0ada5d000)
/lib64/ld-linux-x86-64.so.2 (0x00007fd0af138000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fd0ad83e000)

then startup fails failing to find liblogger.so.

$ cx
WARNING: Traceback (most recent call last):

File "/home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/site-packages/chimerax/core/toolshed/info.py", line 452, in get_module

m = importlib.import_module(self.package_name)

File "/home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/importlib/init.py", line 127, in import_module

return _bootstrap._gcd_import(name[level:], package, level)

File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/site-packages/chimerax/atomic/init.py", line 15, in <module>

import chimerax.atomic_lib

File "/home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/site-packages/chimerax/atomic_lib/init.py", line 34, in <module>

from . import _load_libs

ImportError: liblogger.so: cannot open shared object file: No such file or directory

liblogger.so is at ~/ucsf/chimerax/ChimeraX.app/lib/liblogger.so which I would assume is found by the executable RPATH. Don't know why it doesn't find that.

Time for me to return to my own cross -- trying to build ChimeraX with Python 3.8 -- they got rid of python3.8m, nice but breaks are build all over the place.

comment:14 by pett, 5 years ago

Actually load_libs does directly use libelement, so I updated the Bundle Builder directives to reflect that. 10% chance it fixes the problem.

comment:15 by pett, 5 years ago

Since the problem is not finding libpyinstance, make that 1%

comment:16 by Tom Goddard, 5 years ago

Ok, tried it just now, 0% chance,

cx
WARNING: Traceback (most recent call last):

File "/home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/site-packages/chimerax/core/toolshed/info.py", line 452, in get_module

m = importlib.import_module(self.package_name)

File "/home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/importlib/init.py", line 127, in import_module

return _bootstrap._gcd_import(name[level:], package, level)

File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/site-packages/chimerax/chem_group/init.py", line 14, in <module>

from .chem_group import find_group

File "/home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/site-packages/chimerax/chem_group/chem_group.py", line 18, in <module>

from chimerax.atomic.idatm import type_info, tetrahedral, planar, linear, single

File "/home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/site-packages/chimerax/atomic/init.py", line 15, in <module>

import chimerax.atomic_lib

File "/home/goddard/ucsf/chimerax/ChimeraX.app/lib/python3.7/site-packages/chimerax/atomic_lib/init.py", line 34, in <module>

from . import _load_libs

ImportError: libpyinstance.so: cannot open shared object file: No such file or directory

comment:17 by pett, 5 years ago

I have checked out the source onto plato to try to compile it myself and work on the problem. Don't know how many build tools are missing. I had to install 'yasm'. That's it so far...

comment:18 by pett, 5 years ago

Well, that failed. The compilation/install of Mesa tries to make hard links, which aren't permitted on plato's file system.

in reply to:  19 ; comment:19 by goddard@…, 5 years ago 

Maybe just comment out mesa in prereqs/Makefile.  I think it only used for offscreen rendering.  For your tests all you need is chimerax --nogui right?

comment:20 by pett, 5 years ago

Seems iffy, but I'll give it a try.

comment:21 by pett, 5 years ago

The compiler on plato (GCC 4.8.3, supposedly C++11 compliant) chokes on the code in atomic_lib. Bundle Builder winds up handling the compiler both the -std=gnu99 and the -std=c++11 flags, the former automatically via distutils and the latter explicitly by Bundle Builder. The compiler complains about the -std=gnu99 flag being used with C++ and ignores it (I think). I don't know if it honors the -std=c++11 flag or not. I would _guess_ it would, but it's certainly not clear. The problem really seems to be that their Standard Template Library isn't up to snuff.

comment:22 by pett, 5 years ago

Tomorrow I'm going to try on rbvi-gassho (our Hungarian machine!) where I have more control over the compilation environment.

comment:23 by pett, 5 years ago

With the assistance of our friend rbvi-gassho, I was able to get things to work. Turns out that Bundle Builder supports an undocumented LinkArgument tag that allowed me to insert the necessary (and ugly) linker args to find the various libs at runtime. Will start some builds on plato now.

comment:24 by pett, 5 years ago

Resolution: → fixed
Status: feedback → closed

Appears to work.

Note: See TracTickets for help on using tickets.