Opened 6 months ago

Closed 6 months ago

#17527 closed defect (fixed)

pyproject.toml does not check existence of source files

Reported by: Tristan Croll Owned by: Zach Pearson
Priority: normal Milestone:
Component: Build System Version:
Keywords: Cc: Greg Couch
Blocked By: Blocking:
Notify when closed: Platform: all
Project: ChimeraX

Description 

The following bug report has been submitted:
Platform:        Linux-6.8.0-57-generic-x86_64-with-glibc2.35
ChimeraX Version: 1.10.dev202504300222 (2025-04-30 02:22:20 UTC)
Description
When building an extension using pyproject.toml, it appears there is no check to see if the source files actually exist. While rearranging things to move all C++ source from `src` to `src_cpp` I misplaced the source file for Clipper's `_map_mask` extension - placing it in `src_cpp/_maps/mask.cpp` while the pyproject.toml had:

{{{
[tool.chimerax.extension._map_mask]
sources = [
  "src_cpp/maps/_maps/mask.cpp",
]}}

This led Bundle Builder to silently skip building this extension entirely, leading to some fiddly debugging. Seems worth adding a sanity-check for file existence here?

OpenGL version: 3.3.0 NVIDIA 535.230.02
OpenGL renderer: NVIDIA GeForce RTX 3070/PCIe/SSE2
OpenGL vendor: NVIDIA Corporation

Python: 3.11.4
Locale: en_GB.UTF-8
Qt version: PyQt6 6.8.1, Qt 6.8.2
Qt runtime version: 6.8.2
Qt platform: xcb

XDG_SESSION_TYPE=x11
DESKTOP_SESSION=ubuntu
XDG_SESSION_DESKTOP=ubuntu
XDG_CURRENT_DESKTOP=ubuntu:GNOME
DISPLAY=:1
Manufacturer: Dell Inc.
Model: XPS 8950
OS: Ubuntu 22.04
Architecture: 64bit ELF
Virtual Machine: none
CPU: 20 12th Gen Intel(R) Core(TM) i7-12700
Cache Size: 25600 KB
Memory:
	               total        used        free      shared  buff/cache   available
	Mem:            31Gi        13Gi       4.5Gi       368Mi        12Gi        16Gi
	Swap:          2.0Gi       1.1Gi       900Mi

Graphics:
	0000:01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA104 [GeForce RTX 3070 Lite Hash Rate] [10de:2488] (rev a1)	
	Subsystem: Dell GA104 [GeForce RTX 3070 Lite Hash Rate] [1028:c903]	
	Kernel driver in use: nvidia

Installed Packages:
    alabaster: 1.0.0
    appdirs: 1.4.4
    asttokens: 3.0.0
    auditwheel: 6.3.0
    babel: 2.17.0
    beautifulsoup4: 4.13.3
    blockdiag: 3.0.0
    blosc2: 3.3.1
    build: 1.2.2.post1
    certifi: 2025.4.26
    cftime: 1.6.4.post1
    charset-normalizer: 3.4.1
    ChimeraX-AddCharge: 1.5.18
    ChimeraX-AddH: 2.2.7
    ChimeraX-AlignmentAlgorithms: 2.0.2
    ChimeraX-AlignmentHdrs: 3.6.1
    ChimeraX-AlignmentMatrices: 2.1
    ChimeraX-Alignments: 2.20.1
    ChimeraX-AlphaFold: 1.0.1
    ChimeraX-AltlocExplorer: 1.1.2
    ChimeraX-AmberInfo: 1.0
    ChimeraX-Aniso: 1.1.3
    ChimeraX-Arrays: 1.1
    ChimeraX-Atomic: 1.60.5
    ChimeraX-AtomicLibrary: 14.1.17
    ChimeraX-AtomSearch: 2.0.1
    ChimeraX-AxesPlanes: 2.4
    ChimeraX-BasicActions: 1.1.3
    ChimeraX-BILD: 1.0
    ChimeraX-BlastProtein: 3.0.0
    ChimeraX-Boltz: 1.0
    ChimeraX-BondRot: 2.0.4
    ChimeraX-BugReporter: 1.0.2
    ChimeraX-BuildStructure: 2.13.1
    ChimeraX-Bumps: 1.0
    ChimeraX-BundleBuilder: 1.4.0
    ChimeraX-ButtonPanel: 1.0.1
    ChimeraX-CageBuilder: 1.0.1
    ChimeraX-CellPack: 1.0
    ChimeraX-Centroids: 1.4
    ChimeraX-ChangeChains: 1.1
    ChimeraX-CheckWaters: 1.4
    ChimeraX-ChemGroup: 2.0.2
    ChimeraX-Clashes: 2.3
    ChimeraX-Clipper: 0.25.0
    ChimeraX-ColorActions: 1.0.5
    ChimeraX-ColorGlobe: 1.0
    ChimeraX-ColorKey: 1.5.8
    ChimeraX-CommandLine: 1.2.6
    ChimeraX-ConnectStructure: 2.0.1
    ChimeraX-Contacts: 1.0.1
    ChimeraX-Core: 1.10.dev202504300222
    ChimeraX-CoreFormats: 1.2
    ChimeraX-coulombic: 1.4.5
    ChimeraX-Crosslinks: 1.0
    ChimeraX-Crystal: 1.0
    ChimeraX-CrystalContacts: 1.0.1
    ChimeraX-DataFormats: 1.2.4
    ChimeraX-Dicom: 1.2.7
    ChimeraX-DistMonitor: 1.4.2
    ChimeraX-DockPrep: 1.1.4
    ChimeraX-Dssp: 2.0
    ChimeraX-EMDB-SFF: 1.0
    ChimeraX-ESMFold: 1.0
    ChimeraX-FileHistory: 1.0.1
    ChimeraX-FunctionKey: 1.0.1
    ChimeraX-Geometry: 1.3
    ChimeraX-gltf: 1.0
    ChimeraX-Graphics: 1.4.1
    ChimeraX-Hbonds: 2.5.1
    ChimeraX-Help: 1.3
    ChimeraX-HKCage: 1.3
    ChimeraX-IHM: 1.1
    ChimeraX-ImageFormats: 1.2
    ChimeraX-IMOD: 1.0
    ChimeraX-IO: 1.0.3
    ChimeraX-ItemsInspection: 1.0.1
    ChimeraX-IUPAC: 1.0
    ChimeraX-KVFinder: 1.6.1
    ChimeraX-Label: 1.1.14
    ChimeraX-LinuxSupport: 1.0.1
    ChimeraX-ListInfo: 1.2.2
    ChimeraX-Log: 1.2
    ChimeraX-LookingGlass: 1.1
    ChimeraX-Maestro: 1.9.1
    ChimeraX-Map: 1.3
    ChimeraX-MapData: 2.0
    ChimeraX-MapEraser: 1.0.1
    ChimeraX-MapFilter: 2.0.1
    ChimeraX-MapFit: 2.0
    ChimeraX-MapSeries: 2.1.1
    ChimeraX-Markers: 1.0.1
    ChimeraX-Mask: 1.0.2
    ChimeraX-MatchMaker: 2.2.1
    ChimeraX-MCopy: 1.0
    ChimeraX-MDcrds: 2.10.1
    ChimeraX-MedicalToolbar: 1.1
    ChimeraX-Meeting: 1.0.1
    ChimeraX-MLP: 1.1.1
    ChimeraX-mmCIF: 2.16
    ChimeraX-MMTF: 2.2
    ChimeraX-ModelArchive: 1.0
    ChimeraX-Modeller: 1.5.19
    ChimeraX-ModelPanel: 1.5.1
    ChimeraX-ModelSeries: 1.0.1
    ChimeraX-Mol2: 2.0.3
    ChimeraX-Mole: 1.0
    ChimeraX-Morph: 1.0.2
    ChimeraX-MouseModes: 1.2
    ChimeraX-Movie: 1.0
    ChimeraX-MutationScores: 1.0
    ChimeraX-Neuron: 1.0
    ChimeraX-Nifti: 1.2
    ChimeraX-NMRSTAR: 1.0.2
    ChimeraX-NRRD: 1.2
    ChimeraX-Nucleotides: 2.0.3
    ChimeraX-OpenCommand: 1.14.1
    ChimeraX-OrthoPick: 1.0.1
    ChimeraX-PDB: 2.7.9
    ChimeraX-PDBBio: 1.0.1
    ChimeraX-PDBLibrary: 1.0.4
    ChimeraX-PDBMatrices: 1.0
    ChimeraX-PickBlobs: 1.0.1
    ChimeraX-Positions: 1.0
    ChimeraX-PresetMgr: 1.1.3
    ChimeraX-ProfileGrids: 1.1.1
    ChimeraX-PubChem: 2.2
    ChimeraX-ReadPbonds: 1.0.1
    ChimeraX-Registration: 1.1.2
    ChimeraX-RemoteControl: 1.0
    ChimeraX-RenderByAttr: 1.6.3
    ChimeraX-RenumberResidues: 1.1
    ChimeraX-ResidueFit: 1.0.1
    ChimeraX-RestServer: 1.3.1
    ChimeraX-RNALayout: 1.0
    ChimeraX-RotamerLibMgr: 4.0
    ChimeraX-RotamerLibsDunbrack: 2.0
    ChimeraX-RotamerLibsDynameomics: 2.0
    ChimeraX-RotamerLibsRichardson: 2.0
    ChimeraX-SaveCommand: 1.5.1
    ChimeraX-SchemeMgr: 1.0
    ChimeraX-SDF: 2.0.3
    ChimeraX-Segger: 1.0
    ChimeraX-Segment: 1.0.1
    ChimeraX-Segmentations: 3.5.7
    ChimeraX-SelInspector: 1.0
    ChimeraX-SeqView: 2.17.1
    ChimeraX-Shape: 1.1
    ChimeraX-Shell: 1.0.1
    ChimeraX-Shortcuts: 1.2.0
    ChimeraX-ShowSequences: 1.0.3
    ChimeraX-SideView: 1.0.1
    ChimeraX-SimilarStructures: 1.0.1
    ChimeraX-Smiles: 2.1.2
    ChimeraX-SmoothLines: 1.0
    ChimeraX-SpaceNavigator: 1.0
    ChimeraX-StdCommands: 1.19.1
    ChimeraX-STL: 1.0.1
    ChimeraX-Storm: 1.0
    ChimeraX-StructMeasure: 1.2.1
    ChimeraX-Struts: 1.0.1
    ChimeraX-Surface: 1.0.1
    ChimeraX-SwapAA: 2.0.1
    ChimeraX-SwapRes: 2.5.2
    ChimeraX-TapeMeasure: 1.0
    ChimeraX-TaskManager: 1.0
    ChimeraX-Test: 1.0
    ChimeraX-Toolbar: 1.2.3
    ChimeraX-ToolshedUtils: 1.2.4
    ChimeraX-Topography: 1.0
    ChimeraX-ToQuest: 1.0
    ChimeraX-Tug: 1.0.1
    ChimeraX-UI: 1.45.1
    ChimeraX-Umap: 1.0
    ChimeraX-uniprot: 2.3.1
    ChimeraX-UnitCell: 1.0.1
    ChimeraX-ViewDockX: 1.4.4
    ChimeraX-VIPERdb: 1.0
    ChimeraX-Vive: 1.1
    ChimeraX-VolumeMenu: 1.0.1
    ChimeraX-vrml: 1.0
    ChimeraX-VTK: 1.0
    ChimeraX-WavefrontOBJ: 1.0
    ChimeraX-WebCam: 1.0.2
    ChimeraX-WebServices: 1.1.4
    ChimeraX-Zone: 1.0.1
    colorama: 0.4.6
    comm: 0.2.2
    contourpy: 1.3.2
    coverage: 7.8.0
    cxservices: 1.2.3
    cycler: 0.12.1
    Cython: 3.0.12
    debugpy: 1.8.14
    decorator: 5.2.1
    distro: 1.9.0
    docutils: 0.21.2
    executing: 2.2.0
    filelock: 3.18.0
    fonttools: 4.57.0
    funcparserlib: 2.0.0a0
    glfw: 2.9.0
    grako: 3.16.5
    h5py: 3.13.0
    html2text: 2024.2.26
    idna: 3.10
    ihm: 2.2
    imagecodecs: 2024.6.1
    imagesize: 1.4.1
    iniconfig: 2.1.0
    ipykernel: 6.29.5
    ipython: 8.26.0
    ipywidgets: 8.1.6
    jedi: 0.19.1
    Jinja2: 3.1.6
    jupyter_client: 8.6.3
    jupyter_core: 5.7.2
    jupyterlab_widgets: 3.0.14
    kiwisolver: 1.4.8
    line_profiler: 4.2.0
    lxml: 5.3.1
    lz4: 4.4.4
    MarkupSafe: 3.0.2
    matplotlib: 3.10.1
    matplotlib-inline: 0.1.7
    msgpack: 1.1.0
    ndindex: 1.9.2
    nest-asyncio: 1.6.0
    netCDF4: 1.6.5
    networkx: 3.3
    nibabel: 5.2.0
    nptyping: 2.5.0
    numexpr: 2.10.2
    numpy: 1.26.4
    OpenMM: 8.1.1
    OpenMM: 8.2.0
    openmm-cuda: 8.1.1.11.8
    openvr: 1.26.701
    packaging: 24.2
    ParmEd: 4.2.2
    parso: 0.8.4
    pep517: 0.13.1
    pexpect: 4.9.0
    pickleshare: 0.7.5
    pillow: 10.4.0
    pip: 25.0.1
    pkginfo: 1.11.1
    platformdirs: 4.3.7
    pluggy: 1.5.0
    prompt_toolkit: 3.0.51
    psutil: 7.0.0
    ptyprocess: 0.7.0
    pure_eval: 0.2.3
    py-cpuinfo: 9.0.0
    pycollada: 0.8
    pydicom: 2.4.4
    pyelftools: 0.32
    Pygments: 2.18.0
    pynmrstar: 3.3.5
    pynrrd: 1.0.0
    PyOpenGL: 3.1.9
    PyOpenGL-accelerate: 3.1.9
    pyopenxr: 1.1.4501
    pyparsing: 3.2.3
    pyproject_hooks: 1.2.0
    PyQt6: 6.8.1
    PyQt6-Qt6: 6.8.2
    PyQt6-WebEngine: 6.8.0
    PyQt6-WebEngine-Qt6: 6.8.2
    PyQt6_sip: 13.10.0
    pytest: 8.3.5
    pytest-cov: 6.1.1
    python-dateutil: 2.9.0.post0
    pytz: 2025.2
    pyzmq: 26.4.0
    qtconsole: 5.5.2
    QtPy: 2.4.3
    qtshim: 1.1
    RandomWords: 0.4.0
    requests: 2.32.3
    roman-numerals-py: 3.1.0
    scipy: 1.14.0
    setuptools: 78.1.0
    sfftk-rw: 0.8.1
    six: 1.16.0
    snowballstemmer: 2.2.0
    sortedcontainers: 2.4.0
    soupsieve: 2.7
    Sphinx: 8.2.3
    sphinx-autodoc-typehints: 3.1.0
    sphinxcontrib-applehelp: 2.0.0
    sphinxcontrib-blockdiag: 3.0.0
    sphinxcontrib-devhelp: 2.0.0
    sphinxcontrib-htmlhelp: 2.1.0
    sphinxcontrib-jsmath: 1.0.1
    sphinxcontrib-qthelp: 2.0.0
    sphinxcontrib-serializinghtml: 2.0.0
    stack-data: 0.6.3
    superqt: 0.7.1
    tables: 3.10.2
    tcia_utils: 1.5.1
    tifffile: 2025.3.13
    tinyarray: 1.2.4
    tornado: 6.4.2
    traitlets: 5.14.3
    typing_extensions: 4.13.2
    tzdata: 2025.2
    urllib3: 2.4.0
    wcwidth: 0.2.13
    webcolors: 24.11.1
    wheel: 0.45.1
    wheel-filename: 1.4.2
    widgetsnbextension: 4.0.14
}}}

Change History (3)

comment:1 by pett, 6 months ago

Component: Unassigned → Build System
Owner: set to Zach Pearson
Platform: → all
Project: → ChimeraX
Status: new → assigned
Summary: ChimeraX bug report submission → pyproject.toml does not check existence of source files

Reported by Tristan Croll

comment:2 by Zach Pearson, 6 months ago

Cc: Greg Couch added

Greg, is this the expected behavior of bundle builder? I've noticed that you can build ChimeraX on macOS without having 3DConnexion software installed, but that's necessary to build spacenavigator (which until very recently was an XML bundle).

comment:3 by Zach Pearson, 6 months ago

Resolution: → fixed
Status: assigned → closed

OK, I updated bundle builder so that if it encounters a glob like _sourcedir/*.cpp and that glob doesn't match any files, it complains. It also goes through the results of each glob and tests Pathlib(source_file).is_file() and complains if that returns false. Either error fails a build, unless optional = true for that extension, in which case it's just a warning.

Note: See TracTickets for help on using tickets.