ORCA file: 'utf-8' codec can't decode byte

[steffen@…]

The following bug report has been submitted:
Platform:        Windows-10-10.0.22621
ChimeraX Version: 1.5 (2022-11-24 00:03:27 UTC)
Description
Opening an actual orca output brings this error. OS is windows 11 (german)

Log:
Startup Messages  
---  
note | available bundle cache has not been initialized yet  
  
You can double click a model's Name or ID in the model panel to edit those
fields  
UCSF ChimeraX version: 1.5 (2022-11-24)  
© 2016-2022 Regents of the University of California. All rights reserved.  
How to cite UCSF ChimeraX  

> toolshed show

Downloading bundle SEQCROW-1.6.4-py3-none-any.whl  

Errors may have occurred when running pip:  

pip standard error:  
\---  
  
[notice] A new release of pip available: 22.2.2 -> 23.0.1  
[notice] To update, run: C:\Program Files\ChimeraX\bin\ChimeraX.exe -m pip
install --upgrade pip  
\---  

pip standard output:  
\---  
Looking in indexes: https://pypi.org/simple,
https://cxtoolshed.rbvi.ucsf.edu/pypi/  
Processing c:\users\wotan\downloads\seqcrow-1.6.4-py3-none-any.whl  
Collecting Send2Trash  
Using cached Send2Trash-1.8.0-py3-none-any.whl (18 kB)  
Requirement already satisfied: jinja2 in c:\program
files\chimerax\bin\lib\site-packages (from SEQCROW==1.6.4) (3.1.2)  
Requirement already satisfied: ChimeraX-Core<=1.6,>=1.3 in c:\program
files\chimerax\bin\lib\site-packages (from SEQCROW==1.6.4) (1.5)  
Requirement already satisfied: psutil in c:\program
files\chimerax\bin\lib\site-packages (from SEQCROW==1.6.4) (5.9.1)  
Collecting basis-set-exchange  
Downloading basis_set_exchange-0.9.tar.gz (32.2 MB)  
\---------------------------------------- 32.2/32.2 MB 6.4 MB/s eta 0:00:00  
Preparing metadata (setup.py): started  
Preparing metadata (setup.py): finished with status 'done'  
Requirement already satisfied: scipy in c:\program
files\chimerax\bin\lib\site-packages (from SEQCROW==1.6.4) (1.9.0)  
Collecting jsonschema  
Using cached jsonschema-4.17.3-py3-none-any.whl (90 kB)  
Collecting argcomplete  
Downloading argcomplete-3.0.5-py3-none-any.whl (40 kB)  
\---------------------------------------- 40.2/40.2 kB 2.0 MB/s eta 0:00:00  
Collecting regex  
Downloading regex-2023.3.23-cp39-cp39-win_amd64.whl (267 kB)  
\-------------------------------------- 268.0/268.0 kB 5.5 MB/s eta 0:00:00  
Requirement already satisfied: MarkupSafe>=2.0 in c:\program
files\chimerax\bin\lib\site-packages (from jinja2->SEQCROW==1.6.4) (2.1.1)  
Requirement already satisfied: numpy<1.25.0,>=1.18.5 in c:\program
files\chimerax\bin\lib\site-packages (from scipy->SEQCROW==1.6.4) (1.23.1)  
Collecting pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0  
Downloading pyrsistent-0.19.3-cp39-cp39-win_amd64.whl (62 kB)  
\---------------------------------------- 62.7/62.7 kB 3.3 MB/s eta 0:00:00  
Collecting attrs>=17.4.0  
Using cached attrs-22.2.0-py3-none-any.whl (60 kB)  
Building wheels for collected packages: basis-set-exchange  
Building wheel for basis-set-exchange (setup.py): started  
Building wheel for basis-set-exchange (setup.py): finished with status 'done'  
Created wheel for basis-set-exchange:
filename=basis_set_exchange-0.9-py3-none-any.whl size=34927000
sha256=2a3811b9796b2cd67493c4cdde50d4aebd30a3826527b593c05bb3cf9dcbcf35  
Stored in directory:
c:\users\wotan\appdata\local\pip\cache\wheels\91\3c\d5\a297a1089086aa777b3ccbd3e9ee227d715882482a7e6a1eea  
Successfully built basis-set-exchange  
Installing collected packages: Send2Trash, regex, pyrsistent, attrs,
argcomplete, jsonschema, basis-set-exchange, SEQCROW  
Successfully installed SEQCROW-1.6.4 Send2Trash-1.8.0 argcomplete-3.0.5
attrs-22.2.0 basis-set-exchange-0.9 jsonschema-4.17.3 pyrsistent-0.19.3
regex-2023.3.23  
\---  

Successfully installed SEQCROW-1.6.4 Send2Trash-1.8.0 argcomplete-3.0.5
attrs-22.2.0 basis-set-exchange-0.9 jsonschema-4.17.3 pyrsistent-0.19.3
regex-2023.3.23  
Installed SEQCROW (1.6.4)  

> ui tool show "File Info"

> ui tool show "AaronTools Fragment Library"

> open E:/Arbeit/orca/kalibrierung1/chloroform.out

Summary of feedback from opening E:/Arbeit/orca/kalibrierung1/chloroform.out  
---  
error | unable to open chloroform.out  
  
Traceback (most recent call last):  
File "C:\Program Files\ChimeraX\bin\lib\site-
packages\chimerax\open_command\dialog.py", line 162, in _qt_safe  
run(session, "open " + " ".join([FileNameArg.unparse(p) for p in paths]) + (""  
File "C:\Program Files\ChimeraX\bin\lib\site-
packages\chimerax\core\commands\run.py", line 38, in run  
results = command.run(text, log=log, return_json=return_json)  
File "C:\Program Files\ChimeraX\bin\lib\site-
packages\chimerax\core\commands\cli.py", line 2897, in run  
result = ci.function(session, **kw_args)  
File "C:\Program Files\ChimeraX\bin\lib\site-
packages\chimerax\open_command\cmd.py", line 118, in cmd_open  
models = Command(session, registry=registry).run(provider_cmd_text,
log=log)[0]  
File "C:\Program Files\ChimeraX\bin\lib\site-
packages\chimerax\core\commands\cli.py", line 2897, in run  
result = ci.function(session, **kw_args)  
File "C:\Program Files\ChimeraX\bin\lib\site-
packages\chimerax\open_command\cmd.py", line 193, in provider_open  
models, status = collated_open(session, None, [data], data_format,
_add_models,  
File "C:\Program Files\ChimeraX\bin\lib\site-
packages\chimerax\open_command\cmd.py", line 457, in collated_open  
return remember_data_format()  
File "C:\Program Files\ChimeraX\bin\lib\site-
packages\chimerax\open_command\cmd.py", line 428, in remember_data_format  
models, status = func(*func_args, **func_kw)  
File "C:\Users\wotan\AppData\Local\UCSF\ChimeraX\1.5\site-
packages\SEQCROW\\__init__.py", line 430, in open  
return open_aarontools(  
File "C:\Users\wotan\AppData\Local\UCSF\ChimeraX\1.5\site-
packages\SEQCROW\io.py", line 44, in open_aarontools  
raise e  
File "C:\Users\wotan\AppData\Local\UCSF\ChimeraX\1.5\site-
packages\SEQCROW\io.py", line 36, in open_aarontools  
fr = FileReader(  
File "C:\Users\wotan\AppData\Local\UCSF\ChimeraX\1.5\site-
packages\AaronTools\fileIO.py", line 1535, in __init__  
self.read_orca_out(f, get_all, just_geom)  
File "C:\Users\wotan\AppData\Local\UCSF\ChimeraX\1.5\site-
packages\AaronTools\fileIO.py", line 2189, in read_orca_out  
line = f.readline()  
File "C:\Program Files\ChimeraX\bin\lib\codecs.py", line 322, in decode  
(result, consumed) = self._buffer_decode(data, self.errors, final)  
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0:
invalid start byte  
  
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0:
invalid start byte  
  
File "C:\Program Files\ChimeraX\bin\lib\codecs.py", line 322, in decode  
(result, consumed) = self._buffer_decode(data, self.errors, final)  
  
See log for complete Python traceback.  
  

> help help:quickstart

> open E:/Arbeit/orca/kalibrierung1/chloroform.inp

Unrecognized file suffix '.inp'  

> open E:/Arbeit/orca/kalibrierung1/methanol.out

Summary of feedback from opening E:/Arbeit/orca/kalibrierung1/methanol.out  
---  
error | unable to open methanol.out  
  
Traceback (most recent call last):  
File "C:\Program Files\ChimeraX\bin\lib\site-
packages\chimerax\open_command\dialog.py", line 162, in _qt_safe  
run(session, "open " + " ".join([FileNameArg.unparse(p) for p in paths]) + (""  
File "C:\Program Files\ChimeraX\bin\lib\site-
packages\chimerax\core\commands\run.py", line 38, in run  
results = command.run(text, log=log, return_json=return_json)  
File "C:\Program Files\ChimeraX\bin\lib\site-
packages\chimerax\core\commands\cli.py", line 2897, in run  
result = ci.function(session, **kw_args)  
File "C:\Program Files\ChimeraX\bin\lib\site-
packages\chimerax\open_command\cmd.py", line 118, in cmd_open  
models = Command(session, registry=registry).run(provider_cmd_text,
log=log)[0]  
File "C:\Program Files\ChimeraX\bin\lib\site-
packages\chimerax\core\commands\cli.py", line 2897, in run  
result = ci.function(session, **kw_args)  
File "C:\Program Files\ChimeraX\bin\lib\site-
packages\chimerax\open_command\cmd.py", line 193, in provider_open  
models, status = collated_open(session, None, [data], data_format,
_add_models,  
File "C:\Program Files\ChimeraX\bin\lib\site-
packages\chimerax\open_command\cmd.py", line 457, in collated_open  
return remember_data_format()  
File "C:\Program Files\ChimeraX\bin\lib\site-
packages\chimerax\open_command\cmd.py", line 428, in remember_data_format  
models, status = func(*func_args, **func_kw)  
File "C:\Users\wotan\AppData\Local\UCSF\ChimeraX\1.5\site-
packages\SEQCROW\\__init__.py", line 430, in open  
return open_aarontools(  
File "C:\Users\wotan\AppData\Local\UCSF\ChimeraX\1.5\site-
packages\SEQCROW\io.py", line 44, in open_aarontools  
raise e  
File "C:\Users\wotan\AppData\Local\UCSF\ChimeraX\1.5\site-
packages\SEQCROW\io.py", line 36, in open_aarontools  
fr = FileReader(  
File "C:\Users\wotan\AppData\Local\UCSF\ChimeraX\1.5\site-
packages\AaronTools\fileIO.py", line 1535, in __init__  
self.read_orca_out(f, get_all, just_geom)  
File "C:\Users\wotan\AppData\Local\UCSF\ChimeraX\1.5\site-
packages\AaronTools\fileIO.py", line 2189, in read_orca_out  
line = f.readline()  
File "C:\Program Files\ChimeraX\bin\lib\codecs.py", line 322, in decode  
(result, consumed) = self._buffer_decode(data, self.errors, final)  
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0:
invalid start byte  
  
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0:
invalid start byte  
  
File "C:\Program Files\ChimeraX\bin\lib\codecs.py", line 322, in decode  
(result, consumed) = self._buffer_decode(data, self.errors, final)  
  
See log for complete Python traceback.  
  




OpenGL version: 3.3.0 NVIDIA 531.41
OpenGL renderer: NVIDIA GeForce RTX 3080 Ti/PCIe/SSE2
OpenGL vendor: NVIDIA Corporation

Python: 3.9.11
Locale: de_DE.cp1252
Qt version: PyQt6 6.3.1, Qt 6.3.1
Qt runtime version: 6.3.2
Qt platform: windows

Manufacturer: Alienware
Model: Alienware Aurora R13
OS: Microsoft Windows 11 Pro (Build 22621)
Memory: 34,141,798,400
MaxProcessMemory: 137,438,953,344
CPU: 24 12th Gen Intel(R) Core(TM) i9-12900KF
OSLanguage: de-DE

Installed Packages:
    alabaster: 0.7.12
    appdirs: 1.4.4
    argcomplete: 3.0.5
    asttokens: 2.1.0
    attrs: 22.2.0
    Babel: 2.11.0
    backcall: 0.2.0
    basis-set-exchange: 0.9
    blockdiag: 3.0.0
    build: 0.8.0
    certifi: 2022.9.24
    cftime: 1.6.2
    charset-normalizer: 2.1.1
    ChimeraX-AddCharge: 1.4
    ChimeraX-AddH: 2.2.1
    ChimeraX-AlignmentAlgorithms: 2.0
    ChimeraX-AlignmentHdrs: 3.2.1
    ChimeraX-AlignmentMatrices: 2.0
    ChimeraX-Alignments: 2.6
    ChimeraX-AlphaFold: 1.0
    ChimeraX-AltlocExplorer: 1.0.3
    ChimeraX-AmberInfo: 1.0
    ChimeraX-Arrays: 1.0
    ChimeraX-Atomic: 1.41.5
    ChimeraX-AtomicLibrary: 8.0.3
    ChimeraX-AtomSearch: 2.0.1
    ChimeraX-AxesPlanes: 2.3
    ChimeraX-BasicActions: 1.1.2
    ChimeraX-BILD: 1.0
    ChimeraX-BlastProtein: 2.1.2
    ChimeraX-BondRot: 2.0.1
    ChimeraX-BugReporter: 1.0.1
    ChimeraX-BuildStructure: 2.7.1
    ChimeraX-Bumps: 1.0
    ChimeraX-BundleBuilder: 1.2
    ChimeraX-ButtonPanel: 1.0.1
    ChimeraX-CageBuilder: 1.0.1
    ChimeraX-CellPack: 1.0
    ChimeraX-Centroids: 1.3.1
    ChimeraX-ChangeChains: 1.0.2
    ChimeraX-CheckWaters: 1.3
    ChimeraX-ChemGroup: 2.0
    ChimeraX-Clashes: 2.2.4
    ChimeraX-ColorActions: 1.0.3
    ChimeraX-ColorGlobe: 1.0
    ChimeraX-ColorKey: 1.5.2
    ChimeraX-CommandLine: 1.2.4
    ChimeraX-ConnectStructure: 2.0.1
    ChimeraX-Contacts: 1.0.1
    ChimeraX-Core: 1.5
    ChimeraX-CoreFormats: 1.1
    ChimeraX-coulombic: 1.3.2
    ChimeraX-Crosslinks: 1.0
    ChimeraX-Crystal: 1.0
    ChimeraX-CrystalContacts: 1.0.1
    ChimeraX-DataFormats: 1.2.2
    ChimeraX-Dicom: 1.1
    ChimeraX-DistMonitor: 1.3
    ChimeraX-DockPrep: 1.0
    ChimeraX-Dssp: 2.0
    ChimeraX-EMDB-SFF: 1.0
    ChimeraX-ExperimentalCommands: 1.0
    ChimeraX-FileHistory: 1.0.1
    ChimeraX-FunctionKey: 1.0.1
    ChimeraX-Geometry: 1.2
    ChimeraX-gltf: 1.0
    ChimeraX-Graphics: 1.1
    ChimeraX-Hbonds: 2.4
    ChimeraX-Help: 1.2.1
    ChimeraX-HKCage: 1.3
    ChimeraX-IHM: 1.1
    ChimeraX-ImageFormats: 1.2
    ChimeraX-IMOD: 1.0
    ChimeraX-IO: 1.0.1
    ChimeraX-ItemsInspection: 1.0.1
    ChimeraX-Label: 1.1.7
    ChimeraX-ListInfo: 1.1.1
    ChimeraX-Log: 1.1.5
    ChimeraX-LookingGlass: 1.1
    ChimeraX-Maestro: 1.8.2
    ChimeraX-Map: 1.1.2
    ChimeraX-MapData: 2.0
    ChimeraX-MapEraser: 1.0.1
    ChimeraX-MapFilter: 2.0
    ChimeraX-MapFit: 2.0
    ChimeraX-MapSeries: 2.1.1
    ChimeraX-Markers: 1.0.1
    ChimeraX-Mask: 1.0.1
    ChimeraX-MatchMaker: 2.0.9
    ChimeraX-MDcrds: 2.6
    ChimeraX-MedicalToolbar: 1.0.2
    ChimeraX-Meeting: 1.0.1
    ChimeraX-MLP: 1.1
    ChimeraX-mmCIF: 2.8
    ChimeraX-MMTF: 2.2
    ChimeraX-Modeller: 1.5.6
    ChimeraX-ModelPanel: 1.3.6
    ChimeraX-ModelSeries: 1.0.1
    ChimeraX-Mol2: 2.0
    ChimeraX-Mole: 1.0
    ChimeraX-Morph: 1.0.1
    ChimeraX-MouseModes: 1.1.1
    ChimeraX-Movie: 1.0
    ChimeraX-Neuron: 1.0
    ChimeraX-Nucleotides: 2.0.3
    ChimeraX-OpenCommand: 1.9.1
    ChimeraX-PDB: 2.6.8
    ChimeraX-PDBBio: 1.0
    ChimeraX-PDBLibrary: 1.0.2
    ChimeraX-PDBMatrices: 1.0
    ChimeraX-PickBlobs: 1.0.1
    ChimeraX-Positions: 1.0
    ChimeraX-PresetMgr: 1.1
    ChimeraX-PubChem: 2.1
    ChimeraX-ReadPbonds: 1.0.1
    ChimeraX-Registration: 1.1.1
    ChimeraX-RemoteControl: 1.0
    ChimeraX-RenumberResidues: 1.1
    ChimeraX-ResidueFit: 1.0.1
    ChimeraX-RestServer: 1.1
    ChimeraX-RNALayout: 1.0
    ChimeraX-RotamerLibMgr: 2.0.1
    ChimeraX-RotamerLibsDunbrack: 2.0
    ChimeraX-RotamerLibsDynameomics: 2.0
    ChimeraX-RotamerLibsRichardson: 2.0
    ChimeraX-SaveCommand: 1.5.1
    ChimeraX-SchemeMgr: 1.0
    ChimeraX-SDF: 2.0
    ChimeraX-Segger: 1.0
    ChimeraX-Segment: 1.0
    ChimeraX-SelInspector: 1.0
    ChimeraX-SeqView: 2.7.2
    ChimeraX-Shape: 1.0.1
    ChimeraX-Shell: 1.0.1
    ChimeraX-Shortcuts: 1.1.1
    ChimeraX-ShowSequences: 1.0.1
    ChimeraX-SideView: 1.0.1
    ChimeraX-Smiles: 2.1
    ChimeraX-SmoothLines: 1.0
    ChimeraX-SpaceNavigator: 1.0
    ChimeraX-StdCommands: 1.10
    ChimeraX-STL: 1.0
    ChimeraX-Storm: 1.0
    ChimeraX-StructMeasure: 1.1
    ChimeraX-Struts: 1.0.1
    ChimeraX-Surface: 1.0
    ChimeraX-SwapAA: 2.0.1
    ChimeraX-SwapRes: 2.1.3
    ChimeraX-TapeMeasure: 1.0
    ChimeraX-Test: 1.0
    ChimeraX-Toolbar: 1.1.2
    ChimeraX-ToolshedUtils: 1.2.1
    ChimeraX-Tug: 1.0.1
    ChimeraX-UI: 1.24.3
    ChimeraX-uniprot: 2.2.1
    ChimeraX-UnitCell: 1.0.1
    ChimeraX-ViewDockX: 1.1.4
    ChimeraX-VIPERdb: 1.0
    ChimeraX-Vive: 1.1
    ChimeraX-VolumeMenu: 1.0.1
    ChimeraX-VTK: 1.0
    ChimeraX-WavefrontOBJ: 1.0
    ChimeraX-WebCam: 1.0.1
    ChimeraX-WebServices: 1.1.0
    ChimeraX-Zone: 1.0.1
    colorama: 0.4.5
    comtypes: 1.1.10
    cxservices: 1.2
    cycler: 0.11.0
    Cython: 0.29.32
    debugpy: 1.6.4
    decorator: 5.1.1
    docutils: 0.19
    entrypoints: 0.4
    executing: 1.2.0
    filelock: 3.7.1
    fonttools: 4.38.0
    funcparserlib: 1.0.1
    grako: 3.16.5
    h5py: 3.7.0
    html2text: 2020.1.16
    idna: 3.4
    ihm: 0.33
    imagecodecs: 2022.7.31
    imagesize: 1.4.1
    importlib-metadata: 5.1.0
    ipykernel: 6.15.3
    ipython: 8.4.0
    ipython-genutils: 0.2.0
    jedi: 0.18.1
    Jinja2: 3.1.2
    jsonschema: 4.17.3
    jupyter-client: 7.3.4
    jupyter-core: 5.1.0
    kiwisolver: 1.4.4
    line-profiler: 3.5.1
    lxml: 4.9.1
    lz4: 4.0.2
    MarkupSafe: 2.1.1
    matplotlib: 3.5.2
    matplotlib-inline: 0.1.6
    msgpack: 1.0.4
    nest-asyncio: 1.5.6
    netCDF4: 1.6.0
    networkx: 2.8.5
    numexpr: 2.8.4
    numpy: 1.23.1
    openvr: 1.23.701
    packaging: 21.3
    ParmEd: 3.4.3
    parso: 0.8.3
    pep517: 0.13.0
    pickleshare: 0.7.5
    Pillow: 9.2.0
    pip: 22.2.2
    pkginfo: 1.8.3
    platformdirs: 2.5.4
    prompt-toolkit: 3.0.33
    psutil: 5.9.1
    pure-eval: 0.2.2
    pycollada: 0.7.2
    pydicom: 2.3.0
    Pygments: 2.12.0
    PyOpenGL: 3.1.5
    PyOpenGL-accelerate: 3.1.5
    pyparsing: 3.0.9
    PyQt6-commercial: 6.3.1
    PyQt6-Qt6: 6.3.2
    PyQt6-sip: 13.4.0
    PyQt6-WebEngine-commercial: 6.3.1
    PyQt6-WebEngine-Qt6: 6.3.2
    pyrsistent: 0.19.3
    python-dateutil: 2.8.2
    pytz: 2022.6
    pywin32: 303
    pyzmq: 24.0.1
    qtconsole: 5.3.1
    QtPy: 2.3.0
    RandomWords: 0.4.0
    regex: 2023.3.23
    requests: 2.28.1
    scipy: 1.9.0
    Send2Trash: 1.8.0
    SEQCROW: 1.6.4
    setuptools: 65.1.1
    sfftk-rw: 0.7.2
    six: 1.16.0
    snowballstemmer: 2.2.0
    sortedcontainers: 2.4.0
    Sphinx: 5.1.1
    sphinx-autodoc-typehints: 1.19.1
    sphinxcontrib-applehelp: 1.0.2
    sphinxcontrib-blockdiag: 3.0.0
    sphinxcontrib-devhelp: 1.0.2
    sphinxcontrib-htmlhelp: 2.0.0
    sphinxcontrib-jsmath: 1.0.1
    sphinxcontrib-qthelp: 1.0.3
    sphinxcontrib-serializinghtml: 1.1.5
    stack-data: 0.6.2
    tables: 3.7.0
    tifffile: 2022.7.31
    tinyarray: 1.2.4
    tomli: 2.0.1
    tornado: 6.2
    traitlets: 5.3.0
    urllib3: 1.26.13
    wcwidth: 0.2.5
    webcolors: 1.12
    wheel: 0.37.1
    wheel-filename: 1.4.1
    WMI: 1.5.1
    zipp: 3.11.0

[Eric Pettersen]

Reported by Steffen Thomas

[Tony Schaefer]

 Steffen,

Thanks for reporting this. There's some letter/character in the file that
can't be encoded with a single UTF-8 byte. It would be helpful to have the
input or output file associated with this error, otherwise I'm just
guessing what character is causing the issue, and what decoding would be
needed.

Tony

On Thu, Mar 30, 2023 at 9:38 AM ChimeraX <ChimeraX-bugs-admin@cgl.ucsf.edu>
wrote:

[123steffen@…]

Hello,

Unfortunately, today I work in my office at the university and not at home. And in this particular case I don't have the data synchronized. But I have done some research. And the problem is probably the Windows Powershell (I use Windows 11 myself). It stores files created by >outfile.out in utf-16 format (or any other). You can prevent this in Python by a small change when opening the file, but unfortunately my Python is not that good. However, I found the following workaround:

orca ORCA_Test_0000.inp | out-file foo.out -encoding utf8


for starting orca in the windows powershell avoid this problem.

But I can send you sample files again tonight.


sincerly

Steffen

Am Donnerstag, 30, 03, 2023 23:16:46 schrieb ChimeraX <chimerax-bugs-admin@cgl.ucsf.edu>:
#8746: ORCA file: 'utf-8' codec can't decode byte
----------------------------------+---------------------------
Reporter: steffen@… | Owner: Tony Schaefer
Type: defect | Status: assigned
Priority: normal | Milestone:
Component: Third Party | Version:
Resolution: | Keywords:
Blocked By: | Blocking:
Notify when closed: | Platform: all
Project: ChimeraX |
----------------------------------+---------------------------

Comment (by Tony Schaefer):

{{{
Steffen,

Thanks for reporting this. There's some letter/character in the file that
can't be encoded with a single UTF-8 byte. It would be helpful to have the
input or output file associated with this error, otherwise I'm just
guessing what character is causing the issue, and what decoding would be
needed.

Tony

On Thu, Mar 30, 2023 at 9:38 AM ChimeraX
admin@cgl.ucsf.edu>
wrote:

}}}

--
Ticket URL:
ChimeraX
ChimeraX Issue Tracker

[Tony Schaefer]

ChimeraX devs,

ORCA prints most of the interesting output to stdout, and needs to be
redirected to a file to make the "ORCA output file". The user ran ORCA in
Windows PowerShell, which defaults to UTF-16LE encoding. This is an issue
because Python is trying to use UTF-8 to read this file.

SEQCROW uses a stream that ChimeraX passes to the OpenerInfo for these
types of files. Would it be possible/make sense to have the open command
catch decode errors and try a different encoding if the default doesn't
work? Or maybe there's a better way for SEQCROW/ChimeraX to handle this?

I've attached a sample .gro file that's encoded with UTF-16LE in case it's
helpful. Trying to open this file in ChimeraX results in basically the same
UnicodeDecodeError.

Thanks,

Tony

On Fri, Mar 31, 2023 at 12:06 AM ChimeraX <ChimeraX-bugs-admin@cgl.ucsf.edu>
wrote:

test-utf16.gro​

[Tony Schaefer]

Added by email2trac

[Tony Schaefer]

Steffen,

I've attached a command line script for converting files to UTF-8. It
requires charset_normalizer, which can be installed via pip. The
charset_normalizer module tries to detect what encoding a file has. A
non-UTF-8 or ASCII file will get copied to a new file with '-utf_8' added
to the end, which should be fine to open in ChimeraX. Hopefully a better
solution can be found, but this is what I've got for the moment.

Tony

On Thu, Mar 30, 2023 at 3:16 PM Tony Schaefer <tony.schaefer.chem@gmail.com>
wrote:

utf8_convert.py​

[Tony Schaefer]

Added by email2trac

[Eric Pettersen]

I have made the the generic opener code fall back to utf-16 and then utf-32 in the face of UnicodeDecodeErrors. Only in tomorrow's daily build. Will discuss in today's programmer's meeting if we feel comfortable putting it in the release candidate.

[Eric Pettersen]

We chickened out from putting it in the release candidate, though the decision was close so it could be reversed if you felt strongly it should be in the candidate.

[Tony Schaefer]

Eric,

Would this be in future versions of ChimeraX, just not 1.6? This does seem
like the kind of thing that needs a good amount of testing.

Tony

On Mon, Apr 3, 2023 at 5:06 PM ChimeraX <ChimeraX-bugs-admin@cgl.ucsf.edu>
wrote:

[Eric Pettersen]

That is correct. Assuming it proved not to be problematic in the daily builds, it would be in the 1.7 release and later.