Problem loading file-history cache at startup

[t.laughlin@…]

The following bug report has been submitted:
Platform:        Windows-10-10.0.18362
ChimeraX Version: 0.9 (2019-06-06)
Description
Broken preference directory (attached) that prevented the main interface/GUI from starting upon clicking the executable.

Log:
Startup Messages  
---  
note | available bundle cache has not been initialized yet  
  
UCSF ChimeraX version: 0.9 (2019-06-06)  
© 2016-2019 Regents of the University of California. All rights reserved.  
How to cite UCSF ChimeraX  




OpenGL version: 3.3.0 - Build 25.20.100.6472
OpenGL renderer: Intel(R) HD Graphics 620
OpenGL vendor: Intel
File attachment: ChimeraX_brokenpref.zip

ChimeraX_brokenpref.zip​

[t.laughlin@…]

Added by email2trac

[Eric Pettersen]

Using the provided preferences directory (thanks Thomas!), the traceback is:

NOTE: Traceback (most recent call last):

File

"/Users/pett/src/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py",
line 193, in _run_module_as_main

"main", mod_spec)

File

"/Users/pett/src/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py",
line 85, in _run_code

exec(code, run_globals)

File

"/Users/pett/src/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/ChimeraX_main.py", line 885, in <module>

exit_code = init(sys.argv)

File

"/Users/pett/src/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/ChimeraX_main.py", line 629, in init

sess.ui.build()

File

"/Users/pett/src/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/chimerax/ui/gui.py", line 196, in build

self.main_window = mw = MainWindow(self, self.session)

File

"/Users/pett/src/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/chimerax/ui/gui.py", line 435, in init

filename_size=15, no_hist_text="\n".join(new_user_text))

File

"/Users/pett/src/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/chimerax/ui/file_history.py", line 34, in init

self.update_html()

File

"/Users/pett/src/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/chimerax/ui/file_history.py", line 86, in update_html

html = self.history_html()

File

"/Users/pett/src/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/chimerax/ui/file_history.py", line 41, in history_html

files = file_history(self.session).files

File

"/Users/pett/src/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/chimerax/core/filehistory.py", line 198, in file_history

session.file_history = fh = FileHistory(session)

File

"/Users/pett/src/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/chimerax/core/filehistory.py", line 30, in init

self._files = self.load_history() # Map (file path, database) to FileSpec

File

"/Users/pett/src/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/chimerax/core/filehistory.py", line 108, in load_history

fc = self._file_cache.load()

File

"/Users/pett/src/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/chimerax/core/history.py", line 96, in load

return json.load(f)

File

"/Users/pett/src/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/init.py",
line 296, in load

parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, kw)

File

"/Users/pett/src/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/init.py",
line 348, in loads

return _default_decoder.decode(s)

File

"/Users/pett/src/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py",
line 337, in decode

obj, end = self.raw_decode(s, idx=_w(s, 0).end())

File

"/Users/pett/src/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py",
line 353, in raw_decode

obj, end = self.scan_once(s, idx)

json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 1176559
(char 1176558)

BUG: json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 1176559
(char 1176558)

File
"/Users/pett/src/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py",
line 353, in raw_decode
obj, end = self.scan_once(s, idx)

_See log for complete Python traceback._

[Tom Goddard]

I'm baffled. The file history is a JSON file written by the standard Python json.dump() routine and yet one history entry has a string that is missing its ending quote mark. The Python json module adds the quote marks so I am baffled how that could fail. The string in question is a base64 encoded image so it has every allowable character in base64. Maybe somehow json dump() fails with some weird character sequence. Will need to investigate more. Here is what the incorrect json looks like.

{"path": "C:
Users
Jabberwocky
Desktop
4_13_homologyModels
19Nov25cistemMaps
denmod2_map.ccp4", "format": null, "database": null, "access_time": 1574830711.3832202, "image": "/9j/4...2SeU, "open_options": {}}, ...

where I abbreviated about 500 characters in the "image" value with .... The "image" value is an unterminated string, so the leading quote on "open_options" becomes the terminal quote and then there is no comma, hence the error.

The image value is more than 2x shorter than other image values in the file so it looks like a good bit of it was truncated.

[Tom Goddard]

Conrad suggested that perhaps the file could have been corrupted by two ChimeraX copies trying to save the file history at the same time. ChimeraX only saves the history at exit so it would take exact timing to have the saves happen simultaneously. But maybe two copies were running and the user logged out so both programs quit at the same time. The file history save code uses chimerax.core.safesave.py which first writes the data to a temp file then moves temp file replacing the previous file. But the temp file is just the file name with ".tmp" appended. So both copies would be trying to write the same temporary file. Both copies would open in mode = 'w' which would truncate the file. This seem implausible.

[Tom Goddard]

Fixed.

I put in code to detect if the file history is corrupted. In that case ChimeraX will start and show an error message and copy the corrupted file to a time-stamped backup and ask the user to report it as a bug. If we get more reports of this we'll have to figure out how the file gets corrupted.