Fetched map truncated

[chimerax-bug-report@…]

The following bug report has been submitted:
Platform:        Windows-10-10.0.18362
ChimeraX Version: 0.92 (2020-01-14)
Description
(Describe the actions that caused this problem to occur here)

Log:
UCSF ChimeraX version: 0.92 (2020-01-14)  
© 2016-2019 Regents of the University of California. All rights reserved.  
How to cite UCSF ChimeraX  

> open "C:/Users/Joana Paulino/Downloads/emd_20026.map.gz"

Cannot read compressed CCP4 density map files  

> open emdb:20026

Traceback (most recent call last):  
File "C:\Program Files\ChimeraX\bin\lib\site-
packages\chimerax\map\data\fileformats.py", line 155, in open_file  
data = open_func(apath, **kw)  
File "C:\Program Files\ChimeraX\bin\lib\site-
packages\chimerax\map\data\ccp4\\__init__.py", line 18, in open  
return [CCP4Grid(path)]  
File "C:\Program Files\ChimeraX\bin\lib\site-
packages\chimerax\map\data\ccp4\ccp4_grid.py", line 21, in __init__  
MRCGrid.__init__(self, path, file_type = 'ccp4')  
File "C:\Program Files\ChimeraX\bin\lib\site-
packages\chimerax\map\data\mrc\mrc_grid.py", line 24, in __init__  
d = mrc_format.MRC_Data(path, file_type)  
File "C:\Program Files\ChimeraX\bin\lib\site-
packages\chimerax\map\data\mrc\mrc_format.py", line 39, in __init__  
nc = self.read_values(file, int32, 1)  
File "C:\Program Files\ChimeraX\bin\lib\site-
packages\chimerax\map\data\mrc\mrc_format.py", line 256, in read_values  
% (count, etype.__name__)))  
File "<string>", line None  
SyntaxError: MRC file is truncated. Failed reading 1 values, type int32  
  
During handling of the above exception, another exception occurred:  
  
Traceback (most recent call last):  
File "C:\Program Files\ChimeraX\bin\lib\site-
packages\chimerax\cmd_line\tool.py", line 258, in execute  
cmd.run(cmd_text)  
File "C:\Program Files\ChimeraX\bin\lib\site-
packages\chimerax\core\commands\cli.py", line 2837, in run  
result = ci.function(session, **kw_args)  
File "C:\Program Files\ChimeraX\bin\lib\site-
packages\chimerax\core\commands\open.py", line 44, in open  
db_models, status = _fetch_from_database(session, db_id, db_name, db_format,
name, ignore_cache, **kw)  
File "C:\Program Files\ChimeraX\bin\lib\site-
packages\chimerax\core\commands\open.py", line 153, in _fetch_from_database  
ignore_cache=ignore_cache, **kw)  
File "C:\Program Files\ChimeraX\bin\lib\site-packages\chimerax\core\fetch.py",
line 434, in fetch_from_database  
models, status = df.fetch(session, id, format=format,
ignore_cache=ignore_cache, **kw)  
File "C:\Program Files\ChimeraX\bin\lib\site-packages\chimerax\core\fetch.py",
line 497, in fetch  
return fetch(session, database_id, ignore_cache=ignore_cache, **kw)  
File "C:\Program Files\ChimeraX\bin\lib\site-
packages\chimerax\map\emdb_fetch.py", line 48, in fetch_emdb  
models, status = io.open_data(session, filename, format = 'ccp4', name =
emdb_id, **kw)  
File "C:\Program Files\ChimeraX\bin\lib\site-packages\chimerax\core\io.py",
line 456, in open_data  
models, status = open_func(*args, **kw)  
File "C:\Program Files\ChimeraX\bin\lib\site-packages\chimerax\map\volume.py",
line 3661, in open_map_format  
return open_map(session, path, name=name, format=format, **kw)  
File "C:\Program Files\ChimeraX\bin\lib\site-packages\chimerax\map\volume.py",
line 3248, in open_map  
verbose = kw.get('verbose'))  
File "C:\Program Files\ChimeraX\bin\lib\site-
packages\chimerax\map\data\fileformats.py", line 161, in open_file  
raise FileFormatError(value)  
chimerax.map.data.fileformats.FileFormatError: MRC file is truncated. Failed
reading 1 values, type int32  
  
chimerax.map.data.fileformats.FileFormatError: MRC file is truncated. Failed
reading 1 values, type int32  
  
File "C:\Program Files\ChimeraX\bin\lib\site-
packages\chimerax\map\data\fileformats.py", line 161, in open_file  
raise FileFormatError(value)  
  
See log for complete Python traceback.  
  




OpenGL version: 3.3.0 - Build 23.20.16.4973
OpenGL renderer: Intel(R) UHD Graphics 620
OpenGL vendor: Intel

[Eric Pettersen]

The map is large (~ 130 MB) but worked for me in the daily build.

[Tom Goddard]

The failure reading a fetched MRC file was in reading the first 4 bytes, so it probably was an empty file.

Two people at the Stanford workshop mentioned this problem. And they said they could not fetch the file without getting the same error repeatedly, probably because this empty file was cached.

[Tom Goddard]

The fetch worked fine for me on a Windows machine.

I see from the fetching code that it directly writes the fetched file to the cache directory. Also if the urlopen() response return value read() call returns 0 bytes then there will be no errors and the saved file will have 0 bytes. The reading code assumes that returning 0 bytes means the end of file was reached. Maybe that is wrong. The documentation does not make this 100% clear.

Possibly the server can simply give 0 bytes and some error status code without Python raising an exception. It is not clear. Since I don't have a reproducible test case it is all just guess-work.

I have put in a check that the length of the fetched data matches the http header content length if that value is provided (it is for EMDB and EDS databases). If the size is wrong an error will be raised and the file will not be cached.