macOS Catalina, opening file PermissionError

[goddard@…]

The following bug report has been submitted:
Platform:        Darwin-19.0.0-x86_64-i386-64bit
ChimeraX Version: 0.91 (2019-09-27)
Description
Pressing recent file thumbnail generates this error after updating macOS to Catalina.  Catalina asks when an application first opens a file from a directory whether to allow the app to open files from that directory.  Apparently if this is done with the Python open() function it generates a traceback because it has no way of asking.  If done with the Open File dialog then it does ask and then works.  After that opening files from the directory works without asking.

Log:
UCSF ChimeraX version: 0.91 (2019-09-27)  
© 2016-2019 Regents of the University of California. All rights reserved.  
How to cite UCSF ChimeraX  

> open 3915 fromDatabase emdb

Traceback (most recent call last):  
File
"/Users/goddard/ucsf/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/chimerax/ui/gui.py", line 571, in customEvent  
func(*args, **kw)  
File
"/Users/goddard/ucsf/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/chimerax/ui/widgets/htmlview.py", line 338, in defer  
cxcmd(session, topic)  
File
"/Users/goddard/ucsf/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/chimerax/ui/widgets/htmlview.py", line 459, in cxcmd  
run(session, cmd)  
File
"/Users/goddard/ucsf/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/chimerax/core/commands/run.py", line 31, in run  
results = command.run(text, log=log)  
File
"/Users/goddard/ucsf/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/chimerax/core/commands/cli.py", line 2837, in run  
result = ci.function(session, **kw_args)  
File
"/Users/goddard/ucsf/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/chimerax/core/commands/open.py", line 44, in open  
db_models, status = _fetch_from_database(session, db_id, db_name, db_format,
name, ignore_cache, **kw)  
File
"/Users/goddard/ucsf/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/chimerax/core/commands/open.py", line 153, in _fetch_from_database  
ignore_cache=ignore_cache, **kw)  
File
"/Users/goddard/ucsf/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/chimerax/core/fetch.py", line 434, in fetch_from_database  
models, status = df.fetch(session, id, format=format,
ignore_cache=ignore_cache, **kw)  
File
"/Users/goddard/ucsf/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/chimerax/core/fetch.py", line 497, in fetch  
return fetch(session, database_id, ignore_cache=ignore_cache, **kw)  
File
"/Users/goddard/ucsf/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/chimerax/map/emdb_fetch.py", line 48, in fetch_emdb  
models, status = io.open_data(session, filename, format = 'ccp4', name =
emdb_id, **kw)  
File
"/Users/goddard/ucsf/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/chimerax/core/io.py", line 445, in open_data  
models, status = open_func(*args, **kw)  
File
"/Users/goddard/ucsf/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/chimerax/map/volume.py", line 3598, in open_map_format  
return open_map(session, path, name=name, format=format, **kw)  
File
"/Users/goddard/ucsf/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/chimerax/map/volume.py", line 3197, in open_map  
verbose = kw.get('verbose'))  
File
"/Users/goddard/ucsf/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/chimerax/map/data/fileformats.py", line 153, in open_file  
data = open_func(apath, **kw)  
File
"/Users/goddard/ucsf/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/chimerax/map/data/ccp4/__init__.py", line 18, in open  
return [CCP4Grid(path)]  
File
"/Users/goddard/ucsf/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/chimerax/map/data/ccp4/ccp4_grid.py", line 21, in __init__  
MRCGrid.__init__(self, path, file_type = 'ccp4')  
File
"/Users/goddard/ucsf/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/chimerax/map/data/mrc/mrc_grid.py", line 24, in __init__  
d = mrc_format.MRC_Data(path, file_type)  
File
"/Users/goddard/ucsf/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/chimerax/map/data/mrc/mrc_format.py", line 29, in __init__  
file = open(path, 'rb')  
PermissionError: [Errno 1] Operation not permitted:
'/Users/goddard/Downloads/ChimeraX/EMDB/emd_3915.map'  
  
PermissionError: [Errno 1] Operation not permitted:
'/Users/goddard/Downloads/ChimeraX/EMDB/emd_3915.map'  
  
File
"/Users/goddard/ucsf/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/chimerax/map/data/mrc/mrc_format.py", line 29, in __init__  
file = open(path, 'rb')  
  
See log for complete Python traceback.  
  




OpenGL version: 4.1 ATI-3.0.68
OpenGL renderer: AMD Radeon Pro Vega 20 OpenGL Engine
OpenGL vendor: ATI Technologies Inc.

[Tom Goddard]

Apparently Catalina does not allow an app to open files under Downloads, Documents or Desktop without an explicit granting permission dialog. Using the ChimeraX File / Open... dialog shows that permission dialog, annoying but it works. Using the open command or a file history thumbnail does not show the permission dialog and instead gives a PermissionError. Once you grant permission in a directory (like Downloads/ChimeraX/PDB) using the File / Open... dialog it seems not to ask about that specific directory again.

This is a pretty serious problem. There is a lot of discussion of it with Gimp, where even the File / Open... dialog gets a permission error.

​https://discussions.apple.com/thread/250716806

​https://gitlab.gnome.org/GNOME/gimp/issues/3710

with no solution. I have not seen any analysis that says whether the permission dialog can be circumvented programmatically (seems unlikely since that would defeat this security feature). And I have not seen any discussion of how to make the Python open() command shown the permission dialog (I guess via Qt in our case).

[Tom Goddard]

May want to put a warning on our ChimeraX download page that macOS Catalina causes problems opening files.

[Tom Goddard]

I tried ChimeraX command "open ~/Documents/alfentanil-muor.pdb" in the ChimeraX command-line and it did popup a permission dialog asking to allow access to Documents, and then worked when I granted permission.

The permissions for Downloads, Desktop and Documents folders for each app are shown in macOS Settings under Security and Privacy, use the Privacy tab, and click Files and Folders in the left column. Image attached.

[Tom Goddard]

Security and Privacy settings.

[Tom Goddard]

I would guess that the permission dialog works when the file open system call happens while in a native Mac event handler like a mouse click on the Open File dialog button, or an Enter key press on the ChimeraX command line. The recent file thumbnails are in an HTML widget and the handling of clicks on cxcmd links currently defers executing the command because it receives the link click callback in a different thread (Qt docs call it the IO thread, instead of the UI thread). This code is in ui/widgets/htmlview.py and QWebEngineProfile.setRequestInterceptor() is called to get the link clicks. This Qt call is deprecated in Qt 5.13 and is replaced by setURLRequestInterceptor() which calls the callback in the UI thread. So simply using the new Qt call may be sufficient to get the file permissions dialog to appear when clicking recent file thumbnails. The new call is not in Qt 5.12.5. I think there is a good chance this change will not work because Qt is talking to another process that renders the HTML (Chromium). So the native click event that leads to the callback may have completed by the time the callback is made. We so far have not moved to Qt 5.13 because Qt 5.12 is a long term stability release while Qt 5.13 is not.

[Tom Goddard]

After giving Chimera permissions to access directories these app permission can be reset with command

sudo tccutil reset All edu.ucsf.cgl.chimera

For ChimeraX the command would be.

sudo tccutil reset All edu.ucsf.cgl.ChimeraX

The last argument is the bundle identifier from the app Info.plist file.

[Tom Goddard]

I tested Chimera 1.14 release candidate and opening files using rapid access buttons, menu File / Open..., and the command-line all properly brought up the permissions dialog. Also fetch a PDB file with File / Fetch by ID worked, and using command (open 6igy) without any permission dialog. Also writing to the Desktop with "write #0 ~/Desktop/test3.pdb" worked. These all worked starting from UCSF Chimera having no access permissions (ie not listed in Security and Privacy / Privacy / Files and Folders. So Chimera seems fine with Catalina.

Just starting ChimeraX on Catalina caused it to ask to access files in ~/Desktop even before the splash screen showed. The clicking a file history thumbnail brought up the permission dialog and worked! Also opening a file from command line open ~/Documents/121p.cif also brought up the permission dialog and worked. So now I don't seem to be able to reproduce the PermissionError bug.

[Tom Goddard]

Correction to the previous comment. ChimeraX asks for permission to use files on the Desktop after the splash screen disappears and before the main window appears, not before the splash screen as I claimed before.

[Tom Goddard]

I have not been able to reproduce the Permission Error originally reported. I suspect I may have started ChimeraX from an Emacs shell and that was a necessary step.

Reopen the ticket if the permission error is seen opening a file on Catalina again.