toolshed plugin upload - disk quota exeeded

[wolfgang.lugmayr@…]

hi,

i tried to upload a new starmap release and run into the following error:

OSError at /submit_app/
[Errno 122] Disk quota exceeded: '/usr/local/projects/chimerax/www/production/cxtoolshed3/media/pending_releases/ChimeraX_StarMap-1.2.15-py3-none-any.whl'
Request Method:	POST
Request URL:	https://cxtoolshed.rbvi.ucsf.edu/submit_app/?expect_app_name=ChimeraX_StarMap
Django Version:	4.2.1
Exception Type:	OSError
Exception Value:	
[Errno 122] Disk quota exceeded: '/usr/local/projects/chimerax/www/production/cxtoolshed3/media/pending_releases/ChimeraX_StarMap-1.2.15-py3-none-any.whl'
Exception Location:	/wynton/home/ferrin/cxtoolshed/production-site/lib/python3.8/site-packages/django/core/files/move.py, line 63, in file_move_safe
Raised during:	cxtoolshed3.submit_app.views.submit_app
Python Executable:	/usr/bin/python3
Python Version:	3.8.16
Python Path:	
['/usr/lib64/python38.zip',
 '/usr/lib64/python3.8',
 '/usr/lib64/python3.8/lib-dynload',
 '/wynton/home/ferrin/cxtoolshed/production-site/lib64/python3.8/site-packages',
 '/wynton/home/ferrin/cxtoolshed/production-site/lib/python3.8/site-packages',
 '/usr/lib64/python3.8/site-packages',
 '/usr/lib/python3.8/site-packages',
 '/usr/local/projects/chimerax/www/production']
Server time:	Fri, 30 Jun 2023 04:53:50 -0700

can you please have a look at the problem?

cheers,
wolfgang


-- 
Multi-User Cryo-EM Facility
@ Centre for Structral Systems Biology (CSSB)
@ Deutsches Elektronen-Synchrotron (DESY)
Notkestrasse 85 Gebäude 15 (E.247)
22607 Hamburg, Germany
Tel.: +49 40 8998-87706
Email: wolfgang.lugmayr@cssb-hamburg.de
http://www.cssb-hamburg.de/

Screenshot 2023-06-30 135637.png​

[wolfgang.lugmayr@…]

Added by email2trac

[pett]

I can't upload a new version of the NIH Presets bundle either -- with this exact same error.

[pett]

Scooter has increased cxtoulshed's quota from 1 byte(!) to 100GB. When I get home today I will try to upload the NIH Presets bundle again to verify that this has fixed the problem.

[pett]

Hi Wolfgang,

You should be able to upload your StarMap update now. You should be aware that if you edit the release notes, you will get a wait cursor that doesn't go away when you click Save for your changes, but those changes will actually have worked -- which you can see if you use a different tab in your browser. We're looking into this latter problem.

--Eric

[wolfgang.lugmayr@…]

Hi,

the upload for the wheel worked.

FYI:
I tried to change the HTML text and upload an additional screenshot.
This gets stuck during the picture upload and the corresponding text is not changed.
So as you wrote below do each step indivdually and reload the tab when it gets stuck.
This works.

Cheers & thanks,
Wolfgang

P.S. the new feature of the v1.2 release:
https://pubmed.ncbi.nlm.nih.gov/37253357/


----- Original Message -----
From: "ChimeraX" <ChimeraX-bugs-admin@cgl.ucsf.edu>
To: "Wolfgang Lugmayr" <wolfgang.lugmayr@cssb-hamburg.de>, "gregc" <gregc@cgl.ucsf.edu>
Cc: chimera-programmers@cgl.ucsf.edu, scooter@cgl.ucsf.edu
Sent: Friday, 7 July, 2023 02:04:19
Subject: Re: [ChimeraX] #9278: toolshed plugin upload - disk quota exeeded

#9278: toolshed plugin upload - disk quota exeeded
-----------------------------------------+------------------------
          Reporter:  wolfgang.lugmayr@\u2026  |      Owner:  Greg Couch
              Type:  defect              |     Status:  feedback
          Priority:  critical            |  Milestone:
         Component:  Tool Shed           |    Version:
        Resolution:                      |   Keywords:
        Blocked By:                      |   Blocking:
Notify when closed:                      |   Platform:  all
           Project:  ChimeraX            |
-----------------------------------------+------------------------
Changes (by pett):

 * status:  assigned => feedback

Comment:

 Hi Wolfgang,
         You should be able to upload your StarMap update now.  You should
 be aware that if you edit the release notes, you will get a wait cursor
 that doesn't go away when you click Save for your changes, but those
 changes will actually have worked -- which you can see if you use a
 different tab in your browser.  We're looking into this latter problem.

 --Eric

[Tom Goddard]

Hi Wolfgang,

I tried MEDIC a few days ago outside StarMap and couldn't get it to run and submitted a few issues on the MEDIC Github that Gabriella Reggiano said she will try to fix. At first I tried it from StarMap but I didn't have MEDIC installed (wasn't sure if it was part of Rosetta) -- I like your feature where you can see the scripts StarMap produced in the user interface since that made it clear where my problem was and led me to install MEDIC.

Tom

[wolfgang.lugmayr@…]

Hi Tom,

with MEDIC development we had the problem that Gabi uses Mac and we Windows/WSL & Linux.
Depending on the conda it installs a pytorch where different versions sometimes crash on Mac.

So please have a look at the files on: https://github.com/wlugmayr/chimerax-starmap
medic_functions.source (how i tested it before the release, differs a little from Gabi's install hints) and
copy medic_requirements.txt over the MEDIC GitHub requirements.txt.
If you still have problems (I think you have Mac?) try to set the torch to 1.10 in the new requirements.txt

In the beginning we were discussing if we should make a single MEDIC plugin, but the initial overall goal was to do a checkbox in the MEDIC viewer and go back to the StarMap workflow to improve just these selected regions ('user defined' in StarMap). Unfortunatly the current StarMap Rosetta XML script is not sufficient to do this and needs changes or a new template.

Especially the situation that users need Rosetta and PyRosetta needs a rework of StarMap and the StarMap/MEDIC/improvement workflow on the long run. Let's see if the users like MEDIC and the team of Frank DiMaio has time ;-).

Cheers,
Wolfgang


----- Original Message -----
From: "ChimeraX" <ChimeraX-bugs-admin@cgl.ucsf.edu>
To: "gregc" <gregc@cgl.ucsf.edu>, "Wolfgang Lugmayr" <wolfgang.lugmayr@cssb-hamburg.de>
Cc: "chimera-programmers" <chimera-programmers@cgl.ucsf.edu>, "scooter" <scooter@cgl.ucsf.edu>
Sent: Friday, 14 July, 2023 01:55:09
Subject: Re: [ChimeraX] #9278: toolshed plugin upload - disk quota exeeded

#9278: toolshed plugin upload - disk quota exeeded
-----------------------------------------+------------------------
          Reporter:  wolfgang.lugmayr@\u2026  |      Owner:  Greg Couch
              Type:  defect              |     Status:  feedback
          Priority:  critical            |  Milestone:
         Component:  Tool Shed           |    Version:
        Resolution:                      |   Keywords:
        Blocked By:                      |   Blocking:
Notify when closed:                      |   Platform:  all
           Project:  ChimeraX            |
-----------------------------------------+------------------------
Comment (by Tom Goddard):

 Hi Wolfgang,

   I tried MEDIC a few days ago outside StarMap and couldn't get it to run
 and submitted a few issues on the MEDIC Github that Gabriella Reggiano
 said she will try to fix.  At first I tried it from StarMap but I didn't
 have MEDIC installed (wasn't sure if it was part of Rosetta) -- I like
 your feature where you can see the scripts StarMap produced in the user
 interface since that made it clear where my problem was and led me to
 install MEDIC.

    Tom

[goddard@…]

Hi Wolfgang,

  My problems with MEDIC were caused I think by newer versions of packages where perhaps MEDIC did not specify a required version.  For instance, the first error was numpy.bool does not exist, because the latest 1.25.0 version of numpy removed it -- it had long been deprecated.  Then another missing function error after that one was fixed.

	https://github.com/gabriellareggiano/MEDIC/issues/8

	https://github.com/gabriellareggiano/MEDIC/issues/9

Gabriella says she fixed these this weekend and I will try it soon.  I only tried MEDIC from a shell, need that to work before I can expect it to work in StarMap.

	Tom

[wolfgang.lugmayr@…]

Hi Tom,

we have stopped the development in December last year and I was surprised how many Python dependencies have changed in 6 months until I tried to rebuild a new conda from scratch for my plugin release.

I have spent 3 weeks in finding the actual running combination of libraries and gave up.
This is why I have added my own MEDIC install functions & dependencies on the StarMap github in case of problems.
I run into the same package issues and more as you reported to Gabi.

I have seen that Gabi updated the MEDIC repository similar to mine with the exact package versions in the requirement.txt
This are the package numbers of our December 2022 tests and workes for me on CentOS 7,9 and Ubuntu 20.04 LTS on Windows.

Again if you run into a segementation fault with pytorch on Mac, choose torch 1.10 instead of 1.12.
torch 2.x does not work yet.

My StarMap/MEDIC script template prepares some individual style with ChimeraX commands and presets. Maybe you want different ones and need to change the template. It can be located with 'stmconfig' in the ChimeraX command line (search for the MEDIC_* entry).
If you keep the same name the search order for it is 1. local working directory, 2. $STARMAP_TEMPLATES directory, 3. site-packages location or edit it in the GUI.

Cheers,
Wolfgang



----- Original Message -----
From: "ChimeraX" <ChimeraX-bugs-admin@cgl.ucsf.edu>
To: "Wolfgang Lugmayr" <wolfgang.lugmayr@cssb-hamburg.de>, "gregc" <gregc@cgl.ucsf.edu>
Cc: "chimera-programmers" <chimera-programmers@cgl.ucsf.edu>, "scooter" <scooter@cgl.ucsf.edu>
Sent: Monday, 17 July, 2023 19:48:10
Subject: Re: [ChimeraX] #9278: toolshed plugin upload - disk quota exeeded

#9278: toolshed plugin upload - disk quota exeeded
-----------------------------------------+------------------------
          Reporter:  wolfgang.lugmayr@\u2026  |      Owner:  Greg Couch
              Type:  defect              |     Status:  feedback
          Priority:  critical            |  Milestone:
         Component:  Tool Shed           |    Version:
        Resolution:                      |   Keywords:
        Blocked By:                      |   Blocking:
Notify when closed:                      |   Platform:  all
           Project:  ChimeraX            |
-----------------------------------------+------------------------
Comment (by goddard@\u2026):

 {{{
 Hi Wolfgang,

   My problems with MEDIC were caused I think by newer versions of packages
 where perhaps MEDIC did not specify a required version.  For instance, the
 first error was numpy.bool does not exist, because the latest 1.25.0
 version of numpy removed it -- it had long been deprecated.  Then another
 missing function error after that one was fixed.

         https://github.com/gabriellareggiano/MEDIC/issues/8

         https://github.com/gabriellareggiano/MEDIC/issues/9

 Gabriella says she fixed these this weekend and I will try it soon.  I
 only tried MEDIC from a shell, need that to work before I can expect it to
 work in StarMap.

         Tom

 }}}

[Tom Goddard]

This is a common problem with any software that requires third party libraries and does not specify the exact versions it needs. If you want your distributed software to work reliably, all dependencies should specify exactly the the versions that are known to work. Just letting it take the most recent version is a recipe for having your software constantly breaking if you depend on many dependencies. So ChimeraX specifies all the versions of dependencies it uses.

I'll try Gabriella's fixed MEDIC today if time permits, and if it works from a shell I'll try in StarMap too.