Changes between Initial Version and Version 4 of Ticket #19708


Ignore:
Timestamp:
Jan 21, 2026, 12:54:08 PM (2 weeks ago)
Author:
Tom Goddard
Comment:

Legend:

Unmodified
Added
Removed
Modified

  • Ticket #19708 – Description

    initial v4  
    552) Extremely slow variable performance.  About 50% of the time Boltz predictions run extremely slowly relative to my Linux 4090 lab machine (minsky) and checking stderr output message timestamps shows it takes an extremely long time to load network weights.  For example, an NTCA dimer prediction that runs in 40 seconds on minsky will more often than not take 3-5 minutes on a RunPod VM, and as much as 23 minutes!  This seems to be a network and CPU bottleneck as the GPU has no activity and top shows fluctuating CPU (100% ave).  Some VMs consistently give fast 40 second predictions.  I often see the slow machines have a load average of 10 even though I am not running any jobs.  I suspect the slowness is due to other VMs running on the same hardware hogging the CPUs and network bandwidth.
    66
    7 3) RunPod frequently drops connect to the network shell.  The network shell is convenient since you don't have to set up an ssh key to start it.  But it drops often, rarely staying connected more than 10 minutes.  An ssh connection in a Mac Terminal is much more stable.
     73) RunPod frequently drops connect to the web terminal.  The web terminal is convenient since you don't have to set up an ssh key to start it.  But it drops often, rarely staying connected more than 10 minutes.  An ssh connection in a Mac Terminal is much more stable.
    88
    994) Transferring files to RunPod VMs is a nightmare.  The VMs don't have rsync installed.  Also apt install does not work unless you know to do apt-get update firsts.  And the rsync command with the non-standard ssh port number the VMs use is a pain to type (needs -e "ssh -p portnum") added.  Other methods of file transfer such as using JupyterLab notebook surprisingly don't work, saying uploading directories is not supported.  They absurdly recommend installing their own half-baked runpodctl file transfer app.  Also their documentation on transferring files is missing all the example commands in Safari and Firefox on Mac -- they don't even provide usable documentation.