Long running blastprotein job did not return

[Zach Pearson]

While debugging #5220 attempted to run a job against NR database (47ghpgmq); ChimeraX never retrieved the results.

I thought at first this was because because the time exceeded ~5 minutes, the limit I thought we had for checking job completion in core/tasks.py, but I'm not certain. Other jobs that managed to complete in ~15 minutes did have their results retrieved.

➜  cxservices git:(blast_qt_gui) ✗ curl http://webservices.rbvi.ucsf.edu/cxservices/api/v1/chimerax/files/job_47ghpgmq/_start_time
2021-09-23T00:23:15.368727%
➜  cxservices git:(blast_qt_gui) ✗ curl http://webservices.rbvi.ucsf.edu/cxservices/api/v1/chimerax/files/job_47ghpgmq/_end_time
2021-09-23T00:57:43.588670%
➜  cxservices git:(blast_qt_gui) ✗ curl http://webservices.rbvi.ucsf.edu/cxservices/api/v1/chimerax/files/job_gu83iluc/_start_time
2021-09-13T15:09:08.871013%
➜  cxservices git:(blast_qt_gui) ✗ curl http://webservices.rbvi.ucsf.edu/cxservices/api/v1/chimerax/files/job_gu83iluc/_end_time
2021-09-13T15:24:10.215290%
➜  cxservices git:(blast_qt_gui) ✗ curl http://webservices.rbvi.ucsf.edu/cxservices/api/v1/chimerax/files/job_mjr1i9ki/_start_time
2021-09-13T15:10:29.752294%
➜  cxservices git:(blast_qt_gui) ✗ curl http://webservices.rbvi.ucsf.edu/cxservices/api/v1/chimerax/files/job_mjr1i9ki/_end_time
2021-09-13T15:25:33.781648%

[Zach Pearson]

Applied a hotfix by adding more check-in times to core/tasks.py, but it would be better for the backend to tell the client when to check in next until the job either completed or failed.

[Zach Pearson]

Blocked by ​https://github.com/RBVI/cxwebservices/issues/5

[Zach Pearson]

Closed by ​https://github.com/RBVI/cxwebservices/pull/6. We previously monkeypatched 1.3 to increase the number of times ChimeraX would poll the backend for updates. Now the backend will instruct ChimeraX to poll on an infinitely repeating interval of minutes until a job completes.