Make web page dashboard showing ISOLDE downloads

[tcroll@…]

Some weird new bot behaviour on the toolshed... haven't checked if it's just ISOLDE or all bundles, but ever since mid-last year there's been a ~once-a-month spike where all ISOLDE versions are downloaded a couple of hundred times each. Any idea what might be behind that?

[Eric Pettersen]

Graph of ISOLDE downloads

[Eric Pettersen]

Since run_email2trac isn't working, inserting this comment from Tom Goddard by hand:

Who knows what idiotic robot downloads Toolshed wheels over and over. In the past I saw exactly one download per minute repeated hundreds of times. At any rate, we probably can't spend the time to try to stop that. So you should think about what more useful statistics you could use. I'd suggest the number of unique IP addresses downloading ISOLDE would be a decent measure. I can give you the web server logs for ISOLDE so you can extract that and perhaps we can set up some web page dashboard that shows a graph of unique IP address downloads in the future. Zach is working on making this kind of dashboard for tracking use of ChimeraX web services and when that web log scanner is setup it might be easy to adapt it to the Toolshed.

[Eric Pettersen]

Since run_email2trac continues to not work, inserting this comment from Tristan:

Don't worry on my behalf - looking at these is mostly just an exercise in ego-strokng for me (although it's useful around release time to get a reasonable idea of immediate uptake). If I need to justify impact I can always point to citations and models deposited in the PDB. I did get a little excited when I noticed ISOLDE's 100,000th download should happen in a few months, then deflated when I realised how artificial that number actually is. :)

[Tom Goddard]

Looking at the toolshed logs I see 105523 downloads of ISOLDE for the 6 months Nov 1, 2022 - April 30, 2023 from 2531 unique IP addresses. Looking at just the May 2023 downloads so far (and given that toolshed has been broken some of the 2 weeks I see 2531 downloads from 1341 unique IP addresses. I'll attach the May ISOLDE server logs if you want peruse it more, e.g. to see what platforms people download.

Here are the commands on server wilkins.cgl.ucsf.edu I ran to get these numbers

cd /usr/local/www/logs/cxtoolshed-httpd/cxtoolshed/

zgrep -h isolde/releases cxtoolshed-ssl_access_log.11.gz cxtoolshed-ssl_access_log.10.gz cxtoolshed-ssl_access_\
log.9.gz cxtoolshed-ssl_access_log.8.gz cxtoolshed-ssl_access_log.7.gz cxtoolshed-ssl_access_log.6.gz cxtoolshe\
d-ssl_access_log.5.gz cxtoolshed-ssl_access_log.4.gz cxtoolshed-ssl_access_log.3.gz cxtoolshed-ssl_access_log.2\
.gz cxtoolshed-ssl_access_log.1.gz > ~/isolde_web_log/log

grep -h isolde/releases cxtoolshed-ssl_access_log >> ~/isolde_web_log/log_may2023

cd ~/isolde_web_log

wc log
  105523  2475425 38171898 log

awk '{ print $1 }' log | sort | uniq | wc
   2531    2531   36281

wc log_may2023 
  1341  36799 576313 log_may2023

awk '{ print $1 }' log_may2023 | sort | uniq | wc
    473     473    6780

[Tom Goddard]

Log file for ISOLDE downloads for May 1 - 26, 2023

[Tom Goddard]

Changed this ticket to a request to make a dashboard web page that shows ISOLDE downloads since ISOLDE is a very important part of ChimeraX.

Assigned to Zach because he has been looking at making dashboards for monitoring ChimeraX web service usage.