Update EMDB fetch to not use deprecated ftp protocol.

[Tom Goddard]

ChimeraX EMDB fetch is using these URLs which will be deprecated in Nov 2024 and Sept 2023

​ftp://ftp.wwpdb.org/pub/emdb/structures/EMD-%s/map/%s.gz
​https://ftp.wwpdb.org/pub/emdb/structures/EMD-%s/map/%s.gz

We use the ftp URL because it was significantly faster than using https in the past (tested maybe a couple years ago). The PDB will drop the ftp protocol Nov 1, 2024 (two years away). And they will replace ​https://ftp.wwpdb.org with ​https://files.wwpdb.org in Sept 2023. Currently the fetching uses the ftp url unless the user specifically adds the "transfer https" option to the command.

Here is an August 2022 RCSB news posting about the deprecation

​http://www.wwpdb.org/news/news?year=2022#6303be3b707ecd4f63b3d3d9

Should retest download speed and update URLs. Would be good to look at this for ChimeraX 1.5 since some users may still be using 1.5 in a couple years when ftp no longer works.

Begin forwarded message:

From: Jasmine Young via pdb-l <pdb-l@lists.wwpdb.org>
Subject: pdb-l: Deprecation of FTP File Download Protocol in the PDB Archive
Date: November 3, 2022 at 6:08:39 AM PDT
To: pdb-l@lists.wwpdb.org
Reply-To: Jasmine Young <jasmin@rcsb.rutgers.edu>

Dear PDB-l,

The FTP protocol for file downloads has been losing popularity over the years in favor of HTTP/S. There are many advantages of HTTP/S including speed, statelessness, security (HTTPS), and better support. Importantly during the past 2-3 years the main web browsers (Chrome and Firefox) have dropped support for the FTP protocol, which has effectively discontinued the FTP protocol for non-technical users.

Given that the majority of file download activity on the internet has moved to HTTP/S, wwPDB plans to deprecate FTP download protocol on November 1st 2024.

wwPDB has traditionally supported FTP, together with HTTP/S and RSYNC. Gradual deprecation of the FTP protocol, in favor of the HTTP/HTTPS protocol will be approached while maintaining support for the RSYNC protocol which offers additional functionality compared to the other 2 protocols.

As announced previously <http://www.wwpdb.org/news/news?year=2022#6303be3b707ecd4f63b3d3d9>, we have introduced DNS names that are specific to the protocols:

* files.wwpdb.org for HTTP/S
* ftp.wwpdb.org for FTP. To be deprecated on November 1st 2024. Note
  that from September 2023 this DNS name will not accept HTTP/S traffic.
* rsync.wwpdb.org for RSYNC

Starting September 2023, wwPDB will start enforcing use of these updated DNS names for the preparation of FTP protocol deprecation.

Please contact info@wwpdb.org with any questions.


-- 
Regards,

Jasmine

===========================================================
Jasmine Young, Ph.D.
Biocuration Team Lead
RCSB Protein Data Bank
Research Professor
Institute for Quantitative Biomedicine
Rutgers, The State University of New Jersey
174 Frelinghuysen Rd
Piscataway, NJ 08854-8087

Email:jasmine@rcsb.rutgers.edu
Phone: (848)445-0103 ext 4920
Fax: (732)445-4320
===========================================================
The archive of messages, sent to pdb-l@lists.wwpdb.org, can be found at:
https://lists.wwpdb.org/empathy/list/pdb-l.lists.wwpdb.org

To subscribe via email, send a message with subject or body 'subscribe' to:
pdb-l-request@lists.wwpdb.org
and follow the instructions in the newly received email.

To unsubscribe via email, send a message with subject or body 'unsubscribe' to:
pdb-l-request@lists.wwpdb.org
and follow the instructions in the newly received email.

[Tom Goddard]

Tests downloading using ftp and https

​​ftp://ftp.wwpdb.org/pub/emdb/structures/EMD-%s/map/%s.gz
​​https://files.wwpdb.org/pub/emdb/structures/EMD-%s/map/%s.gz

showed that https was about twice as fast. For example 1.6 Gbyte EMD 15583 took 55 seconds by https and 106 seconds by ftp. EMD 14150 at 240 Mbytes took 12 seconds by ftp and 6 seconds by https.

[Tom Goddard]

Fixed in 1.5 and daily build.

Made wwpdb fetches use https by default and the new files.wwpdb.org url. This is only used for fetches from ".edu" and ".gov" hosts (united states). Europe, Japan and China use local EMDB mirror sites.