Opened 5 years ago
Last modified 3 years ago
#3460 assigned defect
Atom-spec parser slow and possibly leaks memory
Reported by: | Eric Pettersen | Owned by: | Eric Pettersen |
---|---|---|---|
Priority: | moderate | Milestone: | |
Component: | Performance | Version: | |
Keywords: | Cc: | chimera-programmers | |
Blocked By: | Blocking: | ||
Notify when closed: | Platform: | all | |
Project: | ChimeraX |
Description
From chimera-users list:
I'm testing driving ChimeraX (or Chimera) from Jalview, in particular colouring structures using the Jalview alignment residue colouring.
Ideally this should perform well at scale for large numbers of structures and/or colours (for example a graduated colouring based on some property).
I've noticed that ChimeraX appears to do this slower than Chimera.
Is there a reason, or something I should do to make the command faster?
Here's an example like for like comparison using the Chimera/X command line (also attached as a text file)
[for this ticket, just provided as an attachment rather than inline]
Attachments (2)
Change History (24)
comment:1 by , 5 years ago
comment:2 by , 5 years ago
Status and logging are also making running these 383 color commands (one per residue) slow. On my MacBook Pro coloring is done in 6.5 seconds. Commenting out status messages (in MainWindow.status) and the time is 3.0 seconds. Status messages are limited to 60 per-second by the OpenGL refresh rate and each color command is giving a status message. Each of the 383 commands also gets individually logged. Commenting out command logging (in core/commands/cli.py Command.run()) reduces the time from 3.0 to 2.7 seconds.
comment:3 by , 5 years ago
Original email that led to this ticket.
Begin forwarded message:
From: "Mungo Carstairs (Staff)"
Subject: [chimerax-users] Is color command slower in ChimeraX than Chimera?
Date: June 30, 2020 at 2:14:27 AM PDT
To: "chimerax-users@…" <chimerax-users@…>
Hi,
I'm testing driving ChimeraX (or Chimera) from Jalview, in particular colouring structures using the Jalview alignment residue colouring.
Ideally this should perform well at scale for large numbers of structures and/or colours (for example a graduated colouring based on some property).
I've noticed that ChimeraX appears to do this slower than Chimera.
Is there a reason, or something I should do to make the command faster?
Here's an example like for like comparison using the Chimera/X command line (also attached as a text file):
Chimera 1.15 (build 42156)
open cifID:4zho cifID:5h92 cifID:1a70 cifID:4zhp
focus
color #977b64 #0:2.A,2.B;color #e46daf #0:3.A,3.B;color #7dc891 #0:4.A,4.B;color #768daf #0:5.A,5.B;color #bb1691 #0:6.A,6.B;color #1c5355 #0:7.A,7.B;color #5b1873 #0:8.A,8.B;color #d0af91 #0:9.A,9.B;color #6cb737 #0:10.A,10.B;color #8d7c55 #0:11.A,11.B;color #844173 #0:12.A,12.B;color #1f15a0 #0:13.A,13.B;color #a27f91 #0:14.A,14.B;color #19cb64 #0:15.A,15.B;color #484537 #0:16.A,16.B;color #61be37 #0:17.A,17.B;color #84a173 #0:18.A,18.B;color #c76691 #0:19.A,19.B;color #8f31a0 #0:20.A,20.B;color #9eb9af #0:21.A,21.B;color #531564 #0:22.A,22.B;color #707046 #0:23.A,23.B;color #9b80af #0:24.A,24.B;color #ca18a0 #0:25.A,25.B;color #377382 #0:26.A,26.B;color #26a137 #0:27.A,27.B;color #d3b1a0 #0:28.A,28.B;color #e267af #0:29.A,29.B;color #c023af #0:30.A,30.B;color #b18da0 #0:31.A,31.B;color #88bb55 #0:32.A,32.B;color #1fada0 #0:33.A,33.B;color #b22773 #0:34.A,34.B;color #ad19be #0:35.A,35.B;color #6483af #0:36.A,36.B;color #c9de91 #0:37.A,37.B;color #aab2be #0:38.A,38.B;color #59d164 #0:39.A,39.B;color #6d6037 #0:40.A,40.B;color #1cd937 #0:41.A,41.B;color #b5cb82 #0:42.A,42.B;color #e28191 #0:43.A,43.B;color #9537a0 #0:44.A,44.B;color #d6b0a0 #0:45.A,45.B;color #3bcf46 #0:46.A,46.B;color #2c6737 #0:47.A,47.B;color #e586af #0:48.A,48.B;color #e22daf #0:49.A,49.B;color #e29d91 #0:50.A,50.B;color #493582 #0:51.A,51.B;color #cacca0 #0:52.A,52.B;color #7f2855 #0:53.A,53.B;color #86dd91 #0:54.A,54.B;color #b55764 #0:55.A,55.B;color #4ac155 #0:56.A,56.B;color #614a37 #0:57.A,57.B;color #9e69af #0:58.A,58.B;color #5f8855 #0:59.A,59.B;color #958fa0 #0:60.A,60.B;color #9e45af #0:61.A,61.B;color #b9beaf #0:62.A,62.B;color #4add55 #0:63.A,63.B;color #5ba273 #0:64.A,64.B;color #bedf37 #0:65.A,65.B;color #359546 #0:66.A,66.B;color #262d37 #0:67.A,67.B;color #cf5bbe #0:68.A,68.B;color #505c46 #0:69.A,69.B;color #76db37 #0:70.A,70.B;color #3d37be #0:71.A,71.B;color #a28391 #0:72.A,72.B;color #b5cf64 #0:73.A,73.B;color #88b2a0 #0:74.A,74.B;color #bb4a91 #0:75.A,75.B;color #dc8755 #0:76.A,76.B;color #d579a0 #0:77.A,77.B;color #7c9846 #0:78.A,78.B;color #b57b82 #0:79.A,79.B;color #79cd46 #0:80.A,80.B;color #aa1abe #0:81.A,81.B;color #ab4873 #0:82.A,82.B;color #c6df91 #0:83.A,83.B;color #b3b3be #0:84.A,84.B;color #802d91 #0:85.A,85.B;color #d9c4af #0:86.A,86.B;color #c64d91 #0:87.A,87.B;color #857b46 #0:88.A,88.B;color #a26d91 #0:89.A,89.B;color #c62991 #0:90.A,90.B;color #adb1a0 #0:91.A,91.B;color #584991 #0:92.A,92.B;color #93b382 #0:93.A,93.B;color #48e137 #0:94.A,94.B;color #88c56f #1:1.C;color #7b5d60 #1:2.C;color #32d660 #1:3.C;color #339b7e #1:4.C;color #702460 #1:5.C;color #6725ba #1:6.C;color #dc536f #1:7.C;color #a3db7e #1:8.C;color #a8918d #1:9.C;color #d84eba #1:10.C;color #758b7e #1:11.C;color #88d751 #1:12.C;color #75ab7e #1:13.C;color #8e527e #1:14.C;color #cf53d8 #1:15.C;color #dc636f #1:16.C;color #7d288d #1:17.C;color #707460 #1:18.C;color #6fde51 #1:19.C;color #35a960 #1:20.C;color #627d8d #1:21.C;color #bb42ab #1:22.C;color #327060 #1:23.C;color #a3357e #1:24.C;color #28bd8d #1:25.C;color #8f91ba #1:26.C;color #c856d8 #1:27.C;color #795760 #1:28.C;color #c067c9 #1:29.C;color #669b6f #1:30.C;color #adabd8 #1:31.C;color #889d51 #1:32.C;color #c18f9c #1:33.C;color #26bd51 #1:34.C;color #55cdba #1:35.C;color #7e1a60 #1:36.C;color #238451 #1:37.C;color #68678d #1:38.C;color #6da451 #1:39.C;color #a315d8 #1:40.C;color #88256f #1:41.C;color #1f17ba #1:42.C;color #4a456f #1:43.C;color #6da051 #1:44.C;color #a4bfc9 #1:45.C;color #9557ba #1:46.C;color #d6d0ba #1:47.C;color #791d60 #1:48.C;color #e2e1ab #1:49.C;color #1c616f #1:50.C;color #7fda6f #1:51.C;color #8e907e #1:52.C;color #77559c #1:53.C;color #c4bf8d #1:54.C;color #77759c #1:55.C;color #1658d8 #1:56.C;color #add1d8 #1:57.C;color #aa5aba #1:58.C;color #95d3ba #1:59.C;color #adadd8 #1:60.C;color #aa36ba #1:61.C;color #95afba #1:62.C;color #aa74d8 #1:63.C;color #731bd8 #1:64.C;color #442b6f #1:65.C;color #8f1dba #1:66.C;color #882d51 #1:67.C;color #9b2eab #1:68.C;color #764d51 #1:69.C;color #6adb51 #1:70.C;color #359160 #1:71.C;color #e283ab #1:72.C;color #ddb160 #1:73.C;color #cab2ba #1:74.C;color #5559ba #1:75.C;color #8a876f #1:76.C;color #c76aab #1:77.C;color #62a76f #1:78.C;color #793f60 #1:79.C;color #23be51 #1:80.C;color #bab09c #1:81.C;color #99397e #1:82.C;color #a42bc9 #1:83.C;color #53597e #1:84.C;color #165ad8 #1:85.C;color #9d888d #1:86.C;color #d04dab #1:87.C;color #b1d5ba #1:88.C;color #c66dab #1:89.C;color #44a151 #1:90.C;color #df5760 #1:91.C;color #66df6f #1:92.C;color #cf1dd8 #1:93.C;color #a269ab #1:94.C;color #551fba #1:95.C;color #e4a7c9 #1:96.C;color #1f5c8b #2:1.A;color #a68a40 #2:2.A;color #5f3140 #2:3.A;color #76147c #2:4.A;color #9d5140 #2:5.A;color #6c8ed6 #2:6.A;color #37804f #2:7.A;color #8edb31 #2:8.A;color #7b64e5 #2:9.A;color #ab2140 #2:10.A;color #a89a40 #2:11.A;color #b53231 #2:12.A;color #7e247c #2:13.A;color #6125d6 #2:14.A;color #a2265e #2:15.A;color #97bd7c #2:16.A;color #aa556d #2:17.A;color #a9a140 #2:18.A;color #9c3931 #2:19.A;color #9e407c #2:20.A;color #8faa6d #2:21.A;color #166f8b #2:22.A;color #9bd97c #2:23.A;color #76dad6 #2:24.A;color #cd90e5 #2:25.A;color #2628d6 #2:26.A;color #97ce31 #2:27.A;color #e2c07c #2:28.A;color #933a4f #2:29.A;color #cf328b #2:30.A;color #a27e5e #2:31.A;color #b5ca31 #2:32.A;color #946222 #2:33.A;color #8f546d #2:34.A;color #28a040 #2:35.A;color #ab4740 #2:36.A;color #8c1b6d #2:37.A;color #95946d #2:38.A;color #d63b6d #2:39.A;color #76ba5e #2:40.A;color #b5524f #2:41.A;color #8880d6 #2:42.A;color #b3ae8b #2:43.A;color #9acd31 #2:44.A;color #77924f #2:45.A;color #2cc0d6 #2:46.A;color #a9a340 #2:47.A;color #e2867c #2:48.A;color #b5b431 #2:49.A;color #498e4f #2:50.A;color #64627c #2:51.A;color #85188b #2:52.A;color #e4827c #2:53.A;color #1f1a6d #2:54.A;color #a4a27c #2:55.A;color #cdb2e5 #2:56.A;color #60a45e #2:57.A;color #7d2d40 #2:58.A;color #80c45e #2:59.A;color #444422 #2:60.A;color #7ddb40 #2:61.A;color #688240 #2:62.A;color #1fbfd6 #2:63.A;color #46c05e #2:64.A;color #71584f #2:65.A;color #2686d6 #2:66.A;color #353ce5 #2:67.A;color #c85b8b #2:68.A;color #a37a31 #2:69.A;color #3518e5 #2:70.A;color #4882d6 #2:71.A;color #b55631 #2:72.A;color #74487c #2:73.A;color #6149d6 #2:74.A;color #282c40 #2:75.A;color #211e8b #2:76.A;color #22978b #2:77.A;color #7998e5 #2:78.A;color #a66c40 #2:79.A;color #501931 #2:80.A;color #15dd7c #2:81.A;color #ca756d #2:82.A;color #3b94e5 #2:83.A;color #262cd6 #2:84.A;color #7fc322 #2:85.A;color #a88840 #2:86.A;color #2b7a8b #2:87.A;color #486cd6 #2:88.A;color #994031 #2:89.A;color #71ce31 #2:90.A;color #3a8440 #2:91.A;color #ffffff #2:92.A;color #a2c25e #2:93.A;color #cf968b #2:94.A;color #28c440 #2:95.A;color #7b3ee5 #2:96.A;color #6aa8d6 #2:97.A;color #977b82 #3:2.A;color #683191 #3:3.A;color #7dc8af #3:4.A;color #768dcd #3:5.A;color #bb16af #3:6.A;color #1c5373 #3:7.A;color #3727a0 #3:8.A;color #d0afaf #3:9.A;color #6cb755 #3:10.A;color #8d7c73 #3:11.A;color #26d755 #3:12.A;color #1f15be #3:13.A;color #7e3464 #3:14.A;color #df35be #3:15.A;color #484555 #3:16.A;color #9764cd #3:17.A;color #84a191 #3:18.A;color #c766af #3:19.A;color #6f9a55 #3:20.A;color #9eb9cd #3:21.A;color #531582 #3:22.A;color #707064 #3:23.A;color #9b80cd #3:24.A;color #d0dbaf #3:25.A;color #3773a0 #3:26.A;color #26a155 #3:27.A;color #414855 #3:28.A;color #e267cd #3:29.A;color #c023cd #3:30.A;color #b18dbe #3:31.A;color #84d991 #3:32.A;color #1fadbe #3:33.A;color #bc63cd #3:34.A;color #ad19dc #3:35.A;color #6483cd #3:36.A;color #c9deaf #3:37.A;color #aab2dc #3:38.A;color #59d182 #3:39.A;color #6d6055 #3:40.A;color #1cd955 #3:41.A;color #b5cba0 #3:42.A;color #e281af #3:43.A;color #9537be #3:44.A;color #d6b0be #3:45.A;color #3bcf64 #3:46.A;color #2c6755 #3:47.A;color #e586cd #3:48.A;color #e22dcd #3:49.A;color #e29daf #3:50.A;color #4935a0 #3:51.A;color #caccbe #3:52.A;color #d573be #3:53.A;color #e229cd #3:54.A;color #b55782 #3:55.A;color #e449cd #3:56.A;color #614a55 #3:57.A;color #9e69cd #3:58.A;color #5f8873 #3:59.A;color #958fbe #3:60.A;color #9e45cd #3:61.A;color #975564 #3:62.A;color #1cdd73 #3:63.A;color #5ba291 #3:64.A;color #bedf55 #3:65.A;color #359564 #3:66.A;color #262d55 #3:67.A;color #711fa0 #3:68.A;color #505c64 #3:69.A;color #662773 #3:70.A;color #d319be #3:71.A;color #e2a1cd #3:72.A;color #b5cf82 #3:73.A;color #88b2be #3:74.A;color #bb4aaf #3:75.A;color #dc8773 #3:76.A;color #d579be #3:77.A;color #7c9864 #3:78.A;color #9d6c91 #3:79.A;color #79cd64 #3:80.A;color #aa1adc #3:81.A;color #ab4891 #3:82.A;color #1ca373 #3:83.A;color #8b4a73 #3:84.A;color #802daf #3:85.A;color #d9c4cd #3:86.A;color #c64daf #3:87.A;color #857b64 #3:88.A;color #a26daf #3:89.A;color #c629af #3:90.A;color #adb1be #3:91.A;color #5849af #3:92.A;color #93b3a0 #3:93.A;color #48e155 #3:94.A;color #93d3a0 #3:95.A;color #dc4d73 #3:96.A;color #d53fbe #3:97.A;color #6a4f55 #3:98.A
On my Macbook these color commands take about 2 seconds.
Email signature
ChimeraX 1.0 (2020-06-04):
open pdb:4zho pdb:5h92 pdb:1a70 pdb:4zhp
view
color #1/A:2/B:2 #977b64;color #1/A:3/B:3 #e46daf;color #1/A:4/B:4 #7dc891;color #1/A:5/B:5 #768daf;color #1/A:6/B:6 #bb1691;color #1/A:7/B:7 #1c5355;color #1/A:8/B:8 #5b1873;color #1/A:9/B:9 #d0af91;color #1/A:10/B:10 #6cb737;color #1/A:11/B:11 #8d7c55;color #1/A:12/B:12 #844173;color #1/A:13/B:13 #1f15a0;color #1/A:14/B:14 #a27f91;color #1/A:15/B:15 #19cb64;color #1/A:16/B:16 #484537;color #1/A:17/B:17 #61be37;color #1/A:18/B:18 #84a173;color #1/A:19/B:19 #c76691;color #1/A:20/B:20 #8f31a0;color #1/A:21/B:21 #9eb9af;color #1/A:22/B:22 #531564;color #1/A:23/B:23 #707046;color #1/A:24/B:24 #9b80af;color #1/A:25/B:25 #ca18a0;color #1/A:26/B:26 #377382;color #1/A:27/B:27 #26a137;color #1/A:28/B:28 #d3b1a0;color #1/A:29/B:29 #e267af;color #1/A:30/B:30 #c023af;color #1/A:31/B:31 #b18da0;color #1/A:32/B:32 #88bb55;color #1/A:33/B:33 #1fada0;color #1/A:34/B:34 #b22773;color #1/A:35/B:35 #ad19be;color #1/A:36/B:36 #6483af;color #1/A:37/B:37 #c9de91;color #1/A:38/B:38 #aab2be;color #1/A:39/B:39 #59d164;color #1/A:40/B:40 #6d6037;color #1/A:41/B:41 #1cd937;color #1/A:42/B:42 #b5cb82;color #1/A:43/B:43 #e28191;color #1/A:44/B:44 #9537a0;color #1/A:45/B:45 #d6b0a0;color #1/A:46/B:46 #3bcf46;color #1/A:47/B:47 #2c6737;color #1/A:48/B:48 #e586af;color #1/A:49/B:49 #e22daf;color #1/A:50/B:50 #e29d91;color #1/A:51/B:51 #493582;color #1/A:52/B:52 #cacca0;color #1/A:53/B:53 #7f2855;color #1/A:54/B:54 #86dd91;color #1/A:55/B:55 #b55764;color #1/A:56/B:56 #4ac155;color #1/A:57/B:57 #614a37;color #1/A:58/B:58 #9e69af;color #1/A:59/B:59 #5f8855;color #1/A:60/B:60 #958fa0;color #1/A:61/B:61 #9e45af;color #1/A:62/B:62 #b9beaf;color #1/A:63/B:63 #4add55;color #1/A:64/B:64 #5ba273;color #1/A:65/B:65 #bedf37;color #1/A:66/B:66 #359546;color #1/A:67/B:67 #262d37;color #1/A:68/B:68 #cf5bbe;color #1/A:69/B:69 #505c46;color #1/A:70/B:70 #76db37;color #1/A:71/B:71 #3d37be;color #1/A:72/B:72 #a28391;color #1/A:73/B:73 #b5cf64;color #1/A:74/B:74 #88b2a0;color #1/A:75/B:75 #bb4a91;color #1/A:76/B:76 #dc8755;color #1/A:77/B:77 #d579a0;color #1/A:78/B:78 #7c9846;color #1/A:79/B:79 #b57b82;color #1/A:80/B:80 #79cd46;color #1/A:81/B:81 #aa1abe;color #1/A:82/B:82 #ab4873;color #1/A:83/B:83 #c6df91;color #1/A:84/B:84 #b3b3be;color #1/A:85/B:85 #802d91;color #1/A:86/B:86 #d9c4af;color #1/A:87/B:87 #c64d91;color #1/A:88/B:88 #857b46;color #1/A:89/B:89 #a26d91;color #1/A:90/B:90 #c62991;color #1/A:91/B:91 #adb1a0;color #1/A:92/B:92 #584991;color #1/A:93/B:93 #93b382;color #1/A:94/B:94 #48e137;color #2/C:1 #88c56f;color #2/C:2 #7b5d60;color #2/C:3 #32d660;color #2/C:4 #339b7e;color #2/C:5 #702460;color #2/C:6 #6725ba;color #2/C:7 #dc536f;color #2/C:8 #a3db7e;color #2/C:9 #a8918d;color #2/C:10 #d84eba;color #2/C:11 #758b7e;color #2/C:12 #88d751;color #2/C:13 #75ab7e;color #2/C:14 #8e527e;color #2/C:15 #cf53d8;color #2/C:16 #dc636f;color #2/C:17 #7d288d;color #2/C:18 #707460;color #2/C:19 #6fde51;color #2/C:20 #35a960;color #2/C:21 #627d8d;color #2/C:22 #bb42ab;color #2/C:23 #327060;color #2/C:24 #a3357e;color #2/C:25 #28bd8d;color #2/C:26 #8f91ba;color #2/C:27 #c856d8;color #2/C:28 #795760;color #2/C:29 #c067c9;color #2/C:30 #669b6f;color #2/C:31 #adabd8;color #2/C:32 #889d51;color #2/C:33 #c18f9c;color #2/C:34 #26bd51;color #2/C:35 #55cdba;color #2/C:36 #7e1a60;color #2/C:37 #238451;color #2/C:38 #68678d;color #2/C:39 #6da451;color #2/C:40 #a315d8;color #2/C:41 #88256f;color #2/C:42 #1f17ba;color #2/C:43 #4a456f;color #2/C:44 #6da051;color #2/C:45 #a4bfc9;color #2/C:46 #9557ba;color #2/C:47 #d6d0ba;color #2/C:48 #791d60;color #2/C:49 #e2e1ab;color #2/C:50 #1c616f;color #2/C:51 #7fda6f;color #2/C:52 #8e907e;color #2/C:53 #77559c;color #2/C:54 #c4bf8d;color #2/C:55 #77759c;color #2/C:56 #1658d8;color #2/C:57 #add1d8;color #2/C:58 #aa5aba;color #2/C:59 #95d3ba;color #2/C:60 #adadd8;color #2/C:61 #aa36ba;color #2/C:62 #95afba;color #2/C:63 #aa74d8;color #2/C:64 #731bd8;color #2/C:65 #442b6f;color #2/C:66 #8f1dba;color #2/C:67 #882d51;color #2/C:68 #9b2eab;color #2/C:69 #764d51;color #2/C:70 #6adb51;color #2/C:71 #359160;color #2/C:72 #e283ab;color #2/C:73 #ddb160;color #2/C:74 #cab2ba;color #2/C:75 #5559ba;color #2/C:76 #8a876f;color #2/C:77 #c76aab;color #2/C:78 #62a76f;color #2/C:79 #793f60;color #2/C:80 #23be51;color #2/C:81 #bab09c;color #2/C:82 #99397e;color #2/C:83 #a42bc9;color #2/C:84 #53597e;color #2/C:85 #165ad8;color #2/C:86 #9d888d;color #2/C:87 #d04dab;color #2/C:88 #b1d5ba;color #2/C:89 #c66dab;color #2/C:90 #44a151;color #2/C:91 #df5760;color #2/C:92 #66df6f;color #2/C:93 #cf1dd8;color #2/C:94 #a269ab;color #2/C:95 #551fba;color #2/C:96 #e4a7c9;color #3/A:1 #1f5c8b;color #3/A:2 #a68a40;color #3/A:3 #5f3140;color #3/A:4 #76147c;color #3/A:5 #9d5140;color #3/A:6 #6c8ed6;color #3/A:7 #37804f;color #3/A:8 #8edb31;color #3/A:9 #7b64e5;color #3/A:10 #ab2140;color #3/A:11 #a89a40;color #3/A:12 #b53231;color #3/A:13 #7e247c;color #3/A:14 #6125d6;color #3/A:15 #a2265e;color #3/A:16 #97bd7c;color #3/A:17 #aa556d;color #3/A:18 #a9a140;color #3/A:19 #9c3931;color #3/A:20 #9e407c;color #3/A:21 #8faa6d;color #3/A:22 #166f8b;color #3/A:23 #9bd97c;color #3/A:24 #76dad6;color #3/A:25 #cd90e5;color #3/A:26 #2628d6;color #3/A:27 #97ce31;color #3/A:28 #e2c07c;color #3/A:29 #933a4f;color #3/A:30 #cf328b;color #3/A:31 #a27e5e;color #3/A:32 #b5ca31;color #3/A:33 #946222;color #3/A:34 #8f546d;color #3/A:35 #28a040;color #3/A:36 #ab4740;color #3/A:37 #8c1b6d;color #3/A:38 #95946d;color #3/A:39 #d63b6d;color #3/A:40 #76ba5e;color #3/A:41 #b5524f;color #3/A:42 #8880d6;color #3/A:43 #b3ae8b;color #3/A:44 #9acd31;color #3/A:45 #77924f;color #3/A:46 #2cc0d6;color #3/A:47 #a9a340;color #3/A:48 #e2867c;color #3/A:49 #b5b431;color #3/A:50 #498e4f;color #3/A:51 #64627c;color #3/A:52 #85188b;color #3/A:53 #e4827c;color #3/A:54 #1f1a6d;color #3/A:55 #a4a27c;color #3/A:56 #cdb2e5;color #3/A:57 #60a45e;color #3/A:58 #7d2d40;color #3/A:59 #80c45e;color #3/A:60 #444422;color #3/A:61 #7ddb40;color #3/A:62 #688240;color #3/A:63 #1fbfd6;color #3/A:64 #46c05e;color #3/A:65 #71584f;color #3/A:66 #2686d6;color #3/A:67 #353ce5;color #3/A:68 #c85b8b;color #3/A:69 #a37a31;color #3/A:70 #3518e5;color #3/A:71 #4882d6;color #3/A:72 #b55631;color #3/A:73 #74487c;color #3/A:74 #6149d6;color #3/A:75 #282c40;color #3/A:76 #211e8b;color #3/A:77 #22978b;color #3/A:78 #7998e5;color #3/A:79 #a66c40;color #3/A:80 #501931;color #3/A:81 #15dd7c;color #3/A:82 #ca756d;color #3/A:83 #3b94e5;color #3/A:84 #262cd6;color #3/A:85 #7fc322;color #3/A:86 #a88840;color #3/A:87 #2b7a8b;color #3/A:88 #486cd6;color #3/A:89 #994031;color #3/A:90 #71ce31;color #3/A:91 #3a8440;color #3/A:92 #ffffff;color #3/A:93 #a2c25e;color #3/A:94 #cf968b;color #3/A:95 #28c440;color #3/A:96 #7b3ee5;color #3/A:97 #6aa8d6;color #4/A:2 #977b82;color #4/A:3 #683191;color #4/A:4 #7dc8af;color #4/A:5 #768dcd;color #4/A:6 #bb16af;color #4/A:7 #1c5373;color #4/A:8 #3727a0;color #4/A:9 #d0afaf;color #4/A:10 #6cb755;color #4/A:11 #8d7c73;color #4/A:12 #26d755;color #4/A:13 #1f15be;color #4/A:14 #7e3464;color #4/A:15 #df35be;color #4/A:16 #484555;color #4/A:17 #9764cd;color #4/A:18 #84a191;color #4/A:19 #c766af;color #4/A:20 #6f9a55;color #4/A:21 #9eb9cd;color #4/A:22 #531582;color #4/A:23 #707064;color #4/A:24 #9b80cd;color #4/A:25 #d0dbaf;color #4/A:26 #3773a0;color #4/A:27 #26a155;color #4/A:28 #414855;color #4/A:29 #e267cd;color #4/A:30 #c023cd;color #4/A:31 #b18dbe;color #4/A:32 #84d991;color #4/A:33 #1fadbe;color #4/A:34 #bc63cd;color #4/A:35 #ad19dc;color #4/A:36 #6483cd;color #4/A:37 #c9deaf;color #4/A:38 #aab2dc;color #4/A:39 #59d182;color #4/A:40 #6d6055;color #4/A:41 #1cd955;color #4/A:42 #b5cba0;color #4/A:43 #e281af;color #4/A:44 #9537be;color #4/A:45 #d6b0be;color #4/A:46 #3bcf64;color #4/A:47 #2c6755;color #4/A:48 #e586cd;color #4/A:49 #e22dcd;color #4/A:50 #e29daf;color #4/A:51 #4935a0;color #4/A:52 #caccbe;color #4/A:53 #d573be;color #4/A:54 #e229cd;color #4/A:55 #b55782;color #4/A:56 #e449cd;color #4/A:57 #614a55;color #4/A:58 #9e69cd;color #4/A:59 #5f8873;color #4/A:60 #958fbe;color #4/A:61 #9e45cd;color #4/A:62 #975564;color #4/A:63 #1cdd73;color #4/A:64 #5ba291;color #4/A:65 #bedf55;color #4/A:66 #359564;color #4/A:67 #262d55;color #4/A:68 #711fa0;color #4/A:69 #505c64;color #4/A:70 #662773;color #4/A:71 #d319be;color #4/A:72 #e2a1cd;color #4/A:73 #b5cf82;color #4/A:74 #88b2be;color #4/A:75 #bb4aaf;color #4/A:76 #dc8773;color #4/A:77 #d579be;color #4/A:78 #7c9864;color #4/A:79 #9d6c91;color #4/A:80 #79cd64;color #4/A:81 #aa1adc;color #4/A:82 #ab4891;color #4/A:83 #1ca373;color #4/A:84 #8b4a73;color #4/A:85 #802daf;color #4/A:86 #d9c4cd;color #4/A:87 #c64daf;color #4/A:88 #857b64;color #4/A:89 #a26daf;color #4/A:90 #c629af;color #4/A:91 #adb1be;color #4/A:92 #5849af;color #4/A:93 #93b3a0;color #4/A:94 #48e155;color #4/A:95 #93d3a0;color #4/A:96 #dc4d73;color #4/A:97 #d53fbe;color #4/A:98 #6a4f55
These color commands take 15-16 seconds.
Thanks,
Mungo
by , 5 years ago
Attachment: | commands.txt added |
---|
comment:4 by , 5 years ago
The 383 commands also fills the undo history with 383 color undo operations, not ideal. But not saving undo information reduced time only a tiny bit 2.7 seconds to 2.6 seconds.
comment:5 by , 5 years ago
Some additional timing info:
Running the color commands normally on my machine initially takes 6.5 seconds but gets slower with each repetition so that the sixth run takes 8.3 seconds, the slowdown gets worse with each try, i.e. 6.5, 6.8, 6.9, 7.3, 7.7, 8.3 . Also 25-30MB of additional memory gets used with each pass.
Running the commands with logging turned off (but not status messages) stops the progressive slowdown (i.e. all tries roughly 6.5 seconds) but the memory usage increase is still there. So it seems the repetition slowdown is from reloading a progressively bigger log. I wonder if this affects all big logs, or if its because the log is getting reloaded a zillion times here.
Running the commands with both logging and status messages turned off speeds things up to roughly a steady 4.8 seconds. The memory leak goes away. So somehow status message handling is leaking memory.
If I also have no models open when the commands are run, then the time drops to ~3.65 seconds. So actually coloring the structures is about 1.15 seconds and the rest (3.65) is overhead going from the command text to actually running the coloring code. From my previous line profiling and by-hand timings (class methods can't be line profiled), I believe it's largely from parsing the atom specs.
comment:6 by , 5 years ago
Added a regulating timer to the log to effectively collate rapid log messages. This eliminates the progressive slowing of the color commands. The memory leak and other problems are still there.
comment:7 by , 5 years ago
Repeating the color commands 5 times results in an increase in memory usage of 158 MB. With logging turned off, the same. With just the OpenGL part of the status mechanism turned off it drops to 36 MB.
follow-up: 8 comment:9 by , 5 years ago
If all I do is comment out the self._draw_text() call in _StatusBarOpenGL.status(), the memory use increase drops to 37 MB.
comment:10 by , 5 years ago
The routine text_image_rgba() that uses Qt to convert status line text to an rgba image for drawing uses all the memory. It is using Qt objects QImage, QPainter, QFont, QFontMetrics, QColor, QBrush, QPen. I would expect all of these to be released when the Python objects get deleted but that is not what is happening. I will look into whether any of these need some explicit delete or release call.
comment:11 by , 5 years ago
I fixed the status line memory leak. It turns out it was PyOpenGL that leaked memory when handed non-contiguous numpy arrays for textures (or for any other OpenGL call). I reported this PyOpenGL 3.1.5 bug
The reason it used a non-contiguous array is the text needed to be flipped vertically to appear rightside up and that just used a vertically flipped view of the original numpy array. I protected the ChimeraX OpenGL texture and vertex buffer code making sure all numpy arrays are contiguous before calling PyOpenGL.
Still the memory use of about 15 Mbytes per execution of the 383 color commands is excessive. The log increased by 383 lines, maybe 10000 characters or 0.01 Mbytes.
comment:12 by , 5 years ago
Weird that your memory use is roughly 2X mine. Before the leak is plugged, the increase is roughly 32MB/command cycle. After being plugged it's 7MB/cycle whereas for you it's 15MB. Would like to get that near zero of course, but it's still a major improvement.
comment:13 by , 5 years ago
With output to the log commented out (Log.log() -> return True in log/src/tool.py) memory use increases 6 Mbytes each time the 383 color commands are run. Here are total Mbytes used on a MacBook Pro for 9 executions of the 383 commands:
284 296 305 311 317 323 328 335 340 348
With the log not commented out it uses about 8 Mbytes per 383 color commands, so the log is adding 2 more Mbytes per run
286 301 312 316 332 339 350 352 363 361
The above timings are with the 4 models opened (open 4zho 5h92 1a70 4zhp). With no models opened, 10 tries, about 4 Mbytes per run. This suggests about 2 Mbytes per run is being used by command parsing.
223 228 238 245 246 253 251 250 252 259 262
comment:14 by , 5 years ago
My memory use in previous comment is now in line with yours, about 8 Mbytes per 383 color commands with 4 models opened and log enabled. Looks like 2 Mbytes per run for logging, 2 Mbytes per run for command parsing, and 4 Mbytes per run for coloring. All of these would ideally be 0 except a tiny 0.01 for logging.
comment:15 by , 5 years ago
Replacing grako with TatSu results in an 8% speedup of atom spec parsing+evaluation (i.e. about .25 seconds aggregate total). Not breathtaking. Still, why not? Will be tougher to evaluate parsimonious, since it uses a different grammar and different parse-tree handling.
comment:16 by , 5 years ago
On the other hand, TatSu
does not correctly parse our command language, so scratch that (it doesn't correctly parse commands with an empty/omitted argument, e.g. "hb #1 two true" is okay but not "hb two true").
comment:17 by , 5 years ago
Looked at _why_ TatSu doesn't work a little bit. It's because the parser reports the failure position as after the first token rather than before it, so the 'Or' argument parser is fooled into thinking that the atom spec was partially parsed and that it therefore _is_ an atom spec, just mangled, rather than not an atom spec at all. So it never goes on to try the second option for the 'Or', it just reports an error.
comment:18 by , 3 years ago
Seems like we are going to have to use a C++ -based parser. I translated the atomspec syntax rules for Lark, supposedly one of the fastest pure Python parsers, and it takes 3.8 seconds to generate the parse tree. I've attached the test file.
by , 3 years ago
Attachment: | atomspec.lalr added |
---|
comment:21 by , 3 years ago
Yes, it's for the long series of color commands in comment 3 (you can see it in the attachment).
comment:22 by , 3 years ago
My comment about "going to have to use a C++-based parser" was slightly off base. There is no way on God's green earth that we're going to move the entirety of command-line parsing into C++. That would be nigh impossible with aliasing, Annotation inheritance, etc. Moving just the atom-spec parsing into C++ is still worth doing because of long atom specs crashing ChimeraX (#4842) and that may speed up command parsing some, but we may have to live with relative slow command parsing. I guess do #4842 first and see how much that helps.
Did a lot of profiling, and the atom-spec parser takes about 5 seconds to process the color commands, even with no structures open. Also, repeating the command leaks significant memory each time (30-ish MB) and after about 4 repetitions the execution gets slower by about .5 seconds each time, again with no structures open.