Opened 8 days ago

Last modified 8 days ago

#19695 assigned defect

Claude AI prompt to show density near a ligand

Reported by: Tom Goddard Owned by: a.rohou@…
Priority: moderate Milestone:
Component: Input/Output Version:
Keywords: Cc: Zach Pearson, Tom Goddard
Blocked By: Blocking:
Notify when closed: Platform: all
Project: ChimeraX

Description

I just tried the following prompt in a new Claude Desktop chat

Open pdb 9ll0 and show me the density near the ligand.

Results not too pretty. It has been going for 20 minutes, executing 31 command or tool calls so far, the first 16 of them desperately trying to figure out which EMDB map to fetch. Ah, done now! And all the output vanished from Claude Desktop. But below is the ChimeraX view. This seems really bad for such a simple request. I know a special-purpose mcp routine could be added for it, but it is a bit absurd if it needs special routines for everything. I suspect the fact that it lost all the output and was so incredibly slow was probably related to high Claude load. I have the Claude Pro plan $20/month, but the pricing page suggest I need Claude Mac at $100/month for "Priority access at high traffic times" (https://claude.com/pricing).

Change History (5)

comment:1 by Tom Goddard, 8 days ago

Hi Tom,

Thanks for letting me know about this. Yes, please could you create a bug and assign me to it?

I have been working on the MCP bridge pretty intensely the last few days (well, outside of my main job) and I'll have another pull request ready soon. I have made nice progress I think, without having to re-enable the show_hide specialized tool I had written previously. I think fixing the access to documentation and a few other things helped significantly.

Anyway, I tried your prompt with my current version (using Sonnet 4.5) and it didn't get stuck like yours did. It was done in about 1-2 minutes. But it downloaded the wrong map from emdb. Then I asked it how it had decided on that emdb id and it did another web search and downloaded the correct one. So I think we/I just need to improve the open_structure tool that Zach wrote at the very beginning so that the agent knows how to open emdb maps that correspond to pdb structures.

Please open that bug and I hope I can get this working pretty quickly with hopefully pretty minimal changes.

I'll note that this isn't a very pretty cryoEM map. The ligand is not super well resolved, so it's not easy to pick a good isocontour level to make a good looking figure...

Cheers,
Alexis

comment:2 by Tom Goddard, 8 days ago

Hello again,

I am really puzzled as to why your Claude Desktop worked for 20 minutes on this. I just checked out the develop branch (which is sync'ed with upstream) and tested your prompt and it only took a couple of minutes to get to a similar result to yours (this time it worked out successfully how to fetch the EMDB map alongside the PDB).

image.png

See the attached chat transcript (sorry this doesn't show the thinking and tool calls that the agent made, so it's not that useful).

Are you working with a ChimeraX built from the develop branch?

I do not have a Claude license any more. Zach, assuming that you do, can you reproduce the behavior that Tom is seeing with this prompt? I'm wondering if it's a Claude Desktop configuration issue perhaps.

Cheers,
Alexis

comment:3 by Tom Goddard, 8 days ago

Yes my test was with the current ChimeraX develop branch pulled today at about 3 pm. Claude loaded more than one wrong map for me, but did figure out they were wrong and closed them.

I've always observed that Claude Desktop runs a command in ChimeraX and I see it logged and the command completes successfully, and Claude desktop will spin a busy symbol for maybe a minute before it comments "Perfect! Now lets do blah..." So like Zach has mentioned it seems to spend a lot of time just waiting for the server.

I suspect your superior response is that you have a higher-end service with higher priority.

comment:4 by Tom Goddard, 8 days ago

Hi Alexis and Tom,

Here are my results. Ignore the last two commands, I was just setting the background color back to black, then I figured I should give you a screenshot that accurately represents what Claude Code produced (although I zoomed out a bit). It look about ten minutes. Claude tried a few different methods for fetching what it needed from EMDB, including trying to do it on its own in the chat, but it gave up when it realized it needed to use ChimeraX for it and then called out to ChimeraX to do it. It felt like most of the time Claude thought faster than it could type. Results would appear and Claude would give me a blurb about what it did, but it would hang a few words in, I’d look at something else, and a minute later the full text would be there.

— Zach

comment:5 by Zach Pearson, 8 days ago

There's an error in my email. I said 'Claude Code' when I meant 'Claude Desktop'.

Note: See TracTickets for help on using tickets.