ChIMP Meetings (Joint Sali-Ferrin Lab Meetings)

October 15, 2009

Attendees: Conrad, Scooter, yz, TomF, TomG, Keren, Elena, Dina

Preliminary Progress

1. SAXS: TomG & Dina
   • Chimera interface to the SAXS module from IMP
   • Can calculate and plot a SAXS profile for a PDB model
   • Can overlay experimentally derived SAXS profile on the same plot
   • Potential licensing issues with distributing this with Chimera
   • Some potential UI enhancements:
     • Re-use of a plot window to compare experimental profiles vs. changed model
2. Opal: Conrad
   • Web service framework (from SDSC)
   • Easy to deploy web services -- can quickly wrap existing software (legacy command-line programs)
   • Currently running 1.9 on the Plato cluster
   • 2.0 is out, but not much change in functionality (primarily internal changes)
   • Currently using Opal in Chimera for:
     • Template lookup (interactive)
     • BLAST pdb/nr
       • Asynchronous -- can save in sessions, etc.
   • Others
     • GOElite
   • Output is whatever the program outputs
     • Files
     • stdout, stderr

Action Plan

1. Hire postdoc - Done!
2. Hire programmer
   1. Write job description (yz, TomF)
   2. Post through HR
   3. Go through Craig's List?
3. SAXS Interface
   1. Work is known
      • Share plot
      • Use web services
   2. Resolve license
4. Bring yz up to speed
   • Chimera
   • IMP
5. Develop an action plan for
   • MultiFit Interface
   • Protein Homology modeling/loop modeling

Additional Projects

1. Future/new projects
   1. Coarse-grained models (beads)
      • Fitting/optimization of coarse-grained models into density
   2. Flexible fitting
      • Refining data after manipulating structures within densities
   3. Finding components within 2D micrographs
      • Manipulate component in 3D, view on micrographs

Issues

1. Licensing issues with the SAXS module
   • Ben is working with UCSF to get license approved
     • Complicated: not all team members are only at UCSF, and some team members have left UCSF
   • Goal is to release IMP as LGPL
   • Would like to release IMP before releasing an IMP component (e.g. SAXS) separately as part of Chimera
   • Tom thinks that the SAXS component for IMP should be included with Chimera download
   • May need to think about incremental downloads

March 18, 2009

Introduction to IMP (Ben Webb)

1. Aim of IMP:
   • Use structural information from any source (experiment, physics, statistics)
   • Information at different resolutions
   • Generate models at low, high, or mixed resolution
   • Obtain the set of all models that are consistent with the input data
2. Satisfaction of spatial restraints
   • Representation of a system
   • Scoring function
   • Optimization/sampling
3. Example: nuclear pore complex
   • Optimizing individual protein positions (500 proteins)
   • Information:
     • Low-resolution information
     • Protein interactions
4. Example: 26S proteasome
   • Information:
     • EM density map
     • Some model information
     • Gross (very low resolution)
5. Design
   1. Small Kernel + modules
      • Kernel provides some basic C++ classes, utility functions, and a set of abstract bases
      • Modules provide the bulk of the functionality
      • Sali lab provides the kernel and a basic set of modules
   2. Kernel
      • Encodes for:
        • Representation
          • Particle class provides data storage for the system
          • Each Particle object is simply a hash of key/value pairs
            • values can be integers, floats, strings, or pointers to Particles
            • Keys can be added or removed at runtime
            • Eash Particle is an independent hash
          • Model object is a simple contains of Particles (plus Restraints and ScoreStates)
          • e.g. a Model may contains particles with:
            • x,y, z float attributes
            • x,y,z and R float attributes
            • particle 1 and particle2 pointer attributes (bonds)
          • Particle access is usually through Decorators which:
            • wrap Particles
            • provide convenient interface
            • add runtime checks
            • provide a runtime class hierarchy
          • e.g. a Particle with x,y,z keys can be wrapped by an XYZDecorator
          • e.g. a Particle wrapped by a ResidueDecorator could provide residue information
        • Scoring
          • Most simply, the Restraint abstract class takes a set of Particles and returns:
            • a score
            • first derivatives w.r.t. Particle atttributes
          • Known structural information is encoded as implementations of Restraint
          • ScoreState class allows for derived system state information
          • Building blocks
            • UnaryFunction: return the score & derivative of the input w.r.t. some function
            • PairScore: given two Particles, return some pairwise score & derivatives between them
            • PairsRestraint: apply a PairScore to each of a list of Particle pais
          • Example: combine Harmonic UnaryFunction with DistancePairScore to enforce a simple distance restraint
      • Optimization
        • Floating-point Particle attributes can be marked as 'optimizable'
        • The Optimizer class
          • Takes a Model as input
          • Modifies the optimizable attributes to give the best score
          • Yields the final Model state
   3. IMP modules
      • Modules provide useful functionality such as
        • General purpose optimizers (e.g. conjugate gradients, steepest descent)
        • Building block restraints
        • Support for certain representations
        • Special purpose optimizers (e.g. molecular dynamics, inference)
        • Interfaces to other packages (e.g. import of homology-derived restraints from Modeller)
        • Implementations of restraints for specific kinds of structural data (e.g. SAXS, EM, proteomics)
        • Specific applications, e.g. docking
      • Use defined interfaces in the IMP kernal
      • Can be implemented in C++ or Python
      • Live in their own IMP.foo namespace
      • Can be part of IMP or maintained by other labs
      • Can be released under any license compatible with IMP's (LGPL like)
      • Wrapping
        • Kernel and module C++ code wrapped by SWIG
        • Resulting Python code very similar to the C++
6. Compilation
   • Type 'scons'
   • Required dependencies: a C++ compiler, SWIG, boost
   • Optional dependencies: Modeller, CGAL, our EM library
   • Should work with reasonably recent version of g++, MSVC and boost
7. Interfaces with Chimera
   • Python frontend allows easy integration with other Python-based packages
   • e.g. a Chimera extension can simply "import IMP"
   • Alternatively
     • can generate IMP inputs and run IMP in a separate process
     • can read IMP outputs for visualization
   • For loose integration in future, could use the same web server interface framework proposed for ModWeb, ModLoop, etc.

Discussion

• Chimera is interactive, IMP is not necessarily
• Is there a benefit to monitoring an IMP run?
  • Depends on the application
    • NPC was very iterative
  • Might definitely need user interaction during the optimization
  • Biggest category is to have Chimera "check in" on results during optimization
  • If you can visualize the current state of the system, you can change position with Chimera
    • Guide be used to move things out of a local minimum
• Can Chimera be used to setup the constraints/restraints?
  • Could use sequence conservation, etc.
  • Chimera could be to prepare the information for an IMP job
• Chimera <--> IMP: Visualize IMP representation directly in Chimera (translator)
  • Can IMP output an XML-based format? Not yet, but it should be relatively straightforward

February 24, 2009

Introductions

Background

• Goal: leverage work going on in each lab to "make something that is more than the sum of the independent parts"
• History: past attempts have never come together
• Tom and Andre started a renewed conversation to establish the collaboration

Demo

• Chimera and model evaluation
  • Using Render By Attribute to show Modbase error profile (using B-Factor)
  • Question: How does "Render by Attribute" work when you don't have atoms or residues?
    • Answer: Currently it doesn't. Probably something to look at -- how to set up an equivalent mechanism for low-resolution structures?
      • Perhaps we could use volume tracer pseudo-atoms (markers) with render-by-attribute?
  • Question: Can you make worms transparent?
    • Answer: You should be able to...
  • Question: in molmap, could you set the threshold such that the volume within the mesh is a certain value.
    • Answer: The underlying code would support that, but there is no UI, yet. You can also report the volume as you adjust the threshold (which would be an adequate approach)
  • Restraints
    • One solution would be to extend the structure measurements tool (or use something similar)
    • How would you visualize restraints involving multiple atoms?
      • Structure/volume violations
      • Chimera can detect several restraint violations already
      • One solution might be to combine restraints and then plot the first derivative
      • Chimera's new 2D plotting library might be used for constraints that are not spatially correlated.
      • Spatially correlated constraints could be plotted using "normal mode"-style arrows (currently can be done using bild).
        • Getting an interface to do this interactively would be useful
      • Can the tracer pseudo-atom XML file add new attributes?
      • TODO: enhance markers (tracer pseudo-atoms) to support user-defined attributes.
  • Ensembles
    • EnsembleCluster
      • Question: Can EnsembleCluster be used for points in space rather than molecules?
      • Answer: Yes, if you read them in as atoms. Might also be able to to use markers.
      • TODO: enhance markers to be used by EnsembleMatch
      • Suggestion: How to represent objects that represent structures, but are not atomic resolution?
        • Maybe this is one area groups could work together to define standards
        • Currently Chimera uses model, residue, atom as it's hierarchical nomenclature. Would be useful to support additional nomenclature hierarchies that might represent lower resolution structures and other "weird" representations.
  • Animations
    • Continuing to enhance our command set
    • No time-line based UI as of yet
    • Tom's animation example...

Specific Applications

• Multifit - Keren Lasker, Tom Goddard
  • fit multiple proteins in EM maps with steric clash handling.
  • Illustration ​Chimera & MultiFit
  • Interface looks easily tractable, but there will need to be work to develop a general mechanism to support stand-alone runs without freezing Chimera
• SAXS - Dina Schneidman
  • using small angle x-ray scattering as a modeling constraint
  • want interactive 2-d plot of saxs curve and model display
• 2-d EM micrograph restraints - Javier Velazquez-Muriel
  • use 2-d EM micrographs as modeling restraint
  • can Chimera project simulated maps from models and correlate 2-d maps?
• ModWeb

Discussion

• What is the best way to interface between Chimera and other programs?
• One initial possibility would be to use readStdin, but use it "in reverse"
  • Applications: MultiFit, projecting 3D models to 2D, SAXS 2-d plots.
    • Our issues aren't with the simpler algorithms, the time consuming part is the user interface
    • Where we need algorithmic help is in more complicated algorithms (e.g. MultiFit)
  • TODO: improve 3D to 2D projection capabilities
  • TODO: Andre and Tom -- discuss supplemental funding proposal
• Web server interface
  • In the process of building Chimera infrastructure to support asynchronous requests to web servers, including saving and checking on session state.
  • TODO: include Hau, Dina, and Karen with meeting with John Irwin to talk about web services implementation
  • TODO: think about including Chimera<-->ModWeb functionality. Include Ben and Ursula in John Irwin meeting?
    • Key exchange would be the blocking point -- perhaps we could implement a Chimera interface that could be used to get the key automatically

Moving Forward

• Meet every two weeks, but focus on a specific area and advertise the topic in advance.
• Keep content on wiki -- ChIMP project (Chimera-IMP)