= ChIMP Meetings (Joint Sali-Ferrin Lab Meetings) = [[PageOutline]] == October 15, 2009 == Attendees: Conrad, Scooter, yz, TomF, TomG, Keren, Elena, Dina === Preliminary Progress === 1. SAXS: TomG & Dina * Chimera interface to the SAXS module from IMP * Can calculate and plot a SAXS profile for a PDB model * Can overlay experimentally derived SAXS profile on the same plot * Potential licensing issues with distributing this with Chimera * Some potential UI enhancements: * Re-use of a plot window to compare experimental profiles vs. changed model 1. Opal: Conrad * Web service framework (from SDSC) * Easy to deploy web services -- can quickly wrap existing software (legacy command-line programs) * Currently running 1.9 on the Plato cluster * 2.0 is out, but not much change in functionality (primarily internal changes) * Currently using Opal in Chimera for: * Template lookup (interactive) * BLAST pdb/nr * Asynchronous -- can save in sessions, etc. * Others * GOElite * Output is whatever the program outputs * Files * stdout, stderr === Action Plan === 1. ~~Hire postdoc~~ - Done! 1. Hire programmer 1. Write job description (yz, TomF) 1. Post through HR 1. Go through Craig's List? 1. SAXS Interface 1. Work is known * Share plot * Use web services 1. Resolve license 1. Bring yz up to speed * Chimera * IMP 1. Develop an action plan for * !MultiFit Interface * Protein Homology modeling/loop modeling === Additional Projects === 1. Future/new projects 1. Coarse-grained models (beads) * Fitting/optimization of coarse-grained models into density 1. Flexible fitting * Refining data after manipulating structures within densities 1. Finding components within 2D micrographs * Manipulate component in 3D, view on micrographs === Issues === 1. Licensing issues with the SAXS module * Ben is working with UCSF to get license approved * Complicated: not all team members are only at UCSF, and some team members have left UCSF * Goal is to release IMP as LGPL * Would like to release IMP before releasing an IMP component (e.g. SAXS) separately as part of Chimera * Tom thinks that the SAXS component for IMP should be included with Chimera download * May need to think about incremental downloads == March 18, 2009 == === Introduction to IMP (Ben Webb) === 1. Aim of IMP: * Use structural information from any source (experiment, physics, statistics) * Information at different resolutions * Generate models at low, high, or mixed resolution * Obtain the set of all models that are consistent with the input data 1. Satisfaction of spatial restraints * Representation of a system * Scoring function * Optimization/sampling 1. Example: nuclear pore complex * Optimizing individual protein positions (500 proteins) * Information: * Low-resolution information * Protein interactions 1. Example: 26S proteasome * Information: * EM density map * Some model information * Gross (very low resolution) 1. Design 1. Small Kernel + modules * Kernel provides some basic C++ classes, utility functions, and a set of abstract bases * Modules provide the bulk of the functionality * Sali lab provides the kernel and a basic set of modules 1. Kernel * Encodes for: * Representation * Particle class provides data storage for the system * Each Particle object is simply a hash of key/value pairs * values can be integers, floats, strings, or pointers to Particles * Keys can be added or removed at runtime * Eash Particle is an independent hash * Model object is a simple contains of Particles (plus Restraints and ScoreStates) * e.g. a Model may contains particles with: * x,y, z float attributes * x,y,z and R float attributes * particle 1 and particle2 pointer attributes (bonds) * Particle access is usually through Decorators which: * wrap Particles * provide convenient interface * add runtime checks * provide a runtime class hierarchy * e.g. a Particle with x,y,z keys can be wrapped by an XYZDecorator * e.g. a Particle wrapped by a ResidueDecorator could provide residue information * Scoring * Most simply, the Restraint abstract class takes a set of Particles and returns: * a score * first derivatives w.r.t. Particle atttributes * Known structural information is encoded as implementations of Restraint * ScoreState class allows for derived system state information * Building blocks * UnaryFunction: return the score & derivative of the input w.r.t. some function * PairScore: given two Particles, return some pairwise score & derivatives between them * PairsRestraint: apply a PairScore to each of a list of Particle pais * Example: combine Harmonic UnaryFunction with DistancePairScore to enforce a simple distance restraint * Optimization * Floating-point Particle attributes can be marked as 'optimizable' * The Optimizer class * Takes a Model as input * Modifies the optimizable attributes to give the best score * Yields the final Model state 1. IMP modules * Modules provide useful functionality such as * General purpose optimizers (e.g. conjugate gradients, steepest descent) * Building block restraints * Support for certain representations * Special purpose optimizers (e.g. molecular dynamics, inference) * Interfaces to other packages (e.g. import of homology-derived restraints from Modeller) * Implementations of restraints for specific kinds of structural data (e.g. SAXS, EM, proteomics) * Specific applications, e.g. docking * Use defined interfaces in the IMP kernal * Can be implemented in C++ or Python * Live in their own IMP.foo namespace * Can be part of IMP or maintained by other labs * Can be released under any license compatible with IMP's (LGPL like) * Wrapping * Kernel and module C++ code wrapped by SWIG * Resulting Python code very similar to the C++ 1. Compilation * Type 'scons' * Required dependencies: a C++ compiler, SWIG, boost * Optional dependencies: Modeller, CGAL, our EM library * Should work with reasonably recent version of g++, MSVC and boost 1. Interfaces with Chimera * Python frontend allows easy integration with other Python-based packages * e.g. a Chimera extension can simply "import IMP" * Alternatively * can generate IMP inputs and run IMP in a separate process * can read IMP outputs for visualization * For loose integration in future, could use the same web server interface framework proposed for ModWeb, ModLoop, etc. === Discussion === * Chimera is interactive, IMP is not necessarily * Is there a benefit to monitoring an IMP run? * Depends on the application * NPC was very iterative * Might definitely need user interaction during the optimization * Biggest category is to have Chimera "check in" on results during optimization * If you can visualize the current state of the system, you can change position with Chimera * Guide be used to move things out of a local minimum * Can Chimera be used to setup the constraints/restraints? * Could use sequence conservation, etc. * Chimera could be to prepare the information for an IMP job * Chimera <--> IMP: Visualize IMP representation directly in Chimera (translator) * Can IMP output an XML-based format? Not yet, but it should be relatively straightforward == February 24, 2009 == === Introductions === === Background === * Goal: leverage work going on in each lab to "make something that is more than the sum of the independent parts" * History: past attempts have never come together * Tom and Andre started a renewed conversation to establish the collaboration === Demo === * Chimera and model evaluation * Using Render By Attribute to show Modbase error profile (using B-Factor) * Question: How does "Render by Attribute" work when you don't have atoms or residues? * Answer: Currently it doesn't. Probably something to look at -- how to set up an equivalent mechanism for low-resolution structures? * Perhaps we could use volume tracer pseudo-atoms (markers) with render-by-attribute? * Question: Can you make worms transparent? * Answer: You should be able to... * Question: in molmap, could you set the threshold such that the volume within the mesh is a certain value. * Answer: The underlying code would support that, but there is no UI, yet. You can also report the volume as you adjust the threshold (which would be an adequate approach) * Restraints * One solution would be to extend the structure measurements tool (or use something similar) * How would you visualize restraints involving multiple atoms? * Structure/volume violations * Chimera can detect several restraint violations already * One solution might be to combine restraints and then plot the first derivative * Chimera's new 2D plotting library might be used for constraints that are not spatially correlated. * Spatially correlated constraints could be plotted using "normal mode"-style arrows (currently can be done using bild). * Getting an interface to do this interactively would be useful * Can the tracer pseudo-atom XML file add new attributes? * TODO: enhance markers (tracer pseudo-atoms) to support user-defined attributes. * Ensembles * EnsembleCluster * Question: Can EnsembleCluster be used for points in space rather than molecules? * Answer: Yes, if you read them in as atoms. Might also be able to to use markers. * TODO: enhance markers to be used by EnsembleMatch * Suggestion: How to represent objects that represent structures, but are not atomic resolution? * Maybe this is one area groups could work together to define standards * Currently Chimera uses model, residue, atom as it's hierarchical nomenclature. Would be useful to support additional nomenclature hierarchies that might represent lower resolution structures and other "weird" representations. * Animations * Continuing to enhance our command set * No time-line based UI as of yet * Tom's animation example... === Specific Applications === * Multifit - Keren Lasker, Tom Goddard * fit multiple proteins in EM maps with steric clash handling. * Illustration [http://www.cgl.ucsf.edu/home/goddard/temp/1z5s/1z5s.html Chimera & MultiFit] * Interface looks easily tractable, but there will need to be work to develop a general mechanism to support stand-alone runs without freezing Chimera * SAXS - Dina Schneidman * using small angle x-ray scattering as a modeling constraint * want interactive 2-d plot of saxs curve and model display * 2-d EM micrograph restraints - Javier Velazquez-Muriel * use 2-d EM micrographs as modeling restraint * can Chimera project simulated maps from models and correlate 2-d maps? * ModWeb === Discussion === * What is the best way to interface between Chimera and other programs? * One initial possibility would be to use readStdin, but use it "in reverse" * Applications: MultiFit, projecting 3D models to 2D, SAXS 2-d plots. * Our issues aren't with the simpler algorithms, the time consuming part is the user interface * Where we need algorithmic help is in more complicated algorithms (e.g. MultiFit) * TODO: improve 3D to 2D projection capabilities * TODO: Andre and Tom -- discuss supplemental funding proposal * Web server interface * In the process of building Chimera infrastructure to support asynchronous requests to web servers, including saving and checking on session state. * TODO: include Hau, Dina, and Karen with meeting with John Irwin to talk about web services implementation * TODO: think about including Chimera<-->ModWeb functionality. Include Ben and Ursula in John Irwin meeting? * Key exchange would be the blocking point -- perhaps we could implement a Chimera interface that could be used to get the key automatically === Moving Forward === * Meet every two weeks, but focus on a specific area and advertise the topic in advance. * Keep content on wiki -- ChIMP project (Chimera-IMP)