ChimeraX Revision Control
We will use git and git flow for revision control to manage Chimera 2 source code. Development will follow the Vincent Driessen's branching model where the master branch is the release branch and the develop branch is the daily build branch. A team repository is the repository that is used to create release and daily builds, i.e., changes committed in developer repositories but not pushed to the team repository will not appear in either release or daily builds.
This document is not intended as a git tutorial or reference. It has examples for tasks that come up in development of Chimera 2. For a good introduction, read the git book.
There are many web pages describing git flow in various level of details. These two are good places to start:
The remainder of this document describes some common tasks using git. plato refers to the Linux cluster on which the team repository is stored. franklin is the name of a node in the plato cluster.
Initialize git Parameters
Before using git repositories, it is a good idea to set the account and e-mail names that will be recorded. For example, my git parameters are set with:
$ git config --global user.name "Conrad Huang" $ git config --global user.email conrad@cgl.ucsf.eduClone a Developer Repository
To make changes to the source code, a developer must have a working repository. The repository may be on the same host as the team repository, or it may be on a separate host.
The following example shows the steps for creating a repository in the same directory, /usr/local/projects/chimera/git, as the team repository. Developer repositories do not need to be the the same directory as the team repository. The last argument to git clone specifies the location where the developer repository will be created and it may be a full path. However, the web server on plato is configured to simplify accessing files below /usr/local/projects/chimera/git (see NotYetWritten), for example, for viewing changes to HTML files; so it may be more convenient to have developer repositories there than elsewhere.
After cloning the repository, it is necessary to initialize it for git flow, the tool for managing branches. The -d flag specifies that the default branch names for development and feature branches be used. Even if there is no plan to use git flow operations, executing this command is useful since it checks out the develop branch rather than leaving the repository on the master branch, which should only be modified during releases.
franklin:~ conrad$ cd /usr/local/projects/chimerax/git franklin:git conrad$ git clone chimerax.git myrepo Initialized empty Git repository in /usr/local/projects/chimerax/git/myrepo/.git/ franklin:git conrad$ cd myrepo /usr/local/projects/chimerax/git/myrepo franklin:myrepo conrad$ git flow init -d Using default branch names. Which branch should be used for bringing forth production releases? - master Branch name for production releases: [master] Branch name for "next release" development: [develop] How to name your supporting branch prefixes? Feature branches? [feature/] Release branches? [release/] Hotfix branches? [hotfix/] Support branches? [support/] Version tag prefix? [] franklin:myrepo conrad$ ls -l total 36 drwxrwxr-x 4 conrad ferrin 3864 May 3 11:01 docs/ -rw-rw-r-- 1 conrad ferrin 183 May 3 11:01 index.html -rw-rw-r-- 1 conrad ferrin 657 May 3 11:01 Makefile drwxrwxr-x 2 conrad ferrin 3864 May 3 11:01 mk/ drwxrwxr-x 28 conrad ferrin 3864 May 3 11:01 prereqs/ -rw-rw-r-- 1 conrad ferrin 1262 May 3 11:01 README.rst drwxrwxr-x 7 conrad ferrin 3864 May 3 11:01 src/ -rwxrwxr-x 1 conrad ferrin 3589 May 3 11:01 vsvars.sh* drwxrwxr-x 3 conrad ferrin 3864 May 3 11:01 webdemo/Alternatively, a developer may use a repository on his own workstation rather than plato. Below is an example of creating a clone on the remote host, <b>loft</b>, running Cygwin on Windows 7. The <b>ssh</b> agent has already been initialized, so no passwords are required. The steps after git clone are effective the same as the example above.
[loft - /e]$ git clone \ ssh://conrad@plato.cgl.ucsf.edu/usr/local/projects/chimerax/git/chimerax.git \ chimerax Cloning into 'chimerax'... remote: Counting objects: 556, done. remote: Compressing objects: 100% (533/533), done. remote: Total 556 (delta 170), reused 275 (delta 16) Receiving objects: 100% (556/556), 311.17 MiB | 621 KiB/s, done. Resolving deltas: 100% (170/170), done. [loft - /e]$ cd chimerax /e/chimerax [loft - /e/chimerax]$ git flow init -d Using default branch names. Which branch should be used for bringing forth production releases? - master Branch name for production releases: [master] Branch name for "next release" development: [develop] How to name your supporting branch prefixes? Feature branches? [feature/] Release branches? [release/] Hotfix branches? [hotfix/] Support branches? [support/] Version tag prefix? [] [loft - /e/chimerax]$ ls -l total 26 -rw-r--r--+ 1 conrad None 657 May 3 10:51 Makefile -rw-r--r--+ 1 conrad None 1262 May 3 10:51 README.rst drwxr-xr-x+ 1 conrad None 0 May 3 10:51 docs/ -rw-r--r--+ 1 conrad None 183 May 3 10:51 index.html drwxr-xr-x+ 1 conrad None 0 May 3 10:51 mk/ drwxr-xr-x+ 1 conrad None 0 May 3 10:51 prereqs/ drwxr-xr-x+ 1 conrad None 0 May 3 10:51 src/ -rwxr-xr-x+ 1 conrad None 3589 May 3 10:51 vsvars.sh* drwxr-xr-x+ 1 conrad None 0 May 3 10:51 webdemo/Use develop Branch Directly for Development
For short tasks that one expects to complete before moving on to others, a developer may use the develop branch (in his own repository) directly, rather than creating a feature branch (see next section). The example below shows a file being added to the repositories. Note that modifying a file uses exactly the same steps git-wise.
A few items of note:
- the add command stages a file, which means that the repository version of the file will be updated with the current contents of the working file at the next commit.
- git diff may be used to compare the working file with either the staged (using no extra arguments) or the committed (using the HEAD argument) file.
- files may be reverted to the staged version by using the checkout command.
- to unstage a file and return to the committed file, use the reset command.
- a pull is usually done before a commit so that changes committed by others may be incorporated immediately rather than as part of a merge commit.
- the -a flag to commit is used to stage the latest version of changes before committing. This is different from svn where committing a file always commits the latest version. In git, a commit commits the version of the file at the time that it was staged; subsequent changes will not be committed unless the file is explicitly staged again using the add command or the -a flag is specified for the commit command.
- the -m flag to commit specifies a log message for the commit. To use longer log messages, omit the -m flag and a text editor (from environment variable EDITOR) will automatically start.
- the commit command only updates the local repository. To make changes visible to other developers, do not forget to push as well.
franklin:myrepo conrad$ pwd /usr/local/projects/chimerax/git/myrepo franklin:myrepo conrad$ git status # On branch develop nothing to commit (working directory clean) franklin:myrepo conrad$ echo "hello world" > example franklin:myrepo conrad$ git status # On branch develop # Untracked files: # (use "git add <file>..." to include in what will be committed) # # example nothing added to commit but untracked files present (use "git add" to track) franklin:myrepo conrad$ git add example franklin:myrepo conrad$ git status # On branch develop # Changes to be committed: # (use "git reset HEAD <file>..." to unstage) # # new file: example # franklin:myrepo conrad$ git diff franklin:myrepo conrad$ echo "hello conrad" > example franklin:myrepo conrad$ git diff diff --git a/example b/example index 3b18e51..d21055a 100644 --- a/example +++ b/example @@ -1 +1 @@ -hello world +hello conrad franklin:myrepo conrad$ git checkout example franklin:myrepo conrad$ cat example hello world franklin:myrepo conrad$ git diff HEAD diff --git a/example b/example new file mode 100644 index 0000000..3b18e51 --- /dev/null +++ b/example @@ -0,0 +1 @@ +hello world franklin:myrepo conrad$ echo "hello chimerax" > example franklin:myrepo conrad$ git diff diff --git a/example b/example index 3b18e51..addaf36 100644 --- a/example +++ b/example @@ -1 +1 @@ -hello world +hello chimerax franklin:myrepo conrad$ git pull Already up-to-date. franklin:myrepo conrad$ git commit -a -m "example" [develop a3108a9] example 1 files changed, 1 insertions(+), 0 deletions(-) create mode 100644 example franklin:myrepo conrad$ git push Counting objects: 4, done. Delta compression using up to 64 threads. Compressing objects: 100% (2/2), done. Writing objects: 100% (3/3), 278 bytes, done. Total 3 (delta 1), reused 0 (delta 0) Unpacking objects: 100% (3/3), done. To /usr/local/projects/chimerax/git/chimerax.git 1d973d7..a3108a9 develop -> develop
Use Feature Branch for Development
For longer tasks, it is better to use a feature branch. A feature branch is off-shoot of develop that may be used for development of a new feature; the developer may commit changes as desired on the feature branch without affecting the develop branch. Once all the work is completed, the entire set of changes may then be merged back to develop (or discarded if the feature is not useful). git flow has a set of tools for managing feature branches, including creating from and merging back to develop.
A few notes:
- git flow, like git, prints out messages about what commands one may want to use later.
- my_new_feature is the name of the feature branch. Any name may be used, although overlap of branch names and file names should be avoided to minimize confusion.
- git flow only manages the local repository. Changes still have to be pulled from and pushed to the remote repository.
franklin:myrepo conrad$ pwd
/usr/local/projects/chimerax/git/myrepo
franklin:myrepo conrad$ git status
# On branch develop
nothing to commit (working directory clean)
franklin:myrepo conrad$ git flow feature start my_new_feature
Switched to a new branch 'feature/my_new_feature'
Summary of actions:
- A new branch 'feature/my_new_feature' was created, based on 'develop'
- You are now on branch 'feature/my_new_feature'
Now, start committing on your feature. When done, use:
git flow feature finish my_new_feature
franklin:myrepo conrad$ cat example
hello chimerax
franklin:myrepo conrad$ echo "completely different" > example
franklin:myrepo conrad$ git status
# On branch feature/my_new_feature
# Changed but not updated:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: example
#
no changes added to commit (use "git add" and/or "git commit -a")
franklin:myrepo conrad$ git add -A
franklin:myrepo conrad$ git status
# On branch feature/my_new_feature
# Changes to be committed:
# (use "git reset HEAD <file>..." to unstage)
#
# modified: example
#
franklin:myrepo conrad$ git commit -m "half done"
[feature/my_new_feature 8b5e441] half done
1 files changed, 1 insertions(+), 1 deletions(-)
franklin:myrepo conrad$ git status
# On branch feature/my_new_feature
nothing to commit (working directory clean)
franklin:myrepo conrad$ echo "finished product" > example
franklin:myrepo conrad$ git add -A
franklin:myrepo conrad$ git commit -m "my new feature does this"
[feature/my_new_feature 2171ddf] my new feature does this
1 files changed, 1 insertions(+), 1 deletions(-)
franklin:myrepo conrad$ git flow feature finish my_new_feature
Switched to branch 'develop'
Merge made by recursive.
example | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
Deleted branch feature/my_new_feature (was 2171ddf).
Summary of actions:
- The feature branch 'feature/my_new_feature' was merged into 'develop'
- Feature branch 'feature/my_new_feature' has been removed
- You are now on branch 'develop'
franklin:myrepo conrad$ git push
Counting objects: 9, done.
Delta compression using up to 64 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (7/7), 627 bytes, done.
Total 7 (delta 4), reused 0 (delta 0)
Unpacking objects: 100% (7/7), done.
To /usr/local/projects/chimerax/git/chimerax.git
a3108a9..fb5b90a develop -> develop
Stash Modified Files before Switching Branches
git provides a very useful feature where one can save the current state of all working files and return to the committed state. For example, while working some files, a developer may need to update some other files to fix a bug. The already changed files are not ready to be committed into the repository, but they should not be discarded either. Using the stash command, the developer can
- save the current changes,
- return to the committed state,
- make and commit bug fix changes,
- restore the saved changes,
- pull back the bug fix changes, and
- continue development.
[loft - /e/chimerax]$ pwd /e/chimerax [loft - /e/chimerax]$ git status # On branch develop # Changes not staged for commit: # (use "git add <file>..." to update what will be committed) # (use "git checkout -- <file>..." to discard changes in working directory) # # modified: example # no changes added to commit (use "git add" and/or "git commit -a") [loft - /e/chimerax]$ echo "some changes" > example [loft - /e/chimerax]$ git status # On branch develop # Changes not staged for commit: # (use "git add <file>..." to update what will be committed) # (use "git checkout -- <file>..." to discard changes in working directory) # # modified: example # no changes added to commit (use "git add" and/or "git commit -a") [loft - /e/chimerax]$ git stash Saved working directory and index state WIP on develop: fb5b90a Merge branch 'feature/my_new_feature' into develop HEAD is now at fb5b90a Merge branch 'feature/my_new_feature' into develop [loft - /e/chimerax]$ cat example finished product
Here, the developer can make and commit updates without including the earlier changes.
[loft - /e/chimerax]$ git stash list stash@{0}: WIP on develop: fb5b90a Merge branch 'feature/my_new_feature' into develop [loft - /e/chimerax]$ git stash pop # On branch develop # Changes not staged for commit: # (use "git add <file>..." to update what will be committed) # (use "git checkout -- <file>..." to discard changes in working directory) # # modified: example # no changes added to commit (use "git add" and/or "git commit -a") Dropped refs/stash@{0} (9c7ea9292cd81bd1f4d2606dbab129e3cdd5c0d9) [loft - /e/chimerax]$ cat example some changes [loft - /e/chimerax]$ git pull Already up-to-date.
Use plato Repository to Coordinate Development on Multiple Hosts
Sometimes, a developer may use several hosts for development, e.g., a home machine, an office desktop and a laptop. git may be used to coordinate the changes made on different hosts. Rather than cloning directly from the team repository, the developer can create a plato clone first, e.g., myrepo; then local repositories on each host may be cloned from myrepo. Changes may then be pushed and pulled from the remote host to myrepo without making any changes to the team repository. For example, one can
- make and commit a change at the office,
- push the change to myrepo for pick up from home,
- go home and pull the change from myrepo,
- continue development at home,
- commit changes and push to myrepo so that they can be pulled from the office.
Obviously, the coordinating repository does not need to reside on plato. However, it should be on a host that remains accessible from the network at all times so that changes may be pushed or pulled.
In the example below, the myrepo repository on plato is cloned on host huck (a host at work). A shared feature branch, multi_site, is created as is a source file shared. The feature branch is pushed to plato using git flow feature publish. Afterwards, the plato repository is cloned on loft (a host at home). The feature branch is initialized using git flow feature pull. Once the feature branch exists on both hosts, the normal push and pull command from git may be used to synchronized the repositories.
conrad@huck ~$ git clone ssh://conrad@plato.cgl.ucsf.edu/usr/local/projects/chimerax/git/myrepo myrepo
Cloning into 'myrepo'...
remote: Counting objects: 569, done.
remote: Compressing objects: 100% (542/542), done.
remote: Total 569 (delta 175), reused 276 (delta 16)
Receiving objects: 100% (569/569), 311.17 MiB | 20.57 MiB/s, done.
Resolving deltas: 100% (175/175), done.
conrad@huck ~/myrepo$ git checkout master
Branch master set up to track remote branch master from origin.
Switched to a new branch 'master'
conrad@huck ~/myrepo$ git flow init -d
Using default branch names.
Which branch should be used for bringing forth production releases?
- develop
- master
Branch name for production releases: [master]
Which branch should be used for integration of the "next release"?
- develop
Branch name for "next release" development: [develop]
How to name your supporting branch prefixes?
Feature branches? [feature/]
Release branches? [release/]
Hotfix branches? [hotfix/]
Support branches? [support/]
Version tag prefix? []
conrad@huck ~/myrepo$ git flow feature start multi_site
Switched to a new branch 'feature/multi_site'
Summary of actions:
- A new branch 'feature/multi_site' was created, based on 'develop'
- You are now on branch 'feature/multi_site'
Now, start committing on your feature. When done, use:
git flow feature finish multi_site
conrad@huck ~/myrepo$ echo "shared code" > shared
conrad@huck ~/myrepo$ git add shared
conrad@huck ~/myrepo$ git commit -m "shared code" shared
[feature/multi_site da70383] shared code
1 files changed, 1 insertions(+), 0 deletions(-)
create mode 100644 shared
conrad@huck ~/myrepo$ git flow feature publish multi_site
Counting objects: 4, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 283 bytes, done.
Total 3 (delta 1), reused 1 (delta 0)
To ssh://conrad@plato.cgl.ucsf.edu/usr/local/projects/chimerax/git/myrepo
* [new branch] feature/multi_site -> feature/multi_site
Already on 'feature/multi_site'
Summary of actions:
- A new remote branch 'feature/multi_site' was created
- The local branch 'feature/multi_site' was configured to track the remote branch
- You are now on branch 'feature/multi_site'
Subsequently, on another host...
[loft - ~]$ git clone ssh://conrad@plato.cgl.ucsf.edu/usr/local/projects/chimerax/git/myrepo myrepo
Cloning into 'myrepo'...
remote: Counting objects: 569, done.
remote: Compressing objects: 100% (542/542), done.
Receiving objectsremote: Total 569 (delta 175), reused 276 (delta 16)
Receiving objects: 100% (569/569), 311.17 MiB | 586 KiB/s, done.
Resolving deltas: 100% (175/175), done.
[loft - ~]$ cd myrepo/
/e/home/conrad/myrepo
[loft - ~/myrepo]$ git checkout master
Branch master set up to track remote branch master from origin.
Switched to a new branch 'master'
[loft - ~/myrepo]$ git flow init -d
Using default branch names.
Which branch should be used for bringing forth production releases?
- develop
- master
Branch name for production releases: [master]
Which branch should be used for integration of the "next release"?
- develop
Branch name for "next release" development: [develop]
How to name your supporting branch prefixes?
Feature branches? [feature/]
Release branches? [release/]
Hotfix branches? [hotfix/]
Support branches? [support/]
Version tag prefix? []
[loft - ~/myrepo]$ git flow feature pull origin multi_site
Created local branch feature/multi_site based on origin's feature/multi_site.
[loft - ~/myrepo]$ cat shared
shared code
[loft - ~/myrepo]$ echo "more shared code" >> shared
[loft - ~/myrepo]$ git commit -a -m "more code added"
[feature/multi_site f9d8a3d] more code added
1 files changed, 1 insertions(+), 0 deletions(-)
[loft - ~/myrepo]$ git push
Counting objects: 5, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 279 bytes, done.
Total 3 (delta 1), reused 0 (delta 0)
To ssh://conrad@plato.cgl.ucsf.edu/usr/local/projects/chimerax/git/myrepo
da70383..f9d8a3d feature/multi_site -> feature/multi_site
Back to the first host...
conrad@huck ~/myrepo$ git pull
remote: Counting objects: 5, done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 3 (delta 1), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From ssh://plato.cgl.ucsf.edu/usr/local/projects/chimerax/git/myrepo
da70383..f9d8a3d feature/multi_site -> origin/feature/multi_site
Updating da70383..f9d8a3d
Fast-forward
shared | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
conrad@huck ~/myrepo$ cat shared
shared code
more shared code
Delete Unused Branches
To delete a branch from the local repository, use the command:
git checkout develop && git branch -D feature/feature_name
To delete a branch from the remote repository, use the command:
git push origin :feature/feature_name
Create Team Repository
In this section, examples refer to /var/tmp/conrad/example since the actual path /usr/local/projects/chimerax/git is already in use. Examples in other sections of this document refer to the correct path.
Create the bare git repository that will serve as the team repository for ChimeraX source code. Note that the repository is created for sharing within a group. The chgrp command defines which group will have cloning and update privileges. The access permissions such as the setgid bit of directories (meaning files created will inherit the group of the parent directory) in the repository is maintained by git and should not require manual intervention.
franklin:example conrad$ cd /var/tmp/conrad/example franklin:example conrad$ git init --bare --shared=group chimerax.git Initialized empty shared Git repository in /var/tmp/conrad/example/chimerax.git/ franklin:example conrad$ chgrp -R chimdev chimerax.git franklin:example conrad$ ls -l total 4 drwxrwsr-x 7 conrad chimdev 3864 May 1 09:34 chimerax.git/
Create a bootstrap repository so that we can set up the initial configuration for the project. The reason this repository is temporary is that the push/pull linkage between local and remote branches need to be set up when the branches are created, but are handled automatically during cloning after they have been created. Since we will only use master and develop branches in the team repository, we can set them up now and developers who clone later will not have to worry about configuring the push/pull targets.
franklin:example conrad$ git clone --shared chimerax.git bootstrap Initialized empty Git repository in /var/tmp/conrad/example/bootstrap/.git/ warning: You appear to have cloned an empty repository. franklin:example conrad$ cd bootstrap /var/tmp/conrad/example/bootstrap franklin:bootstrap conrad$ ls -la total 12 drwxr-xr-x 3 conrad ferrin 3864 May 1 10:32 ./ drwxrwxr-x 5 conrad ferrin 3864 May 1 10:32 ../ drwxrwxr-x 7 conrad ferrin 3864 May 1 10:32 .git/
Use git flow to set up the branches in the bootstrap repository. After initialization, we are on the develop branch where we add our initial files; xyzzy is our only file in this example. The add -A command adds all modified and untracked files to the staging area; the commit command commits files in the staging area to the local repository; the push command pushes the state of the current branch from the local repository, bootstrap, to the remote team repository, origin, which represents the repository from which we are cloned (chimerax.git in this case).
franklin:bootstrap conrad$ git flow init -d Using default branch names. No branches exist yet. Base branches must be created now. Branch name for production releases: [master] Branch name for "next release" development: [develop] How to name your supporting branch prefixes? Feature branches? [feature/] Release branches? [release/] Hotfix branches? [hotfix/] Support branches? [support/] Version tag prefix? [] franklin:bootstrap conrad$ git status # On branch develop nothing to commit (working directory clean) franklin:bootstrap conrad$ echo hello > xyzzy franklin:bootstrap conrad$ git add -A franklin:bootstrap conrad$ git status # On branch develop # Changes to be committed: # (use "git reset HEAD <file>..." to unstage) # # new file: xyzzy # franklin:bootstrap conrad$ git commit -m "initial setup" [develop ff9b042] initial setup 1 files changed, 1 insertions(+), 0 deletions(-) create mode 100644 xyzzy franklin:bootstrap conrad$ git push origin develop Counting objects: 5, done. Delta compression using up to 64 threads. Compressing objects: 100% (2/2), done. Writing objects: 100% (5/5), 384 bytes, done. Total 5 (delta 0), reused 0 (delta 0) Unpacking objects: 100% (5/5), done. To /var/tmp/conrad/example/chimerax.git * [new branch] develop -> develop
Make the master branch match the develop branch. The checkout command is used to switch from one branch to another; note that this command only uses the local repository and does not require access to the team repository.
franklin:bootstrap conrad$ git checkout master Switched to branch 'master' You have new mail in /var/spool/mail/conrad franklin:bootstrap conrad$ git merge develop Updating 76fd917..ff9b042 Fast-forward xyzzy | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) create mode 100644 xyzzy franklin:bootstrap conrad$ git push origin master Total 0 (delta 0), reused 0 (delta 0) To /var/tmp/conrad/example/chimerax.git * [new branch] master -> master
At this point, the team repository set up is complete. All the files are under revision control and the bootstrap repository may be safely deleted. The team repository, chimerax.git, with master and develop branches, is ready for by developers. Note that developers still have to set up and use git flow after cloning in order to follow the Vincent Driessen's branching model conveniently.
franklin:bootstrap conrad$ cd .. franklin:example conrad$ ls -l total 12 drwxr-xr-x 3 conrad ferrin 3864 May 1 11:19 bootstrap/ drwxrwsr-x 7 conrad chimdev 3864 May 1 09:34 chimerax.git/ franklin:example conrad$ rm -rf bootstrap
![[Chimera Issue Tracking System]](/trac/chimera/chrome/site/chimera_logo.png)