wiki:GalaxyDevelopment
Last modified 4 years ago Last modified on 09/03/13 15:19:04

Cloning galaxy repository for the first-time

  • Create a projects directory in your home directory. The galaxy-code-developer module file assumes following directory structure, so following this convention is important. Alternatively, you can create your own module file and hide your project/directory structure. This is described in the next bullet point.
    $ mkdir -p $HOME/projects/galaxy
    $ cd $HOME/projects/galaxy 
    
  • You can create your own galaxy-code-developer module file as mentioned above.
    • Create .modulefiles directory if necessary and place following galaxy-code-developer module file in it. You can name this file in any way. Following instructions assume file name as 'galaxy-code-developer', but you can use any other file name as well.
      $ mkdir -p ~/.modulefiles
      
      #%Module
      
      # galaxy module file
      # does galaxy even have a version?
      set galaxyver 1.0
      set home $::env(HOME)
      # galaxyinstall - personal install root - CHANGE following line to point to your galaxy clone 
      set galaxyinstall $home/projects/galaxy/galaxy
      # galaxyroot - core/main galaxy root, personal installs will use it's Python and galaxy-tools
      set galaxyroot /share/apps/galaxy
      set url "https://galaxy.uabgrid.uab.edu/"
      set msg "This module adds Galaxy v$galaxyver to various paths\n\nPython Official Site: $url\n"
      
      
      proc ModulesHelp { } {
         puts stderr "$msg"
      }
      
      module-whatis   "$msg"
      
      setenv GALAXY_TOOLS $galaxyroot/galaxy-tools
      setenv GALAXY_PYTHON $galaxyroot/python
      # Use personal install galaxy library files and not main galaxy library files 
      setenv GALAXY_LIB $galaxyinstall/lib
      
      prepend-path PATH $galaxyroot/galaxy-tools/bin
      prepend-path LD_LIBRARY_PATH $galaxyroot/galaxy-tools/lib:$galaxyroot/galaxy-tools/lib64:$galaxyroot/galaxy-tools/lib64/R
      # pythonpath should search for library files in the personal install
      setenv PYTHONPATH $galaxyinstall/lib
      
      # load galaxy sub-modules
      module load galaxy/galaxy-submodule-python-2.6
      module load galaxy/galaxy-submodule-perl-5.18
      module load galaxy/galaxy-submodule-gnuplot-font
      module load galaxy/galaxy-submodule-drmaa
      module load galaxy/galaxy-submodule-java
      module load galaxy/galaxy-submodule-mercurial
      module load galaxy/galaxy-submodule-MACS
      module load galaxy/galaxy-submodule-weblogo
      module load galaxy/galaxy-submodule-meme
      
      
    • Change 'set galaxyinstall <location>' line in above file to your galaxy cloned repository. For example, if you are cloning repository directly in home directory then your galaxyinstall will be '$home/galaxy'.
  • Load galaxy code developer module:
    # See list of available modules first 
    $ module avail
    
    # Load this if you are using generic module file with galaxy code in $HOME/projects/galaxy/galaxy 
    $ module load galaxy/galaxy-code-developer
    
    # Load this if you created a module file in ~/.modulefiles directory. Change module file name if necessary. 
    $ module load galaxy-code-developer
    
    
  • Clone galaxy repository from 'official' galaxy-uab repository
    $ cd $HOME/projects/galaxy/
    $ git clone ssh://<blazerid>@git.uabgrid.uab.edu/home/git/repositories/galaxy.git 
    
  • Checkout develop branch: We will be using galaxy develop branch in our personal as well as deployed install. We can change this convention later, but that's the convention/workflow followed right now.
    $ cd $HOME/projects/galaxy/galaxy
    $ git branch --track develop origin/develop
    $ git checkout develop
    

Configure and access your galaxy instance

  • Configure your instance by running 'personalize.sh' script. The script will change galaxy server port number and file/directory paths. It will not change any user authentication or administration parameters. You will need to modify these parameters manually for now.
    $ ./personalize.sh 
    
  • Start galaxy by running run.sh process. I recommend starting it with '--daemon' option. Note Python Paste server has some bug with Python 2.6 which prevents killing non-daemon server process using Ctrl+C.
    $ ./run.sh --daemon
    
  • Stop galaxy daemon:
    $ ./run.sh --stop-daemon 
    $ ./run.sh --help # See all available options
    
  • Accessing galaxy web interface using port forwarding: The galaxy server will be listening only on localhost, hence you will need to access it using ssh port forwarding. Following ssh command will forward your local system's port to a port on the Cheaha where galaxy service is running. Run this command from your local system and not Cheaha.
    $ ssh -L <local_port>:127.0.0.1:<galaxy_port> <blazerid>@cheaha.uabgrid.uab.edu 
    # Now you can access galaxy by typing - http://localhost:<local_port> in browser 
    

Code development

The distributed (I prefer calling it 'decentralized') repository nature of Git allows numerous development workflows. In case of centralized repositories like Subversion one has to commit changes to the central Subversion server in order to record/version-control changes. However, the decentralized repositories like Git allow you to record your changes locally and share them with other developers. This facilitates some useful workflows as explained graphically here. We are using integration-manager workflow for our galaxy development and following are suggested steps for doing development.

Making and sharing changes

Update develop branch

$ git checkout develop
$ git pull

Create a ticket for your change set

This is optional if you are making changes in your local environment that won't affect others.

Create a new branch for your ticket/changeset

You should always make changes in a non-develop branch unless you are repository integrator and/or you know how to rollback/revert your changes in a clean manner using advanced Git commands like rebase, reset etc. I highly recommend keeping your develop branch read-only and working on a separate issue specific branch. If possible create a branch using ticket number so that it will be convenient (self-describing) while referencing/sharing it with others.

$ git checkout -b <describe-bugfix-branch>

Commit changes to your ticket/changeset

I have not covered (advanced) Git commands that let you add commits interactively or change previous commit contents. But such commands are really useful and common with many git based development. I highly recommend getting familiar with these commands and using it in your development workflow (may add more details on it later). Also, please mention ticket number using trac wiki formatting language in the commit message as a cross-reference. See ProGit's Contributing to a Project chapter for commit guidelines and other useful information on project workflows/contribution.

# git add-commit 
$ git add <filename>
$ git commit -m "Fix for [ticket:12]"

Test changes locally

Sharing you changes

You can share your changes in two ways:

  • Share the branch with other developers
    • For this you will need to modify file system permissions so that other people can access your code directory/repository. Then send access details - repository path and branch name - with other developers.
  • Create a patch for changesets and share it with other users
    # Following command will create a patch file(s) for commits between commit1 and commit2 (displayed after doing a git merge)
    $ git format-patch <commit1>..<commit2>
    
    
  • Email this patch file to galaxy-dev@… and/or pavgi@…
    # Email command example 
    $ /usr/sbin/sendmail galaxy-dev@vo.uabgrid.uab.edu < 0001-*.patch 
    
    
  • Update the ticket information and attach patch file(s).

Incorporating / Merging other user's changes

Will be modified soon - Not wrong, but needs to be clarified

Getting / Pulling changes from develop

Pull develop from the UAB master copy, then rebase your working branch

# pull from master
git checkout develop
git pull

# update my working branch
git checkout ticket_00_branch
git rebase develop

Getting / Pulling changes from other developers

Add remote for the other developer's git repository.

# Add remote for a developer's repository  (-f option performs fetch after remote is added)
$ git remote add -f <blazerid-developer> <repository-URL>  

# Verifying changes/diff against developer's branch 
$ git log -p develop..remotes/<blazerid-developer>/<developer's branch>

# Create a local tracking branch for developer's repo as follows
$ git checkout -b blazerid-develop <blazerid-developer>/develop 

Merging changes obtained from other developers

# Merge developer's branch changes with your 'blessed' develop branch 
$ git checkout develop
# Merge from tracking branch 
$ git merge <blazerid-developer>-develop 
# Alternatively you can merge directly using local ref to remote branch 
$ git merge remotes/<blazerid-developer>/<developer's branch>

# Push changes to the blessed repository 
$ git checkout develop 
$ git push

Reference