Git and Gerrit CodeReview for Scilab developpers

This information is for Scilab developers and code contributors. See GIT for a table of content.

If you are a user of Scilab, you probably don't need to look at this.

Git opens up a number of new options for contributing to Scilab. For the first time, it is easy to review code that is pending addition to the Scilab tree. In fact, reviewing code is one of the best ways to ensure that the releases that Scilab ships remain stable and functional. If you are interested purely in reviewing, then please skip to that section towards the end of this document.

Git also changes the way that developers interact with the Scilab tree. Instead of just having a single version of the tree on your local machine, you have a compressed copy of the entire repository.

While Git is a far more powerful tool than Subversion it is also, inevitably, more complex. This document can only scratch the surface of what's possible with git - there are many, many, documents available that describe git in greater detail, and references to some of them are provided at the end.

Getting the Scilab repository

You can download the entire Scilab repository by running

   1 git clone git://git.scilab.org/scilab.git

to place the clone in a directory called 'scilab' or

   1 git clone git://git.scilab.org/scilab.git <name-of-directory>

to place your clone in a specific directory.

If you have a Gerrit account (you can check your username under your Settings):

   1 git clone ssh://username@git.scilab.org:29418/scilab <name-of-directory>

This will give you a complete copy of the Scilab repository and unless you are exceptionally short of either disk space, or time, is the approach we recommend. Unlike Subversion, you are not just checking out a particular branch, but making a local copy of the project's whole revision history, across all of its branches. This does make the download pretty large - around 670Mbytes at the time of writing.

Checkout a particular branch

The Scilab repository contains many branches, most of which are historical and should not be used for new development.

Current development (new features and bug fixes) for Scilab 6.0 happens in the 'master' branch. Some critical bugs may still be fixed in the '5.5' branch for Scilab 5.5.

A complete list of all branches in the upstream Scilab repository can be obtained by running

   1 git branch -r

If all you wish to do is browse code, then you can directly check out these remote branches, using

   1 git checkout origin/<branch>

For example, to checkout the '5.2' branch:

   1 git checkout origin/5.2

Note that if you wish to do development, you should either make a local branch which tracks the remote one, using something like

   1 git checkout -b 5.2 origin/5.2

or, more simply

   1 git checkout --track origin/5.2

or by creating a topic branch as discussed below.

Checkout a particular release

Every release version of Scilab is marked in the repository by means of a tag.

A complete list of all tags can be obtained by running

   1 git tag

To checkout a particular tag

   1 git checkout 5.2.0

Again, while a direct checkout of a remote tag is fine for code browsing, it should not be used as a place to start development. If you must do development against a tag, then create a local topic branch with it as a starting point, as is discussed below. However, in general, please don't develop from a particular tag, but instead work from a branch tip. It makes it much easier to integrate your changes!

Introducing yourself to git

Before you begin development, you should let git know who you are. This provides it with a name, and email address, that is used to attribute all commits in your repository, and in any that you share code with.

   1 git config user.name "John SMITH"
   2 git config user.email "john.smith@scilab.org"

If you want to make this settings for all of your repositories, then add the --global switch.

   1 git config --global user.name "John SMITH"
   2 git config --global user.email "john.smith@scilab.org"

Note that this email address is the address by which you will be identified in Scilab's revision history - it is also the address to which the gerrit code review tool will send all email related to the review of your code.

Setting up the hooks

Some hooks have been set, in order to indent correctly xcos files or to add id-commits when pushing. To set those, in your scilab repository, type the following commands :

   1 cd .git/
   2 mv hooks /tmp/
   3 ln -s ../git_hooks/ hooks

Git variables settings for XML formatting:

   1 git config --global hooks.xmlindent /usr/bin/xmlindent
   2 git config --add xmlindent.ignored 'scilab/Visual-Studio-settings/*.xml' 
   3 git config --add xmlindent.ignored 'scilab/checkstyle/*.xml' 

Git variables settings for C/C++ formatting:

   1 git config --global hooks.astyle /usr/bin/astyle
   2 git config --add astyle.ignored 'scilab/modules/*/src/jni/*.hxx'
   3 git config --add astyle.ignored 'scilab/modules/*/src/jni/*.cpp'
   4 git config --add astyle.ignored 'scilab/modules/javasci/src/java/org/scilab/modules/javasci/Call_Scilab*.java'
   5 git config --add astyle.ignored 'scilab/modules/helptools/src/java/org/scilab/modules/helptools/SynopsisLexer.java'
   6 git config --add astyle.ignored 'scilab/modules/helptools/src/java/org/scilab/modules/helptools/XML/XMLLexer.java'
   7 git config --add astyle.ignored 'scilab/modules/helptools/src/java/org/scilab/modules/helptools/c/CLexer.java'
   8 git config --add astyle.ignored 'scilab/modules/helptools/src/java/org/scilab/modules/helptools/java/JavaLexer.java'
   9 git config --add astyle.ignored 'scilab/modules/helptools/src/java/org/scilab/modules/helptools/scilab/ScilabLexer.java'
  10 git config --add astyle.ignored 'scilab/modules/scinotes/src/java/org/scilab/modules/scinotes/FunctionScanner.java'
  11 git config --add astyle.ignored 'scilab/modules/scinotes/src/java/org/scilab/modules/scinotes/IndentScanner.java'
  12 git config --add astyle.ignored 'scilab/modules/scinotes/src/java/org/scilab/modules/scinotes/MatchingBlockScanner.java'
  13 git config --add astyle.ignored 'scilab/modules/scinotes/src/java/org/scilab/modules/scinotes/ScilabLexer.java'
  14 git config --add astyle.ignored 'scilab/modules/scicos/src/scicos_sundials/*'

Starting development

We strongly recommend that you do all of your development upon 'topic branches'. This allows you to isolate multiple unrelated changes, and makes it easier to keep your tree in sync with the upstream Scilab one.

Before creating a new topic branch, running

   1 git fetch

will make sure that your repository knows about the latest upstream changes (unlike git pull, this will not update any files that you may have checked out)

To create a new topic branch:

   1 git checkout -b <branch>

For example, to work on a patch to fix printf warnings, based on the current development code, I would do:

   1 git checkout -b fix-printf-warnings origin/master

This puts me on a new branch, ready to start writing code. All new development should be based upon the origin/master branch, submissions based upon other branches are unlikely to be accepted, unless they address issues that are solely present in that branch.

'git add' is used to tell git about any new files you create as part of your patch. If your patch results in any new compilation products (object files, new executables, etc) that git should not be tracking, please make sure that they're caught by the .gitignore mechanism. You can do this by checking that they don't appear in the output from 'git status'

'git mv' and 'git rm' are used to move and delete files respectively.

'git commit -a' is used to commit code to all of the files that git is currently tracking (that is, all of the files that you have checked out from the repository, and all those which you have run git add on)

When you can't see the wood for the trees

If, in the middle of development, you discover that you've gone down a blind alley, and wish to go back to the state of your last commit

   1 git reset --hard

will discard all of the changes you have made since the last commit, or

   1 git checkout -f <file>

will restore <file> to the state it was in at the last commit.

Keeping up with the Jones'

If you're working on a long running development project, you will find that the point you created your topic branch rapidly recedes into history. At some point (and at least before you share your code with us), you'll probably want to update your tree. There are a number of ways of doing this.

If you haven't shared your tree with anyone else, then you can use

   1 git rebase <branch> <topic>

(Where <branch> is the name of the upstream branch - for example origin/master, and <topic> is the name of the topic branch you are currently working on)

Note that git rebase changes your local history (it moves the branch point of your topic branch to the tip of the upstream branch), and is a bad idea if others have cloned your repository. See 'man git-rebase' for more details.

If you can't rebase, then consider either merging the changes onto your local branch, or creating a new topic branch and cherry picking your changes onto it. The man pages for 'git merge' and 'git cherry-pick' provide more detail on these options.

Registering with gerrit

Note : you must provide a valid Scilab's account mail in order to register on gerrit. ATOMS and File Exchange accounts are valid.

To register with gerrit, visit http://codereview.scilab.org/ and log in using your email address and your scilab password.

If, when you introduced yourself to git, you gave it a different email address than the one gerrit currently has listed for you, you need to introduce yourself to gerrit under that name, too. Click on 'Contact Information', and select "Register New Email...". If gerrit thinks you are an "Anonymous Coward" then you can fix that on this page as well.

In order to be able to upload code, you now need to create a ssh key that gerrit can use to identify you, or tell gerrit about one that already exists.

To create a new ssh key, if you don't already have one, run

   1 ssh-keygen -t rsa -f ~/.ssh/id_rsa

The public key for this is now stored in ~/.ssh/id_rsa.pub.

To tell gerrit about your key, log in, and go to Settings. Paste your _public_ key into the box and click on 'Add' to add the new public key. On Profile section, make a note of your 'Username', because you'll need it to push.

To make things easier, set up OpenSSH so that it knows about the defaults for the gerrit server. Edit ~/.ssh/config, and add a section like:

Host git.scilab.org 
User <Username> 
IdentityFile ~/.ssh/id_rsa 
Port 29418

(where Username is what you were told on the the 'Profile' page)

The change id hook

Gerrit introduces the concept of "change IDs". This is a unique reference for a particular change, which remains constant regardless of any changes that are made to the implementation. This allows a single reference to be attached to a given modification, irrespective of any rewrites that may occur as a result of review comments. Manually maintaining change Ids is a pain, so gerrit provides a git hook which can be used to automatically add a change Id to any new modifications you create.

You should already have set this hook at #setting_hooks.

More information: http://gerrit.googlecode.com/svn/documentation/2.1.2/user-changeid.html

Note : you MUST set up this hook BEFORE you do your first commit, as it's done when you commit, and not when you push.

Uploading to gerrit

When submitting to gerrit, it's important to realise that each commit in your branch will become a changeset in the upstream Scilab, typically with no modification at our end. It is therefore important that these commits follow some simple rules...

First, each commit should be complete. The maxim "one change per commit, one commit per change" applies here. Each commit should build and test in its own right. Typically, this means that a change will be in a small number of commits. If, during development, you have created many more than this (for example, you've created a large number of bug fix commits), please use 'git rebase', or cherry pick these commits into a separate tree before uploading them.

Secondly, each commit should have a meaningful revision log. The internals of git means that we can't easily edit these before pushing them into the tree, so we'd like you to get them right for us! A commit message should be comprised of a single 'subject' line (which must *not* end with a full stop), followed by a blank line, followed by one or more paragraphs explaining the purpose of the patch. If it is intended to fix a bug in bugzilla.scilab.org, then the bug number followed by the word 'Fixed' be included on a line of its own.

An example commit message would be

  Xcos: use the Scilab TMPDIR instead of the default one

  Bug 3333 Fixed.

Thirdly, each commit should have a valid changeID. Manually maintaining these is difficult and error prone, so we would strong advise that you install the changeID hook detailed earlier. This will automatically add a ChangeId line to your commit message if it doesn't already contain one.

Once you have commits in this form, use

   1 git log -p origin/<branch>..HEAD

(where <branch> is the upstream branch upon which you are basing this patch).

to check that what you're giving us makes sense. Then, upload them to gerrit using

   1 git push ssh://username@git.scilab.org:29418/scilab HEAD:refs/for/<branch>

(again <branch> is the name of the upstream branch that you are pushing the changes into, not the name of any local branch you may have been developing on)

In this case, refs/for is a literal string. So, if you had been developing against master, you can upload your changes with:

   1 git push ssh://username@git.scilab.org:29418/scilab HEAD:refs/for/master

This can be configured directly using:

   1 git config remote.origin.push master:refs/for/master

Assuming all has gone well, this will have added the entry to the code review queue. The output from git review will give you a change number - this is a unique reference for this particular set of changes. During review you'll be emailed with any comments anyone makes, and can respond to those comments using the gerrit web interface (see the section on reviewing, below). It's possible that issues with your change may be noticed during the review process, and you may be asked to revise it, or update changes to the tip of the tree.

Revising your change

It's possible that your modifications won't be accepted first time. In this case, you need to revise your changes, and resubmit them to gerrit. Please note that this should always be done by modifying your original changeset, _not_ by submitting a new change that makes the required fixes. Either git commit --amend, or git rebase should be used to combine your changes with the original changeset, and then you should push this to gerrit with

   1 git push ssh://username@git.scilab.org:29418/scilab <hash>:refs/changes/<number>

(where <hash> is the sha1 hash of the revised change, and <number> is the change number you received when you originally submitted the patch)

You can obtain the sha1 hash of a commit by using 'git show' (if it is on the tip of your current branch), or 'git log' (if it is in your history)

Other mechanisms of listing the change to push are available - see http://gerrit.googlecode.com/svn/documentation/2.0/user-upload.html for full details.

Updating your change

It's possible that your change may have been made against a tree which is too old for it to apply to the tip. In this case, gerrit will let you know that there is a collision, and request that you update the change to the tip.

You can do this with

   1 git rebase origin/master topic

(assuming your patch is against the 'master' git branch, and lives on the <topic> branch)

And then simply resubmit your change in the same way as if you had been asked to revise it (see notes above)

Reviewing changes

We'll now look at how changes that have made it into gerrit can be reviewed. All code review now happens via the http://codereview.scilab.org interface. You should log in there as detailed above (using your email address and your Scilab password).

You'll be presented with a list of patches requiring review or, if someone has asked, patches you've been explicitly requested to review. There are two types of review - Code Review and Verification. Code Review means that you have read through the code, and are satisified that it works properly, follows the tree's style, and generally doesn't suck. Verification means that you have taken a copy of the patch and tested it. We hope to eventually automate the verification step, but for now both must be perfomed by hand.

To perform a code review, go through each of the diffs in the current changeset for the code you have decided to review. You can double click on a line to leave a comment. Once you have completed commenting, click on the 'Publish Comments' button on the page containing the list of patch sets. You will then be asked to score the patch, with a range from -1 to +1. -1 means that you don't think the code should be applied, +1 means that it is good to apply. You can also leave further, general, comments for the patch submitter.

Note that no matter how many +1 or -1 comments a patch receives, the gatekeepers can override these to either permit or forbid submission. Also, at least one gatekeeper must approve a patch before it can be submitted to the tree.

To verify the code, pull the git copy into your local tree, using the git command listed as part of the patch details. For sanity's sake, we'd strongly recommend you do this pull into a topic branch you've created for the purpose. Build it, test it, and report your results. Note that a single -1 to verification can block patch submission, so please use these options wisely. If in doubt, score 0 and leave your comments. Please indicate when verifying which platforms you have tested on, and what testing you performed.

Note : you check Verified once you've seen that your configure and make have been modified. You check Looks good to me, approved if you are sure that the changes in the operations make them work. You cannot +1 Verified with a No score (0) review

A command line tools is also accessible to apply score :

   1 ssh -p 29418 username@git.scilab.org gerrit approve --verified=+1 --code-review=+1 d15ae2

where :

And use the following to verify, approve and submit the same commit:

   1 ssh -p 29418 username@git.scilab.org gerrit approve --verified=+1 --code-review=+2 --submit d15ae2

Further Reading

[1] Git Magic http://www-cs-students.stanford.edu/~blynn/gitmagic/

[2] Git User's Manual http://www.kernel.org/pub/software/scm/git/docs/user-manual.html

[3] Git Community Book http://book.git-scm.com/

[4] Gerrit Documentation http://gerrit.googlecode.com/svn/documentation/2.0/index.html (only the first 'User Guide' section of this document is relevant)

public: gerrit (last edited 2017-08-22 10:03:42 by paul.bignier@scilab-enterprises.com)