Github Forking Workflows

Owned by Medha Parlikar
Last updated: Sept 16, 2021Version comment
6 min read

Audience

This document is intended for full-time developers, community contributors, and projects managers who all need to understand the details of how to use Github effectively in the CasperLabs' environment. Full-time developers and other contributors must follow these guidelines to the extent possible.

Summary rules of etiquette

  • All work is code reviewed, so please familiarize yourself with that process thoroughly.
  • Name your feature branch after the ticket id, example HWY-240_branch_name
  • Ensure that you are keeping your personal branches up to date with the main dev branch so that your reviewer will only have to see your changes. See below more details.
  • For each release, there must be a branch "release/x.y.z", all release branches must be merged back to "master" branch.
  • Your PR will be the basis for kicking off a code review.  
  • In your PR start with your ticket id followed by colon, example HWY-240: <Title of the PR>
  • When creating your PR use smart commits, and include the Jira issue number.
    • If you add commits to your PR, include the Jira issue number then too.
  • Sign your commits (see How to sign commits)
  • Select at least 2 reviewers.
  • Validate that unit and integration tests passed on your PR.   
    • Tests run in . You can view the test results on the PR page or on Drone by browsing to http://3.16.200.31. Failed tests will block a merge.
    • Notices of failed tests are sent to #alerts in slack.
  • Wait for your reviewer's feedback.
  • Address any feedback.
  • Once approved, merge the PR.
  • 2 Approvals are required in order to merge a pull request.

If you're picking up someone else's pull request, then whatever you do must preserve the commit history. If you can accomplish that by rebasing off someone else's PR, then great. If you need to merge the PR and then create a new PR, then that's also fine. Merging a broken PR and then fixing it "later" is the least preferable, but there are situations where this also works. E.g., if the "fix" is already implemented and will be merged almost immediately.


Please do NOT:

  • Please do not close PRs when there are new changes which result in conflicts. You can add changes and resolve conflicts by merging or rebasing.

The basics of forking

CasperLabs has a variety of repositories under Github and at various times you may be required to contribute to each of them. This is a simple git workflow that keeps all your work on branches in your own fork of the repository. In the general case, no one ever pushes to casper-node/casper-node, which keeps our shared repository clean. All git workflows require diligence to maintain good hygiene around branching, issuing pull requests, and merging. The main rules to keep in mind for this one are:

  1. Never ever mess with master on casper-node/casper-node.
  2. In fact, only ever push to your own fork!

One great piece of advice is to make your personal fork be origin and add the upstream (casper-node/casper-node) fork as a remote. That helps you avoid accidentally impacting the wrong fork by default. Either way you do it, the network diagram will look something like this:



Example process

Setup

Make a personal fork of casper-node/casper-node on github; my user is medha, so I forked to medha/casperlabs. On your development machine, clone your personal branch (medha/casper-node for me) and add casper-node/casper-node) as a remote called upstream.

Clone & Add Remote

<dev:~/src> git clone git@github.com:medha/casper-node
Cloning into 'casper-node'...
remote: Counting objects: 2100, done.       
remote: Compressing objects: 100% (24/24), done.       
remote: Total 2100 (delta 10), reused 34 (delta 7), pack-reused 2058       
Receiving objects: 100% (2100/2100), 1.52 MiB | 10.25 MiB/s, done.
Resolving deltas: 100% (791/791), done.
<dev:~/src> cd casper-node
<dev:~/src/casper-node (master)> git remote add upstream git@github.com:casper-node/casper-node
<dev:~/src/casper-node (master)> git fetch --all
Fetching origin
Fetching upstream
remote: Counting objects: 316, done.       
remote: Compressing objects: 100% (89/89), done.       
remote: Total 316 (delta 89), reused 198 (delta 69), pack-reused 87       
Receiving objects: 100% (316/316), 71.69 KiB | 1.79 MiB/s, done.
Resolving deltas: 100% (100/100), completed with 23 local objects.
From github.com:casper-node/casper-node
 * [new branch]      dev                      -> upstream/dev
 * [new branch]      master                   -> upstream/master
<dev:~/src/casperlabs (master)> git remote -v
origin  git@github.com:medha/casper-node (fetch)
origin  git@github.com:medha/casper-node (push)
upstream    git@github.com:casper-node/casper-node (fetch)
upstream    git@github.com:casper-node/casper-node (push)

Feature branch

Create a new branch, tracking upstream/dev (in other words, branch off of dev in the main casper-node repository). Here I'm working on Jira issue CORE-23:

Create Feature Branch

<dev:~/src/casper-node (master)> git checkout -b CORE-23 upstream/dev --track
Branch CORE-23 set up to track remote branch dev from upstream.
Switched to a new branch 'CORE-23'

Do some work. Ensure the feature branch is up-to-date with the main dev branch, and commit your changes.

Commit Locally

(Note- this is an old example- ignore the fact that it's Scala)

<dev:~/src/casper-node (CORE-23)> git pull --rebase
Already up-to-date.
<dev:~/src/casper-node (CORE-23)> git status
On branch CORE-23
Your branch is up-to-date with 'upstream/dev'.
 
Untracked files:
  (use "git add <file>..." to include in what will be committed)
 
    comm/src/main/scala/coop/node/comm/foo.scala
 
nothing added to commit but untracked files present (use "git add" to track)
<dev:~/src/casper-node (CORE-23)> git add comm/src/main/scala/coop/node/comm/foo.scala
<dev:~/src/casper-node (CORE-23)> git commit -m'Added a foo.'
[CORE-23 2828de5] Added a foo.
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 comm/src/main/scala/coop/node/comm/foo.scala

Really, make sure your feature branch is up-to-date with dev. Then, push the feature branch to your personal fork.

Push Feature Branch

<dev:~/src/casper-node (CORE-23)> git pull --rebase
Already up-to-date.
<dev:~/src/casperlabs (CORE-23)> git remote -v
origin  git@github.com:medha/casper-node (fetch)
origin  git@github.com:medha/casper-node (push)
upstream    git@github.com:casper-node/casper-node (fetch)
upstream    git@github.com:casper-node/casper-node (push)
<dev:~/src/casperlabs (CORE-23)> git push origin CORE-23
Counting objects: 270, done.
Delta compression using up to 12 threads.
Compressing objects: 100% (95/95), done.
Writing objects: 100% (270/270), 62.08 KiB | 31.04 MiB/s, done.
Total 270 (delta 75), reused 246 (delta 63)
remote: Resolving deltas: 100% (75/75), completed with 10 local objects.       
To github.com:medha/casper-node
 * [new branch]      CORE-23 -> CORE-23

Sign your commits

See Signing Commits for information on how to sign your commits.

Pull request

Need to update this to include information for casper-node - 

The casper-node repository does not use a dev branch. Just master.  Build your changes in a fork and create a PR to merge to master.




Keeping up-to-date and resolving merge conflicts

Sometimes, once you've created a pull request, things will happen in the upstream repository before you get a chance to merge. This can cause your local code and your fork to become out of date with the upstream repository. Prefer using git rebase for updating your pull request and if the updating process becomes too painful with many rebase conflicts then consider updating by git merge.

Begin by updating your local repo and checkouting to your local copy of the feature branch

➜ CasperLabs git:(CORE-23) git fetch --all
Fetching origin
Fetching casper-node
Fetching mateusz

➜ CasperLabs git:(CORE-23) git branch
NODE-24-deploy-message
dev
* CORE-23


➜ casper-network git:(CORE-23) git checkout CORE-23
Already on 'CORE-23'


➜  casper-network git:(CORE-23) git remote -v
casper-node git@github.com:CasperLabs/casper-network.git (fetch)
casper-node git@github.com:CasperLabs/casper-network.git (push)
mateusz git@github.com:goral09/casper-network.git (fetch)
mateusz git@github.com:goral09/casper-network.git (push)
origin git@github.com:igor-ramazanov/casper-network.git (fetch)
origin git@github.com:igor-ramazanov/casper-network.git (push)

Try to rebase against the dev branch of the main repository and force push your updated feature branch to your repository

➜ casper-network git:(CORE-23) git rebase casper-node/dev
First, rewinding head to replay your work on top of it...
Applying: Remove models' tests.
➜ casper-network git:(CORE-23) git push origin +CORE-23


At rebasing you may have conflict errors, if resolving process becomes too tricky, then you can abort the rebasing, update the feature branch by merge and push the updated feature branch to your repository.

➜ casper-network git:(c2856dc) ✗ git rebase --abort
➜ casper-network git:(CORE-23) git merge casper-node/dev
Already up to date.
➜ casper-network git:(CORE-23) git push origin CORE-23

Nota Bene: This push will automatically update any pull requests from your fork→casper-node/casper-node. In my case, if I have a pull request open from mparlikar/node→casper-node/casper-node, then the pull request will now contain the differences between the upstream and forked branches.

Git hooks

It might be convenient for having git pre-push hooks, so git will automatically run tests or code analysis tools before actual pushing preventing invalid code to be pushed.

Though, we are not strictly enforcing it because there are some knowable problems on Windows or there can be any other problem that makes it less convenient for a developer.

Below is an example of such pre-push hook, you may want to delete Rust or Scala sections if you are working only on some of them:

# installing rust toolchain and clippy
➜ casper-network git:(dev) cd execution-engine
➜ execution-engine git:(dev) rustup toolchain install $(cat rust-toolchain)
➜ execution-engine git:(dev) rustup component add --toolchain=$(cat rust-toolchain) clippy
➜ execution-engine git:(dev) cd ..
# defining git pre-push hook
➜ casper-network git:(dev) cat > .git/hooks/pre-push <<EOL
#!/bin/bash
set -e
# Check Scala
sbt test
# Check Rust
cd execution-engine
cargo test
cargo clippy -- -D warnings -A renamed_and_removed_lints
EOL

Forking as it relates to releases

The basic approach to forking extends to how we prepare for and release software. Below is a high-level view of the process taken from https://nvie.com/posts/a-successful-git-branching-model. The SRE team supports the merge from dev to release branches and master, and hotfixes as needed.

  • Release branch is cut from Dev at feature complete for the release *(Note, features that are not ready, will be de-scoped from release)
    • Test release branch & bug fix if needed
    • Bug fixes will require 2 Pull Requests
      • 1 to the release branch
      • 1 to dev branch
  • Push release to Master
  • Tag release in Master
  • Release from Master