Propagating Git Repo Changes Downstream

Published 2020-11-30. Last modified 2020-12-21.
Time to read: 3 minutes.

This page is part of the git collection.

I have developed a Jekyll template that I use as the starting point for most of my websites. Whenever I improve the template, I can easily incorporate the changes into all the websites that are based on it. This article describes how I set that up, and the information applies to all templates in general – Jekyll is not required. Templates do not need to have any special characteristics, beyond being generally useful in some sense.

GitHub has a template feature but this article does not require GitHub and works with all git hosts.

This diagram shows the local and hosted versions of a template repository and a project repository based on the template.

Local and Hosted versions of a project and its template

Copy from the Template Repository

To make a new local repository called new_project based on the repository called template from GitHub user with ID mslinn, type the following incantation. Please modify this command to suit your project, which might be hosted on AWS CodeCommit, Bitbucket, GitLab, etc.

Shell

$ git clone git@github.com:mslinn/template.git new_project

I have no git repository called template, so the above is just for explanation purposes.

git automatically sets up a remote origin from the local repository pointing to template for git fetch and git push commands, as we can see from the following:

Shell

$ git remote -v
origin git@github.com:mslinn/template.git (fetch)
origin git@github.com:mslinn/template.git (push)

Define Upstream and Downstream Repositories

To separately obtain updates from the new_project repository tracked at origin and updates from the upstream template, we need to define remote URLs for both the origin and template repositories.

Define Upstream Repository

The template will be an upstream remote repository.

Shell

$ git remote rename origin upstream

To ensure read-only status, we should disable pushing from new_project to the upstream repository, like this:

Shell

$ git remote set-url --push upstream no_push

Create New Downstream Repository

We need to create a new hosted repository for new_project. All the git hosting sites provide a way to do this using a web browser. However, I much prefer to use command line interfaces (CLIs).

AWS CodeCommit

The AWS CLI incantation looks something like this:

Shell

$ aws codecommit create-repository \
  --repository-name new_project \
  --repository-description "My downstream project"

Bitbucket

The Bitbucket CLI incantation looks something like this:

Shell

$ acli bobswift9 \
  --action createRepository \
  --project new_project \
  --repository new_project \
  --name new_project

GitHub

The shiny new official GitHub CLI unfortunately cannot do something that the tried-and-true Hub gh does: decouple the creation of a local git repo from the creation of the remote repo on GitHub. That means we must use Hub, like this:

Shell

$ cd new_project

$ hub create new_project
Existing repository detected
Updating origin
Warning: No xauth data; using fake authentication data for X11 forwarding.
X11 forwarding request failed on channel 0
https://github.com/mslinn/new_project

GitLab

All the GitLab CLIs I found had been abandoned.

All Git Host Sites

We can verify that the remotes for the downstream git project on your computer are now set up appropriately. The output shown shows that I used the GitHub CLI:

Shell

$ git remote -v
origin  git@github.com:mslinn/new_project.git (fetch)
origin  git@github.com:mslinn/new_project.git (push)
upstream        git@github.com:mslinn/template.git (fetch)
upstream        no_push (push)

Updating From the Downstream Repository

We can pull changes from the downstream new_project origin repository into the local copy of the downstream project like this:

Shell

$ git pull
From github.com:mslinn/template
 * branch            master -> FETCH_HEAD
Already up-to-date.

We could have typed this more verbose version, which accomplishes the same thing:

Shell

$ git pull origin
Everything up-to-date

Updating From the Upstream Template

We can pull changes from the upstream template repository into the local copy of the downstream new_project repository like this:

Shell

$ git pull upstream
From github.com:mslinn/template
 * branch            master -> FETCH_HEAD
Already up-to-date.

Pushing Changes

We can push changes from the local copy of the new_project repository to the hosted copy. From the top-level new_project directory, type:

Shell

$ git add -A

$ git commit -m "Commit message goes here"

$ git push origin
Everything up-to-date

However, if we try to push changes from new_project to upstream, we get the following error message. This is good because it means we cannot accidentally modify the upstream template when working on a project derived from the template.

Shell

$ git push upstream
fatal: 'no_push' does not appear to be a git repository
fatal: Could not read from remote repository. Please make sure you have the correct access rights
and the repository exists.

Updating the Upstream Template From a Downstream Repo

It is convenient to use two-way merge utilities to propagate selected changes in a downstream repository with the upstream repository. My favorite such utilities are:

Meld – Available for Linux and Windows. Also, available for Mac without support.
IntelliJ – Available as a command-line utility for Windows, Mac, and Linux. Also integrated with the IDEA GUI.

Once you have propagated selected changes from the downstream project to the upstream template repo, commit the changes to the upstream repo. From the top-level template project directory, type:

Shell

$ git add -A

$ git commit -m "Commit message goes here"

$ git push origin  # the word 'origin' is optional here
Everything up-to-date

Setting Up Another Computer

You might need to work on your project on another computer, and update from upstream the same way you set up the first computer. The process to do this is much the same as what was just described, but with fewer steps. It looks something like this:

Shell

$ git clone git@github.com:mslinn/new_project.git
Cloning into 'new_project'...
Warning: No xauth data; using fake authentication data for X11 forwarding.
X11 forwarding request failed on channel 0
remote: Enumerating objects: xxxx, done.
remote: Total xxxx (delta 0), reused 0 (delta 0), pack-reused 1139
Receiving objects: 100% (xxxx/xxxx), xx.xx MiB | x.x MiB/s, done.
Resolving deltas: 100% (xxx/xxx), done.

$ cd new_project

$ git remote add upstream https://github.com/mslinn/template

$ git remote set-url --push upstream no_push

GitHub Script

The following bash script is an example of how to automate the process of creating a project based on a template, using GitHub as the repository service.

cloneTemplate

#!/bin/bash

# See https://www.mslinn.com/blog/2020/11/30/propagating-git-template-changes.html

function help {
  if [ "$1" ]; then printf "\nError: $1\n\n"; fi
  echo "Usage:

$0 templateUrl newProjectName"
  exit 1
}

if [ -z "$(which git)" ]; then
  echo "Please install git and rerun this script"
  exit 2
fi

if [ -z "$(which hub)" ]; then
  echo "Please install hub and rerun this script"
  exit 3
fi

if [ -z "$1" ]; then help "No git project was specified as a template."; fi
if [ -z "$2" ]; then help "Please provide the name of the new project based on the template"; fi

git clone "$1" "$2"
cd "$2"
git remote rename origin upstream
git remote set-url --push upstream no_push

# Add the -p option to create a private repository
hub create "$2"
git branch -M master
git push -u origin master

Acknowledgement

This posting was inspired by this article.