Mike Slinn
Mike Slinn

Publishing a Draft Article in a Jekyll Collection

Published 2021-12-20.
Time to read: 2 minutes.

This site is categorized under Jekyll.

I often want to have others review articles that I write before I make them visible to the public on my Jekyll-powered web site. There are many ways to do this:

  • Make a PDF (unfortunately, recipients do not see updates, unless a blizzard of emails ensues)
  • Set up a staging server (maintaining another server means more work)
  • Quietly publish to the production server without linking the article into the rest of the site (the subject of this article)

The remainder of this post shows the changes necessary to publish a draft page to a collection, without including it into the collection. The new page is not included in the site map, so the search engines are not notified. The new page is not included in any automatically generated content, so no links to it are inadvertently created. New links from the new page work. Reviewers can see the page if they know the URL.

Set the Flag

Add this to the front matter of any page in a collection that should not be publicly visible:

published: false

Bash Scripts

This is the Bash script I wrote to make this possible. My web site is served from AWS S3 buckets, and the AWS CLI is used to manage AWS services.

#!/bin/bash
#
# Author: Mike Slinn
#
# Before running this script:
# 1) Add "published: false" to front matter for new draft page
# 2) Run _bin/prod to upload images and other dependencies
#
# SPDX-License-Identifier: Apache-2.0

GIT_ROOT="$( git rev-parse --show-toplevel )"
cd "${GIT_ROOT}"
git add .
git commit -m -
source _bin/loadConfigEnvVars
_bin/generate development

cd _site
FILES=$( grep -rl draftPost . | grep '[.]html$' | sed "s|^\./||" | grep -v publishing-drafts )
unset FILES2
for FILE in $FILES; do
  FILES2="$FILES2 /$FILE"
  echo "Uploading https://$DOMAIN/$FILE"
  aws s3 cp \
	--acl public-read \
    --quiet \
	"$FILE" "s3://$DOMAIN/$FILE"
done

# Only invalidate if -I not provided
if [ "$1" != -I ]; then
  aws cloudfront create-invalidation \
      --distribution-id "$AWS_CLOUDFRONT_DIST_ID" \
      --paths $FILES2
fi

No changes were required to my script for defining environment variables from the Jekyll YAML configuration file:

#!/bin/bash
#
# Author: Mike Slinn
#
# Defines the following configuration environment variables when sourced by a bash script:
# AWS_ACCOUNT_ID, AWS_CLOUDFRONT_DIST_ID, AWS_REGION, DOMAIN, LAMBDA_ARN, LAMBDA_IAM_ROLE_ARN, LAMBDA_IAM_ROLE_NAME, LAMBDA_HANDLER, LAMBDA_NAME, LAMBDA_RUNTIME, LAMBDA_ZIP, LAMBDA_PACKAGE_DIR, TITLE and URL
#
# SPDX-License-Identifier: Apache-2.0

function assertEnvVarSet {
  if [ ! -v "$1" ]; then
    echo "Error: no environment variable called $1 is defined"
    return 1
  fi
  return 0
}

function lookupEnvVar {
  assertEnvVarSet "$(readYaml $1)" || { return 1; }
  eval echo -e "\$$(readYaml $1)"
}

function readYaml {
  # $1 - path
  yq eval ".$1" _config.yml
}

function writeYaml {
  # $1 - path
  # $2 - value
  yq eval ".$1 = $2" -i _config.yml
}

GIT_ROOT="$( git rev-parse --show-toplevel )"

if [ -z "$( which yq )" ] || [[ "$( yq -V )" != *4.* ]]; then 
  VERSION=v4.2.0
  BINARY=yq_linux_amd64
  echo "Installing yq"
  # See https://mikefarah.gitbook.io/yq/
  sudo -iH bash <<EOF
  wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY}.tar.gz -O - |\
    tar xz && mv ${BINARY} /usr/bin/yq
EOF
fi

# AWS modifiable values
export GATEWAY_STAGE="$(          readYaml aws.apiGateway.stage          )"

# AWS computed values
export AWS_ACCOUNT_ID="$(         readYaml aws.accountId                 )"
export AWS_CLOUDFRONT_DIST_ID="$( readYaml aws.cloudfront.distributionId )"
export AWS_REGION="$(             readYaml aws.region                    )"
export GATEWAY_ENDPOINT="$(       readYaml aws.apiGateway.endpoint       )"
export GATEWAY_REST_API_ID="$(    readYaml aws.apiGateway.restId         )"
export GATEWAY_RESOURCE_ID="$(    readYaml aws.apiGateway.resourceId )"
export GATEWAY_ROOT_ID="$(        readYaml aws.apiGateway.rootResourceId )"
export GATEWAY_NAME="$(           readYaml aws.apiGateway.name           )"

# Computed AWS Lambda values
export LAMBDA_ARN="$(           readYaml aws.lambda.addSubscriber.computed.arn        )"
export LAMBDA_IAM_ROLE_ARN="$(  readYaml aws.lambda.addSubscriber.computed.iamRoleArn )"

# Modifiable AWS Lambda values
export LAMBDA_HANDLER="$(       readYaml aws.lambda.addSubscriber.custom.handler      )"
export LAMBDA_IAM_ROLE_NAME="$( readYaml aws.lambda.addSubscriber.custom.iamRoleName  )"
export LAMBDA_NAME="$(          readYaml aws.lambda.addSubscriber.custom.name         )"
export LAMBDA_RUNTIME="$(       readYaml aws.lambda.addSubscriber.custom.runtime      )"

export LAMBDA_PACKAGE_DIR="${GIT_ROOT}/_package"
export LAMBDA_ZIP="${LAMBDA_PACKAGE_DIR}/function.zip"

# Misc modifiable values
export TITLE="$( readYaml title )"
export URL="$( readYaml url )"
export DOMAIN="$( echo "$URL" | sed -n -e 's,^https\?://,,p' )"

No changes were required to my script for publishing to production:

#!/bin/bash
#
# Push new Jekyll web site to AWS S3 bucket
#
# Author: Mike Slinn
# SPDX-License-Identifier: Apache-2.0

RED='\033[0;31m'
RESET='\033[0m' # No Color

function brokenLinkCheck {
  # See https://github.com/stevenvachon/broken-link-checker
  BROKEN="$( _bin/checkLinks | grep BROKEN )"
  if [ "$BROKEN" ]; then
    printf "\n${RED}Broken links found, aborting:$RESET"
    echo "$BROKEN"
    printf "\n${RED}=======================$RESET"
    exit 1
  fi
}

function checkDependencies {
  if [ -z "$( which npm )" ]; then
    # See https://github.com/nvm-sh/nvm
    curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash
    export NVM_DIR="$HOME/.nvm"
    [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"  # This loads nvm
    [ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"  # This loads nvm bash_completion
  fi

  if [ -z "$( which node )" ]; then
    nvm install node
  fi

  if [ -z "$( which blc )" ]; then
    yes | sudo -H npm install broken-link-checker -g
  fi

  if [ -z "$( which aws )" ]; then yes | sudo apt install awscli; fi
  if [ -z "$( which jq )" ]; then yes | sudo apt install jq; fi
}

function crawlSiteMap {
  # Ask Google and Bing to crawl the sitemap
  SITEMAP="$DOMAIN/sitemap.xml"

  # See https://developers.google.com/search/docs/guides/submit-URLs
  echo "Notifying Google of new sitemap.xml"
  wget -qO - http://www.google.com/ping?sitemap=https://$SITEMAP > /dev/null

  # See https://www.bing.com/webmaster/help/how-to-submit-sitemaps-82a15bd4
  echo "Notifying Bing of new sitemap.xml"
  wget -qO - http://www.bing.com/ping?sitemap=http%3A%2F%2F$SITEMAP > /dev/null
}

function maybeCreateS3Bucket {
  aws s3 ls "s3://$DOMAIN" 2>/dev/null >/dev/null
  if [[ $? -ne 0 ]]; then # create bucket
    _bin/makeAwsBucket "$DOMAIN"
  fi
}

function waitForInvalidation {
  echo "Waiting for invalidation $2 to complete."
  aws cloudfront wait invalidation-completed \
    --distribution-id "$1" \
    --id "$2"
  echo "Invalidation $2 has completed."
}

function invalidate {
  if [ "$AWS_CLOUDFRONT_DIST_ID" ]; then
    JSON="$( aws cloudfront create-invalidation \
      --distribution-id "$AWS_CLOUDFRONT_DIST_ID" \
      --paths "/*"
    )"
    INVALIDATION_ID="$( jq -r .Invalidation.Id <<< "$JSON" )"
    waitForInvalidation "$AWS_CLOUDFRONT_DIST_ID" "$INVALIDATION_ID" &
  fi
}

function sync {
  # Get rid of any extra files in _site/
  find _site/ -name '*.Identifier' -o -name '.sprockets-manifest*' -delete

  # aws cli does not know the Content-Type for webp files so handle that filetype separately.
  # Do not sync mp4s because they are so large.
  aws s3 sync \
    --acl public-read \
    --exclude '*.ai' \
    --exclude '*.webp' \
    --quiet \
    _site/ s3://$DOMAIN
  aws s3 sync \
    --acl public-read \
    --content-type 'image/webp' \
    --exclude '*' \
    --include '*.webp' \
    --quiet \
    _site/ s3://$DOMAIN
  # Do not sync mp4s because they are so large.
  aws s3 sync \
    --delete \
    --exclude '*.mp4' \
    --quiet \
    _site/ s3://$DOMAIN
}

function publishSite {
  set -b
  #set -e

  # Set current directory to project root
  GIT_ROOT="$( git rev-parse --show-toplevel )"
  cd "${GIT_ROOT}"
  source _bin/loadConfigEnvVars

  checkDependencies
  git pull 3>&1 1>&2 2>&3 | sed -e '/^X11/d'  | sed -e '/^Warning:/d'
  #_bin/setTimes
  _bin/generate production
  if [ $? -ne 0 ]; then exit 1; fi
  if [ $? -ne 0 ]; then exit 2; fi
  brokenLinkCheck
  commit
  maybeCreateS3Bucket
  sync
  _bin/redirects
  _bin/stage -I
  invalidate
  if [ "$DOMAIN" != www.testJekyllTemplate.com ]; then crawlSiteMap; fi
  echo "$( date '+Published %A, %B %d, %Y at %H:%M:%S.' )"
}

publishSite

No changes were required to my script for serving locally:

#!/bin/bash

# SPDX-License-Identifier: Apache-2.0

export PORT_LSB=1
unset FUTURE_POSTS

export FUTURE_POSTS="--future"
export HOST=0.0.0.0
export INCREMENTAL="--incremental"
export LIVE="--livereload"
export LIVE_PORT="--livereload_port 3572$PORT_LSB"
export PORT="400$PORT_LSB"

export DRAFTS="--drafts"
export UNPUBLISHED="--unpublished"
export QUIET="--quiet"
unset VERBOSE
export PORT="400$PORT_LSB"

export OPTIONS="$LIVE_PORT"

function isWindows {
  if [ "$( grep -i Microsoft /proc/version )" ]; then echo yes; fi
}

# Force polling if the script is running on a Windows drive
if [ "$( isWindows )" ]; then
  export OPTIONS="$OPTIONS --force_polling"
fi


function help {
   echo "Runs Jekyll. The default domain:port is localhost:$PORT.
  Options are:
    -F         Disable future posts
    -H domain  Run from given host, defaults to localhost
    -I         Disable incremental compilation
    -L         Disable live reload (use if cannot bind to server/port)
    -c         Clear cache and rebuild site
    -d         Debug mode
    -D         Disable drafts and unpublished
    -g 10      Generate only the last 10 posts
    -p 1234    Run from given port, defaults to $PORT
    -P         Generate a production site
    -q         Set Jekyll logging level to :error (default)
    -v         Set Jekyll logging level to :debug

  For example, to run on http://localhost:$PORT, type:
  _bin/serve
  _bin/serve -c  # Clean out previous build

  When quiet and verbose are both specified, log-level is set to :error
"
   exit 1
}

function installDependencies {
  if [ -z "$( find /usr/lib/x86_64-linux-gnu -name 'libmagic.so*' )" ]; then
    echo "Installing libmagic-dev"
    yes | sudo apt install libmagic-dev
  fi

  if [ -z "$( which ruby )" ]; then
    yes | sudo apt install ruby-full
  fi

  if [ -z "$( which bundle )" ]; then
	  gem install bundler $QUIET
  fi

  bundle install $QUIET
}

function isSiteUp {
  curl -sSf http://$1 > /dev/null 2> /dev/null
}

function makePluginDocs {
  rm -rf jekyll/docs/*
  cd _plugins/ > /dev/null || exit
  if [ "$QUIET" ]; then YARD_QUIET="--no-stats --no-progress"; fi
  yard doc $QUIET $YARD_QUIET \
    jekyll_command_template.rb jekyll_filter_template.rb jekyll_generator_template.rb \
    rawinclude.rb string_overflow.rb symlink_watcher.rb > /dev/null 2> /dev/null
  cd - > /dev/null || exit
}


# Restore default Python environment
if [ "$(declare -f deactivate)" ]; then
  if [ -z "$QUIET" ]; then echo "Deactivating Python virtualenv"; fi
  deactivate
fi

# Set cwd to project root
GIT_ROOT="$( git rev-parse --show-toplevel )"
cd "${GIT_ROOT}" || exit
source _bin/loadConfigEnvVars

export JEKYLL_ENV=development
while getopts "cdDhiFg:hH:ILp:Pqv\?" opt; do
   case $opt in
     D ) unset DRAFTS
         unset UNPUBLISHED
         ;;
     F ) unset FUTURE_POSTS ;;
     H ) export HOST="$OPTARG" ;;
     I ) unset INCREMENTAL ;;
     L ) unset LIVE ;;
     c ) if [ -z "$QUIET" ]; then echo "Removing temporary files from _site/"; fi
         #rm -rf _site/*
         bundle exec jekyll clean $QUIET
         ;;
     d ) set -xv ;;
     g ) export OPTIONS="$OPTIONS --limit_posts=$OPTARG" ;; # generate only the last 10 posts
     p ) export PORT="$OPTARG" ;;
     P ) export JEKYLL_ENV=production ;;
     q ) export QUIET="--quiet" ;;
     v ) export VERBOSE="--verbose" ;;
     * ) help ;;
   esac
done
shift $(($OPTIND-1))

if [ "$HOST" ]; then OPTIONS="$OPTIONS --host $HOST"; fi   # Listen at the given hostname.
if [ "$PORT" ]; then OPTIONS="$OPTIONS --port $PORT"; fi   # Listen at the given port.
OPTIONS="$OPTIONS $FUTURE_POSTS"
OPTIONS="$OPTIONS $INCREMENTAL"
OPTIONS="$OPTIONS $LIVE"
OPTIONS="$OPTIONS $DRAFTS"
OPTIONS="$OPTIONS $UNPUBLISHED"
OPTIONS="$OPTIONS $QUIET"
OPTIONS="$OPTIONS $VERBOSE"

if [ -z "$QUIET" ]; then echo "OPTIONS=$OPTIONS"; fi

source use default > /dev/null
installDependencies
# _bin/make_public_plugin_docs

bundle exec jekyll serve $OPTIONS 2>&1 | \
  grep -Ev 'Using the last argument as keyword parameters is deprecated'

Jekyll Template Change

The Jekyll template I use for my articles contains the following:

{% assign isDraft = post.url | startsWith: '_drafts/' %}
{% if isDraft or page.published == false %}
  <div class="bg_yellow draftPost">— Draft —</div>
{% endif %}

CSS

The CSS contains:

.bg_yellow {
  background-color: yellow;
  padding: 2px;
}

.draftPost {
  text-align: center;
  font-style: italic;
  width: 15ch;
  position: absolute;
  top: 7px;
  left: 0;
}