Published 2020-08-15. Last modified 2022-04-04.
Time to read: 2 minutes.

This site is categorized under Git, Jekyll, Scripting, Ubuntu.

I first launched this website in 1996. Since then, it has been re-incarnated using many different technologies. Presently I use Jekyll to assemble the site, then push the image to a web-enabled AWS S3 bucket that is edge-cached by an AWS CloudFront distribution.

Until yesterday, the site contained images with a mixture of image formats. I decided to convert them all to the new webp format. Because there are hundreds of images in over 120 web pages, I wrote a bash script called toWebP to do the work. This posting provides the toWebP script plus instructions on how you could use it for your website.

The script converts image types gif, jpg, jpeg, png, tif, and tiff. It also modifies the HTML pages, CSS and SCSS that reference those images.

The conversions are set for maximum fidelity (lossless where possible), and maximum compression. This means the images look great and load quickly.

Caveat

The script assumes that all images are local to your website, which makes sense because the converted images need to be stored, and local storage is the only sensible option. It renames all references to images in HTML, CSS and SCSS files to webp format. If the images are remote (for example, on a CDN), they are not converted, but the image file types in the HTML, CSS and SCSS are adjusted anyway. I suppose I could fix the script, but I don't need to do that for myself. If someone needs that feature, go ahead and enhance the script... and please provide me the enhanced script, so I can update this blog posting.

Prerequisites

You need to install the WebP package.

Mac

Use Homebrew or Macports.

Ubuntu (this is the default Linux distribution for Windows Subsystem for Linux)

At a shell prompt type:

Shell

$ yes | sudo apt install webp

Running toWebp

The program may emit warnings when it runs. Those warnings can be safely ignored.

Hopefully, your website is managed by git. I suggest that you commit your work before running the script. That way if something goes wrong you just have to type git stash to return your website to its previous state.

Usage

The general form of the command to convert all images and modify the HTML pages that they are referenced from is:

Shell

$ toWebp <directoryName>

Examples

To convert the website (images, html, scss & css) rooted at the current directory, type:

Shell

$ toWebp .

To convert the website called mySite rooted under your home directory, type:

Shell

$ toWebp ~/mySite

To just convert 1 specific image to webp, type:

Shell

$ toWebp images/blah.jpg

The toWebP Bash Script.

Put this file in one of the directories on your PATH, for example /usr/local/bin:

toWebp

#!/bin/bash

# Convert all images to webp files in place and update HTML, CSS and SCSS to suit
# Usage: toWebp directory|imageFileName
# Example: toWebp .  # convert images, html, scss & css in current directory tree
# Example: toWebp images/blah.jpg   # just convert 1 specific image
#
# SPDX-License-Identifier: Apache-2.0

shopt -s extglob

export CMD="cwebp alpha_q 10 -exact -lossless -m 6 -short -q 100 -z 9"

function checkDependencies {
  if [ -z "$( which cwebp )" ]; then
    echo "Installing webp"
    yes | sudo apt install webp
  fi
}

function convertGifs {
  for F in $( find "$1" -iname '*.gif' ); do
    TOF="${F%.*}.webp"
    gif2webp -loop_compatibility -m 6 -mixed "$F" -q 100 -o "$TOF"
  done
}

function convertMost {
  # See "Extended pattern" extglob for Bash
  # https://wiki.bash-hackers.org/syntax/pattern#extended_pattern_language
  cd "$1"
  for F in $( listImages ); do
    TOF="${F%.*}.webp"
    echo "Converting '$F' to '$TOF'"
    # Warning messages might be emitted, but don't worry
    $CMD "$F" -o "$TOF"
    if [ "$DELETE_OLD" ]; then rm "$F"; fi
  done
  cd -
}

function listImages {
  # Does not return gifs because cwebp cannot handle them
  find . -iregex '.*\.\(jpg\|png\|jpeg\|tif\|tiff\)' -printf '%f\n'
}

function renameImage {
  sed -i "s,\b$1\b,.webp,g" "$2"
}

function swapImages {
  for F in $( find "$1" -iname '*.html' -o -iname '*.css' -o -iname '*.scss' ); do
    for X in .gif .jpg .jpeg .tif .tiff .png; do
      echo "Swapping $X images for .webp in '$F'"
      renameImage "$X" "$F"
    done
  done
}


# Set cwd to project root
GIT_ROOT="$( git rev-parse --show-toplevel )"
cd "${GIT_ROOT}" || exit
source _bin/loadConfigEnvVars

checkDependencies

unset DELETE_OLD   # TODO make this a command line option

if [ -f "$1" ]; then   # just convert a single image to webp
  F="$1"
  $CMD "$F" -o "${F%.*}.webp"
  if [ "$DELETE_OLD" ]; then rm "$F"; fi
elif [ -d "$1" ]; then # process all images, css and scss in directory
  shopt -s globstar
  convertMost "$1"
  convertGifs "$1"
  swapImages "$1"
else
  >&2 echo "Error: you must either specify a valid file or a directory"
fi

Make it Executable

Remember to make the toWebp script executable before trying to use it:

Shell

$ chmod a+x /usr/local/bin/toWebp