Published 2020-08-15.
Last modified 2022-04-04.
Time to read: 2 minutes.
I first launched this website in 1996. Since then, it has been re-incarnated using many different technologies. Presently I use Jekyll to assemble the site, then push the image to a web-enabled AWS S3 bucket that is edge-cached by an AWS CloudFront distribution.
Until yesterday, the site contained images with a mixture of image formats.
I decided to convert them all to the new
webp format.
Because there are hundreds of images in over 120 web pages, I wrote a bash script called toWebP to do the work.
This posting provides the toWebP script plus instructions on how you could use it for your website.
The script converts image types gif, jpg, jpeg, png, tif, and tiff.
It also modifies the HTML pages, CSS and SCSS that reference those images.
The conversions are set for maximum fidelity (lossless where possible), and maximum compression. This means the images look great and load quickly.
I also wrote toPng,
which works in a similar manner as toWebp.
Caveat
The script assumes that all images are local to your website, which makes sense because the converted images need to be stored,
and local storage is the only sensible option.
It renames all references to images in HTML, CSS and SCSS files to webp format.
If the images are remote (for example, on a CDN), they are not converted, but the image file types in the HTML, CSS and SCSS are adjusted anyway.
I suppose I could fix the script, but I don't need to do that for myself.
If someone needs that feature, go ahead and enhance the script...
and please provide me the enhanced script, so I can update this blog posting.
Prerequisites
You need to install the WebP package.
Mac
Ubuntu (this is the default Linux distribution for Windows Subsystem for Linux)
At a shell prompt type:
$ yes | sudo apt install webp
Running toWebp
The program may emit warnings when it runs. Those warnings can be safely ignored.
Hopefully, your website is managed by git.
I suggest that you commit your work before running the script.
That way if something goes wrong you just have to type git stash to return your website to its previous state.
Usage
The general form of the command to convert all images and modify the HTML pages that they are referenced from is:
$ toWebp <directoryName>
Examples
To convert the website (images, html, scss & css) rooted at the current directory, type:
$ toWebp .
To convert the website called mySite rooted under your home directory, type:
$ toWebp ~/mySite
To just convert 1 specific image to webp, type:
$ toWebp images/blah.jpg
The toWebP Bash Script.
Put this file in one of the directories on your PATH,
for example /usr/local/bin,
or for Ubuntu users ~/.local/bin/:
#!/bin/bash
# SPDX-License-Identifier: Apache-2.0
shopt -s extglob
export CMD="cwebp alpha_q 10 -exact -lossless -m 6 -short -q 100 -z 9"
function help {
[[ "$1" ]] && >&2 printf "Error: $1\n\n";
>&2 printf "$( basename $0 ) - Convert all images to webp files in place and update HTML, CSS and SCSS to suit
Usage: toWebp directory|imageFileName
Example: toWebp . # convert images, html, scss & css in current directory tree
Example: toWebp images/blah.jpg # just convert 1 specific image
"
}
function checkDependencies {
if [ -z "$( which cwebp )" ]; then
echo "Installing webp"
yes | sudo apt install webp
fi
}
function convertGifs {
for F in $( find "$1" -iname '*.gif' ); do
TOF="${F%.*}.webp"
gif2webp -loop_compatibility -m 6 -mixed "$F" -q 100 -o "$TOF"
done
}
function convertMost {
# See "Extended pattern" extglob for Bash
# https://wiki.bash-hackers.org/syntax/pattern#extended_pattern_language
cd "$1" || exit 3
for F in $( listImages ); do
TOF="${F%.*}.webp"
echo "Converting '$F' to '$TOF'"
# Warning messages might be emitted, but don't worry
$CMD "$F" -o "$TOF"
if [ "$DELETE_OLD" ]; then rm "$F"; fi
done
cd - || exit
}
function listImages {
# Does not return gifs because cwebp cannot handle them
find . -iregex '.*\.\(jpg\|png\|jpeg\|tif\|tiff\)' -printf '%f\n'
}
function renameImage {
sed -i "s,\b$1\b,.webp,g" "$2"
}
function swapImages {
for F in $( find "$1" -iname '*.html' -o -iname '*.css' -o -iname '*.scss' ); do
for X in .gif .jpg .jpeg .tif .tiff .png; do
echo "Swapping $X images for .webp in '$F'"
renameImage "$X" "$F"
done
done
}
unset DELETE_OLD # TODO make this a command line option
if [[ -f "$1" ]]; then # just convert a single image to webp
F="$1"
$CMD "$F" -o "${F%.*}.webp"
if [ "$DELETE_OLD" ]; then rm "$F"; fi
elif [[ -d "$1" ]]; then # process all images, css and scss in directory
shopt -s globstar
convertMost "$1"
convertGifs "$1"
swapImages "$1"
else
help "You must either specify a valid file or a directory"
fi
Make it Executable
Remember to make the toWebp script executable before trying to use it:
$ chmod a+x /usr/local/bin/toWebp
Help Message
$ toWebp Error: You must either specify a valid file or a directory toWebp - Convert all images to webp files in place and update HTML, CSS and SCSS to suit Usage: toWebp directory|imageFileName Example: toWebp . # convert images, html, scss & css in current directory tree Example: toWebp images/blah.jpg # just convert 1 specific image
The toPng Bash Script.
#!/bin/bash
# Convert all webp images to png in place; does not update HTML, CSS or SCSS
# Usage: toPng directory|imageFileName
# Example: toPng . # convert images in current directory tree
# Example: toPng images/blah.webp # just convert 1 specific image
#
# SPDX-License-Identifier: Apache-2.0
shopt -s extglob
export CMD="dwebp"
function help {
[[ "$1" ]] && >&2 printf "Error: $1\n\n";
>&2 printf "$( basename $0 ) - Convert all images to webp files in place and update HTML, CSS and SCSS to suit
Usage: toWebp directory|imageFileName
Example: toWebp . # convert images, html, scss & css in current directory tree
Example: toWebp images/blah.jpg # just convert 1 specific image
"
}
function checkDependencies {
if [ -z "$(which dwebp)" ]; then
echo "Installing webp"
yes | sudo apt install webp
fi
}
function listImages {
find . -type f -iname '*.webp'
}
function convert {
# See "Extended pattern" extglob for Bash
# https://wiki.bash-hackers.org/syntax/pattern#extended_pattern_language
cd "$1" || exit
for F in $( listImages ); do
TOF="${F%.*}.png"
echo "Converting '$F' to '$TOF'"
# Warning messages might be emitted, but don't worry
$CMD "$F" -o "$TOF"
if [ "$DELETE_OLD" ]; then rm "$F"; fi
done
cd - || exit
}
# Set cwd to project root
GIT_ROOT="$( git rev-parse --show-toplevel )"
cd "${GIT_ROOT}" || exit
[[ -f _bin/loadConfigEnvVars ]] && source _bin/loadConfigEnvVars;
unset DELETE_OLD # TODO make this a command line option
checkDependencies
if [ -f "$1" ]; then # just convert a single image to webp
F="$1"
$CMD "$F" -o "${F%.*}.webp"
if [ "$DELETE_OLD" ]; then rm "$F"; fi
elif [ -d "$1" ]; then # process all images, css and scss in directory
convert "$1"
else
help "You must either specify a valid file or a directory."
fi
Make it Executable
Remember to make the toWebp script executable before trying to use it:
$ chmod a+x /usr/local/bin/toPng
Help Message
$ toPng Error: You must either specify a valid file or a directory. toPng - Convert all images to webp files in place and update HTML, CSS and SCSS to suit. Usage: toWebp directory|imageFileName Example: toWebp . # convert images, html, scss & css in current directory tree Example: toWebp images/blah.jpg # just convert 1 specific image