Brotli is a robust, lossless data compression algorithm developed by Google. We can use Brotli as a Gzip drop-replacement. According to the Can-i-use report, Brotli is widely integrated on major web browsers. However, despite its popularity, Brotli is not included in the nginx core modules. So we have to compile it or use 3rd party nginx packages.
In this post, we will learn how to install and auto upgrade Brotli (ngx brotli) Nginx module on major Linux distributions while we still use the original Nginx version.
Preparations
Installing Nginx Web Server
This process is optional if you have Nginx installed on your server. If you haven’t installed Nginx on your server, run the command below to Install the Nginx web server (maintained by the Ubuntu community):
apt-get update -y && apt-get install nginx -yYou can follow this official manual if you want to install Nginx directly maintained by the Nginx community.
Once the Nginx web server is installed, enable nginx.service to make the Nginx web server automatically start after reboot. To enable and start the Nginx web server, run the command below:
systemctl enable nginx.service && systemctl start nginx.serviceInstalling Nginx Build Dependencies & Tools
Before we compile the nginx module, we have to install Nginx dependencies and the C compiler.
To make the nginx resolves the needed modules, we need to edit /etc/apt/sources.list and remove the preceding # mark on deb-src lines. To automatically do this, we can replace the string using sed command below:
sed -ie 's/^#\sdeb-src/deb-src/g' /etc/apt/sources.listThen, run the command below to fetch the latest package metadata.
apt-get update -yOnce the apt metadata gets updated, run this command to install the required dependencies:
apt-get build-dep nginx Preparing the Installer and Auto Update Script
Once the build-dep task above completed, now we are ready to do the main part. We are using script from https://gist.github.com/pujianto/795e858f11ffc3073a24f60217fcba1a that we have prepared previously.
To do so, as a root user create a file in /usr/local/bin/ngx_brotli with the following content.
#!/bin/bash
# store this file as /usr/local/bin/ngx_brotli
ngx_brotli() {
trap 'printf "Aborted by user. exiting...\n";exit' INT
ngx_brotli_install() {
local TEMP_DIR_INSTALL
TEMP_DIR_INSTALL="$(mktemp -d -t ngx_brotli.XXXXXXXXXX)"
printf "Temp dir: %s\n" "$TEMP_DIR_INSTALL"
trap 'printf "Cleanup...\n Removing %s\n" "${TEMP_DIR_INSTALL}";rm -rf "${TEMP_DIR_INSTALL}";exit' EXIT
local NGX_BROTLI_GIT_URL="https://github.com/google/ngx_brotli.git"
local NGINX_VERSION
local NGINX_LIB_LOCATION
NGINX_VERSION=$(nginx -v 2>&1 | awk '{print $3}' | sed -e 's#/#-#g')
NGINX_LIB_LOCATION=$(nginx -V 2>&1 | grep "modules-path" | sed -e 's#.*--modules-path=\(.[^ ]*\).*##g')
printf "Preparing brotli modules...\n"
mkdir -p "${TEMP_DIR_INSTALL}/nginx"
printf "Downloading nginx source code... (%s) \n" "${NGINX_VERSION}"
curl -s -L "https://nginx.org/download/${NGINX_VERSION}.tar.gz" | tar -xzC "${TEMP_DIR_INSTALL}/nginx" --strip-components 1
printf "Downloading brotli source code...\n"
git clone "${NGX_BROTLI_GIT_URL}" "${TEMP_DIR_INSTALL}/ngx_brotli"
cd "${TEMP_DIR_INSTALL}/ngx_brotli" && git submodule update --recursive --init
printf "Compiling modules...\n"
cd "${TEMP_DIR_INSTALL}/nginx" && ./configure --with-compat --add-dynamic-module=../ngx_brotli && make modules
printf "Installing modules...\n"
cp -vf "${TEMP_DIR_INSTALL}/nginx/objs/ngx_http_brotli"*.so "${NGINX_LIB_LOCATION}/"
}
local REQUIRED_PROGRAMS=("nginx" "git" "curl" "tar" "make")
check_package() {
command -v "" >/dev/null 2>&1 || {
printf >&2 "%s is required but not installed. Aborting.\n" ""
return 1
}
}
for program in "${REQUIRED_PROGRAMS[@]}"; do
check_package "${program}" || return 1
done
# Pass init argument if you want to install brotli modules for the first time.
if [ "" == "init" ]; then
ngx_brotli_install && return 0
fi
if nginx -t 2>&1 | grep -q brotli; then
printf "Upgrading ngx_brotli...\n"
ngx_brotli_install && systemctl restart nginx.service
else
printf "ngx_brotli is already up to date.\n"
fi
}
return 2>/dev/null || ngx_brotli "${@}"After we save the document, we need to change the ngx_brotli file permission to an executable file. To do so, run the following command:
chmod +x /usr/local/bin/ngx_brotliOnce we save it, we need to run ngx_brotli in init mode, so the script will compile ngx_brotli modules and install it on /etc/nginx/modules folder.
To do so, run the following command.
ngx_brotli initIf all dependencies and required programs are installed, now we should have ngx_brotli dynamic modules in /etc/nginx/modules directory.
To verify the installation, run the following command:
ls -la /etc/nginx/modules/ngx_http_brotli_*And you will get a list like this:
root@ubuntu:~# ls -la /etc/nginx/modules/ngx_http_brotli_*
-rwxr-xr-x 1 root root 6821840 Aug 10 02:55 /etc/nginx/modules/ngx_http_brotli_filter_module.so
-rwxr-xr-x 1 root root 87568 Aug 10 02:55 /etc/nginx/modules/ngx_http_brotli_static_module.soAt this point, we have installed ngx_brotli dynamic module into our Nginx webserver. Since this module is dynamic, make sure to load those *.so modules into our /etc/nginx/nginx.conf file. Here is an example config how to load ngx_brotli dynamic modules into main nginx config.
root@ubuntu:~# cat /etc/nginx/nginx.conf
user ubuntu;
worker_processes 4;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
load_module /etc/nginx/modules/ngx_http_brotli_filter_module.so;
load_module /etc/nginx/modules/ngx_http_brotli_static_module.so;
events {
worker_connections 4096;
use epoll;
...Auto Compile Brotli Module After Nginx Upgrade
Since we compiled the ngx_brotli as dynamic modules, there’s a problem. That is when we upgrade the nginx version. If we restart the nginx web server, it will fail. Because the manually loaded dynamic modules are not using the same version. To fix this issue, we need to use Ubuntu’s apt hook feature. In a nutshell, we ask apt-get software to execute our bash script above after an upgrade. If there’s a mismatch version detected, then the ngx_brotli bash script will recompile the modules with the new Nginx version.
To auto compile the ngx_brotli module after Nginx upgrade on Ubuntu, we can do it by creating a /etc/apt/apt.conf.d/100-nginx-post-update file with the following content:
DPkg::Post-Invoke {"/usr/local/bin/ngx_brotli"; };It will auto-upgrade the Ngx brotli version seamlessly.