Resolve Google Lighthouse Audit “does not provide fallback content” with GatsbyJS

Google’s Lighthouse Audit Tool is great for evaluating the performance of a site and for confirming just how awesome static sites created with GatsbyJS + React can be.

A common point reduction seen by Gatsby developers is: Does not provide fallback content when JavaScript is not available, with the description: “The page body should render some content if its scripts are not available”.

This post is here to help you resolve that and get one step closer to a perfect score.

The audit requirement

Google explains: “Your app should display some content when JavaScript is disabled, even if it’s just a warning to the user that JavaScript is required to use the app”.

One might think that React Helmet offers a potential solution, however it’s not applicable in this case. Helmet is specifically a document head manager and even though <noscript> tags are valid inside a document head, the audit rule specifically refers to the page body.

Adding tags to the page body above Components injected by Gatsby

Copy html.js from .cache/default-html.js in your Gatsby project folder to your src/ folder, renaming it to html.js:

cp .cache/default-html.js src/html.js

html.js will now take precendence over Gatsby’s boilerplate version.

Open html.js. Between {this.props.preBodyComponents} and before the <div> that contains Gatsby’s body components, you can insert a tag such as:

<noscript>This website requires JavaScript. To contact us, please send us an email at: <a href="mailto:email@example.com">email@example.com</a></noscript>

Voila, one more checkbox on your Lighthouse audit results!

For more information about html.js see: https://www.gatsbyjs.org/docs/custom-html/

Editing WordPress’ wp-config.php with wp-cli and adding variables with sed

The WordPress CLI (command line interface) is a huge step for enabling developers and devops/sysadmin folks manage their WordPress installations. It’s awesome for writing scripts to automate key tasks.

This post covers editing the WordPress configuration file wp-config.php with the WP-CLI’s wp config command, as well as using the sed command to address a key missing feature of WP-CLI: the ability to add new config variables.

There are plenty of reasons you might want to edit wp-config.php via a script or directly via the command line. For example, developers might appreciate a bash script that sets WP_DEBUG to true, and a devops person might want to create an automated deploy process that ensures key SMTP settings are in place.

The rest of this post will assume your WP-CLI command can be invoked with wp. Depending on how you installed it, the command might be available to you as wp-cli or wp-cli.phar. If you are running wp-cli’s phar file directly, substitute php wp-cli.phar in place of wp in the examples.

Editing config variables with wp-cli’s config command

WP-CLI supports modifying config variables in wp-config.php via wp config.

This is a great feature, albeit with the noted catch that wp config only works for a given variable if that variable is already defined in wp-config.php. I’ll show you how to work around that and add variables with sed in the next section.

The following example uses sudo to run wp config as the _www web server user, the default web server user on MacOS. On Ubuntu and many other linux distros, this user is likely www-data:

sudo -u _www wp config set FS_METHOD 'direct'
sudo -u _www wp config set DISABLE_WP_CRON true
sudo -u _www wp config set WP_DEBUG true
sudo -u _www wp config set WP_DEBUG_LOG true

These are some of the most popular config options that developers and admins want to modify.

Adding config variables to wp-config.php using sed

There are a number of command-line utilities on Linux and Unix-like systems that can edit text files. One of the most popular is sed, the quintessential stream editor. Unix admins have been working with streams forever, long before NodeJS made it cool :).

sed is pre-installed on most systems and can be used directly in the Terminal or inside a bash (or other shell) script.

The following example uses sed to add config variables to wp-config.php right before the well-known “That’s all, stop editing!” comment line found in the file.

This snippet works on MacOS and elsewhere. MacOS and OSX, as well as their related family in the BSD/unix world, generally bundle a classic POSIX-compliant version of the sed command which is more limited vs. the more common and more popular GNU sed that ships with major linux distributions like Ubuntu. If you’re on linux, delete the double quotes '' immediately following the -i flag to be compatible with GNU sed.

Editing wp-config.php with sed:

sed -i '' '/\/\* That.s all, stop editing! Happy blogging. \*\// i\
// FX_SCRIPT FS_METHOD \
define( "FS_METHOD", "direct" ); \
\
// FX_SCRIPT WP_DEBUG \
define( "WP_DEBUG", true ); \
define( "WP_DEBUG_LOG", true ); \
\
// FX_SCRIPT DISABLE_WP_CRON \
define( "DISABLE_WP_CRON", true ); \
\
' wp-config.php

The -i option tells sed to edit the file in-place i.e. modify the file directly. Otherwise, sed lives up to its name and streams output to stdout.

The MacOS version of sed requires a backup file to be specified as the first argument whenever the -i option is used. You can pass empty quotes '' to specify no backup file as demonstrated in the example.

The linux version of sed does not require a backup filename to be specified. You can simply delete the '' arguments as noted above.

The way that single and double quotes are used is very important for getting this command to work. Getting them right is one of the trickiest parts about using sed.

Also note how backslashes are used at the end of each line. This is required to make the command portable and universal: classic sed (BSD/Unix/MacOS) does not recognize \n as a placeholder for the newline character while GNU sed (linux) does. The backslashes enable a trick to use actual newlines instead of placeholders.

Finally, my example adds comment lines that start with // FX_SCRIPT before each change (get it? FX = firxworx, the name of this blog!). I do this to make it easy to search with grep and/or visually look for changes in wp-config.php files that were made by my scripts. You may wish to follow a similar practice. This makes it easier to write other scripts that might find and comment out or delete these entries at a later time.

Adding a custom endpoint to WordPress’ REST API to upload files

This guide provides an overview and examples that demonstrate how to create a custom endpoint for WordPress’ REST API that files can be uploaded to.

The particular examples in this post support a CSV file upload but they are easily modified to accept other types of files.

The ability to upload data files in common formats such as CSV, JSON, XLS, etc. is a common requirement found in many business applications. I hope it is useful to you :).

Authentication

This post assumes the file upload functionality is restricted to administrator-level users that have authenticated by logging into the wp-admin dashboard.

No special auth plugins are required to use the REST API because the user is logged in and assumed to be using their web browser to upload the file.

In order to have the REST API accept file uploads from a remote source such as another website or app, a remote script, or a detached front-end user interface implemented with something like React, consider the various alternative authentication methods available for the WP REST API such as OAuth or Application Passwords that can be supported by installing plugins.

File upload form

For the sake of our example, assume we’ve made a plugin that creates an admin panel that features an HTML form with a file upload input. JavaScript will be used to handle the form submission and send the file to the REST API.

Note the file could be sent from anywhere as long as the server (e.g. CORS) and WordPress is configured correctly (e.g. to support remote authentication using something like OAuth).

The form’s file input element might be something like:

<input id="csv" name="file" type="file" accept=".csv" required />

To accept other file formats, add or change the “.csv” value passed to the accept attribute (prop) above.

We also assume our plugin has employed the wp_localize_script() technique to make the following variables available to JavaScript:

  • pluginConfig.restURL — value set to the output of esc_url_raw( rest_url() )
  • pluginConfig.restNonce — value set to the output of wp_create_nonce( 'wp_rest' )

For more information regarding how and why this is done, see the official REST API Handbook.

In short, JavaScript needs to know the URL of where to send the file and the value of a nonce. The nonce is related to a security measure implemented by WordPress that helps discourage CSRF attacks.

The following JavaScript code makes use of the newer FormData API and uses JQuery’s $.ajax() method to POST the CSV to the API endpoint.

The code assumes a standard HTML form with class “import” (<form class="import"...>) and:

  • a file <input /> tag with id="csv" and name="file"
  • a submit <button> (or <input type="submit"... />) with class="csv-submit"

The specific classes and id’s are used to target the form and obtain the values/data from its inputs.

( function($) {

  'use strict';

  $(document).ready( function() {
    $('.import').on('click', '.csv-submit', function(event) {

      event.preventDefault();
      var file = $('input#csv')[0].files[0];
      var formData = new FormData();

      formData.append( 'file', file );

      // append any other formData 
      // e.g. any id fields, etc as necessary here

      $.ajax({
        url: pluginConfig.restURL + '/import/csv',
        data: formData,
        processData: false,
        contentType: false,
        method: 'POST',
        cache: false,
        beforeSend: function ( xhr ) {
          xhr.setRequestHeader( 'X-WP-Nonce', pluginConfig.restNonce );
        }
      })
     .done(function(data) { 
        // handle success 
      })
      .fail(function(jqXHR, textStatus, errorThrown) {
        // handle failure
      });
    });
  });
})(jQuery);

REST API

Moving into the WordPress side of things, we need to add an endpoint to handle the file upload.

There are tons of ways to structure plugins when it comes to WordPress. The following example isolates the REST-related functionality in its own class called MyPluginRestAPI.

It is assumed that our plugin require()‘s the class file, creates an instance of the class, and then calls the instance’s init() method.

It’s up to you to handle errors. For this example, assume that a subclass of PHP’s Exception class named PluginException exists and that it implements a hypothetical restApiErrorResponse() method. Assume the method sends an error response back to the client by calling WordPress’ rest_ensure_response() function with a WP_Error object as its argument.

Note the following uses some PHP7.1+ syntax. Earlier versions of PHP may not support private constants (which are very new at the time of writing) or the square bracket syntax for defining arrays (use array() instead).

class MyPluginRestAPI {

  private const BASE = 'myplugin/v1';

  public function init() {
    add_action( 'rest_api_init', [ $this, 'initRoutes'] );
  }

  public function initRoutes() {
    register_rest_route( self::BASE, '/import/csv', [
            'methods' => [ 'POST' ],
            'callback' => [ $this, 'importCSVPostRequestHandler' ],
            'permission_callback' => [ $this, 'enforceAdminPermissions' ],
            'args' => [
              // ... 
            ]
    ] );
  } 

  public function enforceAdminPermissions() {
    if ( ! ( current_user_can( 'manage_options' ) || current_user_can( 'administrator' ) ) ) {
      return new WP_Error( 'rest_forbidden', esc_html__( 'Private', 'myplugin' ), array( 'status' => 401 ) );
    }
    return true;
  }

  public function importCSVPostRequestHandler( WP_REST_Request $request ) {

    // if you sent any parameters along with the request, you can access them like so:
    // $myParam = $request->get_param('my_param');

    $permittedExtension = 'csv';
    $permittedTypes = ['text/csv', 'text/plain'];

    $files = $request->get_file_params();
    $headers = $request->get_headers();

    if ( !empty( $files ) && !empty( $files['file'] ) ) {
      $file = $files['file'];
    }

    try {
      // smoke/sanity check
      if (! $file ) {
        throw new PluginException( 'Error' );
      }
      // confirm file uploaded via POST
      if (! is_uploaded_file( $file['tmp_name'] ) ) {
        throw new PluginException( 'File upload check failed ');
      }
      // confirm no file errors
      if (! $file['error'] === UPLOAD_ERR_OK ) {
        throw new PluginException( 'Upload error: ' . $file['error'] );
      }
      // confirm extension meets requirements
      $ext = pathinfo( $file['name'], PATHINFO_EXTENSION );
      if ( $ext !== $permittedExtension ) {
        throw new PluginException( 'Invalid extension. ');
      }
      // check type
      $mimeType = mime_content_type($file['tmp_name']);
      if ( !in_array( $file['type'], $permittedTypes )
          || !in_array( $mimeType, $permittedTypes ) ) {
            throw new PluginException( 'Invalid mime type' );
      }
    } catch ( PluginException $pe ) {
      return $pe->restApiErrorResponse( '...' );
    }

    // we've passed our checks, now read and process the file
    $handle = fopen( $file['tmp_name'], 'r' );
    $headerFlag = true;
    while ( ( $data = fgetcsv( $handle, 1000, ',' ) ) !== FALSE ) { // next arg is field delim e.g. "'"
      // skip csv's header row / first iteration of loop
      if ( $headerFlag ) {
        $headerFlag = false;
        continue;
      }
      // process rows in csv body
      if ( $data[0] ) {
        $field1  = sanitize_text_field( $data[0] );
        $field2  = sanitize_text_field( $data[1] );
        // ... 
        // your code here to do something with the data
        // such as put it in the database, write it to a file, send it somewhere, etc. 
        // ...
      }
    }
    fclose( $handle );
    // return any necessary data in the response here
    return rest_ensure_response( ['success' => true] );
  }

}

Note the series of checks to ensure the uploaded file has the right MIME type, correct extension, was uploaded via POST, and was received with no errors. These are important for both security and file integrity.

The last thing you want to do is enable a possible attacker to upload an executable script (such as a PHP file) and then be able to trigger it (e.g. by visiting the direct URL for the file or finding a way to get a user to access the file).

In terms of methods implemented by the class:

  • initRoutes() calls register_rest_route() to register the custom route
  • enforceAdminPermissions() implements a permission callback to ensure the user is an administrator
  • importCSVPostRequestHandler() handles the POST request

Note that the CSV is assumed to have a header row with column labels. The code is easily revised to accommodate cases where there is no header row.

I hope this helps! If you get stuck, please ask your questions in the comments, and remember that I’m available as a consultant to assist with your project 🙂 .

Installing gulp4 with babel to support an ES6 gulpfile

This guide covers installing gulp4 with babel to support ES6 syntax in your gulpfile.

Gulp is a task automation tool that has emerged as one of the standard build tools to automate the web development workflow. Babel is a compiler/transpiler that enables developers to use next-generation ECMAScript syntax (ES6 and beyond) instead of older JavaScript (ES5) syntax.

Gulp4 and ES6+ work together swimmingly to help you write cleaner, easier-to-read, and more maintainable gulpfile’s.

Installing gulp4

At the time of writing, the default gulp package installs gulp 3.x. The following will install and configure gulp4.

Gulp has two key parts: gulp and the gulp-cli command line tools. The idea is that gulp-cli should be installed globally on a developer’s machine while gulp should be installed locally on a per-project basis. This helps ensure compatibility with different versions of gulp that will inevitably arise when maintaining projects of different vintages.

To use gulp4, cli version 2.0 or greater is required. Check the version on your system with:

gulp -v

If the command returns a command not found error, then you probably don’t have gulp installed at all (or at least don’t have it available in your PATH).

If the command outputs a version lower than 2.0, you may need to uninstall any globally-installed gulp (and/or gulp-cli) and then install the current version gulp-cli before proceeding.

To install gulp-cli globally, run ONE of the following commands, depending on your preference of package manager. npm is the classic node package management tool and yarn is a newer tool developed by Facebook that addresses certain shortcomings with npm.

yarn global add gulp-cli
# OR
npm install gulp-cli -g

Test the install by running gulp -v and ensuring the version output is greater than 2.0. Next, install the gulp@next package. The @next part specifies the next-generation gulp4.

Assuming you have already run npm init or yarn init and have a package.json file, execute the following command in your project’s root directory:

yarn add gulp@next --dev
npm install gulp@next --save-dev

Installing babel

yarn add @babel/core --dev
yarn add @babel/preset-env --dev
yarn add @babel/register --dev
# OR 
npm install @babel/core --save-dev
npm install @babel/preset-env --save-dev
npm install @babel/register --save-dev

Next, create a .babelrc file in your project’s root folder and specify the current version of node as the target:

{
  "presets": [
    ["@babel/preset-env", {
      "targets": {
        "node": "current"
      }
    }]
  ]
}

Create your gulpfile

Create your gulpfile with the filename gulpfile.babel.js. The babel.js suffix ensures that babel will be used to process the file.

The following example demonstrates a few ES6+ features: optional semicolons, import statements, and the “fat arrow” syntax for defining functions:

'use strict'

import gulp from 'gulp'

gulp.task('task-name', () => {
  // example 
  return gulp.src('/path/to/src')
    .pipe(gulp.dest('/path/to/dest'))
})

Gulp4 features a new task execution system that introduces the functions gulp.series() and gulp.parallel() that can execute gulp tasks in either series (one-after-another) or parallel (at the same time). This makes a lot of workflows much easier to define vs. previous versions!

Another nice feature is that gulp4 supports returning a child process to signal task completion. This makes it cleaner to execute commands within a gulp task, which can help with build and deployment related tasks.

The following example defines a default build task that runs two functions/tasks in series using gulp.series(). The build task is defined using the ES6 const keyword and exported as the default function/task for the gulpfile. The example doSomething() and doAnotherThing() functions/tasks are also exported.

'use strict'

import gulp from 'gulp'

export function doSomething {
  // example 
  return gulp.src('/path/to/src')
    .pipe(gulp.dest('/path/to/dest'))
})

export function doAnotherThing {
  // example 
  return gulp.src('/path/to/src')
    .pipe(gulp.dest('/path/to/dest'))
})

const build = gulp.series(doSomething, doAnotherThing)

export default build

Set up MacOS’ built-in Apache + PHP as a LAMP/WordPress Dev Environment

This guide covers configuring MacOS High Sierra, Mojave, and beyond as a local development server for LAMP-stack projects. This is a high-performance option for running projects locally on a Mac and is useful for PHP and WordPress development.

At the end of this guide, you will be able to create new folders within a special virtual hosts directory such as ‘example.com/’ with the sub-folder ‘public_html/’ and the ‘http://example.com.test’ will automatically be available with apache serving files out of the ‘public_html/’ folder.

Apple already bundles many of the necessary pieces with MacOS including apache and PHP. This guide covers configuring them to support WordPress, as well as installing other dependencies such as mysql/mariadb, and useful tools like sequel-pro and wp-cli.

Alternatives

Self-contained solutions for running a LAMP environment on a Mac can be found in products like MAMP and the free XAMPP. They double up what’s already on your Mac but do have some features that you might find appealing.

Virtual Machines are another option. Tools such as Vagrant are great and extremely useful for more complex projects where customized server configuration(s) are required. However, spinning up an entire virtual server with its own CPU, RAM and disk allocation simply to run WordPress is a huge tax on system resources. It’s often a better idea to reserve VM tools for better-suited applications.

A lighter-weight alternative to full VM’s is container virtualization such as with Docker. This brings its own configuration and deployment challenges that may introduce unnecessary effort if Docker doesn’t play a greater role in a given project.

Pre-checks

Confirm your system’s apache version with the command apachectl -v, and its PHP version with the command php -v.

MacOS Sierra and beyond should be running apache 2.4+ and PHP 7+.

Apache configuration

Apple’s default configuration requires several changes to create a suitable development environment.

Backup httpd.conf

First, make a backup copy of apache’s httpd.conf file:

sudo cp /etc/apache2/httpd.conf /etc/apache2/httpd.conf.pre-dev-env

Apache httpd.conf configuration

Open httpd.conf using a text editor such as nano:

sudo nano /etc/apache2/httpd.conf

Each of the following lines are found at different locations within the file. Find each one of them and ensure they are uncommented — delete any preceding # character so they start with a letter instead:

#LoadModule vhost_alias_module libexec/apache2/mod_vhost_alias.so
#LoadModule rewrite_module libexec/apache2/mod_rewrite.so
#LoadModule php7_module libexec/apache2/libphp7.so
#Include /private/etc/apache2/extra/httpd-vhosts.conf

If you’re using the nano editor, its “where” Control+W feature can help you locate each line. It works similar to the “find” command (Control+F) found in most web browsers and word processors.

When you’re done, use Control+O to save (output) your changes and Control+X to exit nano.

The above changes have:

  • enabled the module mod_vhost_alias to enable dynamic virtual hosts;
  • enabled the module mod_rewrite which WordPress uses to rewrite URL paths;
  • enabled PHP7 support so WordPress’ php scripts can execute; and
  • included an extra conf file where we will define a dynamic apache virtualhost for your projects.

Apache dynamic virtual host configuration

In the previous step we referenced a httpd-vhosts.conf file in httpd.conf.

Create a backup of the original file:

sudo cp /private/etc/apache2/extra/httpd-vhosts.conf /private/etc/apache2/extra/httpd-vhosts.conf.pre-dev-env

Now edit that file:

sudo nano /private/etc/apache2/extra/httpd-vhosts.conf

This file is populated by default with a couple of example virtual host definitions: blocks beginning with <VirtualHost *:80> and ending with </VirtualHost>.

Comment out (i.e. prefix each line with #) or delete the two example virtual host definitions.

Next, add your own dynamic virtual host definition to the file.

I’ve decided to use /usr/local/var/www as the base folder to serve projects from. You can modify this to another path if you’d like.

Add the following block:

# Custom configuration for local dev environment

<Directory "/usr/local/var/www">
  Options Indexes MultiViews FollowSymLinks
  AllowOverride All
  Require all granted
</Directory>

<Virtualhost _default_:80>

  # Dynamic virtual hosts for local development
  UseCanonicalName Off
  VirtualDocumentRoot "/usr/local/var/www/%-2+/public_html"

  # Show 404's in the apache error log
  LogLevel info

</Virtualhost>

Save (Control+O) and exit (Control+X) the nano editor.

Explaining the Dynamic Virtual Host definition

In httpd-vhosts.conf as modified above, the <Directory> block is required for Apache 2.4+ and tells Apache that it can serve files from the /usr/local/var/www folder. This explicitly overrides the rigorous security-focused default specified in httpd.conf that tells Apache it can’t serve up any file on the hard disk (/).

The <VirtualHost> block defines a _default_ VirtualHost that listens on port 80.

The VirtualDocumentRoot directive tells Apache where to look for the dynamic VirtualHost’s files. The %-2+ placeholder stands in for “the penultimate (second to last) and all preceding parts” of the server name in the request. Therefore, per the directive, when handling a request for http://example.com.test, Apache will look for that site’s files in /usr/local/var/www/example.com/public_html.

I like to have my project folder names in dev environments mirror my likely production values, such as example.com/ or hypothetical.app/. You may wish to name your project folders differently. You may prefer to use the %0 placeholder for the entire server name (example.com.test), or %1 for the first-part only (example).

To learn more about these directives, refer to the docs:

Create the required paths

Finally, ensure the paths that you specified for your project folders exist. In my case:

mkdir -pv /usr/local/var/www

If you have any project build folders, you can add them (or better yet: create symlinks to them) here!

To give you an idea of how this works:

In /usr/local/var/www if you were to create the folder helloworld.com/ and sub-folder helloworld.com/public_html, and put a basic index.php containing hello world! inside it, at the end of this tutorial Apache will serve it up when you visit http://helloworld.com.test on your machine.

Smoke-check (config check)

As a smoke-check to make sure you haven’t made any typos or introduced any bugs, run the following command to check the syntax of your apache config files:

sudo apachectl configtest

If you followed all of the above steps carefully, it should output “Syntax OK”.

If everything checks out, restart apache so your latest conf changes can take effect:

sudo apachectl restart

Configuring DNS for local domain names

Now we will solve the problem of directing local requests for http://example.com.test to Apache.

One option is to manually add an entry to /private/etc/hosts for every one of your projects. For example, you might add the line 127.0.0.1 myproject.test to enable the local URL ‘http://myproject.test’.

We are going to use a more dynamic approach using a DNS server so that all requests for any URL at the *.test domain will resolve to localhost.

Any DNS server could be employed for this job. We will use dnsmasq, a perfect choice for this type of lightweight local routing.

Follow the steps in my guide Using dnsmasq on MacOS to setup a local domain for development to complete this step.

Customizing PHP settings (php.ini)

MacOS’ PHP uses a default php.ini file based on /private/etc/php.ini.default.

To customize your PHP environment, if a php.ini file doesn’t already exist at /private/etc/php.ini, copy the default template to create a main php.ini file:

sudo cp /private/etc/php.ini.default /private/etc/php.ini

Make any changes you wish to php.ini and restart apache to reload all configuration files:

sudo apachectl restart

If you were to run phpinfo() in a php file from the web server, you should now see that the Loaded Configuration File property now has the value /etc/php.ini.

A very common tweak to the default PHP configuration is to allow larger file upload sizes. The post_max_size and upload_max_filesize properties are only a few megs by default. These limits can be raised as you see fit.

Many developers also tweak the max_execution_time, max_input_time, and memory_limit settings depending on their project.

Always remember to restart apache after making changes to your PHP configuration.

Installing and configuring mysql server

WordPress and many other PHP apps require a MySQL or compatible database such as MariaDB to store its data.

I prefer to use mariadb, which is available via brew. Run the following command to install it for your user:

brew install mariadb

This installation method is recommended in the official docs:

Following installation, manually start the server by running:

mysql.server start

Running mysql as a service

By default you will need to manually start and stop your mysql/mariadb server.

If you want the server to start when you login to your Mac, use Homebrew’s services feature to have it automatically launch in the background:

brew services start mariadb

Homebrew integrates with MacOS’ launchctl to make this happen the correct way as supported by Apple.

Test logging into mysql via the cli client

Once your server has started, you can test logging in as the root user with the Terminal command:

mysql -u root

Issue the quit command to exit the MySQL prompt and return to your Terminal session.

Set a root password and improve security

Even though a default mysql/mariadb configuration only accepts connections from localhost, it is still wise to run the quick interactive mysql_secure_installation command-line-interface script to set a root password and a few other security-minded options.

Troubleshooting: resolve possible socket incompatibility

There may be a slight configuration mismatch between the defaults of brew’s mysql/mariadb packages and the defaults of MacOS’ built-in PHP mysqli extension. It may be fixed in newer packages.

If try to get a LAMP app such as WordPress to connect to mysql on localhost and encounter a “Connection Refused” error even when the database settings are confirmed correct (as defined in wp-config.php for WordPress), then you may need to make a small fix. You can confirm the issue by trying to connect to 127.0.0.1 instead: if it works but localhost doesn’t then it is a socket-related issue. 

For localhost (socket) connections to the database, brew’s package version tends to use the socket file /tmp/mysql.sock. MacOS’ built-in Apache+PHP mysqli extension assumes the socket file is at /var/mysql/mysql.sock (i.e. the default setting of its mysqli.default_socket property). You can verify the value of this property by running phpinfo(); in a php script and finding it in the list.

To resolve, one could edit php.ini or another conf file. A simple solution is to create a symlink at /private/var/mysql/mysql.sock that points to /tmp/mysql.sock so everything works out:

sudo mkdir -p /var/mysql
sudo ln -s /tmp/mysql.sock /private/var/mysql/mysql.sock

On MacOS /var, a classic folder for unix/linux systems, is symlinked to /private/var so to avoid a symlink-on-symlink type situation, the above uses the real path. 

Note that only socket connections (i.e. database connections made to ‘localhost’) would be impacted by this incompatibility. Connections to ‘127.0.0.1’ force connecting over TCP due to the IP address, bypassing any socket concerns.

Install a database management GUI

If you’d like to use a GUI to help manage your mysql databases, one popular choice is sequel-pro. This serves the same purpose as the popular phpmyadmin webapp except it runs as a desktop app on your Mac. To install it:

brew cask install sequel-pro

You can now launch it from your Applications folder.

Install wp-cli

WordPress’ Command-Line-Tools are a huge timesaver for managing both local and remote WordPress installs, and they are essential for writing scripts that automate WordPress deployment, updates, etc.

Install wp-cli for your user with the command:

brew install wp-cli

Confirm your install by running wp --version in Terminal.

Assuming your project folder includes a public_html/ folder that contains your WordPress install, and that you have created a database called “DB_NAME”, you can download and setup WordPress with a handful of wp-cli commands:

cd /Users/username/web-projects/my-project.com/public_html
wp core download
wp config create --dbname=DB_NAME --dbuser=DB_USERNAME --dbpass=DB_PASSWORD --dbhost=localhost
wp core install --url=my-project.com --title="My Project" --admin_name="example_admin" --admin_password="example_password" --admin_email=you@example.com

Check out the command reference and docs/handbook at https://wp-cli.org/

Serving up your project

To make a project available at http://example.com.test you need to:

  • create a folder example.com/ in your chosen base directory (e.g. /usr/local/var/www)
  • create the public_html subfolder: example.com/public_html/
  • copy project files to public_html/ (manually or as an automated task e.g. with gulp)

OR

  • create a symlink from your project location to the chosen base directory

For the symlink option:

ln -s ~/some/project/folder/example.com /usr/local/var/www/example.com

Be sure to replicate the requirement for a public_html/ sub-folder.

When creating symlinks from elsewhere on your system to /usr/local/var/www be careful of permissions issues!

Managing permissions

MacOS’ Apache is running as the user _www by default. That user must have at least read permissions for any folder that you symlink to in order to serve up your project files.

It’s important to note that ~/Documents and any folders created within are not accessible to any group or other users. Apple thankfully assumes that you don’t want other users on a system to be able to access your private documents! As a result, Apache’s _www user will not be able to follow symlinks into the Documents folder belonging to your user.

If you have any web projects in ~/Documents you could save them somewhere else where Apache can read them and symlink to them from there, or you could simply copy your project files over to /usr/local/var/www.

Depending on your desired setup, you may find it convenient to change the group ownership of /usr/local/var/www to _www and set the groupID bit (setgid):

sudo chgrp _www /usr/local/var/www
sudo chmod g+s /usr/local/var/www

The setgid bit ensures that any new files or folders created underneath www/ will inherit the group ID of the directory vs. the default behaviour of having it set to the primary group ID of the user that created the file. This measure can help ensure that Apache’s _www user can access the contents of the www/ folder.

Finally, remember that our particular Apache VirtualHost configuration requires a public_html/ sub-directory under the my-project.com folder for web-servable files. To illustrate: an index.php file in ~/web-projects/my-project.com/public_html will be served up to a local web browser that requests the URL http://my-project.com.test. This works because there is a symlink in /usr/local/var/www named my-project.com that points to the folder ~/web-projects/my-project.com.

Working on a project

In the future, if you haven’t setup apache and mysql server to run automatically as services, you will need to start them:

sudo apachectl start
mysql.server start

To stop or restart apache, use the following commands:

sudo apachectl stop
sudo apachectl restart

You can stop mysql with:

mysql.server stop

Remember: if you make any changes to your apache config, such as adding a new VirtualHost, you will need to restart or reload apache for your changes to take effect.

Finally remember that the settings covered in this guide will serve up your projects to any clients on your network. If you want to hide them from everyone on your local coffee shop’s shared wifi, turn off the servers, harden your firewall (via System Preferences), and/or tweak your Apache conf to only serve to localhost.

Upgrading MacOS

When you upgrade MacOS, the installer will create versions of your conf files with the suffix ~previous before replacing with its own.

You can also create your own backup copies to be safe.

Following an OS update, if the Apache files are changed, you can compare the new versions with your ~previous versions using the diff command to make sure there’s no major updates (very unlikely) then restore your customized versions. After restoring your files, remember to restart apache for the settings to take effect: sudo apachectl restart.

The files customized by following this guide are:

  • /etc/apache2/httpd.conf
  • /private/etc/apache2/extra/httpd-vhosts.conf

Script to Deploy WordPress

Check out this gist for a bash script to quickly deploy a fresh WordPress install:

https://gist.github.com/firxworx/bb9b7f8d71915f9d4e7f8d3a8a531b26

Using dnsmasq on MacOS to setup a local domain for development

This guide covers using dnsmasq as a local DNS server on MacOS to resolve all URL’s with the .test domain name to localhost (127.0.0.1).

This can be very useful to developers wishing to setup an efficient local development and test environment on their machine. For example, if http://myproject.com.test resolved to localhost, a local apache, nginx, or other server could respond to the request.

While any DNS server could be employed for this job, dnsmasq is a perfect choice for this type of lightweight local routing.

Regarding the .test domain, note that this is a recommended choice per the IETF RFC’s (e.g. RFC-2606, RFC-6761). While it’s possible to configure dnsmasq for any arbitrary domain name, only .test, .example, .invalid, and .localhost are reserved for non-Internet use. A popular choice used to be .dev until Google purchased the top-level-domain (TLD) and changed how Chrome handles .dev URL’s.

Install and configure dnsmasq

To get started, use brew to install dnsmasq for your user:

brew install dnsmasq

At the time of writing, the brew package installer creates a sample conf file with all entries commented out. It can be found at ./etc/dnsmasq.conf under brew’s default install folder.

Execute following commands in sequence to configure dnsmasq to resolve *.test requests to 127.0.0.1:

echo "" >> $(brew --prefix)/etc/dnsmasq.conf
echo "# Local development server (custom setting)" >> $(brew --prefix)/etc/dnsmasq.conf
echo 'address=/.test/127.0.0.1' >> $(brew --prefix)/etc/dnsmasq.conf
echo 'port=53' >> $(brew --prefix)/etc/dnsmasq.conf

The above commands simply append the required lines to the dnsmasq.conf configuration file.

Next, use brew’s services feature to start dnsmasq as a service. Use sudo to ensure that it is started when your Mac boots, otherwise it will only start after you login:

sudo brew services start dnsmasq

Add a resolver

Add a resolver for .test TLD’s:

sudo mkdir -pv /etc/resolver
sudo bash -c 'echo "nameserver 127.0.0.1" > /etc/resolver/test'

This tells your system to use localhost (and therefore dnsmasq) as the DNS resolver for the .test domain.

Test your configuration

Flush your DNS cache:

sudo dscacheutil -flushcache
sudo killall -HUP mDNSResponder

Check your configuration by running:

scutil --dns

Inspect the output. One of the entries should specify that for domain test the system will use the nameserver 127.0.0.1. This means that dnsmasq will now be consulted as the nameserver for any .test URLs.

Finally, test the full stack by trying to ping random *.test URL’s such as example.test or myproject.test and confirming that they resolve to 127.0.0.1. For example:

ping -c2 example.test

Provides the output:

PING example.test (127.0.0.1): 56 data bytes
64 bytes from 127.0.0.1: icmp_seq=0 ttl=64 time=0.022 ms
64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.055 ms

OR (in cases where your Mac’s firewall settings are configured to not respond to ping):

PING example.test (127.0.0.1): 56 data bytes
Request timeout for icmp_seq 0
...

The important part to confirm is that example.test resolved to (127.0.0.1).

Using the WordPress REST API with JWT Authentication

The WordPress core now supports a new REST API as of version 4.8.x. Among a sea of new possibilities, one can now build a front-end for a website or app with a framework like React or Vue.js and use WordPress and its familiar admin dashboard to manage the back-end.

This guide covers adding JSON Web Tokens (JWT) authentication support with the JWT Authentication for WP REST API plugin, and sending requests to the API using Postman.

To access the REST interface of a WordPress-powered site append /wp-json/wp/v2/ to the URL. For example, this blog’s REST API is at: https://firxworx.com/wp-json/wp/v2/

Resources:

REST API Authentication

Default cookie authentication

WordPress’ REST API only supports cookie authentication out-of-the-box. This is the same method that WordPress uses by default to authenticate users that use the login form. The idea is that theme and plugin developers can authenticate themselves, write javascript with the JS API, and be on their merry way.

Enabling remote applications with Basic Auth, OAuth, and/or JWT

To support remote applications, we need to add a new REST API authentication method using a plugin. Currently supported options are Basic Auth, OAuth, and JWT:

  • Basic Auth with a username and password is considered insecure and should only be used in development scenarios
  • OAuth is great but it can be a pain to authenticate
  • JWT is awesome and works great with front-end frameworks

JSON Web Tokens are an open industry standard: IETF RFC 7519

Adding JWT Authentication to the REST API

If your WordPress is accessible via the Internet, it is important to enable SSL/https before proceeding.

Start by installing the JWT Authentication for WP REST API plugin but don’t activate it just yet.

Next, ensure your web server supports the HTTP Authorization Header. If you are using a shared host, this is often disabled by default. To enable it, add the following to your WordPress’ .htaccess file:

RewriteEngine on
RewriteCond %{HTTP:Authorization} ^(.*)
RewriteRule ^(.*) - [E=HTTP_AUTHORIZATION:%1]

Edit your wp-config.php file and add the following lines before the comment that says “That’s all, stop editing!”:

define('JWT_AUTH_SECRET_KEY', 'really-secret-key-here');
define('JWT_AUTH_CORS_ENABLE', true);

Change really-secret-key-here in the above to a random string. If you need help obtaining some randomness, copy-and-paste some output from WordPress.org’s secret key service: https://api.wordpress.org/secret-key/1.1/salt/

JWT uses the secret JWT_AUTH_SECRET_KEY to sign JSON Web Tokens. If it is compromised, then your site’s security is compromised! Keep it safe!

The JWT_AUTH_CORS_ENABLE line activates CORs to enhance security. Refer to the plugin docs if you need to modify the default available headers: ‘Access-Control-Allow-Headers, Content-Type, Authorization’

Finally, activate the JWT Authentication for WP REST API plugin!

New endpoints for JWT authentication

The plugin will add the /jwt-auth/v1 namespace with two new endpoints:

  1. /wp-json/jwt-auth/v1/token (POST)
  2. /wp-json/jwt-auth/v1/token/validate (POST)

The first endpoint accepts POST requests that contain a username and password in the body.

  • If the credentials check out, the plugin will issue a 200 response containing a JSON object with: token, user_display_name, user_email and user_nicename.
  • If authentication fails, the response includes an object with the properties: code, data, and message.

The token included in the API’s response can then be included in the HTTP Authorization header of any subsequent requests to WordPress’ REST API. Front-end applications will need to store it somewhere, such as in a cookie or localstorage.

The second endpoint simply validates tokens. If sent a POST request with a valid token in the HTTP Auth header, it will return the following response:

{
  "code": "jwt_auth_valid_token",
  "data": {
    "status": 200
  }
}

You can confirm that JWT is now available by accessing https://yoursite.com/wp-json/ and inspecting the response body to see if the /jwt-auth/v1 and /jwt-auth/v1/token endpoints are available.

Sending requests to the REST API

Unauthenticated request

Start by making a request to the REST API that doesn’t require authentication. This “smoke check” helps confirm that the API is working and that your site’s permalinks are configured correctly.

Open up Postman to send a new GET request to https://your-site.com/wp-json/wp/v2/posts.

You should expect a successful response with a body that contains a JSON representation of your blog’s posts.

If you need to troubleshoot, check your permalink settings in the admin at: Settings -> Permalinks -> Common Settings. Make sure that your server supports URL rewriting (e.g. mod_rewrite for Apache) and that the API’s URL’s resolve correctly.

Authenticating with JWT

In Postman, prepare a new POST request to the jwt-auth/v1/token endpoint (e.g. https://your-site.com/wp-json/jwt-auth/v1/token). Navigate to the Body tab and:

  • ensure the type of request is form-data
  • specify the username and password fields with valid user credentials
  • Fire the request.

If authentication is successful, you’ll get a reply like the following (note: I truncated the token in my example so expect a much longer string).

{
    "token": "eyJ0eXAiOiJKV1QiLC_A_RIDICULOUSLY_LONG_STRING_Ky4Y",
    "user_email": "example@firxworx.com",
    "user_nicename": "example",
    "user_display_name": "Mr Example"
}

Save the token so that you can include it in subsequent requests to the API.

Using the JWT token in subsequent requests

Now send an API request where authentication is required: creating a post.

In Postman, prepare a new POST request to the URL https://your-site.com/wp-json/wp/v2/posts and set the following:

  • in the Headers tab, add the following key value pairs:
    • Content-Type + application/json
    • accept + application/json
    • Authorization + Bearer YOUR_TOKEN_HERE
  • in the Body tab, add the following key value pairs:
    • title + ‘your post title’
    • content + ‘your post body’
    • status + draft (or publish if you prefer)

The crucial entry is the Authorization header. It is imperative that the token be prefixed with the string Bearer followed by a space. Do not forget the space character!

A successful response body will include a JSON object with the new post id, date, and a bunch of other data about your new post.

If you get an error response, double check all of the key value pairs in the Headers and Body. Ensure that you copied your token exactly and in its entirety. You can use the JWT endpoints to validate your token or you could try re-authenticating to obtain a fresh token.

The following example shows how to create a post via the REST API using JS/ES with the fetch() API:

var token = YOUR_TOKEN_HERE;
fetch('https://your-site/wp-json/wp/v2/posts', {
    method: "POST",
    headers: {
        'Content-Type': 'application/json',
        'Accept': 'application/json',
        'Authorization': 'Bearer ' + token
    },
    body: JSON.stringify({
        title: 'Lorem ipsum',
        content: 'Lorem ipsum dolor sit amet.',
        status: 'draft'
    })
}).then(function(response){
    return response.json();
}).then(function(post){
    console.log(post);
});

Troubleshooting

If you find you can’t make authenticated requests, such as when you try to create a post, you may have an issue with the request’s Authorization header not being passed to PHP.

It turns out that many shared web hosts disable the Authorization header by default on Apache (or Apache-like) web servers. This can also be related to using fcgi_module instead of php5_module to support PHP.

A solution is to add the following directive to the top of your WordPress’ .htaccess file. If you are not on a shared hosting environment, other possible locations for this directive are in a VirtualHost configuration or apache server conf:

SetEnvIf Authorization "(.*)" HTTP_AUTHORIZATION=$1

Note that some commenters have reported that the lowercase authorization worked for them vs. Authorization, so if you experience an issue, try the alternate capitalization.

This directive tells Apache to pass the Authorization header to php by setting an environment variable, such that it is available in PHP as $_SERVER['HTTP_AUTHORIZATION'].

If you update a conf file rather than your .htaccess file, don’t forget to restart apache for the setting to take effect!

If that approach fails, another solution to try is adding the following to the top of the .htaccess file in your WordPress’ root folder:

RewriteEngine on
RewriteCond %{HTTP:Authorization} ^(.*)
RewriteRule ^(.*) - [E=HTTP_AUTHORIZATION:%1]

This approach relies on mod_rewrite and may not be as appropriate for some cases because the $_SERVER['REDIRECT_HTTP_AUTHORIZATION'] variable is set for PHP instead of the standard $_SERVER['HTTP_AUTHORIZATION'].

Creating Custom Post Types in WordPress

This guide covers how to create Custom Post Types (CPT’s) in WordPress. CPT’s are important to WordPress developers because they enable the creation of more complex sites + web-apps than is possible with a default WordPress install.

Custom Post Types are frequently defined with additional data fields called meta fields that can be defined and made editable to admins via meta boxes.

Example applications:

  • Jokes — each post contains a joke, which are listed and displayed differently than regular blog posts
  • Job Opportunities — include Salary Range and Location meta
  • Car Listings — registered users post for-sale listings and specify Make, Model, and Year meta via dynamic dropdown menus
  • Beer Reviews — featuring a range of meta fields that include Brewery, Style, and Tasting Score

Custom Post Types can be created (registered) or modified by calling the register_post_type() function within the init action.

Custom Taxonomies and the connection to Custom Post Types

Custom Post Types are closely related to the concept of Custom Taxonomies. Taxonomies are a way to group WordPress objects such as Posts by a certain classification criteria. Developers can define Custom Taxonomies to add to WordPress’ default taxonomies: Categories, Tags, Link Categories, and Post Formats.

Although this guide focuses on CPT’s, its important to note that projects are often implemented using a thoughtful combination of Custom Post Types and Custom Taxonomies.

A classic example of a complementary post type + taxonomy is: Book as a Custom Post Type and Publisher as a Custom Taxonomy.

If your Custom Post Type needs to be related to any Custom Taxonomies, they must be identified via the optional taxonomies argument of the register_post_type() function. This argument only informs WordPress of the relation and does not register any taxonomies as a side-effect. Custom Taxonomies must be registered on their own via WordPress’ register_taxonomy() function.

Registering new Custom Post Types

Registering in a Plugin vs. Theme

Custom Post Types can be registered by plugins or by themes via their functions.php file. It’s generally recommended to go the plugin route to keep a project de-coupled from any particular theme.

In the many cases where CPT’s do not depend on activation or deactivation hooks, they can be defined by a Must-Use Plugin (mu-plugin). This special type of plugin is useful to safeguard against admins (e.g. client stakeholders with admin access) accidentally de-activating any Custom Post Types that are important to their website/app.

If a plugin or theme that registers a CPT becomes deactivated, WordPress’ default behaviour is to preserve the post data in its database, though it will become inaccessible and could break any themes or plugins that assume the CPT exists. The CPT will be restored once whatever plugin or theme that registered it is re-activated.

Basic Definition

Custom Post Types may be registered by calling WordPress’ register_post_type() function during the init action with the following arguments: a required one-word post type key, and an optional array of key => value pairs that specify all optional arguments.

The following example implements a function create_my_new_post_type() that calls register_post_type() to register a CPT called candy. The last line hooks the function to the init action using WordPress’ add_action() function. It could be included as part of a plugin or in a theme’s functions.php.

Some of the most common optional args are specified: user-facing labels for singular and plural, if the CPT is to be public (appear in search, nav, etc) or not, and whether it should have an archive (list of posts) or not.

function create_my_new_post_type() {
    register_post_type( 'candy',
        [
            'labels' => [
                'name' => __( 'Candies' ),
                'singular_name' => __( 'Candy' )
            ],
        'public' => true,
        'has_archive' => true,
        ]
    );
}
add_action( 'init', 'create_my_new_post_type' );

Tip: Namespacing

It is a good practice to namespace any CPT keys by prefixing their names with a few characters relevant to you or your project followed by an underscore, such as xx_candy. This helps avoid naming conflicts with other plugins or themes, and is particularly important if you are planning to distribute your project.

Tip: Use singular form for post type keys

The WordPress codex and Handbooks always use a singular form for post type keys by convention, and WordPress’ default types such as ‘post’ and ‘page’ are singular as well.

Detailed Definition

There are a ton of optional arguments that can be specified when registering a Custom Post Type. The WordPress Developer Documentation is the best source to review all of them: register_post_type().

Some of the more notable options include:

  • labels — array of key => value pairs that correspond to different labels. There are a ton of possible labels but the most commonly specified are ‘name’ (plural) and ‘singular_name’
  • public — boolean indicating if the post type is to be public (shown in search, etc) or not (default: false)
  • has_archive — boolean indicating if an archive (list of posts) view should exist for this post type or not (default: true)
  • supports — array of WordPress core feature(s) to be supported by the post type. Options include ‘title’, ‘editor’, ‘comments’, ‘revisions’, ‘trackbacks’, ‘author’, ‘excerpt’, ‘page-attributes’, ‘thumbnail’, ‘custom-fields’, and ‘post-formats’. The ‘revisions’ option indicates whether the post type will store revisions, and ‘comments’ indicates whether the comments count will show on the edit screen. The default value is an array containing ‘title’ and ‘editor’.
  • register_meta_box_cb — string name of a callback function that will handle creating meta boxes for the CPT so admins have an interface to input meta data
  • taxonomies — an array of string taxonomy identifiers to register with the post type
  • hierarchical — a boolean value that specifies if the CPT behaves more like pages (which can have parent/child relationships) or like posts (which don’t)

The numerous other options enable you to manage rewrite rules (e.g. specify different URL slugs), configure options related to the REST API, and set capabilities as part of managing user permissions.

Adding Meta Fields to a Custom Post Type

Enabling custom-fields

A straightforward way to enable admins to define meta fields as key->value pairs when editing a post is to include the value ‘custom-fields’ in the ‘supports’ array, as part of the args passed to register_post_type().

Adding Meta Boxes to a Custom Post Type

The above ‘custom-fields’ approach works for basic use-cases, however most projects require advanced inputs like dropdown menus, date pickers, repeating fields, etc. and a certain level of data validation.

The solution is defining meta boxes that specify inputs for each of a CPT’s meta fields and handle the validation and save process. Meta boxes must be implemented in a function whose name is passed to register_post_type() via its args as a value of the ‘register_meta_box_cb’ option.

Creating meta boxes can be tricky for the uninitiated… Stay tuned for an upcoming post dedicated solely to them!

In the meantime, I would suggest exploring solutions that simplify the process of creating meta boxes. Two excellent options are the open-source CMB2 (Custom Meta-Box 2) and Advanced Custom Fields (ACF), which offers both free and commercial options. I think the commercial ACF PRO version is well worth the $100 AUD fee to license it for unlimited sites including a lifetime of updates and upgrades.

Displaying a Custom Post Type

Posts belonging to a CPT can be displayed using single and archive templates, and can be queried using the WP_Query object.

Single template: single post view

Single templates present a single post and its content. WordPress looks for the template file single-post_type_name.php for a CPT-specific template and if it doesn’t find it, it defaults to the standard single.php template.

Archive template: list of posts view

Archive templates present lists of posts. A Custom Post Type will have an Archive if it was registered with the optional has_archive argument set to a value of true (default: false).

To create an archive template for your CPT, create a template file that follows the convention: archive-post_type_name.php. If WordPress doesn’t find this file, it defaults to the standard archive.php template.

Using the WP_Query object

WP_Query can be used in widget definitions, in templates, etc. to present posts belonging to a CPT. The following example queries for published posts of the type ‘candy’ and then loops over the results, presenting each one’s title and content as items in a list.

<?php

$args = [
  'post_type'   => 'candy',
  'post_status' => 'publish',
  ]);

$candies = new WP_Query( $args );
if( $candies->have_posts() ) :
?>
  <ul>
    <?php
      while( $candies->have_posts() ) :
        $candies->the_post();
        ?>
          <li><?php printf( '%1$s - %2$s', get_the_title(), get_the_content() );  ?></li>
        <?php
      endwhile;
      wp_reset_postdata();
    ?>
  </ul>
<?php
else :
  esc_html_e( 'No candies... Go get some candy!', 'text-domain' );
endif;
?>

The wp_reset_postdata() call is important to reset WordPress back to the original loop, so other functions that depend on it will work properly. Reference: https://developer.wordpress.org/reference/functions/wp_reset_postdata/

Include the client IP in apache logs for servers behind a load balancer

When servers are behind a load balancer, Apache’s default configuration produces logs that show the load balancer’s IP address instead of the IP of the remote client that initiated the request. Furthermore, if multiple server’s logs are consolidated into one, it can be difficult to determine which server created a given log entry.

This post covers how to improve on Apache’s default logging situation such that:

  • every web server behind the load balancer includes a unique identifier for itself in its log entries, and that;
  • access log entries include the client’s remote IP address as found in the X-Forwarded-For header set by the load balancer.

This setup is more helpful for troubleshooting, configuring monitoring and alerts, etc. and it helps to maximize the value of 3rd-party log aggregation and analysis services like Papertrail or Loggly).

The example commands in this post are applicable to Ubuntu/Debian however they are easily adapted to other environments.

Enable required modules

Start by ensuring that the required Apache modules: env and remoteip, are enabled:

a2enmod env
a2enmod remoteip 
service apache2 restart

Identify each web server

Add a SetEnv directive in the site/app’s apache conf to instruct the env module to set a new environment variable with a value that uniquely identifies each server behind the load balancer.

The example below is a snippet from an Apache VirtualHost’s conf file. It defines a variable called APP_LB_WORKER with the value ‘unique_identifier’. If you were using a devops automation tool such as Ansible, you could use the template module and use a handy variable such as {{ ansible_host }} in place of the example’s hard-coded ‘unique_identifier’ value.

<VirtualHost *:443>
...
    SetEnv APP_LB_WORKER unique_identifier
...
</VirtualHost>

Configure the Apache RemoteIP module

Create a file named /etc/apache2/conf-available/remoteip.conf and use it to set: the RemoteIPHeader to ‘X-Forwarded-For’, any appropriate RemoteIPInternalProxy or other RemoteIP directives, and to define a new LogFormat that includes both the environment variable that contains the server’s identifier and the client’s remote IP as sourced from the X-Forwarded-For header.

The RemoteIPInternalProxy directive tells the RemoteIP module which IP address(es) or IP address blocks it can trust to provide a valid RemoteIPHeader that contains the client’s IP.

The following example’s RemoteIPInternalProxy value is representative of an environment where the load balancer’s internal network IP address belongs to a public subnet with the CIDR block 10.0.1.0/24. Choose an appropriate value (or values) for your environment.

A full list of configuration directives for RemoteIP can be found in the Apache docs: https://httpd.apache.org/docs/2.4/mod/mod_remoteip.html.

The following example names the new LogFormat as “loadbalance_combined”. You can choose any name you like that isn’t already in use.

/etc/apache2/conf-available/remoteip.conf:

RemoteIPHeader X-Forwarded-For
RemoteIPInternalProxy 10.0.1.0/24

LogFormat "%a %v %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" \"%{APP_LB_WORKER}e\"" loadbalance_combined

Finally, enable the conf:

a2enconf /etc/apache2/conf-available/remoteip.conf

The Apache LogFormat is extensively customizable. Learn more about the placeholders and options from here: https://httpd.apache.org/docs/2.4/logs.html. Remote log aggregation service Loggly also has an excellent overview: https://www.loggly.com/ultimate-guide/apache-logging-basics/.

Tell Apache to use the new LogFormat

Next, tell Apache to use the custom LogFormat that we named loadbalance_combined by editing your site/app’s conf file. The following example builds upon the previous example of a VirtualHost conf:

<VirtualHost *:443>
...
    SetEnv APP_LB_WORKER unique_identifier
...
    CustomLog /var/log/app_name/access.log loadbalance_combined
...
</VirtualHost>

The following example is a more elaborate case that uses the tee command to send the log entry to both an access.log file and to the /usr/bin/logger command to include it in the syslog, only if an environment variable named “dontlog” is not set. An example use-case for an env variable like “dontlog” is to set it (e.g. via the SetEnvIf directive) for any requests that correspond to a “health check” from the load balancer to the web server. This helps keep logs clean and clutter-free.

CustomLog "|$/usr/bin/tee -a /var/log/app_name/access.log | /usr/bin/logger -t apache2 -p local6.notice" loadbalance_combined env=!dontlog

Restart Apache

Confirm the validity of your confs using apachectl configtest and restart apache for the new configuration to take effect:

service apache2 restart

Processing payments on WordPress with Stripe and Gravity Forms

This post covers creating a web Form in WordPress using the popular (and excellent) Gravity Forms plugin and configuring it to process payments with Stripe:

  • Gravity Forms is a leading commercial Form Builder plugin for WordPress that many of my clients have had a lot of success with. It is a powerful tool when combined with Payments and it supports integration via its webhooks and API.
  • Stripe is a leading payments processor that is popular due to its ease-of-use and ease-of-integration. Stripe is currently my preferred choice for eCommerce projects.

The following assumes:

  • you are logged into your WordPress (/wp-admin) as an Administrator,
  • your WordPress site has SSL enabled (https://),
  • that Gravity Forms and the Gravity Forms Stripe Add-On plugins have been installed and activated, and
  • you have a valid + verified Stripe account and have inputted all necessary Business Settings.

Get API Keys from Stripe

  • Open a tab and login to your Stripe account.
  • Click on Developers in the left nav. A sub-menu will appear underneath.
  • Click on API Keys in the sub-menu under Developers.
  • Generate a set of Test and Live API Keys. Each comes in a pair consisting of a Private Key and a Publishable Key. The keys themselves are strings of random-looking characters prefixed with: ‘sk_live’, ‘pk_live’, ‘sk_test’, and ‘pk_test’.

Make an effort to keep your Private Keys (those prefixed with ‘sk’) confidential and do not send them to anyone over email or other potentially insecure means.

Setup Gravity Forms for Stripe

In the WordPress admin dashboard:

  • Click on Forms in the left nav. A sub-menu will expand underneath.
  • Click on Settings in the sub-menu under Forms.
  • The resulting Settings page has a series of tabs down the left side. Click on Stripe.
  • On the Stripe page complete the form:
    • API: choose “Live” (or “Test” if you wish all Forms to be in Test Mode)
    • Input the Test Publishable Key, Test Secret Key, Live Publishable Key, and Live Secret Key that you obtained from the Stripe Dashboard
    • Under Stripe Webhooks there are instructions for adding a Webhook within your Stripe account that points to a URL in your WordPress site. Click on the “View Instructions” link to reveal the necessary steps and open a tab to login to your Stripe Dashboard and complete them in Stripe. When that is complete:
    • Tick the “Webhooks Enabled” checkbox
    • Input the Test Signing secret and Live Signing secret values that you obtained from Stripe

Finally, click the Update Settings button to save your settings.

Create a Form and Configure Payments

Create the Gravity Form

Create a new Form and add at least a Product Field, Total field, and Credit Card field. In the Product Field, define your products/services and set their prices.

If they are applicable to your situation, you can also use Option and Quantity fields to gather additional information that can influence the price, and you can include a Shipping field as well.

Add a Stripe Feed to the Form

In your Form’s “Edit” screen, navigate to: Settings > Stripe.

Click on the Add New button to add a new Stripe Feed to this Form.

  • Name textbox: input a descriptive name (e.g. “Event Registration 2018”). Note that its good to include a unique identifier in the Name (e.g. such as the year for an annually recurring event) so you can easily identify payments related to this particular Form + Stripe Feed in the future.
  • Transaction Type: choose “Products & Services” from the drop-down menu (“Subscriptions” is the other available option, but it is not covered this post)

A set of new fields will appear:

  • Payment Amount: choose which Form field you would like Gravity Forms to use to determine the total amount to charge the user. In most cases you’d choose the Form Total field (e.g. “Form Total”) rather than any individual product field.
  • Metadata: optionally choose Form fields that you wish to send to Stripe so it can be included in Stripe Reports. This is an optional step but it can make your life easier down the line because you will be able to see more information on the payment side of things that will help you reconcile and perform accounting and customer support tasks. For each Metadata field you would like to add:
    • Input the field Name (as you’d like it to appear in Stripe) and from the drop-down menu choose which Form Field you would like to use as a value. It is also useful for sending along details about what products or choices the user may have made. Examples:
    • Name: “Entry ID” Value: “Entry ID” — the unique database ID of the form submission in your WordPress + Gravity
    • Name: “Email” Value: “Email” — sends the customer’s email to Stripe
  • Stripe Receipt: Choose whether or not you would like Stripe to automatically send an email receipt to the customer. The default option is “No” but it is often desirable to choose “Email”.
  • Conditional Logic: Optionally add logic to conditionally process payments only if certain values/conditions are met. Most forms do not need to use this option.

When you are done, click on the Update Settings button to save your Stripe Feed.

You should be good to go! Ensure your Stripe Feed is Live and your Form is Live, and then add it to a Post or Page to see it up on your site!

If you want to test your form, you can enable Test mode and use one of the fake/test credit cards that Stripe has published here: https://stripe.com/docs/testing. You will be able to see any Test transactions in your Stripe Dashboard when you toggle to Test vs. Live mode.