Set up MacOS’ built-in Apache + PHP as a LAMP/WordPress Dev Environment

This guide covers configuring MacOS High Sierra, Mojave, and beyond as a local development server for LAMP-stack projects. This is a high-performance option for running projects locally on a Mac and is useful for PHP and WordPress development.

At the end of this guide, you will be able to create new folders within a special virtual hosts directory such as ‘example.com/’ with the sub-folder ‘public_html/’ and the ‘http://example.com.test’ will automatically be available with apache serving files out of the ‘public_html/’ folder.

Apple already bundles many of the necessary pieces with MacOS including apache and PHP. This guide covers configuring them to support WordPress, as well as installing other dependencies such as mysql/mariadb, and useful tools like sequel-pro and wp-cli.

Alternatives

Self-contained solutions for running a LAMP environment on a Mac can be found in products like MAMP and the free XAMPP. They double up what’s already on your Mac but do have some features that you might find appealing.

Virtual Machines are another option. Tools such as Vagrant are great and extremely useful for more complex projects where customized server configuration(s) are required. However, spinning up an entire virtual server with its own CPU, RAM and disk allocation simply to run WordPress is a huge tax on system resources. It’s often a better idea to reserve VM tools for better-suited applications.

A lighter-weight alternative to full VM’s is container virtualization such as with Docker. This brings its own configuration and deployment challenges that may introduce unnecessary effort if Docker doesn’t play a greater role in a given project.

Pre-checks

Confirm your system’s apache version with the command apachectl -v, and its PHP version with the command php -v.

MacOS Sierra and beyond should be running apache 2.4+ and PHP 7+.

Apache configuration

Apple’s default configuration requires several changes to create a suitable development environment.

Backup httpd.conf

First, make a backup copy of apache’s httpd.conf file:

sudo cp /etc/apache2/httpd.conf /etc/apache2/httpd.conf.pre-dev-env

Apache httpd.conf configuration

Open httpd.conf using a text editor such as nano:

sudo nano /etc/apache2/httpd.conf

Each of the following lines are found at different locations within the file. Find each one of them and ensure they are uncommented — delete any preceding # character so they start with a letter instead:

#LoadModule vhost_alias_module libexec/apache2/mod_vhost_alias.so
#LoadModule rewrite_module libexec/apache2/mod_rewrite.so
#LoadModule php7_module libexec/apache2/libphp7.so
#Include /private/etc/apache2/extra/httpd-vhosts.conf

If you’re using the nano editor, its “where” Control+W feature can help you locate each line. It works similar to the “find” command (Control+F) found in most web browsers and word processors.

When you’re done, use Control+O to save (output) your changes and Control+X to exit nano.

The above changes have:

  • enabled the module mod_vhost_alias to enable dynamic virtual hosts;
  • enabled the module mod_rewrite which WordPress uses to rewrite URL paths;
  • enabled PHP7 support so WordPress’ php scripts can execute; and
  • included an extra conf file where we will define a dynamic apache virtualhost for your projects.

Apache dynamic virtual host configuration

In the previous step we referenced a httpd-vhosts.conf file in httpd.conf.

Create a backup of the original file:

sudo cp /private/etc/apache2/extra/httpd-vhosts.conf /private/etc/apache2/extra/httpd-vhosts.conf.pre-dev-env

Now edit that file:

sudo nano /private/etc/apache2/extra/httpd-vhosts.conf

This file is populated by default with a couple of example virtual host definitions: blocks beginning with <VirtualHost *:80> and ending with </VirtualHost>.

Comment out (i.e. prefix each line with #) or delete the two example virtual host definitions.

Next, add your own dynamic virtual host definition to the file.

I’ve decided to use /usr/local/var/www as the base folder to serve projects from. You can modify this to another path if you’d like.

Add the following block:

# Custom configuration for local dev environment

<Directory "/usr/local/var/www">
  Options Indexes MultiViews FollowSymLinks
  AllowOverride All
  Require all granted
</Directory>

<Virtualhost _default_:80>

  # Dynamic virtual hosts for local development
  UseCanonicalName Off
  VirtualDocumentRoot "/usr/local/var/www/%-2+/public_html"

  # Show 404's in the apache error log
  LogLevel info

</Virtualhost>

Save (Control+O) and exit (Control+X) the nano editor.

Explaining the Dynamic Virtual Host definition

In httpd-vhosts.conf as modified above, the <Directory> block is required for Apache 2.4+ and tells Apache that it can serve files from the /usr/local/var/www folder. This explicitly overrides the rigorous security-focused default specified in httpd.conf that tells Apache it can’t serve up any file on the hard disk (/).

The <VirtualHost> block defines a _default_ VirtualHost that listens on port 80.

The VirtualDocumentRoot directive tells Apache where to look for the dynamic VirtualHost’s files. The %-2+ placeholder stands in for “the penultimate (second to last) and all preceding parts” of the server name in the request. Therefore, per the directive, when handling a request for http://example.com.test, Apache will look for that site’s files in /usr/local/var/www/example.com/public_html.

I like to have my project folder names in dev environments mirror my likely production values, such as example.com/ or hypothetical.app/. You may wish to name your project folders differently. You may prefer to use the %0 placeholder for the entire server name (example.com.test), or %1 for the first-part only (example).

To learn more about these directives, refer to the docs:

Create the required paths

Finally, ensure the paths that you specified for your project folders exist. In my case:

mkdir -pv /usr/local/var/www

If you have any project build folders, you can add them (or better yet: create symlinks to them) here!

To give you an idea of how this works:

In /usr/local/var/www if you were to create the folder helloworld.com/ and sub-folder helloworld.com/public_html, and put a basic index.php containing hello world! inside it, at the end of this tutorial Apache will serve it up when you visit http://helloworld.com.test on your machine.

Smoke-check (config check)

As a smoke-check to make sure you haven’t made any typos or introduced any bugs, run the following command to check the syntax of your apache config files:

sudo apachectl configtest

If you followed all of the above steps carefully, it should output “Syntax OK”.

If everything checks out, restart apache so your latest conf changes can take effect:

sudo apachectl restart

Configuring DNS for local domain names

Now we will solve the problem of directing local requests for http://example.com.test to Apache.

One option is to manually add an entry to /private/etc/hosts for every one of your projects. For example, you might add the line 127.0.0.1 myproject.test to enable the local URL ‘http://myproject.test’.

We are going to use a more dynamic approach using a DNS server so that all requests for any URL at the *.test domain will resolve to localhost.

Any DNS server could be employed for this job. We will use dnsmasq, a perfect choice for this type of lightweight local routing.

Follow the steps in my guide Using dnsmasq on MacOS to setup a local domain for development to complete this step.

Customizing PHP settings (php.ini)

MacOS’ PHP uses a default php.ini file based on /private/etc/php.ini.default.

To customize your PHP environment, if a php.ini file doesn’t already exist at /private/etc/php.ini, copy the default template to create a main php.ini file:

sudo cp /private/etc/php.ini.default /private/etc/php.ini

Make any changes you wish to php.ini and restart apache to reload all configuration files:

sudo apachectl restart

If you were to run phpinfo() in a php file from the web server, you should now see that the Loaded Configuration File property now has the value /etc/php.ini.

A very common tweak to the default PHP configuration is to allow larger file upload sizes. The post_max_size and upload_max_filesize properties are only a few megs by default. These limits can be raised as you see fit.

Many developers also tweak the max_execution_time, max_input_time, and memory_limit settings depending on their project.

Always remember to restart apache after making changes to your PHP configuration.

Installing and configuring mysql server

WordPress and many other PHP apps require a MySQL or compatible database such as MariaDB to store its data.

I prefer to use mariadb, which is available via brew. Run the following command to install it for your user:

brew install mariadb

This installation method is recommended in the official docs:

Following installation, manually start the server by running:

mysql.server start

Running mysql as a service

By default you will need to manually start and stop your mysql/mariadb server.

If you want the server to start when you login to your Mac, use Homebrew’s services feature to have it automatically launch in the background:

brew services start mariadb

Homebrew integrates with MacOS’ launchctl to make this happen the correct way as supported by Apple.

Test logging into mysql via the cli client

Once your server has started, you can test logging in as the root user with the Terminal command:

mysql -u root

Issue the quit command to exit the MySQL prompt and return to your Terminal session.

Set a root password and improve security

Even though a default mysql/mariadb configuration only accepts connections from localhost, it is still wise to run the quick interactive mysql_secure_installation command-line-interface script to set a root password and a few other security-minded options.

Troubleshooting: resolve possible socket incompatibility

There may be a slight configuration mismatch between the defaults of brew’s mysql/mariadb packages and the defaults of MacOS’ built-in PHP mysqli extension. It may be fixed in newer packages.

If try to get a LAMP app such as WordPress to connect to mysql on localhost and encounter a “Connection Refused” error even when the database settings are confirmed correct (as defined in wp-config.php for WordPress), then you may need to make a small fix. You can confirm the issue by trying to connect to 127.0.0.1 instead: if it works but localhost doesn’t then it is a socket-related issue. 

For localhost (socket) connections to the database, brew’s package version tends to use the socket file /tmp/mysql.sock. MacOS’ built-in Apache+PHP mysqli extension assumes the socket file is at /var/mysql/mysql.sock (i.e. the default setting of its mysqli.default_socket property). You can verify the value of this property by running phpinfo(); in a php script and finding it in the list.

To resolve, one could edit php.ini or another conf file. A simple solution is to create a symlink at /private/var/mysql/mysql.sock that points to /tmp/mysql.sock so everything works out:

sudo mkdir -p /var/mysql
sudo ln -s /tmp/mysql.sock /private/var/mysql/mysql.sock

On MacOS /var, a classic folder for unix/linux systems, is symlinked to /private/var so to avoid a symlink-on-symlink type situation, the above uses the real path. 

Note that only socket connections (i.e. database connections made to ‘localhost’) would be impacted by this incompatibility. Connections to ‘127.0.0.1’ force connecting over TCP due to the IP address, bypassing any socket concerns.

Install a database management GUI

If you’d like to use a GUI to help manage your mysql databases, one popular choice is sequel-pro. This serves the same purpose as the popular phpmyadmin webapp except it runs as a desktop app on your Mac. To install it:

brew cask install sequel-pro

You can now launch it from your Applications folder.

Install wp-cli

WordPress’ Command-Line-Tools are a huge timesaver for managing both local and remote WordPress installs, and they are essential for writing scripts that automate WordPress deployment, updates, etc.

Install wp-cli for your user with the command:

brew install wp-cli

Confirm your install by running wp --version in Terminal.

Assuming your project folder includes a public_html/ folder that contains your WordPress install, and that you have created a database called “DB_NAME”, you can download and setup WordPress with a handful of wp-cli commands:

cd /Users/username/web-projects/my-project.com/public_html
wp core download
wp config create --dbname=DB_NAME --dbuser=DB_USERNAME --dbpass=DB_PASSWORD --dbhost=localhost
wp core install --url=my-project.com --title="My Project" --admin_name="example_admin" --admin_password="example_password" --admin_email=you@example.com

Check out the command reference and docs/handbook at https://wp-cli.org/

Serving up your project

To make a project available at http://example.com.test you need to:

  • create a folder example.com/ in your chosen base directory (e.g. /usr/local/var/www)
  • create the public_html subfolder: example.com/public_html/
  • copy project files to public_html/ (manually or as an automated task e.g. with gulp)

OR

  • create a symlink from your project location to the chosen base directory

For the symlink option:

ln -s ~/some/project/folder/example.com /usr/local/var/www/example.com

Be sure to replicate the requirement for a public_html/ sub-folder.

When creating symlinks from elsewhere on your system to /usr/local/var/www be careful of permissions issues!

Managing permissions

MacOS’ Apache is running as the user _www by default. That user must have at least read permissions for any folder that you symlink to in order to serve up your project files.

It’s important to note that ~/Documents and any folders created within are not accessible to any group or other users. Apple thankfully assumes that you don’t want other users on a system to be able to access your private documents! As a result, Apache’s _www user will not be able to follow symlinks into the Documents folder belonging to your user.

If you have any web projects in ~/Documents you could save them somewhere else where Apache can read them and symlink to them from there, or you could simply copy your project files over to /usr/local/var/www.

Depending on your desired setup, you may find it convenient to change the group ownership of /usr/local/var/www to _www and set the groupID bit (setgid):

sudo chgrp _www /usr/local/var/www
sudo chmod g+s /usr/local/var/www

The setgid bit ensures that any new files or folders created underneath www/ will inherit the group ID of the directory vs. the default behaviour of having it set to the primary group ID of the user that created the file. This measure can help ensure that Apache’s _www user can access the contents of the www/ folder.

Finally, remember that our particular Apache VirtualHost configuration requires a public_html/ sub-directory under the my-project.com folder for web-servable files. To illustrate: an index.php file in ~/web-projects/my-project.com/public_html will be served up to a local web browser that requests the URL http://my-project.com.test. This works because there is a symlink in /usr/local/var/www named my-project.com that points to the folder ~/web-projects/my-project.com.

Working on a project

In the future, if you haven’t setup apache and mysql server to run automatically as services, you will need to start them:

sudo apachectl start
mysql.server start

To stop or restart apache, use the following commands:

sudo apachectl stop
sudo apachectl restart

You can stop mysql with:

mysql.server stop

Remember: if you make any changes to your apache config, such as adding a new VirtualHost, you will need to restart or reload apache for your changes to take effect.

Finally remember that the settings covered in this guide will serve up your projects to any clients on your network. If you want to hide them from everyone on your local coffee shop’s shared wifi, turn off the servers, harden your firewall (via System Preferences), and/or tweak your Apache conf to only serve to localhost.

Upgrading MacOS

When you upgrade MacOS, the installer will create versions of your conf files with the suffix ~previous before replacing with its own.

You can also create your own backup copies to be safe.

Following an OS update, if the Apache files are changed, you can compare the new versions with your ~previous versions using the diff command to make sure there’s no major updates (very unlikely) then restore your customized versions. After restoring your files, remember to restart apache for the settings to take effect: sudo apachectl restart.

The files customized by following this guide are:

  • /etc/apache2/httpd.conf
  • /private/etc/apache2/extra/httpd-vhosts.conf

Script to Deploy WordPress

Check out this gist for a bash script to quickly deploy a fresh WordPress install:

https://gist.github.com/firxworx/bb9b7f8d71915f9d4e7f8d3a8a531b26

Using dnsmasq on MacOS to setup a local domain for development

This guide covers using dnsmasq as a local DNS server on MacOS to resolve all URL’s with the .test domain name to localhost (127.0.0.1).

This can be very useful to developers wishing to setup an efficient local development and test environment on their machine. For example, if http://myproject.com.test resolved to localhost, a local apache, nginx, or other server could respond to the request.

While any DNS server could be employed for this job, dnsmasq is a perfect choice for this type of lightweight local routing.

Regarding the .test domain, note that this is a recommended choice per the IETF RFC’s (e.g. RFC-2606, RFC-6761). While it’s possible to configure dnsmasq for any arbitrary domain name, only .test, .example, .invalid, and .localhost are reserved for non-Internet use. A popular choice used to be .dev until Google purchased the top-level-domain (TLD) and changed how Chrome handles .dev URL’s.

Install and configure dnsmasq

To get started, use brew to install dnsmasq for your user:

brew install dnsmasq

At the time of writing, the brew package installer creates a sample conf file with all entries commented out. It can be found at ./etc/dnsmasq.conf under brew’s default install folder.

Execute following commands in sequence to configure dnsmasq to resolve *.test requests to 127.0.0.1:

echo "" >> $(brew --prefix)/etc/dnsmasq.conf
echo "# Local development server (custom setting)" >> $(brew --prefix)/etc/dnsmasq.conf
echo 'address=/.test/127.0.0.1' >> $(brew --prefix)/etc/dnsmasq.conf
echo 'port=53' >> $(brew --prefix)/etc/dnsmasq.conf

The above commands simply append the required lines to the dnsmasq.conf configuration file.

Next, use brew’s services feature to start dnsmasq as a service. Use sudo to ensure that it is started when your Mac boots, otherwise it will only start after you login:

sudo brew services start dnsmasq

Add a resolver

Add a resolver for .test TLD’s:

sudo mkdir -pv /etc/resolver
sudo bash -c 'echo "nameserver 127.0.0.1" > /etc/resolver/test'

This tells your system to use localhost (and therefore dnsmasq) as the DNS resolver for the .test domain.

Test your configuration

Flush your DNS cache:

sudo dscacheutil -flushcache
sudo killall -HUP mDNSResponder

Check your configuration by running:

scutil --dns

Inspect the output. One of the entries should specify that for domain test the system will use the nameserver 127.0.0.1. This means that dnsmasq will now be consulted as the nameserver for any .test URLs.

Finally, test the full stack by trying to ping random *.test URL’s such as example.test or myproject.test and confirming that they resolve to 127.0.0.1. For example:

ping -c2 example.test

Provides the output:

PING example.test (127.0.0.1): 56 data bytes
64 bytes from 127.0.0.1: icmp_seq=0 ttl=64 time=0.022 ms
64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.055 ms

OR (in cases where your Mac’s firewall settings are configured to not respond to ping):

PING example.test (127.0.0.1): 56 data bytes
Request timeout for icmp_seq 0
...

The important part to confirm is that example.test resolved to (127.0.0.1).

Encrypting a USB Drive in MacOS, including formatting to JHFS+ or APFS

MacOS Sierra doesn’t feature an option to encrypt a USB drive in Disk Utility or in Finder (at least at the time of writing). This post covers how to format a USB drive to either the JHFS+ or the new APFS filesystem and encrypt it using the Terminal and Disk Utility.

Instructions

First, plug your USB drive into your computer and open the Terminal app.

Use the following command to list your disks:

diskutil list

Look for an entry (or entries) like /dev/disk2 (external, physical) and make absolutely sure that you understand the difference between your system’s hard disk and the external USB drive you want to encrypt.

The “IDENTIFIER” for my USB drive, found at the top of the list, was disk2. My system showed a subsequent entry disk2s1 but note how this still refers to disk2. Only the disk2 part is required. Use whatever diskn number corresponds to your target drive, where n is an integer.

Only proceed if you are certain that you have correctly identified your USB drive!

The following command formats the drive to Apple’s HFS+ with Journaling format, JHFS+. GPT is a crucial argument here to specify the GUID Partition Map option vs. the Master Boot Record option. You can replace the text inside the quoted string (“Flash Drive”) with your desired drive name:

diskutil eraseDisk JHFS+ "Flash Drive" GPT disk2

It is now possible to encrypt the drive with Finder (right-click and choose “Encrypt ‘Flash Drive'”) if you wish to simply keep the JHFS+ file system. If you wish to use the newer APFS file system, do not Encrypt the drive just yet, and read on.

Using the new APFS file system

APFS is Apple’s latest filesystem and it features good support for encryption. Before formatting your drive to APFS, be aware that older Macs (i.e. those without MacOS Sierra and up) will not support it.

To proceed with APFS, open the Disk Utility app.

With the drive formatted to JHFS+, Disk Utility will no longer grey out the “Convert to APFS” option when you right/control+click it.

Find your drive and choose “Convert to APFS”.

Once the file system has been converted to APFS you can go back to Finder, right/control+click on your drive, and choose “Encrypt ‘Flash Drive'” from the menu.

Don’t forget your passphrase 😉

Troubleshooting the fast.ai AWS setup scripts (setup_p2.sh)

Fast.ai offers a well-regarded free online course on Deep Learning that I thought I’d check out.

It seems that a lot of people struggle getting the fast.ai setup scripts running. Complaints and requests for help are on reddit, in forums, etc. This doesn’t surprise me because the scripts are not very robust. On top of that, AWS has a learning curve so troubleshooting following a script failure can be a challenge.

Hopefully this post helps other people that have hit snags. It is based on my experience on MacOS, however should be very compatible for those running Linux or Windows with Cygwin.

Understanding the setup script’s behaviour

It leaves a mess when it fails

If running the setup script fails, which is possible for a number of reasons, it will potentially have created a number of AWS resources in your account and a local copy of an SSH key at ~/.ssh/aws-key-fast-ai.pem. It does not clean up after itself in failure cases.

The setup script doesn’t check for existing fast-ai tagged infrastructure, so subsequent runs can create additional VPC’s and related resources on AWS, especially as you attempt to resolve the reason(s) it failed. The setup script might generate fast-ai-remove.sh and fast-ai-commands.txt but it overwrites these each time its run with only its current values, potentially leaving “orphan” infrastructure.

Thankfully all AWS resources are created with the same “fast-ai” tags so they are easy to spot within the AWS Console.

It makes unrealistic assumptions

The setup script assumes your aws config’s defaults specify a default region in one of its three supported regions: us-west-2, eu-west-1, and us-east-1.

I’m not sure why the authors assumed that a global tech crowd interested machine learning would be unlikely to have worked with AWS in the past and thus no existing aws configuration that might conflict.

The commands in the script do not use the --region argument to specify an explicit region so they will use whatever your default is. If your default happens to be one of the three supported ones, but you don’t have a sufficient InstanceLimit or there’s another problem, more issues could follow.

Troubleshooting

If you encountered an error after running the script, prior to re-running the script, take note of the following checks when attempting to resolve:

Check 1: Ensure you have an InstanceLimit > 0

Most AWS users will have a default InstanceLimit of 0 on P2 instances. You may need to apply for an increase and get it approved (this is covered in the fast.ai setup video).

If a first run of the script gave you something like the following, there was an issue with your InstanceLimit:

Error: *An error occurred (InstanceLimitExceeded) when calling the RunInstances operation: You have requested more instances (1) than your current instance limit of 0 allows for the specified instance type. Please visit http://aws.amazon.com/contact-us/ec2-request to request an adjustment to this limit.* 

InstantLimits are specific to a given resource in a given region. Take note of which region your InstanceLimit increase request was for and verify that it was granted in the same region.

Check 2: Ensure the right region

Verify your current default aws region by running: aws configure get region. The script assumes this is one of three supported regions: us-west-2, eu-west-1, or us-east-1.

The script also assumes that you have an InstanceLimit > 0 for P2 instances in whichever region you would like to use (or T2 instances if you are using setup_t2.sh).

To get things running quickly, I personally found it easiest to make the script happy and temporarily set my aws default to a supported region in ~/.aws/config, i.e.:

[default]
region=us-west-2

Another option is to modify the scripts and add an explicit --region argument to every aws command that will override the default region. If you have multiple aws profiles defined as named profiles, and the profile that you wish to use for fast.ai specifies a default region, you can use the --profile PROFILENAME argument instead.

For example, the following hypothetical aws config file (~/.aws/config) specifies a profile called “fastai”. A --profile fastai argument could then be added to every aws command in the setup script:

[default]
region=ca-central-1

[profile fastai]
region=us-west-2

Check 3: Delete cruft from previous failed runs

This check is what inspired me to write this post!

Delete AWS resources

Review any resources were created in your AWS Console, and delete any VPC’s (and any dependencies) that were spun up. They can be identified because they were created with the “fast-ai” tag which is shown in any tables of resources in the AWS Console.

Cruft resources will have been created in any region that the setup script was working with (i.e. whatever your default region was at the time you ran it).

If you’ve found cruft, start by trying to delete the VPC itself, as this generally will delete most if not all dependencies. If this fails because of a dependency issue, you will need to find and delete those dependencies first.

IMPORTANT: AWS creates a default VPC and related dependencies (subnets, etc.) in every region available to your account. Do NOT delete any region’s default VPC. Only delete resources tagged with “fast-ai”.

Delete SSH keys

Check to see if ~/.ssh/aws-key-fast-ai.pem was created, and if so, delete it before running the script again.

The setup script has logic that checks for this pem file. We do not want the script to find the file on a fresh run.

After a successful run

After the setup script ran successfully, I got output similar to:

{
    "Return": true
}
Waiting for instance start...

All done. Find all you need to connect in the fast-ai-commands.txt file and to remove the stack call fast-ai-remove.sh
Connect to your instance: ssh -i /Users/username/.ssh/aws-key-fast-ai.pem ubuntu@ec2-XX-YY-ZZ-XXX.us-west-2.compute.amazonaws.com

Reference fast-ai-commands.txt for information about your VPC and EC2 instance. An ssh command to connect is in the file, and you can find your “InstanceUrl”.

I suggest picking up the video from here and following along from the point where you connect to your new instance. It guides you through checking the video card with the nvidia-smi command and running jupyter: http://course.fast.ai/lessons/aws.html

Starting and stopping your instance

The fast-ai-commands.txt file outlines the commands to start and stop your instance after the setup has completed successfully, e.g.:

aws ec2 start-instances --instance-ids i-0XXXX
aws ec2 stop-instances --instance-ids i-0XXXX

Its important to stop instances when you are finished using them so that you don’t get charged hourly fees for their continued operation. P2 instances run about $0.90/hr at the time of writing.

Send emails from MacOS Terminal or scripts using Rackspace’s SMTP server

This guide will help configure MacOS so it can send emails from the command-line (Terminal) and shell scripts via an SMTP server. This is useful for enabling scripts, scheduled jobs, etc. to send email notifications.

The example in this post work for Rackspace’s SMTP server. The guide is adaptable to other SMTP servers and email providers, while acknowledging that you may need to adjust certain details. Email providers can vary in how they require users to authenticate and interface with their SMTP servers.

There are lot of examples online for using gmail’s SMTP servers, but since I couldn’t find any complete examples for Rackspace, I decided to write this post!

Configuring postfix on MacOS

MacOS/OSX comes bundled with the postfix mail server. We will configure that to use Rackspace’s SMTP server to send mail.

Prerequisites

To use Rackspace’s SMTP, its a given that you must be a Rackspace customer. Create an email address that you wish to be the “sender” of any notifications. Note this email account’s credentials as you’ll need to include them in your postfix configuration.

I recommend that you use a dedicated email address for sending automated notifications (e.g. no-reply@.., notify@.., etc), rather than an email address with an inbox that is important to your business. Within Rackspace’s control panel, you have the option to configure an auto-responder for your automated notification address so that anyone that replies to a notification or sends anything else to it will receive a canned reply in response.

Step1: Edit main postfix configuration file

Make a copy of your original postfix conf file:

sudo cp /etc/postfix/main.cf /etc/postfix/main.cf.bak

Open postfix’s config file for editing:

sudo nano /etc/postfix/main.cf

In an untouched postfix configuration on MacOS, the following 3x separate lines have the following values:

  • mydomain_fallback = localhost
  • mail_owner = _postfix
  • setgid_group = _postdrop.

If you have previously edited your postfix settings, you may wish to search for each item and reset these back to their default values (first confirm that you aren’t about to break anything!).

Add the following to the very end of the open postfix config file:

relayhost = smtp.emailsrvr.com:587
smtp_sasl_auth_enable=yes
smtp_sasl_password_maps=hash:/etc/postfix/sasl_passwd
smtp_sasl_mechanism_filter = AUTH LOGIN
smtp_sasl_security_options =
smtp_use_tls=yes
smtp_tls_security_level=encrypt

These setting are specific to Rackspace’s requirements. If you are using another SMTP server, you will need to specify different values.

Save and exit the editor.

Step 2: create the sasl_passwd file for postfix to use

Create the /etc/postfix/sasl_passwd file referenced in main.cf with the following line of text, substituting the email@example.com and password placeholders with your own:

smtp.emailsrvr.com:587 email@example.com:password

Next, use the postmap tool to create a lookup table from the sasl_passwd file:

sudo postmap /etc/postfix/sasl_passwd

This will create the file sasl_passwd.db

Step 3: Restart postfix

sudo postfix reload

You might see some warnings, but they should be benign. We don’t need a full mail system, just the ability to send smtp emails. Move on to the next step to test your configuration!

Step 4: Send a test email

The following command sends a test email to your-email@example.com. Give it a shot by substituting your email address:

echo "testing testing test email" | mail -s test your-email@example.com

Note this command does not produce any output when it runs successfully.

Next, check for the test email in your inbox (if its not there, be sure to also check the spam folder). You may want to add your notification email address to your recipient inbox’s address book to ensure better deliverability.

Troubleshooting tips

If you need to troubleshoot, it can be helpful to check the mail queue:

sudo mailq 

You can clear all queued emails with the following command:

sudo postsuper -d ALL

On OSX, postfix logged to the /var/log/mail.log file. On MacOS, this log file doesn’t exist.

For troubleshooting with MacOS, you can view the log in real-time with the command:

sudo log stream --predicate  '(process == "smtpd") || (process == "smtp")' --info

You could have this running in one Terminal tab, and then try to send a test email in another tab, to see what messages you receive.

Acknowledgements

Thanks to this article from <developerfiles.com> which has instructions for using gmail, as well as some commenters on various forums that provided tips to adapt SMTP settings for Rackspace. Rackspace’s documentation was also helpful in realizing a working configuration.

Using rbenv to support multiple versions of ruby on MacOS

This guide covers using the brew package manager to install rbenv on MacOS to enable you to install multiple versions of ruby and switch between them.

Apple bundles ruby with its operating system but the version is generally outdated and often unsuitable for developers working on their latest project. Later versions of Mac OSX through to MacOS Sierra include Ruby 2.0. Earlier OSX variants, specifically the “cats” (Mountain Lion, Lion, Snow Leopard), ship with Ruby 1.8.7. The current stable version at the time of this post is 2.4.x.

rvm is another tool that, similar to rbenv, can be used to switch between multiple versions of ruby. Overall, rbenv has a lighter touch on the MacOS system, and so is the preferred choice for this guide.

Prerequisites

  • brew

Steps

Install rbenv and ruby-build with brew:

brew install rbenv ruby-build

Add rbenv to your bash profile so that it loads every time you open the Terminal app:

echo 'if which rbenv > /dev/null; then eval "$(rbenv init -)"; fi' >> ~/.bash_profile
source ~/.bash_profile

Thanks to: https://gorails.com/setup/osx/10.12-sierra for the snippet.

Next, close and re-open your Terminal app to ensure that rbenv is initialized from your updated bash profile.

Now you can install any ruby version that you like. The following example installs v2.4.0.

rbenv install 2.4.0

rbenv gives you options when choosing which version of ruby you’d like to use:

rbenv shell 2.4.0 temporarily changes the ruby version in your current shell and forces its use over other versions that might be specified by a .ruby-version file in your current working directory. This command works by setting the RBENV_VERSION environment variable in your session. To remove it, run unset RBENV_VERSION.

rbenv local 2.4.0 creates a .ruby-version file in your current directory such that any time ruby is invoked from that directory (or a subtree), rbenv will select the version of ruby that you specified. This is useful for project folders, and you can check the .ruby-version file into your project’s revision control.

rbenv global 2.4.0 changes your default ruby to the specified version in all cases where a version is not specified (e.g. due to a .ruby-version file or if the RBENV_VERSION environment variable is set). Your current Terminal session and any other sessions will be affected.

You can confirm the ruby version in play with the command:

ruby -v

If you don’t see your desired ruby version number in the command’s output (e.g. 2.4.0) and find an older one instead (e.g. 2.0.0 the MacOS default), be sure to close the Terminal and start fresh with a completely terminal window/session, and take note if your working folder has a .ruby-version file set and what the value of the RBENV_VERSION environment variable is (echo $RBENV_VERSION).

Once you are working in a newer ruby environment, you may need to check the gems and other libraries in a project that was started with an earlier version, to ensure they are compatible or need an update.