Stripping leading and trailing slashes from paths in Ansible

This post covers how to use ansible’s regex_replace filter to strip leading and/or trailing slashes from file paths and URL fragments.

The final example demonstrates how to generate a valid filename from a file or URL path by removing leading and trailing slashes, and replacing any remaining slashes with underscores. This could be useful for a variety of applications from backup scripts to web scraping.

Stripping slashes

Regexes and jinja2 expressions in ansible can be a pain in the ass, especially when it comes to escaping the right thing.

The key to the following examples is a double-escape of the forward slash character. They have been tested on ansible v.

Remove leading slashes

{{ variable_name | regex_replace('^\\/', '') }}

Remove trailing slashes

{{ variable_name | regex_replace('\\/$', '') }}

Stripping both leading and trailing slashes

This example makes use of the | (OR) to combine the previous two examples into one regex:

{{ variable_name | regex_replace('^\\/|\\/$', '') 

Here’s a quick debug task that demonstrates the above in action:

- debug:
    msg: "Look Ma! No leading or trailing slashes! -- {{ item | regex_replace('^\\/|\\/$', '') }}"
    - "/home/and/lots/of/folders/trailing_and_leading/"
    - "no_leading/but/there/is/trailing/"
    - "/leading/but/no_trailing"
    - "no_leading/and/no_trailing"

Creating a valid filename from a path

To create a valid filename from a path, we need to remove leading and trailing slashes, then replace any remaining slashes with underscores. This sort of thing can be useful for naming backup files, data obtained from URL scraping, etc.

This is accomplished by adding a second regex_replace to the previous example that replaces all slashes with underscores, e.g. regex_replace('\\/', '_').

The combined regex that creates a valid filename from given a path:

{{ variable_name | regex_replace('^\\/|\\/$', '') | regex_replace('\\/', '_') }}

If variable_name held a value like “/var/www” or “/var/www/” it would result in: “var_www”.

In case you want to replace the slashes in the path with a character other than an underscore, you can adjust the examples to use any valid filename character instead of a slash (e.g. the ‘^’ character).

Working around a regex_replace bug in the shell module

In my working version of ansible (2.3.x) the regex_replace filter is ignored (!) when it is applied to variables in a tasks using the shell module. The value of unfiltered variables are substituted into the shell command fine, and using any other filter works fine too.

Workaround: employ the set_fact module to build a new fact (variable) based on the original variable, applying the regex_replace filter here as required. Reference the new fact in the shell module to take advantage of the pre-filtered values.

In the following example, assume that the hypothetical {{ list_of_paths }} variable contains a list of strings containing file/dir/URL paths. The set_fact module builds the new {{ paths }} fact such that it contains a “pi” item corresponding to every item in the original list. Each of these items has a “stripped” property containing the filtered value and a “path” property containing the original unfiltered value.

- name: workaround example
    paths: "{{ paths|default([]) + [{'pi': {'path': item, 'stripped': item|regex_replace('^\\/|\\/$', '')|regex_replace('\\/', '_')}}] }}"
      with_items: "{{ list_of_paths }}"

When looping over {{ paths }} in a shell task (e.g. via with_items), the filtered slash-free values for items can be referenced via {{ item.stripped }}. The original unfiltered path can be referenced via {{ item.path }}.

Install and secure phpmyadmin: require access via an SSH tunnel

Phpmyadmin is a popular database administration and query tool for MySQL and MariaDB. This guide covers how to install it, plus add an additional layer of security that only permits access via an encrypted ssh tunnel (also known as ssh port forwarding). This guide covers Debian and Ubuntu systems running apache2.

This particular approach is easy to automate and improves on other examples that I’ve come across online. Its the only tutorial that I know of that does things The Debian Way, in this case meaning that phpmyadmin’s customized configuration is applied without editing any of its packaged-managed conf files. This ensures that future package updates do not risk overwriting your security-minded configuration.

Why extra security measures are recommended for phpmyadmin

Phpmyadmin is a popular choice for attackers because it is widely deployed, easy to find, easy to attack, and it provides convenient access to potentially highly sensitive and/or valuable data.

A default package install of phpmyadmin on a public-facing web server is wide open to the Internet. It is often easily found with the default directory pattern “/phpmyadmin”.

A quick look at almost any website’s logs will reveal an endless barrage of attempts to locate phpmyadmin, probing a wide range of likely URL patterns and alternate names. In cases where phpmyadmin is discovered, its practically certain the logs will subsequently reveal attackers attempting to brute force and/or exploit their way in.

Another significant concern with phpmyadmin is that it does not come with ssl configured out-of-the-box. Login credentials and any data that’s accessed can be picked up by anyone that cares to listen in anywhere between your computer and the database server. This is especially a concern for anyone who accesses phpmyadmin via an untrusted network, such as public wifi.

Part 1: install phpmyadmin

First, make sure that you have mysql or mariadb installed and configured, and have taken basic steps to secure the setup. The mysql_secure_installation command is bundled with the mysql + mariadb packages and helps cover the basics.

Install the phpmyadmin package:

sudo apt-get update
sudo apt-get install phpmyadmin

The configuration script will prompt with a number of questions. When asked if you would like to automatically configure phpmyadmin for your web server, choose ‘apache2’.

Once the process is complete, you should be able to access phpmyadmin in a web browser by appending “/phpmyadmin” to your server’s URL, e.g.:

Part 2: Secure phpmyadmin

The following assumes that both apache2 and phpmyadmin were installed via the package manager.

Step 1: create a new conf file

Open your favourite editor and create a file called phpmyadmin-secure.conf in /etc/apache2/conf-available, e.g.:

sudo nano /etc/apache2/conf-available/phpmyadmin-secure.conf

For apache2.4, add the following content:

# ensure that phpmyadmin can only be accessed via localhost 
# remote users must use an ssh tunnel for access
<Directory /usr/share/phpmyadmin>
    Require local

You can double check your apache version with the command: apache2 -v.

For the older apache 2.2.x you need to use an older syntax inside the <Directory> block. If you do not need IPv6 support, delete the last directive in the following block (the entire line containing ::1):

<Directory /usr/share/phpmyadmin>
    Order Deny,Allow
    Deny from all
    Allow from
    Allow from ::1

These directives tell apache that anything in the package install location /usr/share/phpmyadmin should only be accessible by localhost ( Every other IP address, i.e. anything else on the internet, should be forbidden.

Save and exit (control+x in nano).

Step 2: Enable the conf file

Use a2enconf to enable the conf file:

cd /etc/apache2/conf-available
sudo a2enconf phpmyadmin-secure.conf

This command will create a symlink in apache’s conf-enabled/ directory that points to the conf file in the conf-available/ directory.

Note that the a2disconf command also exists and is invoked in a similar manner to a2enconf. It performs the reverse operation and disables the conf file by removing it from conf-enabled/ after it performs some checks.

Step 3: Reload apache and test

Apache must be reloaded or restarted for the updated configuration to take effect:

sudo service apache2 reload

After reloading/restarting apache, try to access phpmyadmin at its URL, e.g.:

You should see an error 403 / forbidden.

Tip: when making changes to apache’s conf files, you can check that the syntax is valid prior to reloading/restarting the server with the command apachectl configtest. Its a good idea to test your configuration before applying it because Apache will refuse to start if it has an invalid config, taking your site(s) offline until you can start it with a valid one.

Tip: if you want to hide the fact that phpmyadmin is even installed on your server at all, you can use apache’s mod_rewrite and/or RedirectMatch directives to produce a 404 ‘not found’ instead of a 403 response code.

Part 3: Access phpmyadmin via an ssh tunnel (ssh port forwarding)

To access your phpmyadmin install, connect to your server with ssh, using its tunnel options.

On your local machine, pick an arbitrary unused port and open a terminal (or an ssh client such as puTTy if your machine is a Windows PC). I use 5050 in the example below, but you can replace this with any available port number. Connect to the remote server via ssh:

ssh -L 5050:localhost:80

The -L option is key to establishing the tunnel. It tells ssh to forward any connections made to your local machine’s port 5050 to the remote host’s port 80 for as long as the ssh session is active.

Any web server (apache in our case) listening on port 80 will perceive that connections via the tunnel are coming from localhost and not from over a network / the internet.

Once you have a successful connection the server, visit the following URL in your browser to confirm you can access phpmyadmin:


All traffic between you and the remote host (including credentials and data) will be encrypted via ssh for as long as your session remains open.

All done!

You can now use phpmyadmin with the knowledge that this powerful and useful but also risky tool is not nearly as exposed to the open internet as it would be with a default install, and that you have taken a small but significant step in the endless battle to protect your credentials and data.

Troubleshooting tips

  • If you can’t access phpmyadmin, one possible gotcha to consider is if your website or app has mod_rewrite redirection rules in place (e.g. to force users to use https, to redirect certain url patterns, etc). These rules may be interfering when you try to access phpmyadmin via the tunnel. If you need to carve out an exception for localhost, but not other hosts, you could add a Rewrite Condition such as: RewriteCond %{HTTP_HOST} !^localhost to exempt it from your other rewrite conditions.

Using rbenv to support multiple versions of ruby on MacOS

This guide covers using the brew package manager to install rbenv on MacOS to enable you to install multiple versions of ruby and switch between them.

Apple bundles ruby with its operating system but the version is generally outdated and often unsuitable for developers working on their latest project. Later versions of Mac OSX through to MacOS Sierra include Ruby 2.0. Earlier OSX variants, specifically the “cats” (Mountain Lion, Lion, Snow Leopard), ship with Ruby 1.8.7. The current stable version at the time of this post is 2.4.x.

rvm is another tool that, similar to rbenv, can be used to switch between multiple versions of ruby. Overall, rbenv has a lighter touch on the MacOS system, and so is the preferred choice for this guide.


  • brew


Install rbenv and ruby-build with brew:

brew install rbenv ruby-build

Add rbenv to your bash profile so that it loads every time you open the Terminal app:

echo 'if which rbenv > /dev/null; then eval "$(rbenv init -)"; fi' >> ~/.bash_profile
source ~/.bash_profile

Thanks to: for the snippet.

Next, close and re-open your Terminal app to ensure that rbenv is initialized from your updated bash profile.

Now you can install any ruby version that you like. The following example installs v2.4.0.

rbenv install 2.4.0

rbenv gives you options when choosing which version of ruby you’d like to use:

rbenv shell 2.4.0 temporarily changes the ruby version in your current shell and forces its use over other versions that might be specified by a .ruby-version file in your current working directory. This command works by setting the RBENV_VERSION environment variable in your session. To remove it, run unset RBENV_VERSION.

rbenv local 2.4.0 creates a .ruby-version file in your current directory such that any time ruby is invoked from that directory (or a subtree), rbenv will select the version of ruby that you specified. This is useful for project folders, and you can check the .ruby-version file into your project’s revision control.

rbenv global 2.4.0 changes your default ruby to the specified version in all cases where a version is not specified (e.g. due to a .ruby-version file or if the RBENV_VERSION environment variable is set). Your current Terminal session and any other sessions will be affected.

You can confirm the ruby version in play with the command:

ruby -v

If you don’t see your desired ruby version number in the command’s output (e.g. 2.4.0) and find an older one instead (e.g. 2.0.0 the MacOS default), be sure to close the Terminal and start fresh with a completely terminal window/session, and take note if your working folder has a .ruby-version file set and what the value of the RBENV_VERSION environment variable is (echo $RBENV_VERSION).

Once you are working in a newer ruby environment, you may need to check the gems and other libraries in a project that was started with an earlier version, to ensure they are compatible or need an update.

Using s3cmd to access S3 buckets from EC2 instances with IAM Role authentication

s3cmd is an open-source command line tool for uploading, retrieving and managing data in Amazon S3 and other providers that implement the S3 protocol (e.g. Google Cloud Storage, DreamHost DreamObjects, etc). It is popular tool with a variety of applications, including backup scripts.

This post covers using s3cmd within an EC2 instance, with authentication to S3 managed via IAM Roles (IAM = Identity and Access Management).

If your project doesn’t have dependencies on scripts and/or pre-existing code that require s3cmd, and you don’t mind locking your project deeper into AWS’ ecosystem, a good alternative is Amazon’s own AWS CLI.

Configuring an IAM Role

When an EC2 instance is launched with an associated IAM role, access keys and secret keys do not need to be stored in config files on the instance itself.

After such an instance is launched, AWS-aware services and scripts on the it can access AWS resources subject to the permissions defined in the IAM role’s associated IAM role policy. Authentication is seamlessly handled via the AWS SDK.

s3cmd versions 1.5.0-alpha2 and above support IAM roles and authentication via the AWS SDK.

IAM roles

Log in to your AWS account and access the AWS IAM Dashboard (also referred to as the IAM Management Console) to manage role-based permissions.

Create an IAM role for your instances if you haven’t already.

IAM role policy

In the IAM Management Console, ensure that your IAM role has an attached policy that provides access to any S3 resource(s) that you want your instance(s) to access via s3cmd. It’s advisable to ensure that role policies are restricted to the minimum amount of scope possible.

Within the IAM Management Console:

  • Choose Roles and select the role that your EC2 instances belong to (or will belong to)
  • Review your existing policies to confirm what your instances can currently access
  • If necessary, use the Create Role Policy button to add a policy with the Policy Generator

For example, suppose you had a an s3 bucket called example-log-bucket and you wanted to configure your EC2 instances to send access logs to it.

Following the order of the Policy Generator’s input fields, you’d create: an Allow policy for the Amazon S3 service regarding the actions s3:ListBucket, s3:PutObject, and s3:PutObjectAcl, and apply them to the ARN arn:aws:s3:::example-log-bucket.

To learn more about how to specify ARNs that identify your various AWS services, see the Amazon docs:

Note that you can use wildcards in ARNs when defining a Role Policy.

Installing and configuring s3cmd on the EC2 instance(s)

s3cmd versions 1.5.0-alpha2 and above support IAM roles.

In earlier versions, an access key and secret key had to be specified in a .s3cfg file in the home directory of the user running the s3cmd command, or in environment variables.

Option A: Install s3cmd with the Ubuntu/Debian apt package manager

If you’re running Ubuntu 16.04 or later (test box at the time of writing) the package version is sufficiently up-to-date (>1.5.0). Install s3cmd with apt-get:

sudo apt-get install s3cmd

Option B: Install s3cmd with the python pip package manager

Pip is an alternative to Debian/Ubuntu apt-get, especially if you are using a different distro or like to use python/pip.

sudo pip install s3cmd

Check the version number with s3cmd --version. If you discover that its <1.5.0, you can specify a particular version for pip to install as follows, substituting in your desired version number (e.g. 1.5.0-alpha3) in place of VERSION_NUMBER below:

pip install s3cmd==VERSION_NUMBER

Configure s3cmd on the EC2 instance(s)

Once an EC2 instance is launched with a suitable IAM Role and associated IAM Role Policy, its easy to get started with s3cmd. The following example is set in the context of the currently logged in user.

Open/create the file ~/.s3cfg using a text editor:

nano ~/.s3cfg

Populate the file with the following content. Rather than filling in hardcoded credentials, leave each of the fields blank (use the example as is; do not specify any values):

access_key =
secret_key = 
security_token =

Save the file and exit the editor.

Later versions of s3cmd that support IAM Roles will auto-magically detect the appropriate authentication values.

s3cmd will now be able to interact with any AWS resource(s) that it is permitted to access, subject to s3cmd’s capabilities and the IAM Role’s active Role Policies.

Use s3cmd

s3cmd has over 60 command line options. Check out the documentation:

How to install Ansible on MacOS using pip

Ansible is a powerful devops/automation tool that helps you control fleets of servers and their applications with ease.

This post covers installing ansible on MacOS using the installation method recommended by the project: pip, the python package manager. There are a few other ways to install ansible on MacOS, including with the brew package manager.

This post assumes that you are interested in installing ansible only for the current user only. If you require a system-wide/multi-user installation, skip to Step 8 to learn more about the MacOS System Integrity Protection (SIP) feature before proceeding.

Installing Ansible with pip

You may be able to skip steps 1, 2, and/or 3 if you already have XCode, XCode Command Line Tools, and/or pip installed on your system.

Step 1: install XCode

Most developers already have XCode installed. If you’re not sure, you can look for it in the Applications folder on your Mac.

XCode is available for free on the App Store. It can be helpful to run XCode at least once after installing it to help it complete its installation process, and get you to agree to any license terms.

The following steps will fail if you have not agreed to Xcode’s license terms.

Step 2: install XCode command line tools

The XCode command line tools include compilers and other tools necessary to proceed. In the MacOS Terminal app, run the command:

xcode-select --install

If this is a fresh install, you will be prompted to agree to Apple’s terms for the command line tools. After you consent, they will download and install.

If you already have the command-line tools installed. The script will let you know and exit.

Once the XCode command line tools are installed, you can use development tools like the gcc compiler. For example, you should be able to run the following command and successfully receive gcc’s version information as output:

gcc --version

If everything looks good and the above command does not throw an error, proceed to the next step.

Step 3: install pip

Pip is the python package manager. It is conceptually similar to other well-known package management tools such as npm (node/javascript) or apt (debian/ubuntu linux distributions).

MacOS comes pre-installed with python, but not pip. To install it, input the following into the Terminal app:

sudo easy_install pip

There is no harm to running this command in cases where pip is already installed.

Step 4: install ansible

To install ansible system-wide, run the following command:

sudo pip install ansible

If you prefer a local install within your user account, you can remove the ‘sudo’ and add the --user flag to the above.

After the installation process has completed, you can verify the install was a success by executing a basic command like: ansible --version. It should run without error.

Step 5: create /etc/ansible

Ansible looks for default configuration files in /etc/ansible. This directory is unlikely to exist on your machine. The following command will create this folder if it doesn’t already exist:

sudo mkdir -p /etc/ansible

Step 6: install optional/helpful packages

It can be helpful to install the python passlib library because MacOS doesn’t use the same types of password hashes as linux does, and you are likely going to want to manage linux machines with ansible. Passlib provides a common and consistent hash generator across operating systems.

To install passlib globally on your Mac:

sudo pip install passlib

If you installed ansible locally, you can use the command pip install --user passlib instead.

Step 7: test out your installation

Per above, if the install was successful, you can now use the ansible command from the Terminal:

ansible --version

A “hello world” of sorts for ansible is pinging yourself:

ansible localhost -m ping

You can also try running an arbitrary command on yourself (such as uname -a or whereis bash):

ansible localhost -a 'uname -a'

To get ansible to detect facts about your machine, try the following. Note the filter argument: it restricts what is returned so you don’t get BLASTED with facts.

By default ansible gathers almost everything there is to know about its target machines and populates them into variables that you can then use in your tasks, roles, and plays. As you learn ansible, you’ll discover this is incredibly useful. Ansible users can customize this behaviour and optionally turn it off to realize performance gains when it isn’t required. For now though, you’re just getting started:

ansible localhost -m setup -a 'filter=ansible_distribution'

Step 8: upgrade ansible

It can be a good idea to upgrade ansible right off the bat to ensure that everything is current, and to understand how the MacOS System Integrity Protection (SIP) feature impacts your installation:

sudo pip install ansible --upgrade 

If this command partially runs and then produces an error, it is because of the SIP feature, originally introduced in OSX El Capitan. You can work around that by adding the --user python parameter to the above command. With this argument, upgrades will be installed for the current user only vs. system-wide, and the upgrade will succeed. The revised command:

sudo pip install ansible --upgrade --user python

Understanding MacOS’ System Integrity Protection (SIP) Feature

SIP was introduced in OSX El Capitan and prevents certain system files, such as those associated with MacOS-bundled python, from being clobbered by anyone — including root.

To avoid SIP-related issues on a system-wide install, you may want to consider using a different python than the one bundled with MacOS. Using a python virtualenv is one option. Another option for multi-user systems is to install user-specific copies of python + pip + ansible within each appropriate user account (e.g. via brew package manager for MacOS).

A more thorough discussion of issues regarding the use of pip on MacOS systems with SIP can be found here:

Step 9: get started!

Congratulations! You have installed ansible on MacOS!

If you’re new to ansible, check out the documentation at to get started!