Using MariaDB / MySQL Service Containers Azure in Devops Pipeline

I've recently been working a Rails application CI to Azure Devops Pipelines feature. As part of the migration, the app is also being bundled into Docker containers (for CI to get started).

One of Pipelines' features is allowing the setup of Docker containers to provide services, such as databases, to a job whilst running the job itself inside a container.

Unfortunately, when I configured a service container to run a MariaDB instance, the pipeline would try to run the jobs before the database had initialised:

Waiting for MySQL database to accept connectionsERROR 2002 (HY000): Can't connect to MySQL server on 'db' (115)
.ERROR 2002 (HY000): Can't connect to MySQL server on 'db' (115)
.ERROR 2002 (HY000): Can't connect to MySQL server on 'db' (115)
...

Service Containers in pipelines do make use of the image's HEALTHCHECKcommand, but at the time of writing, the MariaDB images don't appear to include a healthcheck; and the Pipelines config file doesn't support adding one at runtime, in the same way that you might in a docker-compose.yml file.

As a workaround, I tried sleeping the tests to wait for the database to initialise, but regardless of the sleep time this didn't seem to work. I also wasn't happy including a hard-coded "wait" time which, even if it had worked, might lead to randomly-failing job runs depending on conditions.

Eventually, I found a working solution by using a short step that repeatedly attempts to connect to the the database server.

To set this up for your own project, first, make sure the Docker image you are using for your tests installs the mysql command, e.g.:

# Dockerfile (repository/yourappimage)
FROM ruby 2.6

RUN apt-get update && apt-get install -y mariadb-client

# ...

CMD ["bundle", "exec", "rails", "server", "-b", "0.0.0.0"]

Next, in your azure-pipelines.yml file, configure the service and job containers:

# azure-pipelines.yml
trigger:
- master

resources:
  containers:
  - container: db
    image: mariadb:10.1
    env:
      MYSQL_DATABASE: my_app_test
      MYSQL_ROOT_PASSWORD: superSecret123

jobs:
- job: specs
  displayName: Run the specs

  pool:
    vmImage: 'ubuntu-latest'

  services:
    db: db

  container:
    image: 'repository/yourappimage'
    endpoint: 'your-docker-hub-or-acr-service-connection'
    env:
      RAILS_ENV: test
      RACK_ENV: test
      DATABASE_URL: mysql2://root:superSecret123@db:3306/my_app_test

  steps:
  - script: |
      printf 'Waiting for MySQL database to accept connections'
      until mysql --host db --user=root --password=superSecret123 --execute "SHOW DATABASES"; do
        printf '.'
        sleep 1;
      done;
    displayName: Wait for database to initialise

  - script: bundle exec rails db:create db:migrate
    displayName: Prepare test db

  - script: bundle exec rails test
    displayName: Run specs

The step that checks for connectivity to the database server is at at line 32. It attempts to run a simple SHOW DATABASES command from our job container.

If that command fails, it sleeps for 1 second, and then tries again, repeating this process until the connection is successful (or the job times out).

This prevents the next step - creating the test database schema - from running until our app's container has confirmed that it can connect to the database server.

I hope this is useful for anyone using Azure Devops Pipelines as part of their CI setup. It may also be applicable to other CI services, including the new Github Actions, which allow the use of service containers to provide services to your tests.