Coverage.py has a large test suite that runs in many environments, which can take a while. But some changes don’t require running the test suite at all. I’ve changed the actions to detect when they need to run based on what files have changed, but there were some twists and turns along the way.

The dorny/paths-filter action can check which files have changed for pull requests or branches. I added it to my tests action like this:

jobs:

  changed:
    name: "Check what files changed"
    outputs:
      python: ${{ steps.filter.outputs.python }}
    steps:
      - name: "Check out the repo"
        uses: actions/checkout

      - name: "Examine changed files"
        uses: dorny/paths-filter
        id: filter
        with:
          filters: |
            python:
              - "**.py"

  tests:
    # Don't run tests if the branch name includes "-notests".
    # Only run tests if Python files changed.
    needs: changed
    if: ${{ !contains(github.ref, '-notests') && needs.changed.outputs.python == 'true' }}

The “changed” jobs checks what files have changed, then the “tests” job examines its output to decide whether to run at all.

It’s a little awkward having an output for the “changed” job as an intermediary, but this did what I wanted: if any .py file changed, run the tests, otherwise don’t run them. I left in an old condition: if the branch name includes “-notests”, then don’t run the tests.

This worked, but I realized I needed to run the tests on other conditions also. What if no Python file changed, but the GitHub action file itself had changed? So I added that as a condition. The if-expression was getting long, so I made it a multi-line string:

jobs:

  changed:
    name: "Check what files changed"
    outputs:
      python: ${{ steps.filter.outputs.python }}
      workflow: ${{ steps.filter.outputs.workflow }}
    steps:
      - name: "Check out the repo"
        uses: actions/checkout

      - name: "Examine changed files"
        uses: dorny/paths-filter
        id: filter
        with:
          filters: |
            python:
              - "**.py"
            workflow:
              - ".github/workflows/testsuite.yml"

  tests:
    # Don't run tests if the branch name includes "-notests".
    # Only run tests if Python files or this workflow changed.
    needs: changed
    if: |
      ${{
        !contains(github.ref, '-notests')
        && (
          needs.changed.outputs.python == 'true'
          || needs.changed.outputs.workflow == 'true'
        )
      }}

This seemed to work, but it has a bug that I will get to in a bit.

Thinking about it more, I realized there are other files that could affect the test results: requirements files, test output files, and the tox.ini. Rather than add them as three more conditions, I combined them all into one:

jobs:

  changed:
    name: "Check what files changed"
    outputs:
      run_tests: ${{ steps.filter.outputs.run_tests }}
    steps:
      - name: "Check out the repo"
        uses: actions/checkout

      - name: "Examine changed files"
        uses: dorny/paths-filter
        id: filter
        with:
          filters: |
            run_tests:
              - "**.py"
              - ".github/workflows/testsuite.yml"
              - "tox.ini"
              - "requirements/*.pip"
              - "tests/gold/**"

  tests:
    # Don't run tests if the branch name includes "-notests".
    # Only run tests if files that affect tests have changed.
    needs: changed
    if: |
      ${{
        needs.changed.outputs.run_tests == 'true'
        && !contains(github.ref, '-notests')
      }}

BTW: these commits also update the quality checks workflow which has other kinds of mix-and-match conditions to deal with that you might be interested in.

All seemed good! Then I made a commit that only changed my Makefile, and the tests ran! Why!? The Makefile isn’t one of the checked files. The paths-filter action helpfully includes debug output that showed that only the Makefile was considered changed, and that the “run_test” output was false.

I took a guess that GitHub actions don’t like expressions with newlines in them. Using the trusty YAML multi-line string cheat sheet, I tried changing from the literal block style (with a pipe) to the folded style (with a greater-than):

if: >
  ${{
    needs.changed.outputs.run_tests == 'true'
    && !contains(github.ref, '-notests')
  }}

The literal form includes all newlines, the folded style turns newlines into spaces. To check that I had it right, I tried parsing the YAML files: to my surprise, both forms included all the newlines, there was no difference at all. It turns out that YAML “helpfully” notices changes in indentation, and includes newlines for indented lines. My expression is nicely indented, so it has newlines no matter what syntax I use.

The GitHub actions docs don’t mention it, but it seems that newlines do break expression evaluation. Sigh. My expressions are not as long now as they had gotten during this exploration, so I changed them all back to one line, and now it all works as I wanted.

There are some other things I’d like to tweak: when the tests are skipped, the final status is “success”, but I’m wondering if there’s a way to make it “skipped”. I’m also torn about whether every change to master should run all the workflows or if they should also filter based on the changed files. Currently they are filtered.

Continuous integration and GitHub workflows are great, but they always seem to involve this kind of fiddling in environments that are difficult to debug. Maybe I’ve saved you some grief.