How to Set Up Different Test Profiles in Gradle for Android

Running tests can be resource-intensive, both in terms of compute time and actual costs, especially when tests interact with external services. As the test suite grows, running all tests every time becomes inefficient. By separating tests into different profiles, you can optimize execution time, particularly for integration tests that rely on external services.

Depending on your framework, there might already be more straightforward solutions for this already. However, Android, when combined with Gradle and JUnit 4, doesn’t seem to have native support for this setup.

While Android provides build flavors and allows executing specific tests per flavor, this doesn’t address the problem of selectively running different test types against the same codebase.

Another approach is to extend upon JUnit TestRunner’s as in this blog post. While this is an idiomatic JUnit 4 solution, it involves a lot of boilerplate code, which can be cumbersome to maintain.

Gradle TestFilters and project properties

By leveraging Gradle Test Filters and Project Properties, you can define different test profiles based on naming conventions without adding much complexity to your test code.

With Test Filters, you can separate test profiles by package, class, or method naming conventions. Then, using Project Properties, you can configure these filters to run specific test sets as needed.

In the following example we create our test profiles using class name conventions:

• Integration tests contain IntegrationTest in their class name.
• Special flavors can be appended, e.g. IntegrationTestFlavor1.

You can select a test profile by passing a Gradle project property, such as -PintegrationTests=flavor1.

If no property is passed, Gradle will execute all other tests (e.g., unit tests) by default.

Gradle Execution

Here’s how you can execute the tests with Gradle from the command line:

# Run tests with specific integration test profile
$ gradle -PintegrationTests=flavor1 check

# Run all integration tests
$ gradle -PintegrationTests check

# Run all other tests
$ gradle check

Kotlin Gradle configuration

In the Kotlin-based Gradle configuration, we use a simple when statement to match the integration test flavor based on the property passed.

android {}

tasks.withType<Test> {
    filter {
        if (project.hasProperty("integrationTests")) {
            val testProfile = project.property("integrationTests") as String

            when (testProfile) {
                "flavor1" -> includeTestsMatching("IntegrationTestFlavor1")
                "flavor2" -> includeTestsMatching("IntegrationTestFlavor2")
                else -> includeTestsMatching("*IntegrationTest*")
            }
        } else {
            excludeTestsMatching("*IntegrationTest*")
        }
    }
}

If no property is provided, all non-integration tests will be executed by default.

You can enhance this setup further by dynamically changing the filter based on the property value. But let’s keep it simple.

Groovy Gradle configuration

Here’s the slightly more verbose Groovy version of the configuration:

android {}

tasks.withType(Test).configureEach {
  test {
    filter {
      if (project.hasProperty("integrationTests")) {
        def testProfile = project.property("integrationTests")
        switch (testProfile) {
          case "flavor1":
            includeTestsMatching 'IntegrationTestFlavor1'
            break
          case "flavor2":
            includeTestsMatching 'IntegrationTestFlavor2'
            break
          default:
            includeTestsMatching '*IntegrationTest*'
            break
        }
      } else {
        excludeTestsMatching '*IntegrationTest*'
      }
    }
  }
}

Conclusion

With this setup, you can efficiently manage and execute different test profiles in your Android project, keeping your test runs fast and focused. By using Gradle’s Test Filters and Project Properties, you gain more control over your test suite without adding unnecessary complexity.