How to Write Better Flutter Tests By Using Custom Matchers

Flutter has many advantages, a single language for development, a Flutter unit converter, simplified initial setup, and so on. But there are also the Flutter tests. 

And writing tests can be tedious, and it can seem repetitive when you have to do a lot of related validations in different test cases. This article examines how Flutter developers can improve this situation by demonstrating the use of custom matchers in Flutter unit tests.

Prerequisites:

This article assumes a general knowledge of Dart. Check out https://dart.dev/guides/language/language-tour for a tour if you need an introduction to Dart.

It also assumes a general knowledge of unit testing in Flutter/Dart. Check out https://docs.flutter.dev/testing if you need an introduction to unit testing.

The code in this article is written in Flutter Channel stable, 3.0.2.

Before starting, add the test package to the pubspec.yaml file under the devDependencies section:

dev_dependencies:
test: ^1.21.2

What Are Matchers?

Matchers are abstracted validation logic that are used in tests to ensure that the specific conditions being tested are met.

An example of a matcher is the isEmpty matcher used to check if the object being tested is empty.

The usage is like this below:

 test('Empty string test', () {
      const String emptyString = '';

      expect(emptyString, isEmpty);
    });

This test passes because under the hood, the isEmpty matcher checks if the string satisfies the emptyString.isEmpty condition.

Matchers can be combined to validate a specific behavior. For instance, the isNot matcher can be combined with the isEmpty matcher to check if the test object is not empty.

 test('Non-empty string test', () {
      const String nonEmptyString = 'hello';

      expect(nonEmptyString, isNot(isEmpty));
    });

The flutter test package exports the matcher package that contains several useful matchers that can be found in the matcher documentation.

Why Use Matchers

ABSTRACTION:

They help in abstracting away validation logic and they make tests more concise and more readable.

Let’s see a demonstration.

Say we want to test the GameService class below to validate that its play method sets up the variables of the class correctly.

class GameService {
  bool isRunning = false;
  String title = '';

  void play() {
    isRunning = true;
    title = 'Press Stop To End Game';
  }
}

The test to validate that play() does the correct setup is shown below. Let’s call this Non-Matcher Test.

Non-Matcher Test:

import 'package:flutter_custom_matcher_demo/game_service.dart';
import 'package:test/test.dart';

void main() {
  group('GameService', () {
    test('.play() sets up the game variables correctly', () {
      final GameService gameService = GameService();

      gameService.play();

      expect(gameService.isRunning, true);
      expect(gameService.title, 'Press Stop To End Game');
    });
  });
}

In the test above, we are validating the following:

• that gameService.isRunning returns true, and
• that gameService.title returns “Press Stop To End Game”.

While this is a correct test, this looks like we’re validating the specific properties when we’re actually validating the actual class.

We can improve on this by abstracting it into a custom matcher.

class GameIsSetUp extends Matcher {
  final String expectedTitleString = "Press Stop To End Game";

  @override
  bool matches(Object? item, Map matchState) {
    final GameService gameService = item as GameService;

    final bool isGameRunning = gameService.isRunning;
    final bool isTitleStringCorrect = gameService.title == expectedTitleString;

    return isGameRunning && isTitleStringCorrect;
  }

  @override
  Description describe(Description description) {
    return description.add('GameService variables correctly set up');
  }

  @override
  Description describeMismatch(Object? item, Description mismatchDescription,
      Map matchState, bool verbose) {
    final GameService gameService = item as GameService;

    if (!gameService.isRunning) {
      mismatchDescription
          .add('expected true but got ${gameService.isRunning}\n');
    }

    if (gameService.title != expectedTitleString) {
      mismatchDescription.add(
          'expected "$expectedTitleString" but got "${gameService.title}"\n');
    }

    mismatchDescription.add('Check config for new values');

    return mismatchDescription;
  }
}

The GameIsSetUp matcher contains two required overridden methods (matches and describe) and one optional overriden method(describeMismatch).

The overridden methods perform the following functions:

• matches: This is the method that contains the actual matching logic. It provides the object passed in the expect method in the test in the item variable.
  
  We cast this object into as GameService object, and with that, we can do our actual validation.

• describe: This is the method that holds the description of the matcher. 

The text “GameService variables correctly set up” that we add to the description object describes the matcher’s purpose, which is useful when we have a mismatch.

• describeMismatch: This is the method that holds the description of a mismatch. 

The logic in the method above checks the conditions to see which conditions fail and allows us specify our own custom messages.

Let’s take a look at the usage of the GameIsSetUp in our unit test. Let’s call this Matcher Test.

Matcher Test:

import 'package:flutter_custom_matcher_demo/game_service.dart';
import 'package:test/test.dart';

void main() {
  group('GameService', () {
    test('.play() sets up the game variables correctly', () {
      final GameService gameService = GameService();

      gameService.play();

      expect(gameService, GameIsSetUp());
    });
  });
}

Now we have condensed the logic into one object, the GameIsSetUp() object.

This is particularly useful when you have a very complex validation logic, especially one that has to be repeated in more than one Flutter test automation.

BETTER ERROR MESSAGE ON MISMATCH:

When we encounter a mismatch when we run our tests without custom matchers, we get the default mismatch messages. And if you compare Flutter vs React Native, it’s fair to say that Flutter has the advantage of having these messages.

These messages are great; they help you identify your exact issues, but sometimes you need to add your custom messages and provide more contextual information.

That is where you need the describe method and the describeMismatch method of the custom matcher.

Let’s do a demonstration.

Update the GameService‘s play method to this below:

void play() {
    isRunning = false; // Updated For Demonstration
    title = 'Press Start To End Game'; // Updated For Demonstration
  }

We expect the two validations to fail since we changed two of the conditions required for the test to pass.

Running the Non-Matcher Test gives the result below:

  Expected: <true>
    Actual: <false>

From this result, we see that we only get an error for the first validation that is,  expect(gameService.isRunning, true); even though both validations failed.

Running the Matcher Test gives the result below:

  Expected: GameService variables correctly set up
    Actual: <Instance of 'GameService'>
    Which: expected true but got false
            expected "Press Stop To End Game" but got "Press Start To End Game"
           
            check config for new values

We can see from here that the Expected value is what we return in our describe method, and the Which value contains what we return in our describeMismatch method.

With this, we can return errors for both failed validations and have more context on how to solve this mismatch than we would have by not using a custom matcher.

EASIER MODIFICATION OF VALIDATION LOGIC:

Using custom matchers makes it easier to modify validation logic without necessarily touching the test code.

This is especially important when multiple tests use the same validation logic. It is easier to change the logic in the custom matcher class instead of changing the logic in every tests that uses the logic.

Assume GameService gets updated to include an extra property, numberOfLives and the play method gets updated to assign the int value, 10 to numberOfLives as shown below:

class GameService {
  bool isRunning = false;
  String title = '';
  int numberOfLives = 0;  // Updated

  void play() {
    isRunning = true;
    title = 'Press Stop To End Game';
    numberOfLives = 10;  // Updated
  }
}

If we wrote the test without a custom matcher, that is, Non-Matcher Test, that would require updating the test to include a specific validation for numberOfLives being equal to 10 like this:

import 'package:flutter_custom_matcher_demo/game_service.dart';
import 'package:test/test.dart';

void main() {
  group('GameService', () {
    test('.play() sets up the game variables correctly', () {
      final GameService gameService = GameService();

      gameService.play();

      expect(gameService.isRunning, true);
      expect(gameService.title, 'Press Stop To End Game');
      expect(gameService.numberOfLives, 10);
    });
  });
}

This will become hectic if this validation logic is used in numerous tests. 

Using a Custom Matcher helps improve this experience by only updating the validation logic in the GameIsSetUp while the tests stay untouched.

Our updated GameIsSetUp matcher class will be: 

class GameIsSetUp extends Matcher {
  final String expectedTitleString = "Press Stop To End Game";
  final int expectedNumberOfLives = 10;

  @override
  bool matches(Object? item, Map matchState) {
    final GameService gameService = item as GameService;

    final bool isGameRunning = gameService.isRunning;
    final bool isTitleStringCorrect = gameService.title == expectedTitleString;
    final bool isNumberOfLivesCorrect =
        gameService.numberOfLives == expectedNumberOfLives;  // Updated

    return isGameRunning && isTitleStringCorrect && isNumberOfLivesCorrect;  // Updated
  }

  @override
  Description describe(Description description) {
    return description.add('GameService variables correctly set up');
  }

  @override
  Description describeMismatch(Object? item, Description mismatchDescription,
      Map matchState, bool verbose) {
    final GameService gameService = item as GameService;

    if (!gameService.isRunning) {
      mismatchDescription
          .add('expected true but got ${gameService.isRunning}\n');
    }

    if (gameService.title != expectedTitleString) {
      mismatchDescription.add(
          'expected "$expectedTitleString" but got "${gameService.title}"\n');
    }

    if (gameService.numberOfLives != expectedNumberOfLives) {
      mismatchDescription.add(
          'expected "$expectedNumberOfLives" but got "${gameService.numberOfLives}"\n');
    } // Updated

    mismatchDescription.add('\nCheck config for new values');

    return mismatchDescription;
  }
}

This promotes code reuse and test efficiency.

Conclusion

We have taken a look at matchers in Flutter unit tests, and we have also seen the importance of matchers and how they improve the experience of writing tests by abstracting the validation logic, providing more contextual error messages, and allowing easier modification of validation logic. It’s valuable to know how to test Flutter app, but how to write better Flutter tests by using custom matchers is essential since it can become tedious, and knowing how to improve this situation is beneficial.