What is the proper way for unit testing PHP7 code with PHPUnit 4.1 in Magento 2?

Question

When I'm writing my modules, I'm trying to supply them with unit-tests for the most critical parts of the application. However, there are at the moment (Magento 2.1.3) several ways on how to write unit tests:

Different ways of testing

• Integrate it with bin/magento dev:tests:run unit and run it on top of the default phpunit settings bundled with Magento.
• Write them separately, run them with vendor/bin/phpunit app/code/Vendor/Module/Test/Unit and mock everything that is Magento.
• Write them separately, mock everything and use a system-global version of PHPUnit.
• Write them separately, run them with vendor/bin/phpunit, but still make use of \Magento\Framework\TestFramework\Unit\Helper\ObjectManager.

Magento 2 and PHPUnit

Besides that, Magento 2 comes bundled with PHPUnit 4.1.0, which is not PHP7-compatible. Type-hinting natives (like string and `int) and declaring return types in your signatures will throw errors. For example, an interface / class with a method signature like this:

public function foo(string $bar) : bool;

... will not be able to be mocked by PHPUnit 4.1.0. :-(

My current situation

It's due because of this that I'm now mostly writing my unit tests in the third way (by calling a system-global PHPUnit version).

In my setup, I have PHPUnit 5.6 installed globally, so I can solve write proper PHP7-code, but I have to do some tweaks. For example:

phpunit.xml has to look like this so I can make use of the composer autoloader:

<?xml version="1.0"?>
<phpunit bootstrap="../../../../../../vendor/autoload.php"
         colors="true">
    <testsuites>
        <testsuite name="Testsuite">
            <directory>.</directory>
        </testsuite>
    </testsuites>
</phpunit>

...and in all my setUp()-methods, I have the following check so I can write my tests with forward-compatibility:

// Only allow PHPUnit 5.x:
if (version_compare(\PHPUnit_Runner_Version::id(), '5', '<')) {
    $this->markTestSkipped();
}

This way, when my tests are ran by Magentos' built-in PHPUnit, it doesn't throw an error.

My question

So here's my question: is this a 'healthy' way of writing unit tests? Because it doesn't seem right to me that Magento comes bundled with a whole bunch of tools to aid with testing and I can't use them because I'm using PHP7. I know there are tickets on GitHub that address this issue, but I'm wondering how the community is currently writing it's tests.

Is there a way to write unit tests in Magento 2 so I don't have to 'downgrade' my code and still can use Magentos' built-in helpers for mocking everything the object manager touches? Or is it even bad practice to use the object manager even in your unit tests?

I'm missing a lot of guidance / examples in what's the proper way on how to unit test your own custom modules.

Answer 1

Using the bundled PHPUnit version, even if it's ancient, is probably the best way to go since that will allow running the tests for all modules together during CI.

I think writing tests in a manner that is incompatible with the bundled test framework greatly reduces the value of the tests.
Of course you could set up CI to run your tests with a different version of PHPUnit, but that adds a lot of complexity to the build system.

That said, I do agree with you that it's not worth supporting PHP 5.6. I use PHP7 scalar type hinting and return type hinting as much as possible (plus, I don't care about the market place).

In order to work around the limitations of the PHPUnit 4.1 mocking library there are at least two rather straight forward workarounds that I've used in the past:

1. Use anonymous or regular classes to build your test doubles, for example

   $fooIsFalseStub = new class extends Foo implements BarInterface() {
       public function __construct(){};
       public function isSomethingTrue(string $something): bool
       {
           return false;
       }
   };
   

2. Use the bundled PHPUnit but a third party mocking library that can be included via composer with require-dev, for example https://github.com/padraic/mockery. All mocking libraries I tried can very easily be used with any testing framework, even a very old version of PHPUnit like 4.1.

Neither of those has any technical advantage over the other. You can implement any required test double logic with either.

Personally I prefer using anonymous classes because that doesn't add to the number of external dependencies, and also it's more fun to write them that way.

EDIT:
To answer your questions:

Does Mockery 'solve' the problem of PHPUnit 4.1.0 not being able to properly handle PHP7 type hinting?

Yes, see the example below.

And what are the benefits of anonymous classes over mocking?

Using anonymous classes to create test doubles is also "mocking", it's not really different from using a mocking library such as PHPUnits or Mockery or another.
A mock is just on specific type of test double, regardless of how it is created.
One small difference between using anonymous classes or a mocking library is that anonymous classes don't have an external library dependency, since it's just plain PHP. Otherwise there are no benefits or drawbacks. It's simply a matter of preference. I like it because it illustrates that testing is not about any test framework or mocking library, testing is just writing code that executes the system under test and automatically validates it works.

And how about updating the PHPUnit version in the main composer.json file to 5.3.5 (the latest version supporting PHP7 ánd having public mocking-methods (required by Magento 2's own tests))?

This might be problematic since the tests in other modules and the core are only tested with PHPUnit 4.1, and as such you might encounter false failures in CI. I think it's best to stick with the bundled version of PHPUnit for that reason. @maksek said they will be updating PHPUnit, but there is no ETA for that.

---

Example of a test with a test double of a class that requires PHP7 running with PHPUnit 4.1, using the Mockery library:

<?php

declare(strict_types = 1);

namespace Example\Php7\Test\Unit;

// Foo is a class that will not work with the mocking library bundled with PHPUnit 4.1 
// The test below creates a mock of this class using mockery and uses it in a test run by PHPUnit 4.1
class Foo
{
    public function isSomethingTrue(string $baz): bool
    {
        return 'something' === $baz; 
    }
}

// This is another class that uses PHP7 scalar argument types and a return type.
// It is the system under test in the example test below.
class Bar
{
    private $foo;

    public function __construct(Foo $foo)
    {
        $this->foo = $foo;
    }

    public function useFooWith(string $s): bool
    {
        return $this->foo->isSomethingTrue($s);
    }
}

// This is an example test that runs with PHPUnit 4.1 and uses mockery to create a test double
// of a class that is only compatible with PHP7 and younger.
class MockWithReturnTypeTest extends \PHPUnit_Framework_TestCase
{
    protected function tearDown()
    {
        \Mockery::close();
    }

    public function testPHPUnitVersion()
    {
        // FYI to show this test runs with PHPUnit 4.1
        $this->assertSame('4.1.0', \PHPUnit_Runner_Version::id());
    }

    public function testPhpVersion()
    {
        // FYI this test runs with PHP7
        $this->assertSame('7.0.15', \PHP_VERSION);
    }

    // Some nonsensical example test using a mock that has methods with
    // scalar argument types and PHP7 return types.
    public function testBarUsesFoo()
    {
        $stubFoo = \Mockery::mock(Foo::class);
        $stubFoo->shouldReceive('isSomethingTrue')->with('faz')->andReturn(false);
        $this->assertFalse((new Bar($stubFoo))->useFooWith('faz'));
    }
}

Answer 2

Right now Magento 2 supports next PHP versions:

"php": "~5.6.5|7.0.2|7.0.4|~7.0.6"

It means that all code written by Magento Team works on every supported version.

Hence Magento Team does not use PHP 7 only features. PHP 5.6 features can be covered with PHPUnit 4.1.0.

Writing your own code you can do all you want and write tests in any way you like. But I believe that you will not be able publishing your extension on Magento Marketplace because of requirements violation.