These are quite a lot questions at once, but I can answer at least some:
- Is there any reason why M2 use testing libs (PHPUnit and PHP_CS) from 2014 instead of fresh one?
Major development in Magento 2 started around 2014, so they used the tools available at that time. When PHPUnit 5 came out, there was already a huge number of tests that were not compatible with the new version (see this forum thread for an example), so it's understandable that they postponed an update.
I assume, the reasons to stick with an old PHP_CS version are similar, though I don't have a concrete example here.
- Is normal that output of this test looks like a mess and it's hard to understand what and where something wrong happens? I compare it
with output of code quality tests for CSS/JS and it's nightmare. Is
there any better reporter available or other way to get meaningful
report, instead of something looking like a PHP backtrace?
IDEs like PHPStorm have good integration with these tools, where you can see code sniffer results directly in the source files and get a nice GUI around PHPUnit tests as well.
Besides that, PHPUnit has various output options. For example with the
--testdox argument, you'll get a human readable checklist of passed and failed tests. It provides less infos but a readable overview. You can also get it in HTML format with
--testdox-html=OUTPUTFILE. Similarily you can get the code coverage report in HTML with
But the more useful output formats are XML and JSON formats that can be read by other applications like VisualPHPUnit or CI servers.
PHPUnit parameters for report generation:
Code Coverage Options:
--coverage-clover <file> Generate code coverage report in Clover XML format.
--coverage-crap4j <file> Generate code coverage report in Crap4J XML format.
--coverage-html <dir> Generate code coverage report in HTML format.
--coverage-php <file> Export PHP_CodeCoverage object to file.
--coverage-text=<file> Generate code coverage report in text format.
Default: Standard output.
--coverage-xml <dir> Generate code coverage report in PHPUnit XML format.
--log-junit <file> Log test execution in JUnit XML format to file.
--log-tap <file> Log test execution in TAP format to file.
--log-json <file> Log test execution in JSON format.
--testdox-html <file> Write agile documentation in HTML format to file.
--testdox-text <file> Write agile documentation in Text format to file.
More info: https://phpunit.de/manual/current/en/textui.html
PHP_CS parameters for report generation
PHP_CS also has different report formats:
--report=xml PHP_CS XML format
--report=checkstyle Checkstyle XML format
(other formats: emacs, svnblame, gitblame)
More info: https://github.com/squizlabs/PHP_CodeSniffer/wiki/Reporting
- Is there any reason why it's so slow? It take ~7-8 minutes to analyse template files. Most of front-end tests in worst case take a
few seconds, so there is no way to get live feedback about issues.
I can't tell what the reasons are for PHP_CS to take 8 minutes just for template files, but it should be possible for your watcher to only check changed files. The PHPStorm integration does this quite well.
- How to run this type of tests when we have single module (i.e. theme), not whole Magento 2 instance (CI tests)?
phpcs /path/to/theme to only check files in this directory.
- It looks like PHP_CS has already a simple wrapper for Gulp, but I'm not sure where configuration is stored. It's in /.php_cs file?
It doesn't look like this wrapper includes a file watcher, so I don't see the benefit.
.php_cs file defines which files to check and which coding standards to use. This is a PHP_CS configuration file and independent from the gulp wrapper.