As an integrator, after creating a Magento 2.2 project, should I expect the provided Travis CI configuration to work properly?
The Magento Testing Guide explains how to run individual test suites, but does not provide any guidance for the provided
Why I want this
To meet the requirements of <project>, I'm going to be adding and customizing Magento 2 modules of all kinds: console commands, payment gateways, themes, shipping providers, API endpoints, etc. During this process, I think it would be really nice if an automated build system could tell me, "hey, you goofed and broke this important core thing".
It would also be really nice if there were a standard "Magento 2" way of making this happen, instead of everybody having to reinvent their own square wheel of testing. After creating a project using the given template, one would think that the provided configuration would either (a) work properly, or (b) be documented in such a way that an integrator could get up and running quickly.
Project setup steps
I've created a brand new, fresh Magento 2.2 project by following the "integrator" path, creating a project via composer by fetching the Magento CE metapackage. More specifically, I ran the following command:
$ composer create-project --repository-url=https://repo.magento.com/ magento/project-community-edition testrepo
The resulting project directory comes with (among other things) a
.travis.yml file. I've configured the connection between this project and Travis CI, including valid credentials for the custom Magento composer repository. Unfortunately, the Travis CI build doesn't work out of the box.
Some issues I've run into using the provided configuration (and the workarounds I've tried):
dev/travis/before_*.shscripts are not executable. This causes the build to error out.
Workaround: Set executable bit on scripts.
I created the project on a machine with PHP 7.1.9, so the
composer.lockfile ended up with dependency versions which are incompatible with PHP 7.0. One of the specified PHP versions in the Travis matrix is 7.0, which won't work at all.
Workaround: Remove PHP 7.0 from the build matrix.
composer installruns, it overwrites the dev/ directory. This undoes the changes to my travis scripts, making them non-executable again!
Workaround: Set executable bit on scripts in the Travis config, directly before running them.
After these changes, it works! Sort of...
- JS specs
- Some integration tests
- Functional tests (wow, these take a long time)
- PHP Unit tests
TEST_SUITE=unit(very few failures)
- Static analysis / lint
TEST_SUITE=static(fails with lint errors on core stuff that I can't change)
TEST_SUITE=js GRUNT_COMMAND=static(holy cow, all the errors!)
- Some integration tests
TEST_SUITE=integration INTEGRATION_INDEX=3(fails on reading an XML file from the vendor directory)
Back to the question (and follow-up questions)
.travis.yml intended to be used by integrators?
- Why did I trip over so many hurdles while attempting to set it up?
- What did I do wrong, and/or how do I fix it so that it works as intended?
- Why is the
.travis.ymlfile included in the generated project directory?
- Why is my custom
.travis.ymlfile overwritten with the default one each time I run