I am trying to only run the test StorefrontVerifySecureURLRedirectCheckout. To run it I am using the following command:
vendor/bin/mftf run:test StorefrontVerifySecureURLRedirectCheckout -k
(I have to use -k to skip generating due to an error see the update)
When executing the above command it executes much more tests than just the desired one:
Magento\FunctionalTestingFramework.functional Tests (935) ----------------------
Modules: \Magento\FunctionalTestingFramework\Module\MagentoWebDriver, \Magento\FunctionalTestingFramework\Helper\Acceptance, \Magento\FunctionalTestingFramework\Helper\MagentoFakerData, \Magento\FunctionalTestingFramework\Module\MagentoSequence, \Magento\FunctionalTestingFramework\Module\MagentoAssert, Asserts
--------------------------------------------------------------------------------
AdminAddConfigurableProductWithDropDownAttributeAndCustomSourceTestCest:
Signature: Magento\AcceptanceTest\_MSI_Multi_Mode\Backend\AdminAddConfigurableProductWithDropDownAttributeAndCustomSourceTestCest:AdminAddConfigurableProductWithDropDownAttributeAndCustomSourceTest
Test: tests/functional/Magento/FunctionalTest/_generated/MSI_Multi_Mode/AdminAddConfigurableProductWithDropDownAttributeAndCustomSourceTestCest.php:AdminAddConfigurableProductWithDropDownAttributeAndCustomSourceTest
Scenario --
/******** Beginning execution of MSI_Multi_Mode suite before block ********/
[...]
/******** Execution of MSI_Multi_Mode suite before block complete ********/
[...]
AdminAddConfigurableProductWithDropDownAttributeWithNewSourceTestCest: Admin add configurable product with drop down attribute with new source test
Signature: Magento\AcceptanceTest\_MSI_Multi_Mode\Backend\AdminAddConfigurableProductWithDropDownAttributeWithNewSourceTestCest:AdminAddConfigurableProductWithDropDownAttributeWithNewSourceTest
Test: tests/functional/Magento/FunctionalTest/_generated/MSI_Multi_Mode/AdminAddConfigurableProductWithDropDownAttributeWithNewSourceTestCest.php:AdminAddConfigurableProductWithDropDownAttributeWithNewSourceTest
Scenario --
[..]
It seems like the whole MSI_Multi_Mode suite is being executed.
I've tried it with the example provided in the getting started guide and this works just fine. Only AdminLoginTestCest is executed.
I've compared both tests but I didn't find any difference that could lead to such a behavior.
Update:
I found the following in the official docs:
If
<test>is included in<suite>, it cannot be generated in isolation to the rest of the contents of the suite (see suites for details).
If I understand it correctly is that if a test is included it cannot be run separately.
The test StorefrontVerifySecureURLRedirectCheckout actually is defined in the suite SecureStorefrontURLSuite due to SecureStorefrontURLSuite.xml which includes the group secure_storefront_url.
When executing vendor/bin/mftf run:test StorefrontVerifySecureURLRedirectCheckout without the -k argument as written above it will result in the following error:
In TestGenerator.php line 147:
Cannot reference test configuration for generation without accompanying suite.
How is it possible to run a single test that is included in a suite?
A possible solution might be to create a new suite that only includes this one test but it doesn't seems more like a hack to me and it's quite some effort to always create or change suites just to run a single test. I wonder if there is any other solution for this problem.
Update 2:
I just verified, it is possible to wrap the single test in a suite and then execute the suite with vendor/bin/mftf run:group SingleTestSuiteName --remove but this IS NOT really an solution/answer I am looking for due to the inconveniences it brings with having to keep changing the suite all the time.
