2

TL;DR

I am getting the above error when running integration tests on a CI docker environment.

The problem seems that somewhere within $this->dispatch() the test fails, but does not output any error message or logging (neither var/log nor dev/tests/integration/tmp/sandbox/var/log) The question is how to debug that.


We have set up a CI to run integration tests.

bash-5.0$ cd dev/tests/integration; ../../../vendor/bin/phpunit -v --testsuite 'Our Tests'
PHPUnit 6.5.14 by Sebastian Bergmann and contributors.

Runtime:       PHP 7.2.31
Configuration: /var/www/dev/tests/integration/phpunit.xml

IIPHP Fatal error:  Uncaught PHPUnit\Framework\Exception: User Error: Some transactions have not been committed or rolled back in /var/www/vendor/magento/framework/DB/Adapter/Pdo/Mysql.php:3977.

/var/www/dev/tests/integration/framework/bootstrap.php:144
/var/www/vendor/magento/framework/DB/Adapter/Pdo/Mysql.php:3977

  thrown in /var/www/dev/tests/integration/framework/bootstrap.php on line 144

=== Memory Usage System Stats ===
Memory usage (OS):  148.21M (151.24% of 98.00M reported by PHP)
Estimated memory leak:  50.21M (33.88% of used memory)


Fatal error: Uncaught PHPUnit\Framework\Exception: User Error: Some transactions have not been committed or rolled back in /var/www/vendor/magento/framework/DB/Adapter/Pdo/Mysql.php:3977.

/var/www/dev/tests/integration/framework/bootstrap.php:144
/var/www/vendor/magento/framework/DB/Adapter/Pdo/Mysql.php:3977

  thrown in /var/www/dev/tests/integration/framework/bootstrap.php on line 144

This is not happening locally, so it is a bit hard to debug.

How to get a proper error message to lead to the root cause of the problem?

EDIT I started debugging like this:

vi /var/www/vendor/magento/framework/DB/Adapter/Pdo/Mysql.php

/**                                             
 * Begin new DB transaction for connection
 *                                   
 * @return $this                       
 * @throws \Exception
 */                       
public function beginTransaction()
{
    echo "==== TRANSACTION BEGIN" . PHP_EOL;
    debug_print_backtrace(DEBUG_BACKTRACE_IGNORE_ARGS);

I saw, that indeed two transactions are opened, the first I don't know why:

==== TRANSACTION BEGIN
#0  Magento\Framework\DB\Adapter\Pdo\Mysql->beginTransaction() called at [/var/www/dev/tests/integration/tmp/sandbox-0-f68414eb86eeffb7401faa5c09c05812c4aa222769de918431c0368e2f700667/generated/code/Magento/TestFramework/Db/Adapter/Mysql/Interceptor.php:63]
#1  Magento\TestFramework\Db\Adapter\Mysql\Interceptor->beginTransaction() called at [/var/www/dev/tests/integration/framework/Magento/TestFramework/Db/Adapter/Mysql.php:29]
#2  Magento\TestFramework\Db\Adapter\Mysql->beginTransparentTransaction() called at [/var/www/dev/tests/integration/tmp/sandbox-0-f68414eb86eeffb7401faa5c09c05812c4aa222769de918431c0368e2f700667/generated/code/Magento/TestFramework/Db/Adapter/Mysql/Interceptor.php:24]
#3  Magento\TestFramework\Db\Adapter\Mysql\Interceptor->beginTransparentTransaction() called at [/var/www/dev/tests/integration/framework/Magento/TestFramework/Event/Transaction.php:93]
#4  Magento\TestFramework\Event\Transaction->_startTransaction() called at [/var/www/dev/tests/integration/framework/Magento/TestFramework/Event/Transaction.php:81]
#5  Magento\TestFramework\Event\Transaction->_processTransactionRequests() called at [/var/www/dev/tests/integration/framework/Magento/TestFramework/Event/Transaction.php:46]
#6  Magento\TestFramework\Event\Transaction->startTest()
#7  call_user_func_array() called at [/var/www/dev/tests/integration/framework/Magento/TestFramework/EventManager.php:49]
#8  Magento\TestFramework\EventManager->fireEvent() called at [/var/www/dev/tests/integration/framework/Magento/TestFramework/Event/PhpUnit.php:126]
#9  Magento\TestFramework\Event\PhpUnit->startTest() called at [/var/www/vendor/phpunit/phpunit/src/Framework/TestResult.php:395]
#10 PHPUnit\Framework\TestResult->startTest() called at [/var/www/vendor/phpunit/phpunit/src/Framework/TestResult.php:637]
#11 PHPUnit\Framework\TestResult->run() called at [/var/www/vendor/phpunit/phpunit/src/Framework/TestCase.php:894]
#12 PHPUnit\Framework\TestCase->run() called at [/var/www/vendor/phpunit/phpunit/src/Framework/TestSuite.php:755]
#13 PHPUnit\Framework\TestSuite->run() called at [/var/www/vendor/phpunit/phpunit/src/Framework/TestSuite.php:755]
#14 PHPUnit\Framework\TestSuite->run() called at [/var/www/vendor/phpunit/phpunit/src/Framework/TestSuite.php:755]
#15 PHPUnit\Framework\TestSuite->run() called at [/var/www/vendor/phpunit/phpunit/src/TextUI/TestRunner.php:545]
#16 PHPUnit\TextUI\TestRunner->doRun() called at [/var/www/vendor/phpunit/phpunit/src/TextUI/Command.php:195]
#17 PHPUnit\TextUI\Command->run() called at [/var/www/vendor/phpunit/phpunit/src/TextUI/Command.php:148]
#18 PHPUnit\TextUI\Command::main() called at [/var/www/vendor/phpunit/phpunit/phpunit:53]
==== TRANSACTION BEGIN
#0  Magento\Framework\DB\Adapter\Pdo\Mysql->beginTransaction() called at [/var/www/dev/tests/integration/tmp/sandbox-0-f68414eb86eeffb7401faa5c09c05812c4aa222769de918431c0368e2f700667/generated/code/Magento/TestFramework/Db/Adapter/Mysql/Interceptor.php:63]
#1  Magento\TestFramework\Db\Adapter\Mysql\Interceptor->beginTransaction() called at [/var/www/vendor/magento/framework/Model/ResourceModel/AbstractResource.php:66]
#2  Magento\Framework\Model\ResourceModel\AbstractResource->beginTransaction() called at [/var/www/vendor/magento/framework/Model/ResourceModel/Db/AbstractDb.php:402]
#3  Magento\Framework\Model\ResourceModel\Db\AbstractDb->save() called at [/var/www/app/code/Example/SomeDesign/Model/Design.php:84]
#4  Example\SomeDesign\Model\Design->setDesign() called at [/var/www/app/code/Example/SomeDesign/Test/_files/cdg_design.php:29]
#5  require(/var/www/app/code/Example/SomeDesign/Test/_files/cdg_design.php) called at [/var/www/dev/tests/integration/framework/Magento/TestFramework/Annotation/DataFixture.php:214]
#6  Magento\TestFramework\Annotation\DataFixture->_applyOneFixture() called at [/var/www/dev/tests/integration/framework/Magento/TestFramework/Annotation/DataFixture.php:244]
#7  Magento\TestFramework\Annotation\DataFixture->_applyFixtures() called at [/var/www/dev/tests/integration/framework/Magento/TestFramework/Annotation/DataFixture.php:92]
#8  Magento\TestFramework\Annotation\DataFixture->startTransaction()
#9  call_user_func_array() called at [/var/www/dev/tests/integration/framework/Magento/TestFramework/EventManager.php:49]
#10 Magento\TestFramework\EventManager->fireEvent() called at [/var/www/dev/tests/integration/framework/Magento/TestFramework/Event/Transaction.php:96]
#11 Magento\TestFramework\Event\Transaction->_startTransaction() called at [/var/www/dev/tests/integration/framework/Magento/TestFramework/Event/Transaction.php:81]
#12 Magento\TestFramework\Event\Transaction->_processTransactionRequests() called at [/var/www/dev/tests/integration/framework/Magento/TestFramework/Event/Transaction.php:46]
#13 Magento\TestFramework\Event\Transaction->startTest()
#14 call_user_func_array() called at [/var/www/dev/tests/integration/framework/Magento/TestFramework/EventManager.php:49]
#15 Magento\TestFramework\EventManager->fireEvent() called at [/var/www/dev/tests/integration/framework/Magento/TestFramework/Event/PhpUnit.php:126]
#16 Magento\TestFramework\Event\PhpUnit->startTest() called at [/var/www/vendor/phpunit/phpunit/src/Framework/TestResult.php:395]
#17 PHPUnit\Framework\TestResult->startTest() called at [/var/www/vendor/phpunit/phpunit/src/Framework/TestResult.php:637]
#18 PHPUnit\Framework\TestResult->run() called at [/var/www/vendor/phpunit/phpunit/src/Framework/TestCase.php:894]
#19 PHPUnit\Framework\TestCase->run() called at [/var/www/vendor/phpunit/phpunit/src/Framework/TestSuite.php:755]
#20 PHPUnit\Framework\TestSuite->run() called at [/var/www/vendor/phpunit/phpunit/src/Framework/TestSuite.php:755]
#21 PHPUnit\Framework\TestSuite->run() called at [/var/www/vendor/phpunit/phpunit/src/Framework/TestSuite.php:755]
#22 PHPUnit\Framework\TestSuite->run() called at [/var/www/vendor/phpunit/phpunit/src/TextUI/TestRunner.php:545]
#23 PHPUnit\TextUI\TestRunner->doRun() called at [/var/www/vendor/phpunit/phpunit/src/TextUI/Command.php:195]
#24 PHPUnit\TextUI\Command->run() called at [/var/www/vendor/phpunit/phpunit/src/TextUI/Command.php:148]
#25 PHPUnit\TextUI\Command::main() called at [/var/www/vendor/phpunit/phpunit/phpunit:53]

EDIT2 I added logging also to the commit

#25 PHPUnit\TextUI\Command::main() called at [/var/www/vendor/phpunit/phpunit/phpunit:53]
DESCRIBE `some_design`
INSERT INTO `some_design` (`designer_code`, `designer_variant`, `preview_image_url`, `sku`, `price`, `hash`, `store_id`) VALUES (?, ?, ?, ?, ?, ?, ?)
==== TRANSACTION COMMIT
#0  Magento\Framework\DB\Adapter\Pdo\Mysql->commit() called at [/var/www/dev/tests/integration/tmp/sandbox-0-f68414eb86eeffb7401faa5c09c05812c4aa222769de918431c0368e2f700667/generated/code/Magento/TestFramework/Db/Adapter/Mysql/Interceptor.php:76]
#1  Magento\TestFramework\Db\Adapter\Mysql\Interceptor->commit() called at [/var/www/vendor/magento/framework/Model/ResourceModel/AbstractResource.php:91]
#2  Magento\Framework\Model\ResourceModel\AbstractResource->commit() called at [/var/www/vendor/magento/framework/Model/ResourceModel/Db/AbstractDb.php:426]
#3  Magento\Framework\Model\ResourceModel\Db\AbstractDb->save() called at [/var/www/app/code/Example/SomeDesign/Model/Design.php:84]
#4  Example\SomeDesign\Model\Design->setDesign() called at [/var/www/app/code/Example/SomeDesign/Test/_files/some_design.php:29]
#5  require(/var/www/app/code/Example/SomeDesign/Test/_files/some_design.php) called at [/var/www/dev/tests/integration/framework/Magento/TestFramework/Annotation/DataFixture.php:214]
#6  Magento\TestFramework\Annotation\DataFixture->_applyOneFixture() called at [/var/www/dev/tests/integration/framework/Magento/TestFramework/Annotation/DataFixture.php:244]
#7  Magento\TestFramework\Annotation\DataFixture->_applyFixtures() called at [/var/www/dev/tests/integration/framework/Magento/TestFramework/Annotation/DataFixture.php:92]
#8  Magento\TestFramework\Annotation\DataFixture->startTransaction()
#9  call_user_func_array() called at [/var/www/dev/tests/integration/framework/Magento/TestFramework/EventManager.php:49]
#10 Magento\TestFramework\EventManager->fireEvent() called at [/var/www/dev/tests/integration/framework/Magento/TestFramework/Event/Transaction.php:96]
#11 Magento\TestFramework\Event\Transaction->_startTransaction() called at [/var/www/dev/tests/integration/framework/Magento/TestFramework/Event/Transaction.php:81]
#12 Magento\TestFramework\Event\Transaction->_processTransactionRequests() called at [/var/www/dev/tests/integration/framework/Magento/TestFramework/Event/Transaction.php:46]
#13 Magento\TestFramework\Event\Transaction->startTest()
#14 call_user_func_array() called at [/var/www/dev/tests/integration/framework/Magento/TestFramework/EventManager.php:49]
#15 Magento\TestFramework\EventManager->fireEvent() called at [/var/www/dev/tests/integration/framework/Magento/TestFramework/Event/PhpUnit.php:126]
#16 Magento\TestFramework\Event\PhpUnit->startTest() called at [/var/www/vendor/phpunit/phpunit/src/Framework/TestResult.php:395]
#17 PHPUnit\Framework\TestResult->startTest() called at [/var/www/vendor/phpunit/phpunit/src/Framework/TestResult.php:637]
#18 PHPUnit\Framework\TestResult->run() called at [/var/www/vendor/phpunit/phpunit/src/Framework/TestCase.php:894]
#19 PHPUnit\Framework\TestCase->run() called at [/var/www/vendor/phpunit/phpunit/src/Framework/TestSuite.php:755]
#20 PHPUnit\Framework\TestSuite->run() called at [/var/www/vendor/phpunit/phpunit/src/Framework/TestSuite.php:755]
#21 PHPUnit\Framework\TestSuite->run() called at [/var/www/vendor/phpunit/phpunit/src/Framework/TestSuite.php:755]
#22 PHPUnit\Framework\TestSuite->run() called at [/var/www/vendor/phpunit/phpunit/src/TextUI/TestRunner.php:545]
#23 PHPUnit\TextUI\TestRunner->doRun() called at [/var/www/vendor/phpunit/phpunit/src/TextUI/Command.php:195]
#24 PHPUnit\TextUI\Command->run() called at [/var/www/vendor/phpunit/phpunit/src/TextUI/Command.php:148]
#25 PHPUnit\TextUI\Command::main() called at [/var/www/vendor/phpunit/phpunit/phpunit:53]
SELECT `store`.* FROM `store` WHERE (`store`.`code`='shop_gb_en') ORDER BY `sort_order` ASC

PHP Fatal error:  Uncaught PHPUnit\Framework\Exception: User Error: Some transactions have not been committed or rolled back in /var/www/vendor/magento/framework/DB/Adapter/Pdo/Mysql.php:3985.

/var/www/dev/tests/integration/framework/bootstrap.php:144
/var/www/vendor/magento/framework/DB/Adapter/Pdo/Mysql.php:3985

There is only the commit for the save(), but the first transaction seems to be due to DB isolation which is not working.

edit

https://devdocs.magento.com/guides/v2.3/test/integration/annotations/magento-data-fixture.html states

Do not rely on and do not modify an application state from within a fixture, because application isolation annotation can reset the application state at any moment.

Actually I have code called in the fixture like this:

    public function setDesign(PostInterface $entry)
    {
    try {
        $hash = bin2hex(random_bytes(16));
        $entry['hash'] = $hash;
        $this->resourceModel->save($entry);
    } catch (\Exception $exception) {
        throw new StateException(__('Unable to save design #%1', $entry->getDesignerCode()));
    }
    $storeId = $this->_storeManager->load($entry->getStoreCode())->getId();
    $this->_appEmulation->startEnvironmentEmulation($storeId, \Magento\Framework\App\Area::AREA_FRONTEND, true);
    $url = $this->urlBuilder->getUrl('foodesign/add') . '?hash=' . $hash;
    return $url;
    }

So maybe starting the appEmulation has some side effects on the CI which do not appear on local testing. Still the question is: why

edit

I removed the appIsolation calls from my fixture, problem persists

edit

Actually the problem is not with transactions, the problem is, that my test crashes badly.

I found out by running phpunit --debug and saw that the last test is only started, but not ended.

Then I added lots of

echo __LINE__ . PHP_EOL

to my test and found out that it breaks during

$this->dispatch('example/add/?hash=' . $hash);              

Still try to find out why.

8
  • can you share your integration test code ?As I was having same problem while creating integration test ? Jun 9, 2020 at 6:51
  • My integration test depends on some custom module, so I cannot share it in full. But it is interesting you have the same problem. Can you reproduce it locally or only in docker-based CI?
    – Alex
    Jun 9, 2020 at 7:00
  • As I was working in docker based ddev environment.but with one script it resolved problem Jun 9, 2020 at 7:07
  • There were two major problem for me. Jun 9, 2020 at 7:19
  • $this->dispatch('example/add/?hash=' . $hash) is this is the breakpoint ?if yes can you check url mentioned is proper in your code ? Jun 9, 2020 at 10:21

3 Answers 3

3

TL;DR

Magento integration tests do not seem to be prepared for the case that a secure base URL is set in the integration testing database. In this case they run into an exit statement when trying to redirect to the secure bsae URL, because running from the CLI can never be from a secure currently URL, because there is no current URL in the CLI at all.


Turns out it works if I uncomment

vendor/magento/framework/App/Router/Base.php:296

This line

$this->_checkShouldBeSecure($request, '/' . $moduleFrontName . '/' . $actionPath . '/' . $action);

There is an exit in there

protected function _checkShouldBeSecure(\Magento\Framework\App\RequestInterface $request, $path = '')
{
    if ($request->getPostValue()) {
        return;
    }

    if ($this->pathConfig->shouldBeSecure($path) && !$request->isSecure()) {
        $url = $this->pathConfig->getCurrentSecureUrl($request);
        if ($this->_shouldRedirectToSecure()) {
            $url = $this->_url->getRedirectUrl($url);
        }

        $this->_responseFactory->create()->setRedirect($url)->sendResponse();
        // phpcs:ignore Magento2.Security.LanguageConstruct.ExitUsage
        exit;
    }
}

I opend a Magento issue for this: https://github.com/magento/magento2/issues/28636

1

This might be because nested transactions are not supported.

So if transactions are used within a test, you should use

@magentoDbIsolation disabled
8
  • Thanks Fabian, do you have a clue why it works locally but not on the CI ? different MariaDB versions maybe? CI: 10.4.13 Local (working): 10.1.44-MariaDB-0ubuntu0.18.04.1
    – Alex
    May 29, 2020 at 12:43
  • Maybe, I would try to reproduce it on the same version, and also make sure you start with the same test db state. But I don't know for sure May 29, 2020 at 12:47
  • Tried locally with the same MariaDB docker container as we have on the CI and it still works and so the CI problem is not reproducible
    – Alex
    May 29, 2020 at 13:36
  • Is it really true that nested transactions are not supported? I mean on DB level this is correct --- There is this counter in Magento which is increased and decreased and does only the outermost transaction. When I enable the MySQL Log I also only see START TRANSACTION and ROLLBACK .. this seems to be correct, but somehow PhpUnit and Magento seems to get confused with counting
    – Alex
    May 29, 2020 at 13:58
  • /var/www/vendor/magento/framework/DB/Adapter/Pdo/Mysql.php:3985 is the classes'__destruct call .. unfortunately I cannot find out who calls it
    – Alex
    May 29, 2020 at 14:00
0
+300

There were two major problem for me.

  1. First Was everytime we run integration test db of integration test should be blank.because when we use fixture it creates entries in blank db.So we need to have blank db at the time running integration test.Second there should be order.
  2. The Magento-specific annotations for integration tests are applied in the following order:

@magentoAppIsolation @magentoDbIsolation @magentoDataFixture @magentoComponentsDir @magentoAppArea @magentoCache @magentoConfigFixture

Above 2 things solved my issue.

Following annotations I used.

/**
     * @magentoAppIsolation enabled
     * @magentoDbIsolation enabled
     * @magentoDataFixture testProductFixture
     * @magentoAppArea frontend
     */
5
  • Did you use all that annotations on your test?
    – Alex
    Jun 9, 2020 at 7:46
  • 1
    see I edited my answer with my used annotations Jun 9, 2020 at 8:45
  • 1
    @Alex : need to run this command before u start ur integration test Jun 9, 2020 at 8:52
  • 1
    @Alex : rm -Rf ./dev/tests/integration/tmp/sandbox* Jun 9, 2020 at 8:53
  • Does not help :-( Actually I think my test is crashing somewhere (see updated question) but I don't know why as I do not get any error messages
    – Alex
    Jun 9, 2020 at 9:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.