I have set up EcomDevs' Magento testing solution using this manual. I have found it a bit of a struggle to get everything that I need to work, so when I ran phpunit in the root of my project and it returned

FAILURES! Tests: 32, Assertions: 43, Failures: 14.

along with all the details of those failures I have been left wondering have I set it up incorrectly or is it areas that the framework is not able to test? I cant find anything online about this problem, so i'm guessing i've done something wrong.

Looking through the list of problems they all seem very similar (install and upgrade arrays are the only area that are failing, example below):

14)     EcomDev_PHPUnitTest_Test_Lib_Constraint_Config_Resource_Script::testGetVersionScriptsDiff with data set "to_1.0.5_one_invalid" ('directory_one_invalid', 'scheme', NULL, '1.0.5')
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
 'expected' => Array (
-        0 => 'install-1.0.0.php'
-        1 => 'upgrade-1.0.0-1.0.1.php'
-        2 => 'upgrade-1.0.1-1.0.5.php'
+        0 => 'install-1.0.5.php'
 'actual' => Array (
-        0 => 'upgrade-1.0.0-1.0.1.php'


Obviously I dont want to ignore failures, so how do I resolve this issue?


I found the cause of my problem by reviewing this link (don't know how i missed it)

installation (step 4)

If it shows that there was no tests found, it means that extension was successfully installed. If it shows some errors, then it means, that your customizations has install scripts that relay on your current database data and you should fix them. Or use your dev database as a base for the tests, but prior first time running the suite.

After reading this I

  • deactivated all custom modules (unsuccessful)
  • re-copied the dev db and applies it for the test db (incase of any errors) (unsuccessful)
  • tried to use the dev db for my tests (returns: "Test DB cannot be the same as the live one. You can change this option by running ecomdev-phpunit.php with magento-config action")(unsuccessful and can only find a reference to magento-config, but can't find the action in any file)
  • set my site to the testdb and applied the devdb in local.xml.phpunit and the same failures returned

I am now out of ideas

  • 1
    Use the develop branch! Commented Mar 7, 2014 at 10:52
  • two days of struggling and it was simply the wrong branch....arrrgghh
    – tony09uk
    Commented Mar 7, 2014 at 11:25
  • 1
    Oh, we all did that :-) I recommend to join #magento on freenode to ask short questions easier :-) Commented Mar 7, 2014 at 11:27
  • 1
    @FabianBlechschmidt fancy add an answer here :) Commented May 22, 2014 at 16:19

2 Answers 2


EcomDev_PHPUnit is refactored at the moment. Sadly broke the master branch, but the dev branch is great, working and useable!

So use this: https://github.com/EcomDev/EcomDev_PHPUnit/tree/dev


Just going to leave this here, in case someone needs help.

If you want to use dev branch, instead of the default modman script
modman clone git://github.com/EcomDev/EcomDev_PHPUnit.git
modman clone git://github.com/EcomDev/EcomDev_PHPUnit.git -b dev

If you still encounter the same error Test DB cannot be the same as the live one.

You need to configure PHPUnit to ignore same DB errors by setting <allow_same_db>1</allow_same_db> in app/etc/local.xml.phpunit

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.