7

Does Magento enterprise edition 1.14 have pre written functional automation tests? We use Magento 1.14 and I have to decide if I have to start building functional end-to-end user journey tests from scratch using say testng or can I start using the functional tests which come with Magento 1.14 (if any) and build on it?

My other question is isn't it enough I do end-to-end user journey tests like logging in, placing order, payment for order, cancel order, etc .. which will implicitly test the units(code) underneath from a QA tester point of view?

I am also thinking if I don't use Magento framework I will be able to use the same automation tests for some other store which doesn't use Magento at all.

Share your thoughts.

2

The short version of the answer to your first questions is, sadly, no. There is no publicly available suite of pre-developed functional tests for Magento EE 1.14 that you can use right now. The comments on this answer clarify it a bit.

That being said, there are some resources you can leverage for creating functional tests for your Magento version. The Magento Test Automation Framework is officially deprecated because Magento is focusing all efforts on the new Magento Testing Framework, but that is only supporting Magento 2 with no plans to backport to older versions at present. The Official MTAF User Guide does include some code samples for functional tests that should work for your current version and provides guidance on how to extend these beyond the code that is included. The 2013 Bugathon Wiki also has some really informative guides on QA and Functional Testing. This tutorial provides additional samples that you may be able to use immediately as well.

There has also been some great samples and utilities published on using CasperJS to perform front-end testing of Magento 1.X versions on both EE and CE platforms. Finally, if you really want to accelerate your functional testing and leverage a support team to assist you when needed, you may want to try BugBuster's Magento Functional Test Services. For a low monthly fee you can leverage their platform and support expertise to develop and extend functional tests of Magento. They have a quite a few integration points and a recorder app that helps you develop your functional tests more quickly than developing them by hand. I'm evaluating it now after researching their offerings for this answer but don't have enough knowledge yet to offer pros / cons.

Regarding your second question, you may be correct if your perspective is strictly focused on the traditional QA point of view. I would argue that if you are developing your own Magento extensions, you'd be BEST served by developing unit tests to cover critical functionality and at LEAST a few functional tests or integration tests if interfacing with 3rd party web services. The main reason is that functional tests are incredibly fragile and don't provide a tight enough feedback loop to provide value on their own, at least that's the Googler perspective. New Relic recently agreed noting they were the most expensive and difficult to maintain, but agrees that they DO provide a lot of value....

The key takeaway is you need to determine what is your best investment of your resource now....and later when you've finished that first investment, revisit the decision. Perhaps you start with functional tests only to satisfy a business requirement. Perhaps later as you add custom functionality you implement unit testing as well as end-level functional testing. Do what your business drivers require and hopefully you can leverage the first effort to build consensus on your team and your business partners on what's the best next step.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.