Automated tests: orders, product views and usability for Magento 2

Question

I'm looking for a way to automate testing on multiple websites. Now I'm pretty new to the testing part of Magento 2 and there seems to be a lot of ways to test in Magento 2, like unit testing, functional testing, etc.

They all seem to be more code based testing. That part we already covered. I am looking for a way to test the part on the frontend, more automated.

Things like open productspage and report errors in JS and PHP, same for categorypage, CMS pages, create testorder and see if adding to cart until checking out causes any error.

A lot of terms are on the internet. Unit Testing, Performance Testing, Automated Regression Testing and the meaning of the terms vary a lot, depending which blog/post/site you are visiting.

What I need: test features like add product to cart, view cart, go to checkout, actual finish the order from checkout, create account as new customer etc. Frontend usability.

Question 1: what kind of testing am I looking for, e.g. unit test?

Question 2: should I use Magento standard testing methods or are there proper 3rd party testingsolutions which might meet my requirements?

Sorry for tl;dr, but I'm a noob when it comes to automated testing.

Answer 1

What I need: test features like add product to cart, view cart, go to checkout, actual finish the order from checkout, create account as new customer etc. Frontend usability.

In Magento terms, this is functional testing. The functional tests in Magento are automated browser tests with Selenium, using a "headless" browser.

As a third-party alternative to the Magento testing framework, I can suggest Magium. I only used it for Magento 1 yet, but it should work for Magento 2, too.

report errors in JS and PHP

This particular feature is not covered with the functional tests. If there are PHP errors, the tests should discover that, because the site will break. For JS errors, it is similar: if they break functionality and you are testing that functionality, the tests report it.

But the tests only see, what's in the browser, so you won't get reports about error messages. Maybe you are looking for a monitoring tool that reports errors on the production site?