End-to-end (e2e) tests are slow and flaky. They don't have to be, but effort to fix breakage starts consuming most available time.
One idea is to separate scraping from verification. The latter would run very fast and be reliable: it only tests against stored state.
Then scraping is just procedural, clicking things, waiting for page loads, and reading page elements into a database.
Some consequences are needing integrity checks to ensure data has been read (first name field selector was updated but not populated), self-healing selectors (AI, et al), and certifying test results against known versions (fixing the scraper amid UI redesign).
A lot of effort is saved by using screenshot diffing of, say, React components, especially edge cases. It also (hopefully) shifts-left test responsibility to the devs.
Ideally, we only have some e2e tests, mostly happy paths, that also act as integration tests.
We could combine these ideas with "stacked databases" from the article and save on duplication.
Finally, the real trick is knowing, in the face of changes, which tests don't have to run, making the whole run take less time.
If they are slow, it means your application is slow. Good thing your tests make you realize it so you can work on it.
If they are flaky either your application is flaky or your UI is hard to use. Anyway that's something your tests tell you you have to fix.
And last: if your tests are all independents, why not run them all in parallel? With IaC you should be able to provision one instance of your architecture per test (or maybe dozen tests) easily.
Even in parallel, there are tradeoffs: run them all in one container, or chunk them out to available workers. The first runs into resource constraints; the latter takes up everything in the shared pool.
With IaC, emulating a constellation of all dependent services along with the site is technically feasible. (There are other possible constraints.)
What's your ideal scenario? For example, k8s + cloud, ephemeral db, auto-merged IaC file of a thousand services, push-button perf testing, regression suite with a hundred bots, etc.
On premise k8s cloud where you can deploy many instances of the exact same services you have in prod. Let's say a E2E test takes 5s to run, your deployment takes 2mn and you want to stay under the 5mn line for running your test suite: deploy an instance of all your services and their databases per batch of 30 tests.
I can understand this not really being possible at an Amazon scale. But for most businesses? A good beefy server should be enough.
One idea is to separate scraping from verification. The latter would run very fast and be reliable: it only tests against stored state.
Then scraping is just procedural, clicking things, waiting for page loads, and reading page elements into a database.
Some consequences are needing integrity checks to ensure data has been read (first name field selector was updated but not populated), self-healing selectors (AI, et al), and certifying test results against known versions (fixing the scraper amid UI redesign).
A lot of effort is saved by using screenshot diffing of, say, React components, especially edge cases. It also (hopefully) shifts-left test responsibility to the devs.
Ideally, we only have some e2e tests, mostly happy paths, that also act as integration tests.
We could combine these ideas with "stacked databases" from the article and save on duplication.
Finally, the real trick is knowing, in the face of changes, which tests don't have to run, making the whole run take less time.