CLI E2E Tests Frameworks

[LeoBorai]

As part of testing HTTP Server, support for E2E CLI tests would be great, which allows us to
verify changes keep behavior consistent.

We must define a Framework to use for this purpose.

[Antosser]

Imo making e2e testing isn't worth it. There are severavl reasons:

• e2e testing would require an additional toolchain. It would be a pain to make it work on both Windows, Linux and Mac.
• There isn't a good framework for testing UIs. How would you verify that a web site is giving you what you want it to programmatically? Would you use a regular expression? An HTML parsing library? There isn't any good option. And when the UI changes, we would have to rewrite the tests
• BATS is a CLI testing program. Testing websites would be very complicated
• Manual testing would be so much easier and edge cases are just as likely to happen as in an overly complicated testing framework.
• In case we miss something the it user could just open an issue. If we missed it, it's probably not so bad

Imo, e2e programmatic testing takes up way too much time and slows down development a lot. Manual e2e testing would be much easier and wouldn't cost us much reliability.

I'm still all in for making unit tests

[Antosser]

Any updates?

[LeoBorai]

Hi @Antosser! For E2E testing Im coming up with a solution.

On the meantime I think we can:

• Move forward with pending PR/Feature Requests
• Prepare E2E Solution

Once the E2E solution is ready, I can jump in and write test cases the HTTP Server.

I will keep this discussion open until we have these E2E tests up and running.