Consolidate our internal example apps (waspc/examples) about wasp HOT 6 OPEN

@sodic this all makes sense to me, except for not wanting to test the external example apps. Why wouldn't we have e2e tests for them? For every release, we need to make sure that these apps work correctly. Isn't it much easier to have e2e tests that we can run then testing these apps manually? If we don't have too many example apps + we share the setup for e2e tests (which would be doable I think), then maintaing those e2e tests sounds better than manually testing all those apps each time to me.

from wasp.

For every release, we need to make sure that these apps work correctly.

@Martinsos That's true. I just don't know how we'd write the kind of tests that would spare us from having to run it manually.

If we can do it and it doesn't take a lot of time, I'm in.

For every release, we need to make sure that these apps work correctly.

@Martinsos That's true. I just don't know how we'd write the kind of tests that would spare us from having to run it manually.

If we can do it and it doesn't take a lot of time, I'm in.

Just Playwright tests, like we have for open-saas and for that headless-test/examples/todoApp, that would run the app, do some clicking around, stuff like that. Hm I guess you are not sure if these can test those apps well enough, taking into account stuff can go wrong from the Wasp side? That is maybe a good question. Ok, but certainly worth trying, we can see how it goes, as you said.

from wasp.

Nice initiative 👍 the rising number of example apps makes cutting a new release more work than needed, so I think this is a good thing to tackle.

I think we should have one dev app that's a "kitchen sink" (name commonly used for these types of apps) which has all the features. Ideally, we cover all of them with headless tests then. Which gives us confidence that everything works and everything is tested all the time. Some of the stuff like logging in with Google might not be possible with headless tests (I'm not sure, tbh) but that's a small subset of features.

About the external apps: the best external app we have now is the Open SaaS one. It's the most useful one. I guess the other apps had a stronger purpose before like to demonstrate you can build more complex apps with Wasp. Now, maybe we no longer need the Trello clone or the Tableau clone because we have Open SaaS.

What's useful for us?

• Being able to work on new stuff.
• Have automated tests.

What's useful for the users:

• Having working code that they can copy paste.
• Seeing which features are available and how they work (live demos).
• Seeing various integrations e.g. different UI libs or 3rd party APIs.

I'd merge the headless tests app and the dev app. I'd probably delete most of the external apps and then show off the dev app as the "kitchen sink" app for the users to see working code. But I'm less certain about the second thing.

from wasp.

Great analysis, I'll quote several points and say what I think and why.

Requirements

I'm on board with all the requirements you listed, except maybe this one:

We want e2e tests for all example apps. Especially internal ones, but we should probably also have them for external ones.

Depending on how many example apps we have and how we use them, having e2e tests for each one of them might be more trouble than it's worth.

I'm more in favor of one big e2e-tested app with all functionalities, and keeping the "public" example apps simple and untested, as if they were written by someone who's not on the Wasp team.

Which brings me to...

Questions/ideas

Could we have just one set of example apps? So they serve both as internal and external apps?

I'd rather keep them separate. The two sets of example apps have different responsibilities (showcasing features vs. testing and working wasp), and I feel merging them would cause problems. It's no accident we naturally evolved into our current state where these two sets of example apps are separate.

This ties into my first response - I prefer running e2e tests exclusively on internal apps.

Can we write e2e tests setup just once and use it for all the apps? So we don't have to define that again and again for every app?

You mean like having a e2e testing framework? I think it's cool, but also think we don't need it (since I am a strong supporter of only running e2e tests on a single app, in which case this becomes a moot point).

What about the test we have for waspc/examples/todoApp, that checks if it compiles and builds. Should we make that test universal and run it for every (internal) example app in the CI?

This we could do (for both public and private apps). It's simple enough.

Do we need so many external example apps? We have quite a few at the moment.

Probably not, we could remove some of them from the repo.

Some options in Wasp are mutually exclusive [...] Could dope.sh help?

Yeah, this is a problem. I would wait and see how dope.sh does with OpenSaas before extending it here. Maybe we should start by testing only a single of the mutually exclusive options? I think that should be enough.

Possible solutions

It seems quite clear that we should merge waspc/examples/ and headless-test/examples.

Yes, we all agree on this one :)

As for the solutions, I'd like to propose a third one (it's a modified version of your first proposed solution). I'll copy your text and modify it (the main differences are in bold):

Option 3

We have two sets of apps: waspc/examples and examples.

In waspc/examples, we would have one "mega" app, TheTestApp, that would have all the stuff in it and would be the
only one we use to develop, test and e2e test stuff.

We would also have another, BasicApp, which would have just some very basics in it, making it somewhat easier to try out stuff. Maybe this BasicApp is not even needed.

The CI ensures that TheTestApp compiles, builds, and e2e tests it on every run. This is the only app that's e2e tested.

In examples, we would have one bigger, more complex app, real world or something like that. We would also have our todo app tutorial app in there. And finally, maybe one or two more apps that showcase specific features -> but we should try to have as few of these as possible, and also consider just merging these specific features into one app if possible. These apps wouldn't be covered with e2e tests, but we would tests to see if they compile and build. These would run only on release in CI.

from wasp.

@infomiho that makes sense to me, although I think there is some benefit to having example apps that are a bit smaller also -> a big one is a lot to grasp. Although TodoApp for tutorial is maybe also a good smaller example app? Hm. Maybe different use cases, for example one for live chat, one for ecommerce, one for dashboard, stuff like that. Yeah probably that would be most valuable. OpenSaas is one hand example app, yeah that is true. But it is also quite big which I think makes it harder to take a look and say "aha this is what Wasp works like". Hm. Ok I think we need to figure this out yet.

from wasp.

For every release, we need to make sure that these apps work correctly.

@Martinsos That's true. I just don't know how we'd write the kind of tests that would spare us from having to run it manually.

If we can do it and it doesn't take a lot of time, I'm in.

from wasp.