Export util modules about projen HOT 9 CLOSED

I've only started building on top of a small part of what's exposed, and things that I found incredibly useful right away, but aren't made available by other means are synthSnapshot, and directorySnapshot.

Agree. Synth for files and directories (without writing to disk) comprise the bulk of my local test utils for Projen related things. They are mostly cut/paste from Projen but it would be nicer to just import from Projen directly. Being able to snapshot a synth'd file output is a great cheap way to know what's changed during synth without jumping through hoops.

from projen.

We're not keen on this because it creates a lot of maintenance surface the is completely unrelated to the core of the project.

E.g. why can't you use lodash.merge?

The reason people are using these, is because projen currently doesn't actively prevent the imports by not setting an experts field. But we will eventually and I think many people are aware that path imports are bad practice.

Please add a warning to your workaround comment that this will break in future releases.
Edit: Added the warning. Closing the issue for now. Happy to discuss it on here.

from projen.

I've only started building on top of a small part of what's exposed, and things that I found incredibly useful right away, but aren't made available by other means are synthSnapshot, and directorySnapshot.

Things that seem to become important quite quickly are exec, writeFile. There's other things surfacing from the GitHub-wide search, for packages that will break with ESM changes.

I guess these aren't really something that's essential, but they are imbued with certain particularities, and it does save others from re-implementing these, and in a way that's not consistent with how projen behaves.

from projen.

I'll leave this as a workaround when authoring your own changes:

⚠️ Be aware that this will stop working in a future release and is not encouraged

import { createRequire } from 'node:module'

const require = createRequire(import.meta.url)
const { deepMerge } = require('projen/lib/util')

from projen.

@mrgrain that makes sense.

I have to think how I wouldn't have started publishing anything that's built on top of Projen if I couldn't have leveraged things like xSnapshot in my packages for writing tests on day 1, though. And, I think it natural for @cdklabs and others to repeat the same patterns and use the same abstractions from Projen.

I wonder if you would reconsider making a plan of publishing these separately, eventually. It's not that these things can't be reverse engineered and alternatives provided, but it doesn't make a great foundation for building on top of Projen if this also has to be in the list of things that need to be solved before commencing actual work.

from projen.

This is a good point. But I think we should be talking about which parts of utils we want to include explicitly vs. the parts that are easily available outside of projen (like deepMerge). I definitely see the need for projen testing helpers being readily available.

Maybe if you could start with a more focused proposal on what exactly of the current utils we should export?

from projen.

I've only started building on top of a small part of what's exposed, and things that I found incredibly useful right away, but aren't made available by other means are synthSnapshot, and directorySnapshot.

Yep, that's the beginning of a testing library.

Things that seem to become important quite quickly are exec, writeFile. There's other things surfacing from the GitHub-wide search, for packages that will break with ESM changes.

The issue I have with those is what are they even used for? These are low level helpers used by the proper features like File constructs and tasks. Why are the high-level features not used?

from projen.

@mrgrain I found that since Projen provides no way to hook into modifying the contents of files as they are written by components outside one's control, nor there being support for replacing contents of FileBase instances one has a reference to, we all simply have to share process and disk space, from the way I understand things.

If providing these is contentious, then it's hinting that something could use some thinking through from the grounds up, perhaps by providing these hooks, and offering us to interject asynchronously as well, if we need to. Then there wouldn't be need to reach for helpers to mutate the disk, or running child processes for ad-hoc tasks, but as things stand now these might be needed for anything beyond the basics from what I can tell.

from projen.

Thanks for your response @langri-sha ! I don't want to put this all on you, just because you are the person who brought it up. And I very much appreciate you for starting this discussion. So whenever I mention users, please know that I'm really not just talking about you!

@mrgrain I found that since Projen provides no way to hook into modifying the contents of files as they are written by components outside one's control,

Yes, this is the main problem with OOP and even worse with the Constructs Programming Model that really wants to have correctness at construction time. The AWS CDK suffers from the same problem.

nor there being support for replacing contents of FileBase instances one has a reference to, we all simply have to share process and disk space, from the way I understand things.

In projen everything ends up being something that's written to the disk. That's why we already have the low level object file APIs. I'm aware we are missing one for plain text files. Tasks have their own extension API, but can also be changed using files. Other than that - what use cases are still missing here?

If providing these is contentious, then it's hinting that something could use some thinking through from the grounds up, perhaps by providing these hooks, [...].
Then there wouldn't be need to reach for helpers to mutate the disk, or running child processes for ad-hoc tasks, but as things stand now these might be needed for anything beyond the basics from what I can tell.

Yes absolutely spot on. We technically already have pre and post synth hooks, but they are not very extensible at the moment. What I'm missing from this discussion is some concrete examples of things users cannot do right now, and thus pushes them to use these helpers.

One thing that I came across for deepMerge is merging options with defaults. I was hoping to provide an explicit feature for this in projen.

Do you know of anything else?

and offering us to interject asynchronously as well, if we need to.

Async is unfortunately a completely different discussion. It's just not possible in a constructor (which doesn't matter for hooks) and not (well) supported in jsii, which itself is a non-negotiable core tenet of projen.

from projen.