A Pragmatic, Not-Really-Typed Errors Proposal about typescript HOT 16 OPEN

@fatcerberus My instinct is that there are at least some cases where it’d be useful to distinguish between the different built-in error types, but I don’t think that making them structurally distinct is a requirement or blocker for this proposal. If this proposal gets any traction, I imagine there’ll be a lot of testing on real-world code before anything lands, and that real world testing should make it clearer whether making the built-in errors distinct is actually worth it.

from typescript.

@RyanCavanaugh I don’t know enough type theory to fully engage the issues here, but let me take a stab at a constructive response — and forgive me if I miss things that ought to be obvious.

What I take you to be saying, essentially, is: because so much of TS’ underlying logic relies on types being in a hierarchy/partial order, the concept of "mutual subtypes that are nevertheless distinct" is somewhere between “very tricky to implement” and “conceptually incoherent”. Since open-ended unions try to make mutual subtypes out of "blue" | string and string, they’re gonna run into lots of problems. The same thing applies to trying to put unknown/UnknownError into a union that doesn’t reduce, and to building mutually-assignable function types containing that union.

Do I have all that right?

If so, I guess I see three directions for trying to advance the DX goals of this proposal (which are very valuable imo):

1. Stick with the prior issue's conclusion, that this should be implemented outside the type system. Thinking open-mindedly, maybe "outside the type system" wouldn't have to mean "outside of TypeScript", given that inclusion in TS might be critical for adoption.

   However, beyond adoption, I think the fundamental issue is that some type system involvement probably is necessary to make this all work. As much as the UnknownError/open-ended union portion of this proposal is a weird fit for the type system, the work to figure out the non-UnknownError constituents of the ErrType requires very standard type system machinery, especially for generic error types. Consider promiseThatCanRejectWithX.then(functionThatCanThrowY). Any analysis should conclude that that's a promise whose known rejection types include X and Y. Similarly, on your example of const x = e ? TA : TB, the analysis has to conclude that the known error types of x include A and B. That all feels very type system-y.

2. Try again to somehow solve the open-ended union problem in a way that's consistent/coherent/predictable. I accept that this is hard or maybe impossible (but seems very interesting, so I might try to help). At the very least, it probably requires devising a bunch of new typing/type inference rules, even if @DanielRosenwasser's recent idea here is in the right direction.

3. Try to come up with a simpler, and more-targeted (albeit less complete) solution that would be suffice for this proposal. I'm inclined towards this approach.

Here’s a sketch of one idea I’ve been noodling on in that direction, which I’m hoping you can sanity check:

• As in the OP, every function type (and promise type) would have an associated type for its errors. However, rather than trying to make this type represent every possible error (with an open-ended union), it would represent just the known errors. Let’s call it the KnownErrType. In () => void throws A, A would now be the KnownErrType.
 Open-ended unions are totally gone.

• The error variable that reaches a catch block would continue to be typed as unknown. But, these catch block variables would have a special tag that effects how their type is shown in IDE popups. Specifically, the IDE popup would show both the variable's actual type, and its known error types (derived, as per the OP, from CFA on the corresponding try block). Maybe this is with some sort of newly-formatted popup; maybe it's a hack where the language server reports that the type of the variable is unknown | KnownError1 | ... | KnownErrorN, even though that's obviously equivalent to unknown and the "real" type would simply be unknown. If the type of the catch block variable narrows, the same narrowings would apply to the known error types. So, in other words, catch block variables would have a type at each location and a set of known error cases at each location, and the IDE popup would show both.

• Since we still want to keep function types assignable to each other regardless of errors, the KnownErrType of a function/promise type would be ignored in almost all circumstances; i.e., types with different KnownErrTypes wouldn’t be distinct for the purposes of almost all compiler algorithms. The idea, basically, is to lean into the idea of the KnownErrType being a kind of documentation/metadata, and keep it out of most type-system logic. As mentioned above, though, it can't be kept out of all type system logic (e.g., it needs to be elicitable in conditional types with throws infer E; when TS infers specially marked error type parameters per the OP, for parametric error cases; etc.). But the idea would be that there are specific cases where a function's known errors need to be elicited and, in these cases, the KnownErrType is used and behaves just like a regular type, as it no longer has the weird UnknownError in it.

• The only real requirement, then, is that TS preserve/update a function's KnownErrType as needed, so that it can serve its ultimate documentation purpose in a catch block. To that end, when two function types (or two promise types) “combine” to produce a new type, their KnownErrTypes combine as well to produce the KnownErrType of the new type. Specifically:

  • When subtype reducing two function/promise types in a union, the resulting type’s KnownErrType would be the union of the two original type’s KnownErrTypes.

  • When unifying two function or promise types into a new function or promise type, the resulting type’s KnownErrType would be the union of the two original type’s KnownErrTypes.

  • Maybe there are other cases too? Again, I don’t know enough type theory or TS compiler details to know all the ways that the KnownErrType information might get lost. But I hope the basic idea here is simpler, in that two function types which differ only in their known errors would no longer have to be preserved indefinitely as distinct types that are mutual subtypes; instead, they can immediately be collapsed into one type (but preserving the original types' known error metadata).



So, in your example of const x = e ? TA : TB, the unification of () => void throws A and () => void throws B would always be () => void throws A | B.

from typescript.

Today, almost all the built-in Error types have structurally-identical definitions.

Outside of custom error subclasses (which users can make structurally distinct themselves if they need to), is there any real use case for distinguishing between, say, TypeError and ReferenceError? Unlike, say, C#, the built-in error classes in JS are very general and often fall at awkward boundaries. In other words you probably wouldn't ever write in JS

try {
    // do a thing
}
catch (e) {
    if (e instanceof TypeError) {
        console.log("invalid data was received by some function");
        // error handled, recover
    } else {
        throw e;  // not a type error, rethrow
    }
}

because you might get a RangeError instead of a TypeError due to the exact same cause (bad data passed to function, e.g.). So IMO it doesn't really matter that they're not structurally distinct.

from typescript.

I'm still not clear on what's being gained from doing this in type space, where it has no type effects and will misbehave in all sorts of ways, instead of doing this in the JS Doc space? If the feature isn't going to work well under indirection anyway, then it just seems obviously fine to implement this through walking up to JS Doc declarations the same way we do to provide documentation tooltips.

I read your question in a couple different ways.

The first read, which probably isn't what you mean, is: "Could this be done using the JSDoc annotations that code actually has today?" I think the answer there is pretty clearly no: the @throws annotations on existing code are woefully incomplete and/or out of date, and probably always will be if developers are being asked to maintain these annotations manually.

So then the second read is: "Could some analysis tool automatically identify a function's anticipatable errors and serialize that set of errors to JSDoc? Then, that JSDoc could be consumed by the TS language server to power IDE popups."

I'm assuming that the analysis to identify each function's anticipatable error types needs to happen on TS source code, rather than on the published, possibly-minified JS code: by the time the code is converted to JS, it seems like too much type information is lost.¹ So, if the analysis is happening on the raw TS source, there needs to be some way to serialize the results of the analysis and publish it with the compiled code, and the question is just: is JSDoc adequate for that?

My first thought is that, if Typescript is doing this analysis (which I think it should be for reasons discussed below), then it seems a little weird for TS to modify a function's JSDoc on emit. There could also be conflicts if the function already has @throws annotations.

The bigger limitation of JSDoc is that it doesn't support any parametricity. E.g. in,

function f(arr: unknown[]) {
  try {
    return arr.map(xxxx);
  } catch (e) { /* … */ }
}

Clearly, the known error types for e should be the same as the known errors for xxxx, but there's no JSDoc that one can write for map to make that happen.

But maybe that's not a show-stopper. Maybe a workaround would be to say: any time f calls a function and passes it a function as an argument, assume that function argument is gonna get called and its thrown errors are gonna propagate. So e would end up with xxxx's error types as possibilities, even though TS knows nothing about map's handling of errors or whether it even calls xxxx. Something analogous would happen when a Promise is passed as an argument. (I.e., TS would assume that the promise is awaited in the function its passed to, and that its rejection propagates.)

There are lots of cases where that heuristic won't work right, but it's a conservative and maybe-not-horrible assumption?

Still, part of what I was going for with my original syntax was that it would be possible to be a bit more precise about things like this. E.g., to do:

interface PromiseRejectedResult<E> {
  status: "rejected";
  reason: WithKnownCases<E, unknown>;
}

type PromiseSettledResult<T, E> = PromiseFulfilledResult<T> | PromiseRejectedResult<E>;

interface PromiseConstructor {
  // all() propagates errors. `GetRejectsWith` extracts a promise's known rejection types.
  // The heuristic above would give identical behavior for Promise.all, but wouldn't work for allSettled.
  all<T extends readonly unknown[] | []>(values: T): 
    Promise<{ -readonly [P in keyof T]: Awaited<T[P]>; }, GetRejectsWith<T[number]>>;

  // allSettled() preserves known errors in the settlement results, but removes all known
  // errors on the returned Promise, by leaving off its second type parameter (which 
  // represents the known rejection types and would default to never).
  allSettled<T extends readonly unknown[] | []>(values: T): 
    Promise<{ -readonly [P in keyof T]: PromiseSettledResult<Awaited<T[P]>, GetRejectsWith<T[P]>>; }>;
}

When you say "if the feature isn't going to work well under indirection anyway...", I guess I was hoping that it could work well under indirection, and that was my motivation for a more-complex syntax than what JSDoc supports. My "when types combine..." proposal was an attempt, without really knowing how TS is implemented, to preserve the ability for error typing to work under indirection at least reasonably well (certainly better than it would with JSDoc).

But, let's assume that doing anything with errors in the type system is not worth it; that the JSDoc syntax is sufficiently expressive; and that the parametric error cases can be handled well enough with some heuristics. Then...

1. Some analysis still needs to happen on TS source code before its compiled/published, to identify relevant errors, with the results being serialized to JSDoc comments.

2. If every library author has to manually enable that analysis for their library — say, by installing some TS compiler plugin — many libraries won't, so consumers won't have access to nearly as much error information.

3. Therefore, this proposal becomes a request for TS to do that analysis so that all consumers benefit. TS would analyze every function's error handling according to the ErrType inference rules in the OP and serialize the results to JSDoc on emit. And then use this info in IDE popups.

from typescript.

Without a fully-thought out response, the way I've often alikened something like this is to #26277 ("open-ended union types"), where there is some partially-known set of constituents that you want to handle (for some definition of what "handle" is).

from typescript.

Open-ended unions sounds like what people are often shooting for when they try to write stuff like "foo" | "bar" | string (if not to aid with completions).

from typescript.

@DanielRosenwasser I hadn’t seen that issue, but this proposal would absolutely leverage open-ended union machinery if it were to exist! Obviously, that machinery alone isn’t enough to cover all the functionality here (e.g., for ErrType inference), but it's very complimentary.

from typescript.

I really like this proposal.

declaration file improvements should come quickly, whenever a TS-based library is recompiled and republished, because a huge amount of this error information will be inferred;

As far as I understood you want to put the inferred types automatically into generated .d.ts files (like declare function doSomething(): void throws RangeError;).
This is a new syntax so old typescript compilers will not be able to parse these files.
Not every consumer of .d.ts files are quick in upgrading, so IMO we should be able to disable this emit.
Our customers are often consuming our .d.ts files using typescript 3.9.x :-(
We guarantee this compatibility with a CI build step.
As long as we do not use new syntax (like declare function fancyGetter(name: `get${string}`): number;, new in TS4.1) in an API this is possible.

from typescript.

IIRC from what maintainers have said, backward compatibility for .d.ts emit is not guaranteed in general; you're expected to have a downleveling step in your toolchain if you need your declaration files to work with older TS versions than the one you're using.

from typescript.

@DanielRosenwasser If this proposal seems promising, what would the next step be here? Is it the type of thing where the TS team would want to see more community input before anything else? Or is the (long) previous discussion in #13219 already a signal of sufficient community demand? Are there specific issues with the proposal that I could maybe help to address? Or is it more a matter of the TS team talking internally first to figure out how/whether this would cohere with other features TS might add (like open-ended unions), how valuable error typing would be, how hard it'd be to implement, etc?

from typescript.

I think we'd have to allocate some time among the team to get a sense of everything you just listed (e.g. difficulty of implementation, feel, future-compatibility against other possible language features, etc.). Part of it is just a timing/availability thing.

from typescript.

Got it; makes total sense.

Whenever you and the team do have time to talk about it, I’m excited to hear what the outcome is :)

from typescript.

What we've discovered trying (multiple times!) to implement non-workarounded completion lists for open-ended unions is that the instant something doesn't have type system effects, it tends to disappear nearly immediately, or cause huge problems.

Example: having two types () => throws A (call it TA) and () => throws B (call it TB) be mutual subtypes seems fine, but it isn't. It means, for example, that given const x = e ? TA : TB, x has to be one of those types, but can't be a union, so it means TA or TB would get randomly picked. Once it gets randomly picked, then it's a huge hazard because people will inevitably try to fish out the throws type with something like type GetErr<F> = F extends (() => unknown throws infer E) ? E : never, so then type K = GetErr<typeof x> randomly gets you A or B and causes different errors to appear or not appear.

Putting in place type system features which never affect assignability is thus very dangerous, because it means that the behavior of your program becomes chaotic, or you're not allowed to use that feature in any way where it's observable, which becomes its own can of worms. Like you might just say "Oh, well, it's just illegal to write throws infer E, but that doesn't solve the problem, because you can do a trivial indirection:

const Throws<T> = () => unknown throws T;
type TA = Throws<TA>;

type Unthrows<T> = T extends Throws<infer E> ? E : never;

where now you have a situation where you can't do nominal inference of Unthrows<TA> because it would cause the illegal observation of the throws type. So now you have to have a separate system to track "type parameters where it's legal to observe them" and "type parameters where it's not legal to observe them" and come up with sensible error behavior anyone someone tries to cross that invisible line. You can keep stacking on more ad-hoc rules to try to ban this, but you'll always either end up at some extremely inconsistent (and defeatable) set of rules, or prevent the feature from being used in the way it was originally intended in the first place.

Ultimately this doesn't sound like a type system feature, for basically the same reason that type system features to create documentation descriptions doesn't exist. At the end of the day you can't add things to the type system that don't do anything because people will still find ways to observe the thing that's happening, and without some well-reasoned ordering of subtypes, that will cause a lot of unpredictable behavior.

from typescript.

The solution above — treating the catch block variable specially — isn't really a complete solution. E.g., it doesn't support communicating the known error types to a promise.catch callback. But it probably gives the vast majority of the real world DX value, and it doesn't block a more comprehensive open-ended union solution in the future, which would change the type of the catch block variable from unknown to an open-ended union containing/bound by unknown.

It also probably would make it harder for code to opt-in, on a case-by-case basis, to a TS check that all the known errors had been handled. (Though a compiler flag or a lint rule could enable check that globally for codebases that really want it.)

from typescript.

To that end, when two function types (or two promise types) “combine” to produce a new type

It's notable that there is no existing "combine" mechanism where two types get synthesized into a new type. For example, let's say you have

declare function choose<T>(a: T, b: T): T;

if we call choose(throwsA, throwsB), generic inference needs to produce a single output type T from the presented candidates (throwsA and throwsB). Synthesizing a new type from the two of them has never been done before, because there's never been a correct reason to do it before. This invariant has been useful and correct for the entire lifetime of TypeScript so changing it now should have some really strong justification.

Obviously nothing is impossible and invariants have been removed in the past, but breaking this invariant only for the sake of making catch variables get better IDE pops-ups is not really compelling from a trade-offs perspective.

I'm still not clear on what's being gained from doing this in type space, where it has no type effects and will misbehave in all sorts of ways, instead of doing this in the JS Doc space? If the feature isn't going to work well under indirection anyway, then it just seems obviously fine to implement this through walking up to JS Doc declarations the same way we do to provide documentation tooltips.

from typescript.

@RyanCavanaugh Any further thoughts here?

from typescript.