Scala 3 feature parity about cats-tagless HOT 38 OPEN

Hey 👋 , we've (with @dos65) recently tired to play around with lampepfl/dotty#11685 to start working on the cats-tagless macros migration; very good and exciting news is that it looks like it works 🎉 🎉 🎉 ! (Scala 3.2.0-RC4 is required)

It lives in a separate repo https://github.com/pomadchin/tagless-derivation at this point, with some basic macro functions for FunctorK, SemigroupalK, InvariantK, and ApplyK derivation.

Hopefully I'll be able to push it more towards making it a part of this repo, but don't hesitate to grab / contribute / start work on bringing it into this repo already.

from cats-tagless.

@joroKr21 @smarter
lampepfl/dotty#11685 is now merged into dotty (it was noted as a blocker on an earlier message of joro's). Is it now possible to support this library on Scala 3?

from cats-tagless.

Hey @jchapuis yea, just a bit slow due to the lack of time 😞

I finished the POC a ~ day ago in a separate repo (https://github.com/pomadchin/tagless-derivation); hopefully will have time soon to start working on a PR against this repo.

from cats-tagless.

I've been mostly interested in the ApplyK derivation for the simple aglebras composition; I have macro porotytpes for the FunctorK, SemigroupalK, ApplyK, InvariantK, ContravariantK only.

Don't hesitate to create PRs against https://github.com/pomadchin/tagless-derivation with macro implementations for other concepts!

I'll try to create the initial PR against this repository so it can be picked up if needed.

from cats-tagless.

@bpholt i can split my PR in two if that’s really needed; also it can be helpful in case we don’t want to merge poor persons macro implementations as is (macros work but don’t support all the cases) cc @joroKr21

from cats-tagless.

Just doing some evaluation for scala 3. We use it in some projects and we would like to know if we could use it when Scala 3 comes out? Or do you plan to port the most used macros first? FunctorK and Tupled2K are essential, aspect weavng is nice but can be added later.

It's on my todo list to work on this. I can't promise that it will be done by the time Scala 3 final out though. But - if we manage to port one macro, we can port them all. The implementation is very consistent. The only odd ones are Instrument and Aspect. Note that there are no macro annotations on Scala 3 so they will have to be replaced by support for derives FunctorK etc. clauses.

All the macros need to be rewrite right? I am not an expert :) Contributed only by looking at the examples of the work of the maintainers. Let me know if I can help a hand.

Embarrassingly, the most demotivating part of the work is setting up the build and CI for cross publishing 🙈 We also need to apply simulacrum-scalafix before we can start because we use @typeclass. So here are two self-contained tasks we need to do first:

1. Apply simulacrum-scalafix
2. Setup build and CI to cross-publish the core module (it doesn't contain any macros)

from cats-tagless.

We are blocked by lampepfl/dotty#11685 - looks like we will have to wait until Scala 3.1 😞
I didn't think such a feature could be missing from the new macros.

from cats-tagless.

@ScalaWilliam the current scala3 macro capabilities are sufficient enough to work on the cats-tagless scala 3 support; check out the fully functional POC here https://github.com/pomadchin/tagless-derivation

I still didn't have a chance to fully commit into porting this POC into this repository.

from cats-tagless.

Let me take a look first. It might take a while though. I'm really busy these days because I changed jobs recently and I'm also in the process of changing apartments.

from cats-tagless.

https://scalacenter.github.io/scala-3-migration-guide/docs/macros/migration-tutorial.html

from cats-tagless.

This library uses macro annotations which don't exist in Dotty, but things like @autoFunctorK trait Foo[F[_]] should be rewritable as trait Foo[F[_]] derives FunctorK using https://dotty.epfl.ch/docs/reference/contextual/derivation.html. In fact it looks like shapeless-3 already has support for that! (there's an example in the readme)

from cats-tagless.

@smarter the macro annotations are not essential. They just generate an implicit instance in the companion object but the heavy lifting is done by blackbox macros. If we want to use shapeless 3 we would need to somehow generate mirrors for traits.

from cats-tagless.

First of all, great work with this library.

Just doing some evaluation for scala 3. We use it in some projects and we would like to know if we could use it when Scala 3 comes out? Or do you plan to port the most used macros first? FunctorK and Tupled2K are essential, aspect weavng is nice but can be added later.

All the macros need to be rewrite right? I am not an expert :) Contributed only by looking at the examples of the work of the maintainers. Let me know if I can help a hand.

from cats-tagless.

We also need to resolve #192 - our build is really outdated 😕

from cats-tagless.

Out of curiosity, what part of cats-tagless requires generating arbitrary class implementations? I would have though that at least some part of cats-tagless could already be implemented today given that shapeless-3 supports derives FunctorK for example which I assume is equivalent to @autoFunctorK

from cats-tagless.

@smarter on a high level think Mirror for traits.
Given trait UserRepo[F[_]] we want to have FunctorK[UserRepo].
shapeless-3 supports case class UserRepo[F[_]] or enum but that is not quite the same.

from cats-tagless.

Would lampepfl/dotty#11686 be relevant?

from cats-tagless.

No, what we are talking about here is the equivalence between a trait and the Mirror.Product of all it's methods as functions. BTW I'm not sure, does every method type in Scala 3 have a corresponding function type? Can you mix polymorphic functions and dependent functions and contextual functions arbitrarily? But even then type members of the trait would be a problem.

from cats-tagless.

You can mix them yes, but you can't represent method with default parameters or methods with varargs losslessly as functions.

from cats-tagless.

You can mix them yes, but you can't represent method with default parameters or methods with varargs losslessly as functions.

That's not a problem. We are just forwarding to an underlying instance and modifying the arguments and/or the result. Varargs just have to be passed correctly but otherwise can be viewed as a Seq. Default parameters are a property of the interface, not the implementation so they don't matter.

So in theory, if we had Mirror.Product[trait, tuple of functions] we could use shapeless-3

from cats-tagless.

But then you have edge cases like what about abstract vs concrete methods?
You shouldn't really have the AnyRef methods in that pile.

from cats-tagless.

Well, someone has to go and try to port the macros to find that out :). That would be a good exercise since this feature is still experimental in the compiler (only available in nightlies) and user feedback would help us decide if it's good enough to stabilize.

from cats-tagless.

How about we first make it work without any cross-building and release it as cats-tagless3 - since configuring a build is super complicated, this could be decoupled?

I had a go, the following compiles core (except some places I had to comment). I am not particularly knowledgeable on HKTs, so this might be completely wrong.

• ScalaWilliam@5914b5f
• ScalaWilliam@62f6e25

It fails to compile when running testsJVM / testQuick - in particular, in Derive.scala under the macros project, with an error like:

[error] java.lang.StackOverflowError
[error] dotty.tools.dotc.typer.ImportSuggestions.distinctRefs(ImportSuggestions.scala:277)
[error] dotty.tools.dotc.typer.ImportSuggestions.distinctRefs$(ImportSuggestions.scala:23)
[error] dotty.tools.dotc.typer.Typer.distinctRefs(Typer.scala:107)
[error] dotty.tools.dotc.typer.ImportSuggestions.distinctRefs(ImportSuggestions.scala:278)
...

from cats-tagless.

That stackoverflow is fixed in 3.1.1 but your build uses 3.1.0 for some reason.

from cats-tagless.

@smarter yep that works. compilation is Ok in core but not in macrosJVM and lawsJVM. 138 errors

from cats-tagless.

Yeah we need to reimplement the macros in Scala 3 - a good place to start is with FunctorK

from cats-tagless.

Hey 👋 , we've (with @dos65) recently tired to play around with lampepfl/dotty#11685 to start working on the cats-tagless macros migration; very good and exciting news is that it looks like it works 🎉 🎉 🎉 ! (Scala 3.2.0-RC4 is required)

It lives in a separate repo pomadchin/tagless-derivation at this point, with some basic macro functions for FunctorK, SemigroupalK, InvariantK, and ApplyK derivation.

Hopefully I'll be able to push it more towards making it a part of this repo, but don't hesitate to grab / contribute / start work on bringing it into this repo already.

Great news

from cats-tagless.

Hey 👋 , we've (with @dos65) recently tired to play around with lampepfl/dotty#11685 to start working on the cats-tagless macros migration; very good and exciting news is that it looks like it works 🎉 🎉 🎉 ! (Scala 3.2.0-RC4 is required)

It lives in a separate repo https://github.com/pomadchin/tagless-derivation at this point, with some basic macro functions for FunctorK, SemigroupalK, InvariantK, and ApplyK derivation.

Hopefully I'll be able to push it more towards making it a part of this repo, but don't hesitate to grab / contribute / start work on bringing it into this repo already.

Great! are you still working on this front?

from cats-tagless.

lampepfl/dotty#11685 seems closed but not sure if all the pieces are in place for this yet.

from cats-tagless.

@smarter @joroKr21 is there anything else missing for this? I understand @nicolasstucki has a PR lampepfl/dotty#15024 but I am not sure if this will be sufficient to fix it.

from cats-tagless.

That is fantastic @pomadchin, I will not have the time to spare to contribute, but am hugely interested in using this. Also had set myself as as a watcher for the repo.

from cats-tagless.

Yes I think we have all the tools necessary, it's just a matter of making the time work on this. The beginning is most difficult because we have to setup all the infrastructure. But we can do this one type class at a time, starting with FunctorK and ApplyK as the most useful ones.

from cats-tagless.

There's been a lot of work on this, which is really great! I've been wondering, as an interim step, could the core module be published for Scala 3 while work on the macros continues? It obviously would be tedious to use, since you'd have to write out algebra instances manually, but it would unlock downstream Scala 3 support for things like libraries that use the typeclasses and assume their users make the instances available. (Our async-utils and natchez-tagless libraries are examples.)

from cats-tagless.