Champion "Covariant Return Types" (VS 16.8, .NET 5) about csharplang HOT 69 OPEN

I find myself wishing C# supported covariant returns very often, and often with code that isn't particularly "clever".

from csharplang.

Covariant returns would be very useful for typed cloning methods, preventing the need to manually create unique protected cloning methods.

from csharplang.

@MohammadHamdyGhanem

Why didn't this happen yet?
It seems popular enough.

Because you haven't written it yet.

Seriously, there are a million "good" ideas, modifying the language is a very expensive process and the team has finite resources. Things take time.

from csharplang.

Why didn't this happen yet?
It seems popular enough.

from csharplang.

@HaloFour

Because you haven't written it yet.

If I can write "it", I would rather write a new S# (Smart Sharp) language, where I do whatever I want :).

Anyway, may God help the team :)

from csharplang.

Prioritization of runtime support for covariant return types is being tracked internally at https://devdiv.visualstudio.com/DevDiv/_queries/edit/1009909

from csharplang.

For a long time I didn't realise that the problems I had with many of my inheritance/interface heirarchies nearly always came back to this common problem - the lack of return type covariance to allow overriding methods to narrow their return types. This then leads to duplication of methods and properties to satisfy both the base and the subclass contracts, which is a whole lot of general messyness.

So I am very much in favour of this proposal.

from csharplang.

The spec leaves open the question of whether override would be supported for interface implementators to have covariant return types, has a decision been made on this?

example, the spec mentions

interface I1 { I1 M(); }
interface I2 { I2 M(); }
interface I3: I1, I2 { override I3 M(); }

But currently override is not allowed on interface properties as far as I can tell.

from csharplang.

@gafter
I think this feature is an extremely important one, and would be willing to put some work towards implementing it. I know @ldematte did some work on this, and I can build on his proposal.

If you think this is likely to succeed, I will go ahead. Some pointers on the process I will need to take would be helpful.

@ldematte, I assume you are no longer working on this, but if you are, please let me know, and we can discuss if I can help. Also, if you've achieved anything useful so far, it would be great if I could build on that.

Also @sharwell and @HaloFour were arguing on how best to implement this, and no decision seemed to be made. Should I just go for this with Method Shadowing, or do we need to finish that discussion first?

from csharplang.

I have created #2844 as a draft specification for this feature.

from csharplang.

@HaloFour That is the most likely approach, and something corresponding for interfaces too.

from csharplang.

I think its funny, that Roslyn itself suffers from the lack of this feature: AST modification methods in it, like MethodDeclarationSyntax.WithModifiers could have been virtual.

from csharplang.

Im trying to sum up current state of this feature, please correct me if im wrong:

1. Sufficiently good(for poking roslyn with a stick) proposal is in place
2. There was a successful attempt had been made with manual IL changes as one proposed in feature
   https://github.com/YairHalberstadt/csharplang/blob/master/proposals/covariant-returns-exploration.md
3. There was a successfull attempt at making it work with vs 2017
   #49 (comment)
4. There is info which roslyn methods are need to be poked to make this feature work
   dotnet/roslyn#357 (comment)
5. These isn't a PR for the feature

This rises a question:
Is PR for roslyn with this feature welcomed at this stage of discussion?

from csharplang.

One example of a breaking change that we would have if we add contravariant parameters:

class Base
{
    public virtual void M(string o) { }
    public virtual void M(object o) { }
}
class Derived : Base
{
    public override void M(object o) { } // oops, now overrides both methods
}

from csharplang.

My two cents.

Repurpose the as keyword in the method return type specification.

For example:

public abstract class Foo {
    public abstract object DoStuff();
}

public class ProposedBar : Foo {
    public override string as object DoStuff() => "stuff"; // covariantly overrides Foo.DoStuff()
}

The presence of the as keyword can trigger some compiler magic that will help the compiler understand which interface method or base class abstract method it is supposed to be implementing covariantly.

The use of as keyword can be made optional. It is only required if the compiler do not have enough information to figure out automatically.

from csharplang.

I confirm you that, with the .NET 5 RC2 published yesterday, Covariant Return Types is working correctly now

from csharplang.

Current proposals to implement this require using compiler tricks. That was because till now, the runtime has been considered immutable.

However with the move towards .Net Core, runtime changes have been put back on the table.

Would it be possible to look into if this can be supported at a runtime level?

from csharplang.

Are there any plans to make it simpler to write functions that returns typeof(this)? It seems like when we have covariant return types, just some syntactic sugar needs to be added to support this. This is one of the most common scenarios for covariant returns I have encountered. E.g.

abstract class FluentBuilder
{
    public virtual thistype DoSomething() // new keyword - will return FluentBuilder in this class, but subclass type in subclasses
    {
        // ...
        return this; // Can only be allowed to "return this" for it to work(?)
    }
}
class SomeFluentBuilder : FluentBuilder { } // will not need to override DoSomething to make it return SomeFluentBuilder

instead of (C#9)

abstract class FluentBuilder
{
    public virtual FluentBuilder DoSomething()
    {
        // ...
        return this;
    }
}
class SomeFluentBuilder : FluentBuilder
{
    public override SomeFluentBuilder DoSomething()
    {
        base.DoSomething();
        return this;
    }
}

from csharplang.

Great work @YairHalberstadt , although your designs do not address properties. As initially proposed by @gafter, the implementation should include properties (thus associated getters) as well as method return values. IE this should be allowed:

case l:

interface IA
{

}

public class A : IA
{

}

interface IB
{
     IA PropertyA { get;  }
}

class B : IB                    //Should compile!
{
    public A PropertyA { get; set; }  
}

Furthermore, expanding on your "case k", what about when generics are nested (ie containers). Where this really starts getting interesting and very powerful is if implicit casting is also supported into covariant type interfaces (ie out interfaces):

case m:

interface IA
{

}

public class A : IA
{

}

interface IB
{
    IReadOnlyList<IA> GetList();
}

class B : IB
{
    public List<A> GetList()  
    {
        return new List<A>();       
    }
}

  //Nb implicit conversion of List<A> -> IReadOnlyList<IA> since IReadOnlyList is a generic <out IA>

If both of the above were allowed then one could use interfaces to provide read-only access to an entire object model without the end-user referencing or knowing anything about the implementing classes... this would be extremely powerful and have many applications.

from csharplang.

@Logerfo

Depends on how covariant return types are implemented. The CLR has always considered the return type of a method to be a part of its signature and for overloads that only differ by return type. Covariant return types could build on top of this by using an overload to define the override with the covariant return type and emitting a bridge method as the actual overload:

public abstract class Foo {
    public abstract object DoStuff();
}

public class Bar : Foo {
    public override string DoStuff() => "stuff";
}

// compiler emits
public class Bar : Foo {
    public new string DoStuff() => "stuff";

    override object Foo.DoStuff() => this.DoStuff();
}

I used an explicit implementation-like syntax above as C# doesn't currently support a syntax that would enable this. But such syntax is fitting as the CLR also allows for explicit overrides of virtual methods (complete with the ability to change the accessibility and the name of the overridden method).

As for CRTP I think the question is how it would be implemented. Auto-generating covariant returning methods seems easy enough, but I don't think it would work well without generics being involved, and that would either require CLR support or some janky C# voodoo which would probably be difficult to prevent from leaking.

I don't think that covariant returns are limited to the CRTP case so it makes sense to enable them as a standalone language feature, especially since they also don't require any CLR changes or even really any changes to the C# grammar.

from csharplang.

@gafter @Joe4evr @JonHanna @TylerBrinkley @srivatsn
Or better:

public interface IQux : IBar
{
    this Get();
}

or better #1566 :

public interface IQux : IBar
{
    [IBaz] Get(); 
}

from csharplang.

To save you a little time, fell free to recycle the work I did when I was looking at feasibility.
There is already a mock C# syntax (which is really simple), target IL compiled with ILASM and tested with the C# and VB compilers. And with ILASM/DASM and reflector.
A tool that needed modification for sure was Visual Studio: intellisense got confused, producing strage/non intuitive suggestions, but it worked (no crashes).
Do you have a link at my work/gists?

from csharplang.

I have just tried it with VS2017 and it works flawlessly, proposing them as overloads (even it is just the return type changing).
I still have a test project with a compiled dll in my dropobox if you want to try it out.

from csharplang.

How much more work would supporting contravariant parameters involve?

Based on my limited experience, I think that those have way more limited use-cases than covariant return types while at the same time bringing more issues to the table.

from csharplang.

I don't know about 2-4. I should also point out that the LDM and compiler team haven't decided to prioritize this for any particular release. Having said that, a full prototype would give us more information about whether this is worth whatever it might turn out to cost. So sure, a prototype would be helpful.

from csharplang.

How much more work would supporting contravariant parameters involve?

It would involve extreme breaking changes in common scenarios. In other words, impossible.

@gafter Could you please explain (just small example), what breaking changes you mean?

from csharplang.

Please also extend this feature so it doesn't have to be strictly covariant in the currently used sense but rather also allows value type returns to implement methods/properties that expect an object return type or an interface return type, i.e:

#296

I believe the strict definition of "covariant" currently only applies to reference types, right?

from csharplang.

One question about Covariant Return Types
I am a little confused with it

Is this feature ready to be used in the last version of NET 5 (5.0.100-rc.1.20452.10 version) or is in progress yet?

It's implemented but RC1 contains a bug with using derived type in abstract causing a TypeLoadException. Hit me. RC2 has it fixed.

from csharplang.

@Daniel15 such issues are closed only after spec is added (look for Needs ECMA spec badge)

from csharplang.

It's a bit unclear from this thread and other notes. Is it done using "CLR hack" of just allowing covariant return types, or the compiler double-method emittance?

from csharplang.

@TahirAhmadov

The CLR was modified to allow the overriding method to have a covariant return type. I don't see why that would be considered a "hack", though.

You'll laugh but that's how it was characterized by a member of the language team about 10 years ago when I first brought it up, before GitHub.

from csharplang.

This would apply to methods and properties, and be supported in classes and interfaces.

The part about support in interfaces, does this mean something like the following?

public interface IFoo { }
public interface IBar
{
    IFoo Get();
}
public interface IBaz : IFoo { }
public interface IQux : IBar
{
    override IBaz Get(); //or would it be 'override IBaz IBar.Get();'?
}

I guess it's lucky that discussions around Default Interface Implementations have pushed back a little bit on using override as a keyword for just that. Even if it wasn't, there probably could've been a way for the two to co-exist, but if you're talking about adding one new thing to interfaces, it would be good to consider the other things that are proposed to be added to interfaces, and make sure that one design choice doesn't preclude any other features entirely.

from csharplang.

Would this need CLR changes, or compiler changes only?

from csharplang.

According to linked proposal document it is purely compiler magic. It is mentioned that compiler will spit two methods instead of one. One virtual override and another who will call this override and cast result to more specific instance so nothing fancy. Though for that reason alternative mentioned in proposal SEEMS to be better for performance but that will come at the cost of ugliness in code (method duplications).

I wonder if both couldnt be combined into one proposal? So compiler will produce same code to alternative but source code will look like original proposal.

from csharplang.

I think this will require some (perhaps substantial) design work before the desired/best implementation techniques are decided/agreed.

from csharplang.

@gafter
What would that involve? Would a proposal containing C# and target IL for a variety of examples be sufficient?

from csharplang.

@YairHalberstadt That would help. It would also help to understand how this would affect any tools that consume IL, including compilers for other languages.

from csharplang.

Ok. I will get started on a design proposal.

from csharplang.

dotnet/roslyn#357 (comment)

https://gist.github.com/ldematte/39b9f3119a92dc058799

Are the relevant parts

from csharplang.

Thanks Idematte. I'll definitely use whatever I can from your work.

from csharplang.

A tool that needed modification for sure was Visual Studio: intellisense got confused, producing strage/non intuitive suggestions, but it worked (no crashes).

Can you be more specific? What strange/unintuitive suggestions are you referring to? Thanks!

from csharplang.

Honestly, I can't recall.
Mind you, the test was done a couple of years ago, meaning VisualStudio 2015, I don't know how VS2017 could react. I probably still have the dlls though, I can try them again (or recreate them from .il)

from csharplang.

from csharplang.

https://www.dropbox.com/sh/hw0sgrae83i93lm/AAC_YXcNxjY3q70JExCoAWfOa?dl=0

There is also a draft proposal I forgot I wrote :)
And a console application with some performance tests too (the stub was designed to work efficiently across multiple subclasses)

from csharplang.

@ldematte Thank you, your draft proposal is very helpful!

from csharplang.

@gafter, @Idematte, @HaloFour, @sharwell
I have written a relatively detailed exploration of 3 different designs for implementing covariant return types in C#.
I would be interested in hearing what you think, and receiving any feedback you have.
I hope it's helpful.
https://github.com/YairHalberstadt/csharplang/blob/master/proposals/covariant-returns-exploration.md

from csharplang.

Sorry, I have just changed the file name slightly to https://github.com/YairHalberstadt/csharplang/blob/master/proposals/covariant-returns-exploration.md
The old link will give you a 404 error

from csharplang.

@m00bh000
Thanks.
All the designs can be mapped to properties directly - there is no reason they should not work. Similarly for your case m.

I don't currently have the time to add them to the exploration, but if anyone does, I'll happily take a PR.

from csharplang.

@YairHalberstadt

By that you mean having the runtime allow for a method with a more specific return type to be used in an override slot, rather than having the compiler emit bridge methods?

If that were to be considered I think it should be bidirectional and also allow contravariant parameters.

from csharplang.

@HaloFour
Yes, and I agree.

from csharplang.

While we're at it, might it even be possible to extend "return type covariance"/"parameter type contravariance" to generic arguments when they're in the appropriate position?

// return type constrained by interface, or shape detection at runtime
public Foo<Base> M()
{
    // currently produces CS0029: Cannot implicitly convert type 'Foo<Derived>' to 'Foo<Base>'
    return new Foo<Derived>();
}

from csharplang.

@Joe4evr I'd prefer in/out variance be specifiable on a class, with appropriate checking that the shape of the type fits those rules.

Note: in the interim, you could always get what you've written by using interfaces. This also ensures you can separate out the part of shape of your type that you want to be variant, without restricting other parts of your type (say, your private methods) that won't need those rules applied to them.

from csharplang.

How much more work would supporting contravariant parameters involve?

It would involve extreme breaking changes in common scenarios. In other words, impossible.

from csharplang.

Well then I guess we don't need to even consider it? 🤷‍♂️

from csharplang.

I like this because it solves some cases where CRTP would be needed. But this doesn't solve all the cases... First of all: does this feature require CLR changes?
The problem with CRTP is that I cannot have polymorphism without actually knowing the constructed type, which kinda breaks its usefulness. For example:

class Animal<T> where T: Animal<T>, new()
{
   public T[] Parents { get; private set; }
   public T Breed(T other) => new T
   {
      Parents = { this, other }
   };
}
class Dog : Animal<Dog> { }
class Person
{
   public Animal[] Pets { get; } //not possible
}

But what if C# had this T type for the constructed type out of the box, without the need for an actual generic enclosing type. Like this:

class Animal
{
   public TConstructed[] Parents { get; private set; }
   public TConstructed Breed(TConstructed other) => new TConstructed
   {
      Parents = { this, other }
   };
}
class Dog : Animal { }
class Person
{
   public Animal[] Pets { get; } //now possible
}

It doesn't matter how TConstructed is called (can't think of a non-breaking syntax).
The new() constraint is debatable but that's just an example anyway.

Does my suggestion require CLR changes? If both requires or if both doesn't, I think the cost-benefit of my suggestion would fit better IMO, since it could be used for virtual methods as well.

from csharplang.

@HaloFour I see. I opened another issue (#2839) to split up the suggestions then.

from csharplang.

Just a note: the LDM is proceeding with this feature with the premise that we can do it with runtime support rather than through the generation of bridge methods. Our current exploration will help us understand how feasible this approach really is.

from csharplang.

@gafter

By doing it through runtime support you mean having the CLR allow for the .override clause to be allowed when the method has a signature that only differs by return type as long as the return type has identity or implicit reference conversion to the return type of the overridden method?

.class public auto ansi beforefieldinit A extends [mscorlib]System.Object
{
    .method public hidebysig newslot virtual instance object M () cil managed 
    {
        // yadda yadda yadda
    }
}

.class public auto ansi beforefieldinit B extends A
{
    .method public hidebysig virtual instance string M () cil managed 
    {
        .override method instance object A::M()
        // yadda yadda yadda
    }
}

from csharplang.

FYI, the draft specification is at https://github.com/dotnet/csharplang/blob/master/proposals/covariant-returns.md

from csharplang.

@kinchungwong The compiler doesn't need any "help" to understand which method is being overridden. The return type was never part of that calculation.

from csharplang.

There are other proposals that cover that.

from csharplang.

One question about Covariant Return Types
I am a little confused with it

Is this feature ready to be used in the last version of NET 5 (5.0.100-rc.1.20452.10 version) or is in progress yet?

from csharplang.

I was writing this because my Covariant Return Types code was not working as expected and I was confused about it
Thanks a lot for the details about the issue with it @amis92

from csharplang.

It sounds like this is now available in C# 9.0 (ref https://devblogs.microsoft.com/dotnet/c-9-0-on-the-record/#covariant-returns) so this issue can probably be closed?

from csharplang.

@TahirAhmadov

The CLR was modified to allow the overriding method to have a covariant return type. I don't see why that would be considered a "hack", though.

from csharplang.

Perhaps they thought of the solution as a "hack" since it required users to target a new CLR as opposed to being backwards compatible with older CLR's like that of the .NET Framework.

from csharplang.

@TahirAhmadov
The CLR was modified to allow the overriding method to have a covariant return type. I don't see why that would be considered a "hack", though.

You'll laugh but that's how it was characterized by a member of the language team about 10 years ago when I first brought it up, before GitHub.

Yea, things changed a lot in that time. I understand that the main reason they think differently now is the fact that changing the CLR no longer introduces the risk of automatically breaking every .NET application on your PC.

from csharplang.

The spec leaves open the question of whether override would be supported for interface implementators to have covariant return types, has a decision been made on this?

example, the spec mentions

interface I1 { I1 M(); }
interface I2 { I2 M(); }
interface I3: I1, I2 { override I3 M(); }

But currently override is not allowed on interface properties as far as I can tell.

One thing I would love, would be to write GetEnumerator() once to implement IEnumerable and IEnumerable<T>. Would override-support in interfaces be required for this?

from csharplang.

@gafter @MadsTorgersen

What is happening with doing this for interfaces as well? Is that going ahead? Is there another issue tracking that?

If not, what are the arguments against doing so? I didn't see any in this thread.

from csharplang.