Git Product home page Git Product logo

icerpc-csharp's Introduction

IceRPC

IceRPC for C#

CI License Report Generator Coverage coveralls.io Coverage Status

Getting started | Examples | NuGet packages | Documentation | API reference | Building from source

IceRPC is a modular RPC framework that helps you build networked applications with minimal effort.

Built for QUIC

IceRPC was built from the ground up to take advantage of QUIC, the new multiplexed transport that underpins HTTP/3.

QUIC is ideally suited for RPCs: an RPC maps to a request/response pair carried by a bidirectional QUIC stream. Multiple request/response pairs can proceed in parallel inside the same QUIC connection without interfering with each other.

IceRPC uses its own application protocol, icerpc, to exchange connection settings, transmit requests and responses, and ensure an orderly shutdown. This new RPC-focused protocol is a thin layer over QUIC.

Not only for QUIC

The primary transport for IceRPC is QUIC, but we're still in the early days of QUIC, so being QUIC-only is not practical.

To bridge this gap, IceRPC provides a multiplexing adapter called Slic. Slic implements a QUIC-like multiplexed transport over any duplex transport such as TCP. This way, you can use IceRPC with QUIC, with TCP (via Slic), and with various other traditional transports such as Bluetooth and named pipes1.

Modern C# and .NET

IceRPC for C# takes full advantage of the latest C# syntax and .NET features to offer a truly modern C# API.

Chief among these features is async/await. Async/await allows you to utilize threads efficiently when making calls that wait for I/O, and RPCs are all about network I/O. Async/await also makes your code easier to read and maintain: you can see immediately when you make an RPC versus a local synchronous call since all RPC calls have Async APIs that are usually awaited. For example:

// Synchronous code (old RPC style)

// It's unclear if this is a remote call that takes milliseconds or a local call that takes
// at most a few microseconds. In any case, this call is holding onto its thread until it
// completes.
string greeting = greeter.Greet(name);
// Asynchronous code with await (modern RPC style)

// We see it's a special call thanks to await and the Async suffix. GreetAsync releases the
// thread while waiting for the response from the peer and it's just as easy to write as
// the synchronous version.
string greeting = await greeter.GreetAsync(name);

With IceRPC, all calls that make network I/O are Async and only Async. IceRPC does not provide a parallel blocking synchronous API.

IceRPC leverages System.IO.Pipelines for maximum efficiency. This allows IceRPC to rent all its byte buffers from the same configurable memory pool.

IceRPC naturally supports cancellation just like all modern C# libraries, with trailing cancellation token parameters. This cancellation works "across the wire": when you cancel an outstanding RPC invocation, the remote service is notified and can in turn cancel further processing.

Modular and extensible

When you make an RPC with IceRPC, your request and response travel through an invocation pipeline (on the client side) and a dispatch pipeline (on the server side):

---
title: Client-side
---
flowchart LR
    subgraph pipeline[Invocation pipeline]
        direction LR
        di[Deadline\ninterceptor] --> ri[Retry\ninterceptor] --> connection[network\nconnection] --> ri --> di
    end
    client -- request --> di
    client -- response --- di
Loading
---
title: Server-side
---
flowchart LR
    subgraph pipeline [Dispatch pipeline]
        direction LR
        lm[Logger\nmiddleware] --> dm[Deadline\nmiddleware] --> service --> dm --> lm

    end
    connection[network\nconnection] -- request --> lm
    connection -- response --- lm
Loading

These pipelines intercept your requests and responses and you decide what they do with them. If you want to log your requests and responses, add the Logger interceptor to your invocation pipeline or the Logger middleware to your dispatch pipeline. If you want to retry automatically failed requests that can be retried, add the Retry interceptor to your invocation pipeline. IceRPC provides a number of interceptors and middleware for compression, deadlines, logging, metrics, OpenTelemetry integration, and more. You can also easily create and install your own interceptors or middleware to customize these pipelines.

Since all this functionality is optional and not hard-coded inside IceRPC, you can choose exactly the behavior you want. For example, you don't need the Compress interceptor if you're not compressing anything: if you don't install this interceptor, there is no compression code at all. Less code means simpler logic, fewer dependencies, faster execution and fewer bugs.

This modularity and extensibility is everywhere in IceRPC. You can easily implement a new duplex or multiplexed transport and then plug it in IceRPC. All the transport interfaces are public and fully documented.

And you can use IceRPC with a DI container—or not. It's all opt-in.

Choose your IDL

IceRPC provides a first-class byte-oriented API that allows you to make RPCs with the IDL and serialization format of your choice.

IceRPC comes with full support for two IDLs: Slice (described below) and Protobuf. You can use either Slice or Protobuf to define the contract between your clients and servers.

Slice

The Slice IDL and serialization format help you define RPCs in a clear and concise manner, with just the right feature set. Slice itself is not tied to IceRPC: you can use Slice without any RPC framework, or with a different RPC framework.

This repository provides an IceRPC + Slice integration that allows you to use IceRPC and Slice together seamlessly.

Defining the customary Greeter interface in Slice is straightforward:

// Interface Greeter is implemented by a service hosted in a server.
interface Greeter {
    // The greet request carries the name of the person to greet and
    // the greet response carries the greeting created by the service
    // that implements Greeter.
    greet(name: string) -> string
}

You don't need to craft special request and reply message types: you can specify all your parameters inline.

The Slice compiler for C# then generates readable and succinct C# code from this Greeter interface:

  • a client-side IGreeter interface with a single GreetAsync method.
  • a client-side GreeterProxy that implements IGreeter by sending requests / receiving responses with IceRPC
  • a server-side IGreeterService interface that you use as a template when writing the service that implements Greeter

Slice also supports streaming in both directions. For example:

interface Generator {
    // Returns a (possibly infinite) stream of int32
    generateNumbers() -> stream int32
}

interface Uploader {
    // Uploads an image (can be very large)
    uploadImage(image: stream uint8)
}

A stream of uint8 is mapped to a C# PipeReader while a stream of any other type is mapped to an IAsyncEnumerable<T>.

Slice provides common primitives types with easy-to-understand names:

  • string
  • bool
  • fixed-size integral types (int8, int16, int32, int64, uint8, uint16, uint32, uint64)
  • variable-size integral types (varint32, varint62, varuint32, varuint62)
  • floating point types (float32, float64)

You can define new types with struct, enum, and custom. You can construct collections with Sequence<T> and Dictionary<Key, Value>. And you can have your Slice operation return different types upon success or failure with Result<Success, Failure>.

custom allows you to send any C# type you wish through Slice, in keeping with IceRPC's mantra of modularity and extensibility. You just need to provide methods to encode and decode instances of your custom type.

Protobuf

Protocol Buffers, or Protobuf for short, is a popular IDL and serialization format developed by Google. It's the preferred IDL for a number of RPC frameworks, including gRPC.

The IceRPC + Protobuf integration allows you to call and implement Protobuf services using IceRPC with only a few lines of code.

Ice interop

IceRPC for C# provides a high level of interoperability with Ice. You can use IceRPC to write a new C# client for your Ice server, and you can call services hosted by an IceRPC server from an Ice client.

IceRPC for Ice users provides all the details.

Build Telemetry

IceRPC collects anonymous telemetry data during the build process. Participation in this program is optional, and you may opt-out if you do not wish to share any information. Detailed information about what data is collected and how to opt-out can be found in the Build Telemetry README.

License

IceRPC is licensed under the Apache License version 2.0, a permissive open-source license.

This license allows you to use IceRPC in both open-source and closed source applications, free of charge. Please refer to the license for the full terms and conditions.

Footnotes

  1. IceRPC for C# currently provides two duplex transport implementations: TCP (with or without TLS), and Coloc (an in-memory transport for testing). Future releases may add additional transports.

icerpc-csharp's People

Contributors

bentoi avatar bernardnormier avatar dependabot[bot] avatar externl avatar insertcreativityhere avatar pepone avatar reecehumphreys avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

icerpc-csharp's Issues

Admin tool without IceGrid

It would be useful to provide admin tools that work with regular servers, not just IceGrid-managed servers.

The server (or app) would just need to enable its admin object. We already have most of the required code in icegridadmin and the IceGridGUI ... where it works only with IceGrid-managed servers.

ICE-5853

Eliminate invocation mode

I propose to eliminate invocation mode and replace it with a simple IsOneway / oneway bool.

The only problematic value is Datagram, which we can compute automatically:
a) when an OA publishes proxies, it would make these proxies "datagram" when all the published endpoints are UDP
b) when making a twoway invocation (operation not marked oneway, proxy that is not IsOneway) on a proxy, UDP endpoints are filtered out during connection establishment. As long as an existing non-UDP connection is cached on this proxy, UDP endpoints would not be used (e.g. a oneway call could use an existing tcp connection).
c) making a oneway invocation on a proxy (void-returning operation with oneway metadata or IsOneway proxy), UDP endpoints are considered as usual, and the proxy can end up with a UDP connection. Subsequent two way calls on this proxy would fail as long as it caches this UDP connection.
d) proxies with only UDP endpoints are not considered for colocation optimization

The only slight downside is the behavior in (c) above, for application that mix tcp and udp endpoints in the same proxies. It shouldn't affect many applications!

Untangle Communicator and ObjectAdapter

An object adapter truly needs a communicator only for proxy unmarshaling. I believe all other uses are artificial.
See also #51.

I propose to separate them out, i.e.:

  • a communicator manages outgoing connections, but not object adapters. In particular, it does not keep track of object adapters that use this communicator.
  • as a result, there is no communicator shutdown vs destroy=dispose. Dispose=shutdown, and there is no destroy anymore.
  • an application needs to shutdown/dispose all the object adapters it creates - it can't just shutdown/dispose the communicator
  • an object adapter can still function when the communicator is shutdown. Obviously some things may not work, like umarshaling proxies.
  • the communicator no longer holds or provides various server-side defaults like ServerName, TlsServerOptions, AcceptNonSecure to the object adapters created with this communicator
  • object adapters are created using their ctor, not a factory method on Communicator
  • object adapter names are no longer unique "within a communicator". These names are used only for logging and typically (but not necessarily) to lookup configuration

Merge Reference into ObjectPrx

Concrete class ObjectPrx currently has a single get-only IceReference property. And most IObjectPrx methods simply delegate to Reference via this IceReference property.

This indirection is unnecessary and ObjectPrx could simply be the reference (and implement IObjectPrx methods directly).

This distinction would make some sense if it was common for several proxies to share the same reference, but in practice this is something very uncommon. It only occurs when you "(un)checked cast" (clone) a proxy into a proxy with a different type and keep everything else identical.

Rename ObjectAdapter to Server

The term "object adapter" comes from CORBA and its famous "portable object adapter" (POA).

I think using server like gRPC is better: simpler, shorter, easier to grasp.

Extended struct type that supports optional data members

See http://www.zeroc.com/forums/help-center/6189-any-reason-not-supporting-optional-values-structures.html

We should consider adding a new struct type that allow optional members. This struct type would be encoded with one additional byte.

Do we need a new encoding for this new struct type? What would be the name ("xstruct", "extendedstruct", "structopt", ...)?

AFAICT, we don't necessarily a new encoding, a type can't be read without its associated Slice so only Ice versions supporting the new type will be able to read it.

Add the concept of session

I propose to add the concept of sessions both at the Ice2 protocol level and in the API. The concept of session is similar to the concept of HTTP session which is a well understood concept that makes the Web what it is today.

DISCLAIMER: This is rough proposal for now so please don't pay too much attention to the API or the details. We can think about these some more if we decide to spend more time on this proposal.

On the client side

A client session is similar to a gRPC channel: it's created for an endpoint. A session can have different states (connecting, connected, disconnected, ...) and credentials are attached to the session.

Like gRPC, proxies are created from sessions.

Unlike Ice, a request doesn't live out of thin air: it is always attached to a session. This simplifies request retries. It's not individual requests which are retried like in Ice, it's the session that retries re-establishing the connection and when the connection is restored, it sends its queued requests again over the new network connection.

Callback services can be attached to a client session to allow a server to perform requests on callback services implemented by a client. A service can optionally be registered with an identity.

There could load balancing and replication policies attached to a session, similar to HTTP sessions. Sessions could also be created using an indirection mechanism (similar to adapter IDs). These depend on the container used to provide the service and are not defined here for now.

Example of what the session API could look like:

var session = new Session("ice://myhost:10000");
var proxy = IHelloPrx.Create(session);
await proxy.SayHelloAsync();

// Add a callback service and ask the server to invoke the callback service
session.Add(new CallbackServiceImpl());
await proxy.InvokeCallback();

Proxies can still be transmitted over-the-wire. There are different proxy types: session (similar to fixed), direct and indirect proxies.

On the server side

Servers support a SessionFactory which is created for a given endpoint. A session factory listens on the endpoint to create new server sessions. Like client sessions, server sessions also support different states (connected, disconnected, etc). The application can be notified when new server sessions are created to check the credentials of the client.

Service implementations (servants) are attached to the session factory with an optional identity. Method implementations have access to the session object that was used to receive the request. They can create proxies to call client services through the server session.

var factory = new SessionFactory("ice://myhost:10000");
factory.Add(new HelloServiceImpl());
await factory.Completion;

Session connectivity

A session outlive the network connection. A network connection can only be attached to a single session. The failure of a network connection leaves a session in the "disconnected" state. The session can try establishing a new network connection to restore connectivity with the server. It will retry the network connection either on the same server or possibly on another server depending on replication policy.

TLS 0-RTT can be used for re-establishing the network connection with the server.

Protocol support

Sessions require the protocol to support a session ID in the Initialize frame. With the Ice1 protocol where there's no such ID, a session would be bound to the Ice connection, the session dies when the connection dies.

C# improve generated Equals

For Ice.Identity we currently generate

public override bool Equals(object? other)
{
    if(object.ReferenceEquals(this, other))
    {
        return true;
    }
    if(other == null)
    {
        return false;
    }
    if(GetType() != other.GetType())
    {
        return false;
    }
    Identity o = (Identity)other;
    if(this.name == null)
    {
        if(o.name != null)
        {
            return false;
        }
    }
    else
    {
        if(!this.name.Equals(o.name))
        {
            return false;
        }
    }
    if(this.category == null)
    {
        if(o.category != null)
        {
            return false;
        }
    }
    else
    {
        if(!this.category.Equals(o.category))
        {
            return false;
        }
    }
    return true;
}

I think will be a bit cleaner to generate

public override bool Equals(object? other)
{
    if (object.ReferenceEquals(this, other))
    {
        return true;
    }

    if (other == null || this.GetType() != other.GetType())
    {
        return false;
    }

    Identity o = (Identity)other;

    if (!string.Equals(name, o.name))
    {
        return false;
    }

    if (!string.Equals(category, o.category))
    {
        return false;
    }

    return true;
}
  • Using the <type>.Equals static for the data members avoids the extra null checks

Consider providing asynchronous version of blocking Ice APIs

We still have few APIs which can potentially block:

  • object adapter creation
  • object adapter activation/deactivation (only if a locator is enabled I believe)
  • communicator initialization

While it can be useful in some circumstances, I don't think making createObjectAdapter non-blocking is a top-priority, typically object adapters are created at the application initialization where it's ok to make blocking calls. The activate/deactivate methods however can typically be used from servant dispatch and here it would be desirable to make sure these 2 don't block (we currently only provide the guarantee that hold() won't block).

For the API, we should decide whether or not we want "async" versions of the createObjectAdapter/active/deactivate methods or if having waitForCreate/waitForActivate/waitForDeactivate is sufficient (we currently have waitForHold/waitForDeactivate already so it would be more consistent to go with waitForCreate/waitForActivate).

The OA creation might require to block for several reasons right now:

  • resolving DNS names for the OA endpoints (Endpoints property)
  • getting the router server endpoints for routed OAs

The waitForCreate method would be useful to wait for the completion of the OA creation... We would have to decide how createDirectProxy (potentially indirectly called by addXxx methods or createProxy depending on whether or not the OA is direct/indirect) behaves if it's called before the OA creation completes. It could raise a new Ice::ObjectAdapterNotInitializedExcetion exception (but this could potentially break existing applications...). This wouldn't be a problem if we provided an async version of createObjectAdapter instead.

The same goes for communicator initialization. Right now, it could block on file access (e.g.: loading certificates, property file), creation of the admin adapter, etc. On some platforms (UWP), we have to workaround some APIs that check for blocking calls from the UI thread... to allow initializing the communicator.

Deobjectify Ice

I propose to rename Object to Service in IceRPC.

Object-oriented programming was popular 30 years ago. For the past 10+ years, the popular model is service oriented, with micro services.

The terminology would become:

  • a client uses a service proxy (proxy for short) to send requests to a (remote) service
  • a service can implement one or more interfaces
  • a service has an identity (with a name and a category)
  • a servant incarnates a service within a server (see ObjectAdapter => Server proposal)
  • a client can send or receive a service proxy as part of a request

I think this works fine and matches gRPC's notion of service.

The alternative definition for a "service" is something much coarser like a "DNS service" or "IceGrid service", and we can live with this dissonance.

Consider adding destroyAsync on Communicator

We could a new destroyAsync operation on Communicator, that destroys the communicator asynchronously. This implementation would most likely spawn a thread to run destroy.

Use case: Java servers
If you implement your shutdown operation implementation (including Process::shutdown) with the new communicator.destroyAsync instead of communicator.shutdown, you no longer need to keep the main thread around.

Currently you need the main thread to wake up after shutdown and then destroy the communicator. If you just let the main thread complete without calling exit, nobody destroys the communicator and the server keeps running (because of client thread pool threads).

ICE-8684

Disable collocation optimization for multicast?

If an application creates an object adapter with a multicast endpoint, and also creates a proxy for that same endpoint, Ice by default will use collocation optimization for invocations on that proxy. This means the request is never actually broadcast.

It's easy enough to work around it by disabling collocation optimization for the proxy, but the behavior is surprising.

consider using C# tuples for multiple return values

Currently for synchronous operations out parameter map to C# out parameters, and for asynchronous operations a struct OpName_Result is generated when there is more than one output parameter.

public struct Example_OpResult
{
    public Example_OpResult(double returnValue, bool outp1, long outp2) { ... }
  
    public double returnValue;
    public bool outp1;
    public long outp2;
}

public interface ExamplePrx : Ice.ObjectPrx
{
    System.Threading.Tasks.Task<Example_OpResult>
    opAsync(....)
}

We can replace Example_OpResult with a tuple (double returnValue, bool outp1, long outp2) and use that with the task

As for synchronous operations I will like to be able to use tuples there too

(string fistName, string secondName) = prx.GetFullName();

// better than
string secondName;
string firstName = proxGetFullName(out secondName);

Ability to intercept in and out marshaled parameters in dispatch interceptor

It would be helpful to be able to intercept in and out parameters in dispatch interceptors.

Use-case: cached responses: with the ability to store in and out parameters, it's possible to intercept and cache large/time-consuming responses to some operations.

(Benoit's comment)
With collocation optimization working for the blobject API, I think it's not too difficult to use blobject to efficiently achieve this.

ICE-4284

Remove support for Slice local definitions

The local Slice definitions are too constraining to define the API for local Ice operations, we keep adding metadata an local Slice only features to catch up with the different language features, by moving away from defining the local API in Slice we will be free to define the local API in a way that is more natural to each language mapping.

Preserve optional members

We provide a feature to preserve slices from a class which aren't known to the receiver. This allows to re-marshall the object with the unknown slices. However, we don't currently support preserving the optionals which aren't known to the receiver. As a result, those optionals won't be re-marshalled with the object. It could be good to also support this for feature completeness.

Location Resolver

Other proposals (like #62, #68) are about deconstucting Communicator - split it in smaller objects and eliminating the need for a "central do-it-all" object.

This proposal is about extracting LocatorInfo out of Communicator into a separate object.

  1. A proxy holds an optional location resolver object, e.g. in C#
public interface IServicePrx
{
    public ILocationResolver? LocationResolver { get; }
}

This location resolver object is a local object that resolves locations (and well-known proxies) and maintains a cache of previous resolutions.

Its C# interface is:

public interface ILocationResolver
{
    public void ClearCache(IServicePrx proxy);
    
    public async ValueTask<(EndpointList Endpoints, TimeSpan EndpointsAge)> ResolveAsync(
            IServicePrx proxy,
            TimeSpan endpointsMaxAge,
            CancellationToken cancel);
}

(basically the same API as LocatorInfo, which we can refine later on)

  1. The IceRPC runtime provides 2 implementations of ILocationResolver
  • an implementation based on a locator proxy:
public interface ILocationResolver
{
    // locator.Connection must be non-null
    public static ILocationResolver FromLocator(LocatorPrx locator);
}
  • the DiscoveryServer implementation
public class DiscoveryServer
{
    public ILocationServer LocationResolver { get; }
}

Initially DiscoveryServer could use an internal/hidden coloc locator proxy for its implementation.

Fix Discovery multicast interface parsing/logic

The introduction of server name completely broke the Discovery multicast interface parsing.

This code:

            // Create one lookup proxy per endpoint from the given proxy. We want to send a multicast datagram on
            // each of the lookup proxy.
            // TODO: this code is incorrect now that the default published endpoints are no longer an expansion
            // of the object adapter endpoints.
            foreach (Endpoint endpoint in _lookup.Endpoints)
            {
                if (!endpoint.IsDatagram)
                {
                    throw new ArgumentException("options.Lookup can only have udp endpoints", nameof(options));
                }

                ILookupPrx key = _lookup.Clone(endpoints: ImmutableArray.Create(endpoint));
                if (endpoint["interface"] is string mcastInterface && mcastInterface.Length > 0)
                {
                    Endpoint? q = lookupReply.Endpoints.FirstOrDefault(e => e.Host == mcastInterface);
                    if (q != null)
                    {
                        _lookups[key] = lookupReply.Clone(endpoints: ImmutableArray.Create(q));
                    }
                }

                if (!_lookups.ContainsKey(key))
                {
                    // Fallback: just use the given lookup reply proxy if no matching endpoint found.
                    _lookups[key] = lookupReply;
                }
            }

Unify Nullable and Optional

In modern programming languages such as C# 8, Kotlin and Swift, T? means an optional T or a nullable T, and the notion of nullable and optional are synonymous. Similarly in Java (which is slower to evolve), the Optional generic cannot hold a null value.

As of Ice 3.7, Slice has two nullable types: proxies and classes. And Ice provides a mechanism to mark any parameter or data member as optional with the optional(tag) prefix.

We could unify nullable and optional in Slice as follows:

  • T? means an optional/nullable T, where T can be any type (string, int, interface, class, struct...) with the exception of exception types. The language mapping is straightforward – whatever 3.7 used for optional(tag) T
  • T means a non-null (non-optional) instance or value. For example, when T is class, a parameter of type T would mean a "non-null/non-optional T".
  • when T is an interface, T* is a synonym for T?
  • the preferred syntax for specifying optional parameters and data members is to use the T? type, e.g.:
optional(1) string? x; // preferred syntax
optional(2) Intf? y; // equivalent to optional(2) Intf* y
  • for backwards compatibility, "optional(tag) string x" is equivalent to "optional(tag) string? x" but is a deprecated syntax. Likewise for other types, except interfaces: if T is an interface, optional(tag) T x is rejected as a syntax error.
  • optional(tag) no longer has any effect on the language mappings: it's purely about how/where the optional parameter/data member gets marshaled and unmarshaled.

For example, the language mappings for these two operations would be identical:

interface Foo
{
    void sendGreeting(optional(1) string? message);
    void sendGreeting2(string? message);
}
  • since optional is now only about marshaling/unmarshal, we would rename it to tag. So
void sendGreeting(optional(1) string? message);
void sendGreeting(tag(1) string? message); 

and synonymous, but the first form is deprecated.

Effect of this proposal on backwards compatibility:

  • In 3.7, it was possible to send an optional(tag) proxy or class instance with a null value, but this didn't work in all language mappings. In particular, this was messy in Java and Swift where the mapping doesn't support this distinction. With Slice 4.0, this distinction disappears.
  • Ice 3.7, when T is class, a parameter or date member of type T was nullable. This would no longer be true in 4.0 - the parameter could not be null. Ice would enforce this non-null value when marshaling or unmarshaling the parameter or data member, if doing it with a special non-nullable type is not doable.

Review C# mapping for structs

Ice for .NET supports two different mappings for Slice structures. By default, Slice structures map to C# structures if they (recursively) contain only value types. If a Slice structure (recursively) contains a string, proxy, class, sequence, or dictionary member, it maps to a C# class.

There is also cs:class metadata that allows to force the class mapping for a Slice struct.

For struct with class members, this is desirable as the unmarshaling of class data members can happen after the struct read has completed, in this case we will be patching the original and not the copy as struct are value types

self.s2 = Test.SmallStruct.ice_read(is_);

In the code above if the patching of class data members happens after ice_read returns, it will not patch s2 data members but the original struct created inside ice_read

But there is no reason to force the class mapping when the struct doesn't contains classes, we should keep the struct mapping for the rest of the cases.

User will still be able to map struct to classes by using cs:class metadata

Encoding not checked when resolving a well-known proxy from the locator cache

When resolving a well-known proxy and the well-known proxy is not in the locator cache, the Ice runtime queries the locator which might return a proxy with an incompatible encoding. The proxy with the incompatible encoding is still cached however.

Upon a second request to resolve the well-known proxy, the well-known proxy from the cache is used and the encoding is not checked leading to eventually sending a request to a server that doesn't support the client encoding.

Given that the locator cache is per-encoding on the client side, the best is probably to not add proxies with incompatible encoding to the cache and assume that the proxies from the cache are only proxies with encodings compatible with the encoding associated to the cache.

oneway operation metadata

Proposal: add a new oneway operation metadata or keyword. An operation with this metadata/keyword would be sent oneway (i.e. no response) even when called on a proxy with an invocation mode set to two-way.

With this enhancement, most applications would not need to create oneway proxies, even if they make oneway calls.

Related PR: zeroc-ice/ice#746

Re: metadata vs keyword, in practice it would be very similar to idempotent (a keyword) - a bool/enum sent with each invocation. The main difference is idempotent expresses a guarantee given by the server, and the server verifies the client does not incorrectly believe a non-idempotent operation is idempotent (i.e. there is an idempotent mismatch in the client and server Slice definitions). For oneway, the server implementation should usually behave the same for oneway and twoway calls, or can easily support both invocation modes; a mismatch between the client and server definitions would not break anything. So I think metadata is more appropriate for oneway.

Make shutdown and deactivate non-blocking?

Communicator::shutdown calls ObjectAdapter::deactivate which can make a sync call to the IceGrid registry. As a result, these operations can block.

We should make shutdown (and deactivate?) non-blocking if possible.

ICE-8700

Default path

An ice+transport URI with no path is equivalent to an URI with a default path, e.g.:

ice+tcp://host

is equivalent to

ice+tcp://host/default # identity = default, category is empty

which is itself equivalent to

ice+tcp://host//default # explicit empty category

This default mechanism does not apply for plain ice URI, i.e.

ice:

is not valid. To use the well-known identity "default" with an ice: URI, you need to type ice:default.

Using default is also required when an ice+transport URI includes a location. For example:

ice+tcp://host/loc1/loc2/loc3//default # identity = <empty>/default

On the server side, the object adapter may or may not provide a servant for this default identity.

And with ice1 stringified proxies, there is no default identity.

C# Ice/ami test failure on master (4.0)

That is failing with VS 2019 Jenkins VMs build 1044

testing graceful close connection with wait... System.AggregateException: One or more errors occurred. ()
 ---> ZeroC.Ice.ConnectionLostException
 ---> System.Net.Sockets.SocketException (10054): An existing connection was forcibly closed by the remote host.
   at ZeroC.Ice.StreamSocket.FinishRead(ArraySegment`1& buffer, Int32& offset) in C:\Users\vagrant\workspace\ice\master\csharp\src\Ice\StreamSocket.cs:line 250
   --- End of inner exception stack trace ---
   at ZeroC.Ice.Proxy.<InvokeAsync>g__InvokeAsync|9_0(IObjectPrx proxy, OutgoingRequestFrame request, Boolean oneway, Boolean synchronous, IProgress`1 progress, CancellationToken cancel) in C:\Users\vagrant\workspace\ice\master\csharp\src\Ice\Proxy.cs:line 536
   at ZeroC.Ice.OutgoingRequest.<InvokeAsync>g__ReadVoidReturnValueAsync|3_0(ValueTask`1 task, Boolean oneway) in C:\Users\vagrant\workspace\ice\master\csharp\src\Ice\OutgoingRequest.cs:line 167
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait()
   at ZeroC.Ice.Test.AMI.AllTests.allTests(TestHelper helper, Boolean collocated) in C:\Users\vagrant\workspace\ice\master\csharp\test\Ice\ami\AllTests.cs:line 680
   at ZeroC.Ice.Test.AMI.Client.Run(String[] args) in C:\Users\vagrant\workspace\ice\master\csharp\test\Ice\ami\Client.cs:line 23
   at Test.TestDriver.RunTest[T](String[] args) in C:\Users\vagrant\workspace\ice\master\csharp\test\TestCommon\TestHelper.cs:line 176

Bidir ObjectAdapter

Currently to setup "bidir", you create an outgoing connection and later set your object adapter on this connection. This has several downside:

  • the outgoing connection is typically created as a result of an outgoing request. What if this outgoing request triggers a bidir callback before the application has time to call SetAdapter on the new connection?
  • when the connection drops - which by default the application does not notice - the application needs to re-register its object adapter (with SetAdapter)

A simpler solution would be to provide a bidir object adapter. Such an object adapter has no Endpoint or PublishedEndpoint and registers itself automatically with all outgoing connections of its communicator as soon as the connection is established. It would be straightforward to use.

One potential downside is the bidir OA could be attached to too many outgoing connections, in particular some connections that should not be able to callback on the OA's objects. So maybe we can include a filtering mechanism, based on endpoints, or maybe based on a callback from the communicator during connection establishment - "do you want to attach to this new connection"?

Refactor IceSSL to be a core plug-in built-in Ice library

Now that IceSSL use the default SSL implementation for each OS and doesn't depend on third party libraries will be worth bundle it with Ice library.

One thing to keep in mind is that we want to support separate IceSSL implementation, specially on Windows we support SChannel (default system impl) and OpenSSL

We want to do this for all languages that support IceSSL

see [ICE-3247]

Convert struct and class into struct, compact struct and 1.1-only class

Proposal: replace Ice Slice struct and class by IceRPC Slice struct, compact struct and ref struct.

In IceRPC Slice:

  • a struct is always a copy: if you pass the same value as two args for the same operation, you get 2 separate copies in the arg payload
  • a ref struct has reference semantics for Slice i.e. it's always marshaled as an index + value, unless the referenced struct was already marshaled in the encapsulation (in this case, the marshaled representation is only an index).
  • a compact struct is just like a struct but with a more compact marshaled encoding (saves 1 byte). A compact struct, unlike a struct or ref struct, cannot have tagged members.

Since a struct has "by value" semantics and can hold tagged member it has no equivalent in Ice Slice. Ice Slice struct corresponds to a compact struct, while Ice Slice class corresponds to a ref struct.

For the terminology, I prefer struct, ref struct and compact struct because it expresses that they are all very similar and the default (struct) is the one you should use most often. With struct and class, the preferred default is less obvious.

Then:

  • ref struct supports inheritance but only with the 1.1 (and 1.1+) encoding
  • AnyClass (Value in 3.7) is renamed AnyRefStruct
  • in C#, struct/compact struct are mapped by default to a C# struct, while a ref struct is always mapped to a C# class. We also add back the cs:class metadata for struct and compact struct.

nonnull keyword for proxies and classes

The new nonnull Slice keyword tells Ice that a proxy or class parameter/container element cannot be null. For example:

sequence<nonnull Object*> ObjectSeq;  // a sequence of non-null proxies

class V { ... }

interface A
{
    void op(nonnull V v); // v cannot be null 
}

nonnull does not apply to data members: proxy and class data members are always nullable.

nonnull can be implemented (enforced) in the generated code without changing the Ice 1.1/1.0 encodings.

In addition, an optional proxy or class can no longer be nullable:
optional(1) Object* obj means obj can be either not set or a non-null proxy. The third option where obj can be null is removed. This introduces a (small) breaking change compared to Ice 3.7, where this optional-set-to-null is available in most programming languages (Java is a notable exception).

With the 1.1 encoding, when unmarshaling an optional proxy or class, we would tolerate a null value and translate it to a not-set optional. The reverse would not be provided: the application could not marshal an optional-set-to-null. This could potentially create some interop problems with older applications.

Syntactically, optional(1) Object* obj means optional of a nonnull proxy. Adding nonnull is not allowed, e.g. this is a syntax error:

void op(optional(1) nonnull Object* obj); // syntax error: nonnull applies only to non-optional proxies and classes

C# Ice/threading test hangs on Windows

The Ice/threading tests hangs on Windows VMs, seems like the ThreadPool stops deque new Jobs even when there is still free threads in the thread pool.

Normalize case for generated identifiers

Each programming language has its own preferred case convention for identifiers, in particular for member functions/methods and data members/fields/properties/instance variables:

  • PascalCase in C#
  • camelCase in Java, JavaScript/TypeScript, MATLAB, PHP and Swift
  • snake_case in Python
  • PascalCase, camelCase or snake_case in C++ depending on which coding style you use

The Slice compilers currently don't change the case of identifiers in the generated code: if you create an interface MyInterface in Slice, you'll get MyInterface in all programming languages. Likewise, if you define an operation myOp (camelCase), you will get a myOp function/method for your proxies and skeletons in all programming languages, and it will look odd in C# and Python.

The Slice naming convention used by ZeroC for Ice and the Ice demos is:

  • camelCase for operations, parameter names, data members
  • PascalCase for module names, types names (interface, class, struct, enum etc.), constant and enumerators

Ice users are of course free to use whichever naming convention they prefer. For example, it's fairly common to use PascalCase for operation names in Slice and get the same case in the resulting methods/functions.

It would be nice if the generated identifiers could respect the preferred naming convention of each language, instead of generating good looking identifiers in some languages and odd-looking names in other languages.

Proposed solution
Add file metadata directives that "normalizes" the case of generated identifiers. The default (without metadata) would be the current behavior: keep the identifiers as-is.

To keep things reasonably simple, I propose a single per-language switch: either the Slice compiler keeps all the identifiers as is, or it normalizes all the identifiers. For example, you would not be able to tell the compiler to normalize the generated identifiers for your operation names but not for your data member names. Furthermore:

  • module names would not be transformed. We would introduce instead module metadata to set the name of the corresponding namespace, package, module etc.
  • type names and operation parameter names would not be transformed

In other words, this new metadata would affect only the mapping for operation names, data members names, enumerators and constants. In most languages, there would be only one option, for example:

[["java:normalize-case"]] // mapped methods and fields are camelCase
[["cs:normalize-case"]] // mapped methods and properties are PascalCase
[["swift:normalize-case"]] // mapped functions and properties are camelCase
[["python:normalize-case"]] // mapped methods and instance variables are snake_case

In C++, where there is no universal naming convention, the metadata would include the desired convention:

["cpp:normalize-case:camelCase"]] // mapped functions and data members are camelCase
["cpp:normalize-case:PascalCase"]] // mapped functions and data members are PascalCase
["cpp:normalize-case:snake_case"]] // mapped functions and data members are snake_case

Rename Slice module Ice to module IceRpc

Alongside this change, all the "core" code would move to namespace IceRpc.Core, and the IceRpc.Core.dll assembly, and eventually the IceRpc.Core Nuget package.

This requires changing the Slice compiler to generate ::Ice::xxx type IDs for classes and exceptions defined in module IceRpc::Core, for interop with Ice.

idempotent and tuple return type

This currently does not work on master - I get a strange syntax error.

For example:

idempotent (EndpointDataSeq endpoints, StringSeq newLocation) resolveLocation();

ShutdownAsync/DisposeAsync pattern

I believe the correct pattern for async disposable objects like Communicator and ObjectAdapter is as follows:

public sealed class Xxx : IAsyncDisposable
{

    public ValueTask DisposeAsync() => new(ShutdownAsync());

    // if cancel receives a cancellation request, the implementation terminates all calls "as soon as possible" resulting in
    // possibly ungraceful connection closures
    public Task ShutdownAsync(CancellationToken cancel = default) { .... }

}

Disposing the object with await using means graceful shutdown. However an application that uses ShutdownAsync directly can decide to timeout very quickly and not "hang" on shutdown.

Optional of type class in Compact Format (Requires encoding update!)

When we unmarshal in "compact format", an unrecognized optional parameter or data member of type class can break the unmarshaling (result in a no exception factory exception) because we don't know how to skip this optional value.

We document this limitation, which is really an oversight and not something we did on purpose. For all other types, we encode the number of bytes to skip with the optional tag in case we don't know the type of this optional value.

We should fix this bug in the next revision of the Ice encoding.

ICE-8526

Continue simplifying the streaming API

There are several features in the stream APIs that Ice no longer use or never used:

  • nested encapsulations, none of our Slice types use that
  • encapsulation-less streaming (it's possible to stream a type without calling startEncaps/endEncaps)

I believe the later is still used by Freeze for writing data to the databases. It might also be used by the lmdb layer (to be verified).

Removing support for these features would simplify the streaming API and could even improve performances a bit. We could for example get rid of the encapsulation structure and stack if we remove support for nested encapsulations, we would no longer need to allocate a separate encapsulation structure with the output/input stream.

Removing this means that we could potentially break user code that somehow rely on these features so we need to consider this as well before deciding to remove them.

Replace Value's ice_postUnmarshal and ice_preMarshal

Values provide two marshal/unmarshal interception points, in all language mappings.
For example in C++:

class ICE_API Value
{
public:
   [ ... ]

    /**
     * The Ice run time invokes this method prior to marshaling an object's data members. This allows a subclass
     * to override this method in order to validate its data members.
     */
    virtual void ice_preMarshal();

    /**
     * The Ice run time invokes this method after unmarshaling an object's data members. This allows a
     * subclass to override this method in order to perform additional initialization.
     */
    virtual void ice_postUnmarshal();

These interceptors are called all the time during marshaling and unmarshaling and by default do nothing. They are potentially useful only when the application uses custom values created through custom value factories. And in practice, few applications do so.

I propose to move the interception points to avoid calling them with default generated value implementations:

  • post-unmarshal becomes a separate optional parameter that the application supplies alongside the value factory when registering a custom value factory. For example
// Slice
interface ValueFactoryManager
{
     void addWithInterceptor(ValueFactory factory, PostUnmarshalInterceptor interceptor, string id);
}
  • pre-marshal is replaced by renaming the existing _iceWrite method to ice_write; the application that needs to intercept post-marshaling can then override ice_write to insert the interception code and then call its parent's ice_write.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.