Support asynchronous block callback arguments in methods about edistantobject HOT 19 OPEN

can you provide how NSXPCConnection handles async callback, API etc?

when you mentioned block, you mean the actual block type, the anonymous function, right?

eDO doesn't have future but can provide API that will integrate with future.

from edistantobject.

Yes, block = lambda = anon function.

Let's look at a simple example:

I have a shared protocol:

@protocol TestServiceProtocol

- (void)upperCaseString:(NSString *)aString withReply:(void (^)(NSString *))reply;
    
@end

On the server (listener), when a new connection arrives, the protocol is explicitly given to the XPC system:

- (BOOL)listener:(NSXPCListener *)listener shouldAcceptNewConnection:(NSXPCConnection *)newConnection
{
    newConnection.exportedInterface = [NSXPCInterface interfaceWithProtocol:@protocol(TestServiceProtocol)];
    newConnection.exportedObject = self;
    [newConnection resume];
    return YES;
}

Finally on the client, the protocol is again provided explicitly. Without providing this, calling a remote method fails with unknown selector.

NSXPCConnection* c = [x initWithServiceName:@"com.LeoNatan.XPCTester.XPCService"];
c.remoteObjectInterface = [NSXPCInterface interfaceWithProtocol:@protocol(TestServiceProtocol)];
[c resume];

[[c synchronousRemoteObjectProxyWithErrorHandler:^(NSError * _Nonnull error) {
	NSLog(@"%@", error);
}] upperCaseString:@"hello" withReply:^(NSString *aString) {
	NSLog(@"Result string was: %@", aString);
}];

On the server, when the method is invoked by the connection, the supplied block is a block created at runtime during decoding of the XPC connection:

(lldb) po [reply debugDescription]
<__NSStackBlock__: 0x7000031e1378>
 signature: "v16@?0@8"
 invoke   : 0x7fff309dd3a2 (/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation`__58-[NSXPCConnection _decodeAndInvokeMessageWithEvent:flags:]_block_invoke.93)

Interestingly, if I use protocol_copyMethodDescriptionList to obtain the protocol methods, the returned argument types are the usual ObjC types:

2019-09-23 12:38:16.612025+0300 ExampleApp[7233:315365] getSpringboardRegionOverride:reply:: v32@0:8@16@?24
2019-09-23 12:38:18.252195+0300 ExampleApp[7233:315365] getServerAnswerForQuestion:reply:: v32@0:8@16@?24
2019-09-23 12:38:19.229449+0300 ExampleApp[7233:315365] getAppleTVMode:: v24@0:8@?16
2019-09-23 12:38:22.786022+0300 ExampleApp[7233:315365] rebuildCache:: v24@0:8@?16

But if I use _protocol_getMethodTypeEncoding, a more interesting types are available:

2019-09-23 13:17:06.789312+0300 ExampleApp[7418:337846] getSpringboardRegionOverride:reply:: v32@0:8@"NSString"16@?<v@?@"NSNumber">24
2019-09-23 13:17:06.789477+0300 ExampleApp[7418:337846] getServerAnswerForQuestion:reply:: v32@0:8@"NSString"16@?<v@?@"NSDictionary">24
2019-09-23 13:17:06.789604+0300 ExampleApp[7418:337846] getAppleTVMode:: v24@0:8@?<v@?@"NSDictionary">16
2019-09-23 13:17:06.789714+0300 ExampleApp[7418:337846] rebuildCache:: v24@0:8@?<v@?B>16

I'm guessing this is the reason they require a protocol, because this type of type metadata is only output for protocols if I recall correctly.

Once the reply block is called on the server, it is serialized and the originally provided client block is then called using NSInvocation.

from edistantobject.

There also appears to exist _Block_signature which returns the extended signature of an arbitrary block object.

from edistantobject.

- (void)upperCaseString:(NSString *)aString withReply:(void (^)(NSString *))reply;

are you saying that you declare your own method signature where you have a callback block? then in your impl in the server side, the callback is, for the purpose of this discussion, invoked after the method returns. and NSXPCConnection is able to invoke a remote block as is?

I'm guessing this is the reason they require a protocol, because this type of type metadata is only output for protocols if I recall correctly.

I also found in typedefs, more detailed encoding info is provided compared to the encoding directly retrieved from the method or class itself. I would guess this is just how the runtime is programmed right now, protocol or typedef is more typed driven so it would contain more type info? but in either case, the signatures look good to me, and it actually doesn't matter how detail the block signature is, eDO also forwards block invocation but doesn't require the detailed info from encoding, there is a runtime structure that eDO to use to construct the NSInvocation, or to be more precisely, to let objc runtime to construct NSInvocation. And this might be the reason why it didn't work for you as I see your callstack was coming from a dylib.

from edistantobject.

I only declare a protocol. The system deduces the type of the block and creates a proxy. NSXPC provides two types of proxy objects for clients, an async one and a sync one. The sync one blocks the client call until the reply block is called on the server; the reply block is called on the same thread as method call in client process. The async one does not block, and the reply block is called on a background queue.

Regarding +load, +initialize and attr(constructor), I've hit similar issues of order.
I've resorted to hacks:
https://github.com/wix/DetoxSync/blob/ff1e278cac69fcac2e88195f59ceeef3beeaa71c/DetoxSync/DetoxSync/DetoxSync.pch#L24
https://github.com/wix/DetoxSync/blob/ff1e278cac69fcac2e88195f59ceeef3beeaa71c/DetoxSync/DetoxSync/SyncManager/DTXSyncManager.m#L124

I'm not sure this hack specifically will fit the usecase here, as eDO is using a category. But the current behavior is a bug.

from edistantobject.

I will open a new issue regarding the load order.

from edistantobject.

I only declare a protocol.

Did you declare the last reply: in your protocol as well?

Regarding +load, +initialize and attr(constructor), I've hit similar issues of order.

Thanks for the sharing, it's a smart workaround. I don't know if there is an order issue in eDO, because when it forwards a block, it should add those two forwarding methods to NSBlock, unless it doesn't because you load it from dylib? because it's a dummy empty block placeholder, it doesn't do anything and just pass through quietly.

from edistantobject.

The issue here is that +load is called earlier in the start process, before attr(constructor). So the functionality there hasn't had a chance to run yet.

from edistantobject.

but that only matters after the block is invoked remotely? as long as by the time you invoke a remote block, the attr(constructor) is set, you should be good, right?

from edistantobject.

It's not the behavior I observed. In my case, the remote call was blocking (so blocking in a +load), and so the start process was blocked.

from edistantobject.

Oh I see. Let's continue the discussion on a different thread, and focus on the async call for this one. Both look like legitimate issues to resolve to me.

from edistantobject.

Regarding the reply:, yes, it is part of the method. I'm not sure how Apple deals with methods that have two blocks, for example, but it's an edge case that is less interesting. The normal use-case is to have a reply block (with or without arguments), to be called asynchronously when the information is available or task is completed remotely.

from edistantobject.

oh so you are saying, Apple forces the method to be a pattern that has a reply: in the end with the return value as its argument?

from edistantobject.

Right, forgot to mention. In NSXPX, method return value can only be void or NSProgress. Reply block must be used to return a value, but the block doesn’t necessarily have to include arguments.

from edistantobject.

I read some of the doc here, looks like the way NSXPC supports block is different than eDO.

eDO works more like native ObjC, so it doesn't require you to change your signature to have reply:. One idea to support async call is to have a category API in NSObject like -asyncWithCallback:(void(^)(NSInvocation *))block, very similar to passByValue, it signals the object that the future calls will be async and return immediately. So you would have something like this:

[[remoteTestObject asyncWithCallback:^(NSInvocation *invocation) {
  __unsafe_unretained NSString *result;
  [invocation getReturnValue:&result];
}] upperCaseString];

This somehow looks really ugly to me.

from edistantobject.

I’m not sure I understand why this is necessary. It does look ugly indeed.

One idea I had was to use the remote block proxy’s lifecycle to control when the connection ends. So if a method is called with block arguments, on the server create proxy block arguments, and add some dealloc handler to each proxy block (using associated objects). Then, as long as these blocks are alive (proxy blocks have not been released), keep the connection open. If blocks are called on server, forward to client. Once all block proxies are released, close connection. This then is exactly like normal Objective C methods/blocks.

from edistantobject.

The remote object can hold on to the connection/EDOHostService, but currently it doesn't, I worry that this can stress the service because eDO can generate many remote objects as well as "temporary services", the service only created dynamically to wrap a local argument in a short period of time. but on the other hand, if you do manually create a EDOHostService, the remote object or the block can be invoked even after the method returns.

my earlier comment was only meant for an organic way to support async call, such that your method would be only -upperCaseString and it turns itself into a future return automatically.

from edistantobject.

Your last comment is not a feasible scenario in many cases, especially when GCD is invoked. upperCaseString is a trivial example, not indicative of real-world usage. Completion handlers are needed in many cases where the server needs to perform asynchronous tasks.

NSXPC (through XPC) retains connections too, and it is the most used IPC mechanism on macOS and iOS. I am not familiar with any issues that has. I guess a major difference is the usage of a bad socket rather than a mach port.

from edistantobject.

The completion handler should be already supported with an exception that the service is not held by the remote object such that you have to create a service explicitly. NSXPC API has a parent NSXPCConnection interface for the client so it can be bi-directional channel, whereas in eDO, there will be two services and two separate client/service pairs.

The one feature that eDO has is to automatically turn your local objects to remote objects, but it also hides a lot of details that become harder to debug and understand when there is any issue like this. Maybe making a paired service would work better or easier to understand and API-wise. At least at a very minimum, the block is supported and there is no type restriction unlike XCP where you can only send POD or NSCoding objects.

from edistantobject.