maxxfrazer / multipeerhelper Goto Github PK

A light swift wrapper around the iOS MultipeerConnectivity framework. Including an example project using RealityKit's MultipeerConnectivityService.

Home Page: https://maxxfrazer.github.io/MultipeerHelper/documentation/multipeerhelper/

License: MIT License

Swift 97.29% Shell 2.71%

realitykit multipeer-connectivity multipeerconnectivity ios swift swiftpm augmented-reality arsession arkit arkit3

multipeerhelper's Introduction

MultipeerHelper

MultipeerConnectivity can be a big pill for developers to swallow. This package aims to simplify the creation of a multi-peer experience, while still delivering the full power of Apple's API.

Installation

This is a Swift Package, and can be installed via Xcode with the URL of this repository:

[email protected]:maxxfrazer/MultipeerHelper.git

For more information on how to add a Swift Package using Xcode, see Apple's official documentation.

Usage

To use this package, all you have to do is import MultipeerHelper and initialise the object:

self.multipeerHelp = MultipeerHelper(
  serviceName: "helper-test"
)

Because MultipeerConnectivity looks over your local network to find other devices to connect with, there are a few new things to include since iOS 14.

The first thing, is to include the key NSLocalNetworkUsageDescription in your app's Info.plist, along with a short text explaining why you need to use the local network. For example "This application needs access to the local network to find opponents.".

As well as the above, you also need to add another key, NSBonjourServices. Bonjour services is an array of Bonjour service types. For example, if your serviceName is "helper-test", you will need to add _helper-test._tcp and _helper-test._udp.

The two above keys are included in the Example Project.

See full documentation here:
https://maxxfrazer.github.io/MultipeerHelper/documentation/multipeerhelper/

RealityKit

To extend this to RealityKit's synchronization service, simply add the following:

self.arView.scene.synchronizationService = self.multipeerHelp.syncService

And also make sure that your ARConfiguration's isCollaborationEnabled property is set to true.

To make sure RealityKit's synchronizationService runs properly, you must ensure that the RealityKit version installed on any two devices are compatible.

By default, any OS using MultipeerHelper that can install RealityKit (iOS, iPadOS and macOS) will have a key added to the discoveryInfo. To use this easily, you can add the shouldSendJoinRequest method to your MultipeerHelperDelegate, and make use of the checkPeerToken which is accessible to any class which inherits the MultipeerHelperDelegate. Here's an example:

extension RealityViewController: MultipeerHelperDelegate {
  func shouldSendJoinRequest(
    _ peer: MCPeerID,
    with discoveryInfo: [String: String]?
  ) -> Bool {
    self.checkPeerToken(with: discoveryInfo)
  }
}

This method is used in the Example Project.

Initializer Parameters

serviceName

This is the type of service to advertise or search for. Due to how MultipeerConnectivity uses it, it should have the following restrictions:

Must be 1–15 characters long
Can contain only ASCII lowercase letters, numbers, and hyphens
Must contain at least one ASCII letter
Must not begin or end with a hyphen
Must not contain hyphens adjacent to other hyphens.

sessionType (default: `.both`)

This lets the service know if it should be acting as a service host (advertiser), peer (browser), or in a scenario where it doesn't matter, both. The default for this parameter is both, which is the scenario where all devices want to just connect to each other with no questions asked.

peerName (default: `UIDevice.current.name`)

String name of your device on the network.

encryptionPreference (default: `.required`)

encryptionPreference is how data sent over the network are encrypted.

delegate (default: `nil`)

This delegate object will inherit the MultipeerHelperDelegate protocol, which can be used for all the handling of transferring data round the network and seeing when others join and leave.

multipeerhelper's People

Contributors

Stargazers

Watchers

Forkers

cyhunter chen-zhu peastar yunusemrebakir productinfo alexbuce xueselangman91

multipeerhelper's Issues

Fatal error while trying to create sync service

Hello - Thank you for your hard work on this project!

I've setup my ARViewController based on the provided example project. I have all of the correct entries in my info.plist. When running the app, the build succeeds but immediately fails in the func setupMultipeer() while trying to set RealityKit syncronization during:

guard let syncService = multipeerHelp.syncService else { fatalError("could not create multipeerHelp.syncService") }

Could you provide any insight as to why this may be happening?

Does this work with iOS 12

Hi,

Does this library work with iOS 12?

Socket Connection Error

Hi, first off, thanks for making this package as the abstraction substantially decreases the complexity of implementing collaboration in AR.

I'm running into this error on my iPad:

Metal API Validation Enabled
BrowserDelegate <MCPeerID: 0x281b707f0 DisplayName = Michael’s iPhone>
2020-02-22 17:15:55.464060-0500 virtual-estate[2555:367808] Compiler failed to build request
2020-02-22 17:15:55.464205-0500 virtual-estate[2555:367808] [Graphics] makeRenderPipelineState failed [output of type ushort is not compatible with a MTLPixelFormatR16Float color attachement.].
2020-02-22 17:15:55.464234-0500 virtual-estate[2555:367808] [Graphics] makeRenderPipelineState failed.
2020-02-22 17:15:56.626214-0500 virtual-estate[2555:368242] [] nw_resolver_start_query_timer_block_invoke [C2.3] Query fired: did not receive both families in time for 19003cc7-c970-4045-9c49-cf2f96965b4e.local.:50357@awdl0
2020-02-22 17:15:59.469842-0500 virtual-estate[2555:368133] [] nw_socket_handle_socket_event [C1:1] Socket SO_ERROR [54: Connection reset by peer]
2020-02-22 17:15:59.474126-0500 virtual-estate[2555:368133] [MCNearbyServiceAdvertiser] Data from peer [Michael’s iPhone,15600260] received with error Connection closed.
2020-02-22 17:16:00.699231-0500 virtual-estate[2555:367808] [Technique] Error getting pose for participant anchor: CMMapNotAvailable

I'm running the latest versions on my iPhone and iPad (13.3.1). They're both running on the same Wifi network. I also tried running my iPhone as a hotspot and my iPad connecting to it. Below is the code I used:

class ViewController: UIViewController {
    

    @IBOutlet var arView: ARView!
    let multipeerHelp = MultipeerHelper(
      serviceName: "climate-ar",
      sessionType: .both
    )

    override func viewWillAppear(_ animated: Bool) {
        super.viewWillAppear(animated)
    
        let configuration = ARWorldTrackingConfiguration()
    
        configuration.isCollaborationEnabled = true
        
        configuration.environmentTexturing = .automatic
    
        self.arView.scene.synchronizationService = multipeerHelp.syncService
        
        self.arView.session.run(configuration)
        
    }

   override func viewDidLoad() {
        super.viewDidLoad()

   let artBookAnchor = try! Experience.loadArtBook()
  arView.scene.anchors.append(artBookAnchor)
   }
}

Example project not working

Hi!
Thanks great some repository, and articles always.

I tried this project's example with two real devices.
But not working.
Of course, Can you work it yourselves?
Specifically, Entity can not synchronization.

This is my situation.

- Two devices is connected each other. (Turned on Bluetooth and Wi-fi both of them)
- I tap screen on Device1, and put red box on surface, But in Device2 not appeared the box.

Device info
- Device1: iOS 13.3.1, iPhone Xs Max
- Device2: iOS14.0 beta, iPhoneX

This is log of Device2 (Receiver of red box)

2020-06-29 22:52:13.521758+0900 MultipeerHelper+Example[67559:1883010] Metal GPU Frame Capture Enabled
2020-06-29 22:52:13.521980+0900 MultipeerHelper+Example[67559:1883010] Metal API Validation Enabled
2020-06-29 22:52:14.414965+0900 MultipeerHelper+Example[67559:1883010] Compiler failed to build request
2020-06-29 22:52:14.415091+0900 MultipeerHelper+Example[67559:1883010] [Graphics] makeRenderPipelineState failed [output of type ushort is not compatible with a MTLPixelFormatR16Float color attachement.].
2020-06-29 22:52:14.415117+0900 MultipeerHelper+Example[67559:1883010] [Graphics] makeRenderPipelineState failed.
2020-06-29 22:52:14.555203+0900 MultipeerHelper+Example[67559:1883365] [Technique] World tracking performance is being affected by resource constraints [3]
2020-06-29 22:52:14.555255+0900 MultipeerHelper+Example[67559:1883365] [Technique] VIO error callback: 222737.830150, 3, Frame time stamps are either out of order or repeating
BrowserDelegate <MCPeerID: 0x280db4350 DisplayName = Kazumasa>
2020-06-29 22:52:14.869577+0900 MultipeerHelper+Example[67559:1883365] [Technique] VIO error callback: 222738.129133, 3, Frame time stamps are either out of order or repeating
2020-06-29 22:52:15.007852+0900 MultipeerHelper+Example[67559:1883365] libMobileGestalt MobileGestalt.c:5777: Taking legacy CameraOffset_2D path
2020-06-29 22:52:15.008189+0900 MultipeerHelper+Example[67559:1883365] libMobileGestalt MobileGestalt.c:5777: Taking legacy CameraOffset_2D path
2020-06-29 22:52:15.734233+0900 MultipeerHelper+Example[67559:1883263] [] nw_resolver_start_query_timer_block_invoke [C1.3] Query fired: did not receive both families in time for 35780577-a78c-49fa-a6f0-d94f50ecb914.local.:64971@awdl0
new peer has joined: Kazumasa
㶒峍ꏝ犢｣潭⹡灰汥⹲敫楴⹨敬汯
2020-06-29 22:52:16.245227+0900 MultipeerHelper+Example[67559:1883010] [Network] Mismatched protocol, received 0x6 expecting 0x5
2020-06-29 22:52:16.245310+0900 MultipeerHelper+Example[67559:1883010] [Network] Received corrupt message from 'YnBsaXN0MDDUAQIDBAUGBwpYJHZlcnNpb25ZJGFyY2hpdmVyVCR0b3BYJG9iamVjdHMSAAGGoF8QD05TS2V5ZWRBcmNoaXZlctEICVRyb290gAGkCwwTFFUkbnVsbNMNDg8QERJUbmFtZVJpZFYkY2xhc3OAAhOQzGr4j2RaRYADWEthenVtYXNh0hUWFxhaJGNsYXNzbmFtZVgkY2xhc3Nlc1hNQ1BlZXJJRKIXGVhOU09iamVjdAgRGiQpMjdJTFFTWF5lam10dn+Bio+ao6yvAAAAAAAAAQEAAAAAAAAAGgAAAAAAAAAAAAAAAAAAALg='. Disconnecting.
2020-06-29 22:52:16.245374+0900 MultipeerHelper+Example[67559:1883010] [Network] Attempt to receive data from connection 'YnBsaXN0MDDUAQIDBAUGBwpYJHZlcnNpb25ZJGFyY2hpdmVyVCR0b3BYJG9iamVjdHMSAAGGoF8QD05TS2V5ZWRBcmNoaXZlctEICVRyb290gAGkCwwTFFUkbnVsbNMNDg8QERJUbmFtZVJpZFYkY2xhc3OAAhOQzGr4j2RaRYADWEthenVtYXNh0hUWFxhaJGNsYXNzbmFtZVgkY2xhc3Nlc1hNQ1BlZXJJRKIXGVhOU09iamVjdAgRGiQpMjdJTFFTWF5lam10dn+Bio+ao6yvAAAAAAAAAQEAAAAAAAAAGgAAAAAAAAAAAAAAAAAAALg=' that is disconnected
2020-06-29 22:52:25.803059+0900 MultipeerHelper+Example[67559:1883304] [GCKSession] Not in connected state, so giving up for participant [0F645A45] on channel [0].
2020-06-29 22:52:25.812846+0900 MultipeerHelper+Example[67559:1883304] [GCKSession] Not in connected state, so giving up for participant [0F645A45] on channel [1].
2020-06-29 22:52:25.822362+0900 MultipeerHelper+Example[67559:1883304] [GCKSession] Not in connected state, so giving up for participant [0F645A45] on channel [2].
2020-06-29 22:52:25.833197+0900 MultipeerHelper+Example[67559:1883304] [GCKSession] Not in connected state, so giving up for participant [0F645A45] on channel [5].
2020-06-29 22:52:25.840927+0900 MultipeerHelper+Example[67559:1883304] [GCKSession] Not in connected state, so giving up for participant [0F645A45] on channel [6].
hello! from Kazumasa
hello! from Kazumasa
hello! from Kazumasa
2020-06-29 22:52:45.765924+0900 MultipeerHelper+Example[67559:1883285] [Graphics] Failed to find reflection for buffer clusterIndexTable

Can you solve this problem?

Cannot use Entity Gestures with Collaborative Session

Hi,
I know this issue may be a little bit out of the scope of this repo, but want to ask you for help after reading your amazing posts about RealityKit

In the official example (https://developer.apple.com/documentation/arkit/creating_a_collaborative_session), the users can only add a cube to the scene without interacting with it.
I try to add gesture supports to the cubes, so that the multiple users can move or rotate the cubes (whose movements/rotations will then be synced to all paired users).

But I found that if I do not add SynchronizationService, the movements of a cube in one user's scene will not be synced to others.
If I add SynchronizationService, the cube won't be able to move (with error msg CollaborativeSession [Collision] Bad paramater (SphereRadius), value = 0.000000, passed to shape creation).

This is the repo of my code: https://github.com/chenzhutian/collaboration-interaction
Could you please help? Do you have any idea of fixing this issue?

Collaborative session with physics

Hi,
I really wanted to thank you for the effort you put into this, this is really helpful.

What your example (and basically every other RealityKit example but SwiftStrike) did not include is physics.

I set up the collaborative session with your MultipeerHelper and did get the connectivity to work. Both devices can see the blocks placed by the other device. Everything works fine until I add physics. With physics, everything starts to behave strangely. Blocks are jumping around and I will not have the same behavior on both systems.

I then tried to separate host and client, so that only the host places ModelEntities. So I now have a TapGestureRecognizer, that places ARAnchors

@objc func handleTap(recognizer: UITapGestureRecognizer){
    let cameraPosition = arView.cameraTransform.matrix
    let translate = float4x4(
        [1,0,0,0],
        [0,1,0,0],
        [0,0,1,0],
        [0,0,-0.4,1]
    )
    let blockPosition = cameraPosition * translate
    
    let arAnchor = ARAnchor(name: "block", transform: blockPosition)
    arView.session.add(anchor: arAnchor)
}

And I implemented the delegate method to let the host add a ModelEntity, once an ARAnchor was added:

func session(_ session: ARSession, didAdd anchors: [ARAnchor]) {
    for anchor in anchors {
        if anchor.name == "block" {
            
            // Check for host or not
            // If host: add block to the scene
            if isHost {
                let anchorEntity = AnchorEntity(anchor: anchor)
                
                let model = ModelEntity(
                    mesh: .generateBox(size: [0.3, 0.05, 0.2]),
                    materials: [SimpleMaterial(color: .cyan, isMetallic: false)]
                )
                model.generateCollisionShapes(recursive: true)
                model.physicsBody = PhysicsBodyComponent(
                    shapes: model.collision!.shapes,
                    mass: 1,
                    material: .generate(friction: 10, restitution: 0.1),
                    mode: .dynamic
                )
                
                anchorEntity.addChild(model)
                
                anchorEntity.synchronization?.ownershipTransferMode = .autoAccept
                arView.scene.addAnchor(anchorEntity)
            }
        }
    }
}

On the host device, everything works fine. But on the physics behave completely different. Blocks are jumping around. I can see, that when I add a new block it seems to try to synchronize both worlds again, and for a short moment on the client system the blocks are at the same place as on the host system, but then the blocks jump out of each other again.

Have you ever faced similar problems? Maybe I am missing something or doing something wrong.

Thanks in advance!

Problem using RealityComposer

Hi, thank you very much for your work and this help me a lot! Please help me with my code...
I use your code to Connect a rcProject file. I modify the code
else if anchor.name == "Anchor for object placement" { print("anchor.name:" ,anchor.name ) self.scene.anchors.append(boxAnchor) }
In my rcProject file ,I have four Notification activities which controlled by a UIButton, each time I click the button, it will change to the next activity.
xxActivity.post()
Now I can successfully connect this file with two iPhone. But the problem is: when I click button and send this step index to another iPhone, the other iPhone called:
`func receivedData(_ data: Data, _ peer: MCPeerID) {

    guard let myData = try? NSKeyedUnarchiver.unarchivedObject(ofClasses: [NSDictionary.classForKeyedUnarchiver()], from: data) as? Dictionary<String, String> else{
        return
    }
    print("------------receivedStepIndex:", myData)

    if(myData["type"] == "step"){
        xxActivity.post()
    }
}`

But this code cause a crash in the rcProject file's code inside. The same error happens in the Offical sample code too.("CreatingACollaborativeSession")

Can you help? Thank you!

Session doesn't connect to peers

Hi,
I have this running on 2 physical devices but when I try to connect them, I simply get this in the Xcode cmd output:
Not in connected state, so giving up for participant [48c78cc] on channel [0]. Not in connected state, so giving up for participant [48c78cc] on channel [1]. Not in connected state, so giving up for participant [48c78cc] on channel [2].

The peers never connect to each other. Is there anything I need to consider aside from initializing the session? I am currently doing it like in the sample app (which works fine on both devices)

import Foundation
import SwiftUI
import Foundation
import MultipeerConnectivity


struct tutorialView: View {
    @State var delegate = multidelegate()
    var body: some View {
        Text("Peers connected: \(delegate.multipeerHelp?.connectedPeers.count ?? 999)").onAppear(){
            delegate.setupMultipeer()
        }
    }
}

class multidelegate: MultipeerHelperDelegate{
    var multipeerHelp: MultipeerHelper!
    func setupMultipeer() {
        multipeerHelp = MultipeerHelper(
            serviceName: "demo",
            sessionType: .both,
            delegate: self
        )
    }
    func receivedData(_ data: Data, _ peer: MCPeerID) {
      print(String(data: data, encoding: .unicode) ?? "Data is not a unicode string")
    }

    func peerJoined(_ peer: MCPeerID) {
      print("new peer has joined: \(peer.displayName)")
    }
}