Make sure Expo[https://expo.dev] and CocoaPods[https://cocoapods.org] are installed.

1. Clone the project.
2. Open the project and run expo install
3. Open the iOS directory in the project and run pod install
4. Open RoboflowExpoExample.xcworkspace in Xcode
5. Click on the project name on the left, then Signing and Capabilites, then select your team.
6. Open RoboflowFrameProcessorPlugin.swift in XCode and add your apiKey, model name, and model version from Roboflow[https://roboflow.com]
7. Run the app on a device using XCode

Start by setting up an expo project if you haven't already using the expo init <ProjectName> command.

Roboflow relies on native code which means you have to eject your expo project out of the managed workflow into the bare workflow. Note, this means you either have to use the EAS build system or compile the iOS app yourself in xcode. Eject using this command expo prebuild.

This will build the naive iOS and Android projects in the ios and android folders now present in your project. Roboflow only supports iOS for the time being so we will focus there first.

Roboflow is distributed using cocoapods, a package manager for native iOS apps. In order to install the app first open the Podfile in the iOS directory.

Under the line that says config = use_native_modules! add the Roboflow pod using the line pod 'Roboflow'.

Roboflow requires a minimum iOS version of 15.4. Edit the line at the top of the Podfile that says platform :ios, '12.0' and change it to platform :ios, '15.4'

Next, in a terminal window, run pod install to install the Roboflow package into your iOS project.

Now that you have Roboflow installed you can use it in any native file, the remainder of the example will focus on using Roboflow to process a camera stream using the react-native-vision-camera package.

Make sure you are in the root of your expo project (not the ios folder) and run expo install react-native-vision-camera react-native-reanimated to install react-native-vision-camera and its dependency react-native-reanimated.

These npm dependencies also come bunled with native dependencies which we must also install. Run pod install in the ios directory.

react-native-vision-camera lets you use custom frame processors to process frames in the videos tream with high performance. Roboflow should be used in a swift frame processor like the one in this example called RoboflowFrameProcessorPlugin.swift.

You should first open the .xcworkspace file in the ios folder in XCode for easier development on this part.

In XCode, open Info.plist. At the bottom click the little plus symbol while hovering over View controller-based status bar appearance. In the new row that appears, type Privacy - Camera Usage Description into the key column. Then type a custom message in the value column that will show when someone opens your app for the first time.

Either create a swift file called RoboflowFrameProcessorPlugin.swift or copy the one provided in this example. If XCode prompts you to create a bridging header file click create. Inside this frame processor we will implement roboflow for processing images in the incoming frames. First, we need to connect our swift file to the objective-c vision-camera library.

In the Bridging-Header.h file add these two lines to the top.

#import <VisionCamera/FrameProcessorPlugin.h>
#import <VisionCamera/Frame.h>

Next create a file called RoboflowFrameProcessorPlugin.m with the following lines to tell expo where our function is.

#import <Foundation/Foundation.h>
#import <VisionCamera/FrameProcessorPlugin.h>

@interface VISION_EXPORT_SWIFT_FRAME_PROCESSOR(roboflowDetect, RoboflowFrameProcessorPlugin)
@end

In RoboflowFrameProessorPlugin, we first define an instance of RoboflowWrapper with our apiKey. Insert your apiKey, found at roboflow.com. The loadRoboflowModelWith function loads your model from roboflow using the model name and version, set those values to your custom ones from roboflow.com.

Finally, custmize the confidence threshold, and overlap threshold to your desired values.

In a new JS file, create a function called roboflowDetect which connects to your native function like so.

import "react-native-reanimated";

export const roboflowDetect = (frame) => {
  "worklet";
  return [__roboflowDetect(frame), frame.width, frame.height];
};

Open the babel.config.js file to expose the native function to your react native app using react-native-reanimated. Make sure your file includes the plugins section from below.

module.exports = function (api) {
  api.cache(true);
  return {
    presets: ["babel-preset-expo"],
    plugins: [
      [
        "react-native-reanimated/plugin",
        {
          globals: ["__roboflowDetect"],
        },
      ],
    ],
  };
};

In App.js add these imports.

import {
  useCameraDevices,
  Camera,
  useFrameProcessor,
} from "react-native-vision-camera";
import { roboflowDetect } from "./roboflowDetect";
import {
  runOnJS,
  useSharedValue,
} from "react-native-reanimated";

Then add these lines to get the camera device.

const devices = useCameraDevices("wide-angle-camera");
const device = devices.back;
if (device == null) return <Text>Loading...</Text>;

Next Add the the following lines to your return to add the camera.

<Camera
    style={StyleSheet.absoluteFill}
    device={device}
    isActive={true}
/>

To connect the camera to our frameprocessor lets first build a worklet function to access the native frame processor.

const frameProcessor = useFrameProcessor((frame) => {
    "worklet";
    const [detections, width, height] = roboflowDetect(frame);
    console.log(`Detections in Frame: ${detections.length}`);
}, []);

Then set the camera to use this worklet by adding frameProcessor={frameProcessor} to the <Camera /> tag.