FieldDay iOS SDK

Installation

You can add the FieldDay iOS SDK using Swift Package Manager.

• From the File menu, select Add Packages...
• Enter the repo's URL

http://github.com/fieldday-ai/fieldday-ios-sdk.git

Prepare Xcode Project

• Add the FieldDay Package
• Provide Camera Usage Description
  • Project > Build Settings
  • Scroll to Info.plist Values
  • Find "Privacy - Camera Usage Description"
  • Provide a suitable description of why you wish to access the Camera
• [Optional] Lock the supported screen orientations
  • Target > General > Deployment Info
  • iPhone Orientation
  • Check only Portrait

Setup FieldDay Project

There are two methods to use your FieldDay project with the iOS SDK. To get started, open your project in the FieldDay app. Make sure you have trained at least one model in the project.

Method 1: Publish your project

FieldDay allows you to publish your project so it can be fetched over the internet. Using this method, you can ensure your project is always kept up-to-date with any changes you make. The package caches project information, so make sure to set the appropriate cache policy as per your requirements (see Caching).

• Tap the share button at the top right corner of the screen.
• Select the Swift tile.
• Scroll down to the section where it says "Share Code".
• Copy the alphanumeric share code that appears by tapping on the box.

Usage

import SwiftUI
import FieldDay

struct ContentView: View {
    var body: some View {
        FDViewfinderView(
            /// Enter the share code that we copied from FieldDay
            shareCode: "________________________________"
        )
    }
}

Method 2: Use a CoreML .mlmodel file

Alternatively, you can also package a model file with your app. This has the advantage of always working offline. But you will need to update it manually to include changes from your FieldDay project.

• Tap the share button at the top right corner of the screen.
• Select the CoreML option.
• Tap "Export .mlmodel". This will give you two files – Categories.swift and Project Name.mlmodel.
• Add the two files to the Xcode project.

Usage

import SwiftUI
import FieldDay

struct ContentView: View {
    var body: some View {
        FDViewfinderView(
            modelURL: Project_Name.urlOfModelInThisBundle,
            // A class like "Project_Name" is automatically generated by Xcode for your model.
            // Open the `.mlmodel` file in Xcode to find the class name.
            categories: categories
            // This array is defined in the Categories.swift file.
            // You can edit the category names or colours.
        )
    }
}

Advanced Usage

Handle prediction events

At the moment, the FieldDay SDK supports handling the following events:

• When the model makes a prediction
• When a prediction pill is tapped

A prediction pill is the element at the bottom of the screen, showing the category name for the prediction

These events can be handled via the onPrediction and onPredictionTap modifiers on the ViewfinderView. They can be used as follows.

FDViewfinderView(...)
    .onPredictionTap { category in
        print(category.name)
    }
    .onPrediction { prediction, category in
        print(category.name, prediction.confidence)
    }

FDCategory

struct FDCategory {
    var id: String
    var name: String
    var color: Color
    var isBackground: Bool
}

• id - The unique identifier of the category
• name - The category's name (defined in the FieldDay App)
• color - The category's color (defined in the FieldDay App)
• isBackground - Indicates whether the category is the default "Background" category

FDModelPrediction

struct FDModelPrediction {
    var identifier: String
    var confidence: Float
    var results: [VNClassificationObservation]
}

• identifier - The identifier returned associated with the CoreML prediction
• confidence - The confidence of the prediction, normalized from 0...1
• results - Contains confidence values for all categories in the model

Modify the Viewfinder

Viewfinder Size

Use the viewfinderSize modifier to adjust size. This ensures the underlying camera preview is sized correctly.

FDViewfinderView(...)
    .viewfinderSize(width: 200, height: 200)

Viewfinder Mode

Use the viewfinderMode modifier to select between .photo and .video. This selects the appropriate aspect ratio and zoom levels for the device. The SDK defaults to .video mode.

FDViewfinderView(...)
    .viewfinderMode(.photo)

Disable Model Feedback UI

You can choose to hide the predictions UI with the feedbackUIHidden modifier.

FDViewfinderView(...)
    .feedbackUIHidden()

Prediction Smoothing

The SDK smooths the results of the FieldDay model to reduce jitter in the results. If your model doesn't require this, or the performance doesn't match the FieldDay app, you can disable the smoothing.

FDViewfinderView(...)
    .predictionSmoothingDisabled()

Caching

When using project keys, FieldDay has an option to cache network data to offer limited offline functionality. By passing in a FDCachePolicy in the FDViewfinderView intializer, you can customize how the cache is used. If no policy is passed in, the default is .cacheThenNetwork.

FDViewfinderView(
    ...,
    cachePolicy: .ignoreCache
)

FDCachePolicy

enum FDCachePolicy {
    case cacheThenNetwork
    case ignoreCache
    case ignoreCacheCompletely
}

• cacheThenNetwork - Fetches from the cache, then the network, and only uses the network result if it differs from the cache. This is the default policy.
• ignoreCache - Fetches from the network only, but still writes the result to the cache.
• ignoreCacheCompletely - Fetches from the network only, and does not write the result to the cache.

Manual Cache Clearing

If needed, the clearFDCache() extension on UserDefaults lets you manually clear the FieldDay cache.

UserDefaults.standard.clearFDCache()

Debugging

To view debugging logs that expose error messages for various operations - just add the .debug() modifier to your FDViewfinderView. FieldDay logs will be prefixed with "🤖⚠️".

FDViewfinderView(...)
    .debug()