FaceNet PyTorch to CoreML Converter

This script converts the pre-trained FaceNet PyTorch model from timesler/facenet-pytorch to CoreML format for use in iOS applications.

Model Information

• Source: FaceNet PyTorch implementation (VGGFace2 trained)
• Repository: https://github.com/timesler/facenet-pytorch
• Model: InceptionResnetV1 trained on VGGFace2 dataset
• Input: 160×160×3 RGB image
• Output: 512-dimensional face embedding vector
• Accuracy: 99.65% on LFW benchmark
• Use Case: Face recognition and verification

Prerequisites

1. Install Python 3.11

# Install Python 3.11 via Homebrew
brew install python@3.11

# Verify installation
python3.11 --version
# Should output: Python 3.11.x

2. Create Virtual Environment

# Create virtual environment
python3.11 -m venv facenet-env

# Activate environment
source facenet-env/bin/activate

# Install dependencies
pip install torch torchvision facenet-pytorch coremltools

3. Download FaceNet PyTorch Model

The model can be downloaded from the facenet-pytorch repository:

• Visit: https://github.com/timesler/facenet-pytorch
• Download: 20180402-114759-vggface2.pt

Usage

Convert PyTorch Model to CoreML

# Activate virtual environment
source facenet-env/bin/activate

# Run conversion script
python convert_facenet_pytorch_to_coreml.py /path/to/20180402-114759-vggface2.pt

# Or specify output path
python convert_facenet_pytorch_to_coreml.py /path/to/model.pt FaceNet.mlpackage

Expected Output

✅ SUCCESS! Model saved to: FaceNet.mlpackage

Model details:
  Input: 160×160×3 RGB image (normalized 0-1)
  Output: 512-dimensional embedding vector
  Size: ~90 MB

Using the Model in Xcode

1. Add Model to Xcode Project

Method A: Drag and Drop

1. Open your Xcode project
2. In Project Navigator, select where to add the model (e.g., Resources/MLModels/)
3. Drag FaceNet.mlpackage from Finder into Xcode
4. In the dialog:
   • ✅ Check "Copy items if needed"
   • ✅ Select your app target
   • Click "Finish"

Method B: Add Files

1. Right-click in Project Navigator
2. Choose "Add Files to [ProjectName]..."
3. Select FaceNet.mlpackage
4. Ensure target is selected
5. Click "Add"

2. Verify Model in Xcode

1. Click on FaceNet.mlpackage in Project Navigator
2. Xcode will show the model details:
   • Inputs: input - Image (Color 160×160)
   • Outputs: var_2167 or similar - MultiArray (Float32 512)
3. Xcode automatically generates Swift wrapper class

3. Use Model in Swift Code

import CoreML
import Vision

// Load the model
guard let model = try? VNCoreMLModel(for: FaceNet(configuration: MLModelConfiguration()).model) else {
    print("Failed to load FaceNet model")
    return
}

// Create Vision request
let request = VNCoreMLRequest(model: model) { request, error in
    guard let results = request.results as? [VNCoreMLFeatureValueObservation],
          let embedding = results.first?.featureValue.multiArrayValue else {
        return
    }

    // embedding is a 512-dimensional vector (MLMultiArray)
    print("Generated embedding with \(embedding.count) dimensions")
}

// Process image
let handler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer, options: [:])
try? handler.perform([request])

Model Performance

• Embedding generation: ~50-100ms per face on iPhone
• Memory: ~90MB model size
• Accuracy: 99.65% on LFW benchmark
• Vector size: 512 floats = 2KB per face embedding

Face Matching

Once you have embeddings, compare faces using cosine similarity:

func cosineSimilarity(_ a: [Float], _ b: [Float]) -> Float {
    let dotProduct = zip(a, b).map(*).reduce(0, +)
    let normA = sqrt(a.map { $0 * $0 }.reduce(0, +))
    let normB = sqrt(b.map { $0 * $0 }.reduce(0, +))
    return dotProduct / (normA * normB)
}

// Similarity > 0.6-0.7 typically indicates same person
let similarity = cosineSimilarity(embedding1, embedding2)
if similarity > 0.75 {
    print("Same person!")
}

Resources

• FaceNet PyTorch: https://github.com/timesler/facenet-pytorch
• CoreML Tools Documentation: https://coremltools.readme.io/
• Apple ML Documentation: https://developer.apple.com/documentation/coreml

License

• This script: MIT License
• FaceNet model: Check timesler/facenet-pytorch repository for licensing
• CoreML Tools: Apache 2.0

How the Script Works

The conversion script performs the following steps:

Step 1: Load PyTorch Model

model = torch.load(pt_file_path, map_location='cpu')

• Loads the .pt file which contains model weights (state dict)
• Uses map_location='cpu' to avoid GPU requirements

Step 2: Reconstruct Model Architecture

from facenet_pytorch import InceptionResnetV1
facenet_model = InceptionResnetV1(pretrained=None, classify=False)
facenet_model.load_state_dict(model, strict=False)

• The .pt file only contains weights, not architecture
• Uses facenet-pytorch library to create the InceptionResnetV1 architecture
• classify=False creates embedding-only mode (no classification head)
• strict=False ignores logits.weight and logits.bias (classification layer weights we don't need)

Why this is necessary:

• FaceNet was trained with a classification head (predicting person IDs)
• For face recognition, we only need the embedding layer (512D vector)
• The classification layer is dataset-specific (VGGFace2 has 9000 identities)
• Embeddings are universal and can match any face

Step 3: Validate Model

example_input = torch.randn(1, 3, 160, 160)
example_output = model(example_input)
print(f"Output shape: {example_output.shape}")  # Should be [1, 512]

• Tests model with random 160×160×3 RGB image
• Verifies output is 512-dimensional embedding
• Catches architecture loading errors before conversion

Step 4: Trace Model for Export

traced_model = torch.jit.trace(model, example_input)

• PyTorch JIT tracing converts dynamic model to static computation graph
• Required by CoreML Tools for conversion
• Records all operations performed during the example forward pass

Step 5: Convert to CoreML

mlmodel = ct.convert(
    traced_model,
    inputs=[ct.ImageType(
        name='input',
        shape=example_input.shape,
        scale=1/255.0,  # Normalize to 0-1
        bias=[0, 0, 0]
    )],
    minimum_deployment_target=ct.target.iOS15
)

• Converts PyTorch computation graph to CoreML ML Program format
• ct.ImageType tells CoreML this input is an image (enables optimizations)
• scale=1/255.0 automatically normalizes pixel values from 0-255 to 0-1
• minimum_deployment_target=iOS15 uses modern .mlpackage format

Step 6: Add Metadata

mlmodel.short_description = "FaceNet face recognition model"
mlmodel.author = "FaceNet PyTorch (converted to CoreML)"
mlmodel.license = "MIT"
mlmodel.version = "vggface2-pytorch"

• Adds metadata visible in Xcode
• Helps document model source and version

Step 7: Save as ML Package

mlmodel.save('FaceNet.mlpackage')

• Saves as .mlpackage bundle (iOS 15+ format)
• Contains:
  • model.mil - Model code in MIL (Model Intermediate Language)
  • Data/weights/ - Model weights
  • Manifest.json - Package metadata

Key Technical Details

Why classify=False?

The FaceNet model has two modes:

• classify=True: Outputs class probabilities (for training on specific dataset)
• classify=False: Outputs 512D embedding (for face recognition)

For face recognition, embeddings are more useful because:

• They generalize to unseen people
• Can be compared with cosine similarity
• Work for verification (is this the same person?) and identification (who is this?)

Why strict=False?

The downloaded model includes weights for:

• Base network (convolutional layers, residual blocks) ✅ We need these
• Classification head (logits.weight, logits.bias) ❌ We don't need these

Using strict=False allows loading base network weights while ignoring the classifier.

What's an Embedding?

An embedding is a 512-dimensional vector (512 floats) that represents a face's unique characteristics:

• Captures facial geometry, features, and identity
• Two photos of the same person produce similar vectors
• Photos of different people produce different vectors
• Compared using cosine similarity: similarity = dot(v1, v2) / (norm(v1) * norm(v2))

Notes

• Output is normalized L2 embedding vector (ready for cosine similarity)
• Model expects faces to be pre-cropped and aligned to 160×160
• Input images should be RGB (not BGR or grayscale)
• Face detection must be performed separately (use Vision Framework)