From acbf8445152dc13783cfb3ea97050fa425f612ff Mon Sep 17 00:00:00 2001 From: Jon Samp Date: Thu, 4 Jun 2026 10:45:01 -0500 Subject: [PATCH] adds skills from agent and release skill --- plugins/expo/skills/apple-targets/SKILL.md | 261 ++++++++ .../expo/skills/apple-targets/account-auth.md | 260 ++++++++ plugins/expo/skills/apple-targets/action.md | 197 ++++++ .../expo/skills/apple-targets/app-intent.md | 175 +++++ .../apple-targets/authentication-services.md | 230 +++++++ .../expo/skills/apple-targets/bg-download.md | 168 +++++ .../apple-targets/broadcast-setup-ui.md | 187 ++++++ .../skills/apple-targets/broadcast-upload.md | 234 +++++++ .../skills/apple-targets/call-directory.md | 167 +++++ .../skills/apple-targets/classkit-context.md | 192 ++++++ plugins/expo/skills/apple-targets/clip.md | 299 +++++++++ .../skills/apple-targets/content-blocker.md | 227 +++++++ .../apple-targets/credentials-provider.md | 217 +++++++ .../apple-targets/device-activity-monitor.md | 156 +++++ .../apple-targets/entitlements/README.md | 50 ++ .../apple-targets/entitlements/app-groups.md | 601 ++++++++++++++++++ .../skills/apple-targets/file-provider-ui.md | 196 ++++++ .../skills/apple-targets/file-provider.md | 365 +++++++++++ .../expo/skills/apple-targets/intent-ui.md | 180 ++++++ plugins/expo/skills/apple-targets/intent.md | 250 ++++++++ plugins/expo/skills/apple-targets/keyboard.md | 213 +++++++ .../skills/apple-targets/location-push.md | 177 ++++++ plugins/expo/skills/apple-targets/matter.md | 160 +++++ .../skills/apple-targets/message-filter.md | 183 ++++++ .../skills/apple-targets/network-app-proxy.md | 258 ++++++++ .../skills/apple-targets/network-dns-proxy.md | 252 ++++++++ .../apple-targets/network-filter-data.md | 234 +++++++ .../apple-targets/network-packet-tunnel.md | 223 +++++++ .../apple-targets/notification-content.md | 238 +++++++ .../apple-targets/notification-service.md | 272 ++++++++ .../skills/apple-targets/photo-editing.md | 285 +++++++++ .../skills/apple-targets/print-service.md | 142 +++++ .../skills/apple-targets/quicklook-preview.md | 261 ++++++++ .../apple-targets/quicklook-thumbnail.md | 242 +++++++ plugins/expo/skills/apple-targets/safari.md | 217 +++++++ plugins/expo/skills/apple-targets/share.md | 165 +++++ .../skills/apple-targets/shield-action.md | 157 +++++ .../skills/apple-targets/shield-config.md | 190 ++++++ .../expo/skills/apple-targets/smart-card.md | 263 ++++++++ .../apple-targets/spotlight-delegate.md | 163 +++++ .../expo/skills/apple-targets/spotlight.md | 117 ++++ .../apple-targets/unwanted-communication.md | 278 ++++++++ .../apple-targets/virtual-conference.md | 144 +++++ plugins/expo/skills/apple-targets/watch.md | 493 ++++++++++++++ plugins/expo/skills/apple-targets/widget.md | 536 ++++++++++++++++ plugins/expo/skills/release-app/SKILL.md | 101 +++ .../references/app-store-metadata.md | 94 +++ .../release-app/references/eas-build.md | 70 ++ .../release-app/references/eas-submit.md | 162 +++++ .../release-app/references/marketing.md | 68 ++ .../release-app/references/monetization.md | 52 ++ .../release-app/references/privacy-policy.md | 124 ++++ .../release-app/references/rejection-scan.md | 83 +++ 53 files changed, 11229 insertions(+) create mode 100644 plugins/expo/skills/apple-targets/SKILL.md create mode 100644 plugins/expo/skills/apple-targets/account-auth.md create mode 100644 plugins/expo/skills/apple-targets/action.md create mode 100644 plugins/expo/skills/apple-targets/app-intent.md create mode 100644 plugins/expo/skills/apple-targets/authentication-services.md create mode 100644 plugins/expo/skills/apple-targets/bg-download.md create mode 100644 plugins/expo/skills/apple-targets/broadcast-setup-ui.md create mode 100644 plugins/expo/skills/apple-targets/broadcast-upload.md create mode 100644 plugins/expo/skills/apple-targets/call-directory.md create mode 100644 plugins/expo/skills/apple-targets/classkit-context.md create mode 100644 plugins/expo/skills/apple-targets/clip.md create mode 100644 plugins/expo/skills/apple-targets/content-blocker.md create mode 100644 plugins/expo/skills/apple-targets/credentials-provider.md create mode 100644 plugins/expo/skills/apple-targets/device-activity-monitor.md create mode 100644 plugins/expo/skills/apple-targets/entitlements/README.md create mode 100644 plugins/expo/skills/apple-targets/entitlements/app-groups.md create mode 100644 plugins/expo/skills/apple-targets/file-provider-ui.md create mode 100644 plugins/expo/skills/apple-targets/file-provider.md create mode 100644 plugins/expo/skills/apple-targets/intent-ui.md create mode 100644 plugins/expo/skills/apple-targets/intent.md create mode 100644 plugins/expo/skills/apple-targets/keyboard.md create mode 100644 plugins/expo/skills/apple-targets/location-push.md create mode 100644 plugins/expo/skills/apple-targets/matter.md create mode 100644 plugins/expo/skills/apple-targets/message-filter.md create mode 100644 plugins/expo/skills/apple-targets/network-app-proxy.md create mode 100644 plugins/expo/skills/apple-targets/network-dns-proxy.md create mode 100644 plugins/expo/skills/apple-targets/network-filter-data.md create mode 100644 plugins/expo/skills/apple-targets/network-packet-tunnel.md create mode 100644 plugins/expo/skills/apple-targets/notification-content.md create mode 100644 plugins/expo/skills/apple-targets/notification-service.md create mode 100644 plugins/expo/skills/apple-targets/photo-editing.md create mode 100644 plugins/expo/skills/apple-targets/print-service.md create mode 100644 plugins/expo/skills/apple-targets/quicklook-preview.md create mode 100644 plugins/expo/skills/apple-targets/quicklook-thumbnail.md create mode 100644 plugins/expo/skills/apple-targets/safari.md create mode 100644 plugins/expo/skills/apple-targets/share.md create mode 100644 plugins/expo/skills/apple-targets/shield-action.md create mode 100644 plugins/expo/skills/apple-targets/shield-config.md create mode 100644 plugins/expo/skills/apple-targets/smart-card.md create mode 100644 plugins/expo/skills/apple-targets/spotlight-delegate.md create mode 100644 plugins/expo/skills/apple-targets/spotlight.md create mode 100644 plugins/expo/skills/apple-targets/unwanted-communication.md create mode 100644 plugins/expo/skills/apple-targets/virtual-conference.md create mode 100644 plugins/expo/skills/apple-targets/watch.md create mode 100644 plugins/expo/skills/apple-targets/widget.md create mode 100644 plugins/expo/skills/release-app/SKILL.md create mode 100644 plugins/expo/skills/release-app/references/app-store-metadata.md create mode 100644 plugins/expo/skills/release-app/references/eas-build.md create mode 100644 plugins/expo/skills/release-app/references/eas-submit.md create mode 100644 plugins/expo/skills/release-app/references/marketing.md create mode 100644 plugins/expo/skills/release-app/references/monetization.md create mode 100644 plugins/expo/skills/release-app/references/privacy-policy.md create mode 100644 plugins/expo/skills/release-app/references/rejection-scan.md diff --git a/plugins/expo/skills/apple-targets/SKILL.md b/plugins/expo/skills/apple-targets/SKILL.md new file mode 100644 index 0000000..175eb42 --- /dev/null +++ b/plugins/expo/skills/apple-targets/SKILL.md @@ -0,0 +1,261 @@ +--- +name: apple-targets +description: Use for anything related to multi-target Apple/iOS apps including widgets (home screen, Lock Screen, Live Activities), App Clips, share extensions, Safari extensions, watch apps, notification extensions, keyboard extensions, or any Apple extension type. Trigger phrases include "widget", "share extension", "App Clip", "watchOS app", "multi-target", "apple-targets", "bunx create-target", "bun create target", "iOS extension". +version: 1.0.0 +license: MIT +--- + +# Apple Extension Target Skills Reference + +This directory contains comprehensive skill documents for all supported Apple extension and target types. Each skill document provides everything an AI agent needs to understand, implement, and troubleshoot a specific Apple extension type, including: + +- Apple documentation links and WWDC session references +- Runtime behavior and lifecycle explanation +- Real-world use cases and implementation patterns +- Production-ready Swift code examples +- Platform availability and version requirements +- Common gotchas and debugging tips + +> **Tip:** For a focused, end-to-end iOS **App Clip** setup, a dedicated `add-app-clip` skill is also available. Use this `apple-targets` skill for the full catalog of Apple extension and target types. + +## How to Use These Skills + +**For AI Agents:** + +1. **When a user asks about a specific extension type** (e.g., "How do I build a widget?" or "I need to add a share extension"), read the corresponding skill document to get complete implementation guidance. + +2. **When planning a feature** that might require an extension (e.g., "Add home screen presence" -> read `widget.md`), consult the skill document to understand requirements and constraints. + +3. **When debugging extension issues**, refer to the "Gotchas" section of the relevant skill document -- these capture common pitfalls from Apple Developer Forums and real-world experience. + +4. **When uncertain which extension type fits a use case**, scan the "Use Cases" sections across related skills to find the best match. + +**Structure of Each Skill Document:** + +Every skill document follows this template: +- **Frontmatter**: Title, one-line description, minimum OS version +- **Apple Documentation**: Official reference links +- **WWDC History**: Session links showing evolution of the feature +- **What It Does**: Step-by-step runtime behavior +- **Use Cases**: Real-world scenarios organized by industry/app type +- **Key Classes**: Table of essential framework types +- **Implementation**: Production-ready Swift code with inline comments +- **Using with @bacons/apple-targets**: Setup instructions for this plugin +- **Platform Availability**: OS version matrix +- **Gotchas**: Common mistakes, memory limits, and silent failure modes + +## Skills by Category + +### User Interface Extensions + +| Skill | Extension Type | When to Read | +|-------|---------------|--------------| +| [widget.md](widget.md) | `widget` | User wants home screen widgets, Lock Screen widgets, Live Activities, StandBy mode UI, or Control Center controls (iOS 18+). Timeline-based SwiftUI views. | +| [clip.md](clip.md) | `clip` | User wants instant-launch mini-apps triggered by NFC, QR codes, App Clip Codes, Safari banners, or Messages links. Not an extension--a size-constrained application target. | +| [share.md](share.md) | `share` | User wants their app in the system share sheet to receive URLs, images, text, or files from other apps. | +| [action.md](action.md) | `action` | User wants context menu actions in Safari, Photos, or Files that transform content and optionally return modified data. | +| [keyboard.md](keyboard.md) | `keyboard` | User wants a custom system keyboard with specialized input (emoji pickers, GIF search, language-specific layouts, calculators). | +| [photo-editing.md](photo-editing.md) | `photo-editing` | User wants to add filters, adjustments, or effects to photos/videos directly in the Photos app editing UI. | +| [quicklook-preview.md](quicklook-preview.md) | `quicklook-preview` | User wants rich previews of custom file types in Quick Look with interactive controls and multiple pages. | +| [quicklook-thumbnail.md](quicklook-thumbnail.md) | `quicklook-thumbnail` | User wants to generate thumbnail images for custom file types shown in Finder, Files app, and Spotlight. | + +### Notifications + +| Skill | Extension Type | When to Read | +|-------|---------------|--------------| +| [notification-service.md](notification-service.md) | `notification-service` | User wants to download images, decrypt payloads, or modify push notification content before display. Runs in background with 30-second time limit. | +| [notification-content.md](notification-content.md) | `notification-content` | User wants custom SwiftUI/UIKit UI for expanded push notifications with interactive controls and rich media. | + +### Siri & Intents + +| Skill | Extension Type | When to Read | +|-------|---------------|--------------| +| [intent.md](intent.md) | `intent` | User wants Siri voice commands for built-in domains (messaging, payments, workouts, media playback, ride booking). Uses legacy SiriKit framework. | +| [intent-ui.md](intent-ui.md) | `intent-ui` | User wants custom UI in the Siri interface when confirming or handling intents. Companion to the `intent` extension. | +| [app-intent.md](app-intent.md) | `app-intent` | User wants modern App Intents (iOS 16+) for Shortcuts, Spotlight, Siri, Focus Filters, and Control Center widgets. Replaces custom `.intentdefinition` files. | + +### Networking & VPN + +| Skill | Extension Type | When to Read | +|-------|---------------|--------------| +| [network-packet-tunnel.md](network-packet-tunnel.md) | `network-packet-tunnel` | User wants to build a VPN client that tunnels all device traffic through a custom protocol. Classic and modern Personal VPN. | +| [network-app-proxy.md](network-app-proxy.md) | `network-app-proxy` | User wants app-level network filtering (per-app VPN) where specific apps route traffic through a custom proxy. Requires MDM or supervised devices. | +| [network-dns-proxy.md](network-dns-proxy.md) | `network-dns-proxy` | User wants to intercept and respond to DNS queries system-wide, enabling DNS filtering, logging, or custom resolution. | +| [network-filter-data.md](network-filter-data.md) | `network-filter-data` | User wants deep packet inspection for content filtering (parental controls, enterprise security). Requires special Apple entitlement. | + +### File Management + +| Skill | Extension Type | When to Read | +|-------|---------------|--------------| +| [file-provider.md](file-provider.md) | `file-provider` | User wants to expose cloud storage or remote file systems in the Files app with on-demand download, search, and sync. | +| [file-provider-ui.md](file-provider-ui.md) | `file-provider-ui` | User wants custom UI for file actions (sharing, permissions) in the document picker. Companion to File Provider extension. | + +### Communication & Telephony + +| Skill | Extension Type | When to Read | +|-------|---------------|--------------| +| [message-filter.md](message-filter.md) | `message-filter` | User wants to filter incoming SMS/MMS messages to classify spam, transactions, or promotions into separate tabs. | +| [call-directory.md](call-directory.md) | `call-directory` | User wants to block phone numbers or provide caller ID labels for incoming calls. Updates a system-managed SQLite database. | +| [unwanted-communication.md](unwanted-communication.md) | `unwanted-communication` | User wants a UI for users to report unwanted calls, messages, or FaceTime attempts to a reporting service. Requires special entitlement. | + +### Content & Media + +| Skill | Extension Type | When to Read | +|-------|---------------|--------------| +| [broadcast-upload.md](broadcast-upload.md) | `broadcast-upload` | User wants to stream ReplayKit screen recordings to a live streaming service (Twitch, YouTube Live). Receives video/audio buffers in real time. | +| [broadcast-setup-ui.md](broadcast-setup-ui.md) | `broadcast-setup-ui` | User wants custom UI for configuring a live broadcast (selecting stream key, privacy settings). Companion to broadcast upload extension. | +| [safari.md](safari.md) | `safari` | User wants Safari Web Extensions (Manifest V3) with content scripts, background pages, and native messaging to the containing app. | +| [content-blocker.md](content-blocker.md) | `content-blocker` | User wants rule-based ad blocking, tracker blocking, or content hiding in Safari using JSON declarative rules. No JavaScript execution. | + +### Education & Screen Time + +| Skill | Extension Type | When to Read | +|-------|---------------|--------------| +| [classkit-context.md](classkit-context.md) | `classkit-context` | User wants to report student progress in educational apps to Apple's Schoolwork app via ClassKit framework. | +| [device-activity-monitor.md](device-activity-monitor.md) | `device-activity-monitor` | User wants to monitor app/website usage and trigger actions when Screen Time limits are reached. Parental control and focus apps. | +| [shield-action.md](shield-action.md) | `shield-action` | User wants custom UI when a user attempts to bypass a Screen Time shield (blocked app/website). Handle or deny the request. | +| [shield-config.md](shield-config.md) | `shield-config` | User wants custom shield UI (blocking screen) shown when apps or websites are restricted by Screen Time rules. | + +### Authentication & Security + +| Skill | Extension Type | When to Read | +|-------|---------------|--------------| +| [authentication-services.md](authentication-services.md) | `authentication-services` | User wants AutoFill for passwords and passkeys in Safari and apps. Modern replacement for `credentials-provider`. Supports passkey creation/assertion (WebAuthn). | +| [credentials-provider.md](credentials-provider.md) | `credentials-provider` | User wants legacy password AutoFill (iOS 12+). For new projects, use `authentication-services` instead, which supports both passwords and passkeys. | +| [smart-card.md](smart-card.md) | `smart-card` | User wants to authenticate with smart cards or hardware tokens (YubiKey, PIV cards) via the CryptoTokenKit framework. Enterprise use. | +| [account-auth.md](account-auth.md) | `account-auth` | User wants to modify authentication requests for Kerberos or other enterprise single sign-on (SSO) protocols. Requires special entitlement. | + +### Location & Background Services + +| Skill | Extension Type | When to Read | +|-------|---------------|--------------| +| [location-push.md](location-push.md) | `location-push` | User wants silent push notifications that wake the app for high-accuracy location updates. Used for fleet tracking, delivery apps, and location-based alerts. | +| [bg-download.md](bg-download.md) | `bg-download` | User wants on-demand resources or large asset downloads managed by the system's background asset downloader. Used for games and media apps. | + +### Smart Home & Matter + +| Skill | Extension Type | When to Read | +|-------|---------------|--------------| +| [matter.md](matter.md) | `matter` | User wants to add Matter smart home device support for device commissioning via Home app. Requires MFi enrollment and Matter certification. | + +### Printing + +| Skill | Extension Type | When to Read | +|-------|---------------|--------------| +| [print-service.md](print-service.md) | `print-service` | User wants to discover and print to network printers without AirPrint, or provide custom print rendering for specialized printers. | + +### Search & Indexing + +| Skill | Extension Type | When to Read | +|-------|---------------|--------------| +| [spotlight.md](spotlight.md) | `spotlight` | User wants to index app content in Spotlight, provide rich search results, and handle taps from Spotlight search. Uses Core Spotlight and `NSUserActivity`. | +| [spotlight-delegate.md](spotlight-delegate.md) | `spotlight-delegate` | User wants a background extension for large-scale Spotlight indexing that runs periodically on a system-managed schedule. macOS only (not iOS). | + +### Collaboration & Conferencing + +| Skill | Extension Type | When to Read | +|-------|---------------|--------------| +| [virtual-conference.md](virtual-conference.md) | `virtual-conference` | User wants to integrate a video conferencing app with the system call UI and CallKit, enabling Handoff, picture-in-picture, and system call controls. | + +### Apple Watch + +| Skill | Extension Type | When to Read | +|-------|---------------|--------------| +| [watch.md](watch.md) | `watch` | User wants a standalone watchOS app with complications, workout sessions, HealthKit integration, and Watch Connectivity for iPhone sync. Not an extension--a full application target. | + +## Target Types vs. Extensions + +**Application Targets** (not extensions): +- **App Clip** (`clip`) -- uses product type `com.apple.product-type.application.on-demand-install-capable` +- **watchOS App** (`watch`) -- uses product type `com.apple.product-type.application` + +These are full applications with their own `Info.plist`, app icons, and lifecycle. They do not use `NSExtensionPointIdentifier`. + +**All other types are app extensions** that use product type `com.apple.product-type.app-extension` and declare an extension point identifier in their `Info.plist`. + +## Quick Reference: Common Use Cases + +| User Says... | Read This Skill | +|-------------|----------------| +| "Add a home screen widget" | [widget.md](widget.md) | +| "Add a share extension" | [share.md](share.md) | +| "Build an instant app / App Clip" | [clip.md](clip.md) | +| "Integrate with Siri" | [intent.md](intent.md) or [app-intent.md](app-intent.md) | +| "Modify push notifications" | [notification-service.md](notification-service.md) | +| "Custom notification UI" | [notification-content.md](notification-content.md) | +| "Build a VPN app" | [network-packet-tunnel.md](network-packet-tunnel.md) | +| "Password manager AutoFill" | [authentication-services.md](authentication-services.md) | +| "Block spam calls" | [call-directory.md](call-directory.md) | +| "Filter SMS spam" | [message-filter.md](message-filter.md) | +| "Add to Files app" | [file-provider.md](file-provider.md) | +| "Custom keyboard" | [keyboard.md](keyboard.md) | +| "Photo filter extension" | [photo-editing.md](photo-editing.md) | +| "Safari ad blocker" | [content-blocker.md](content-blocker.md) | +| "Safari extension" | [safari.md](safari.md) | +| "Live streaming / screen recording" | [broadcast-upload.md](broadcast-upload.md) | +| "watchOS app" | [watch.md](watch.md) | +| "Screen Time parental controls" | [device-activity-monitor.md](device-activity-monitor.md) | +| "Smart home / Matter" | [matter.md](matter.md) | +| "Print to network printer" | [print-service.md](print-service.md) | +| "Spotlight search integration" | [spotlight.md](spotlight.md) | + +## Creating a New Extension with @bacons/apple-targets + +Every skill document includes a "Using with @bacons/apple-targets" section with the three-step setup: + +```sh +# 1. Run the CLI to scaffold the extension +bunx create-target + +# 2. Ensure the plugin is in app.json (automatic with CLI) +{ + "expo": { + "plugins": [["@bacons/apple-targets"]] + } +} + +# 3. Generate the native project +bunx expo prebuild --clean -p ios +``` + +The `` parameter corresponds to the extension type slug shown in each skill document's title (e.g., `widget`, `share`, `clip`, `notification-service`). + +### When to Run Prebuild + +Because target source files live outside the `/ios` directory and are **linked** into the Xcode project (not copied), Swift code changes are automatically synchronized. You don't need to re-run prebuild when editing code. + +- **Run `prebuild --clean`** when: + - Initial setup after creating a new target + - Changing `expo-target.config.js` (icon, colors, entitlements, bundle ID, frameworks) + - Changing `app.json` plugin configuration + - Adding or removing targets + - Xcode project corruption or mysterious build failures + +- **Run `prebuild` (without `--clean`)** when: + - You need to regenerate the project but want to preserve manual Xcode modifications + - Updating the plugin version with minor project changes + +- **No prebuild needed** for: + - Editing Swift code in `targets//` directories -- files are linked, changes appear immediately in Xcode + - Adding/modifying Swift files within an existing target + - Most day-to-day development work on target implementations + +This architecture allows you to iterate on extension code without constantly regenerating the entire Xcode project, speeding up development and preserving manual project customizations. + +## Entitlements & Capabilities Guides + +The `entitlements/` directory contains guides for iOS entitlements and capabilities that apply across multiple extension types: + +| Guide | When to Read | +|-------|--------------| +| [entitlements/app-groups.md](entitlements/app-groups.md) | User needs to share data between the main app and extensions (widgets, share, notifications, App Clips). Covers naming conventions, UserDefaults suite, shared containers, Keychain access groups, and automatic EAS provisioning. | + +**Common questions answered by entitlements guides:** +- "How do I share data between my app and widget?" -> Read `app-groups.md` +- "My extension can't access UserDefaults from the main app" -> Read `app-groups.md` +- "How does EAS handle App Groups code signing?" -> Read `app-groups.md` + +## Additional Resources + +- **Expo Documentation**: https://docs.expo.dev/config-plugins/introduction/ +- **Continuous Native Generation**: https://docs.expo.dev/workflow/continuous-native-generation/ diff --git a/plugins/expo/skills/apple-targets/account-auth.md b/plugins/expo/skills/apple-targets/account-auth.md new file mode 100644 index 0000000..04d32ed --- /dev/null +++ b/plugins/expo/skills/apple-targets/account-auth.md @@ -0,0 +1,260 @@ +--- +title: Account Authentication Modification Extension +description: Lets password managers and apps offer one-tap upgrades from weak passwords to Sign in with Apple or strong auto-generated passwords, triggered from iCloud Keychain security recommendations and AutoFill prompts. +version: iOS 14.0+, macOS 11.0+ +--- + +# Account Authentication Modification Extension (`account-auth`) + +An extension that integrates with iCloud Keychain to offer users one-tap account security upgrades. When the system detects a weak, reused, or breached password for your app, it can present your extension to silently upgrade the credential to **Sign in with Apple** or a **strong auto-generated password** -- no manual password reset flow required. + +## Apple Documentation + +- [Upgrading Account Security With an Account Authentication Modification Extension](https://developer.apple.com/documentation/authenticationservices/upgrading-account-security-with-an-account-authentication-modification-extension) +- [ASAccountAuthenticationModificationViewController](https://developer.apple.com/documentation/authenticationservices/asaccountauthenticationmodificationviewcontroller) +- [ASAccountAuthenticationModificationExtensionContext](https://developer.apple.com/documentation/authenticationservices/asaccountauthenticationmodificationextensioncontext) +- [convertAccountToSignInWithAppleWithoutUserInteraction](https://developer.apple.com/documentation/authenticationservices/asaccountauthenticationmodificationviewcontroller/3650342-convertaccounttosigninwithapplew) +- [changePasswordWithoutUserInteraction](https://developer.apple.com/documentation/authenticationservices/asaccountauthenticationmodificationviewcontroller/changepasswordwithoutuserinteraction(for:existingcredential:newpassword:userinfo:)) + +## WWDC History + +- **[WWDC 2020, Session 10666 -- One-Tap Account Security Upgrades](https://developer.apple.com/videos/play/wwdc2020/10666/)** -- Introduced the extension. The primary reference for implementation. Covers both upgrade paths, the non-interactive and interactive flows, and associated domain setup. +- **[WWDC 2020, Session 10173 -- Get the Most Out of Sign in with Apple](https://developer.apple.com/videos/play/wwdc2020/10173/)** -- Covers the Sign in with Apple upgrade path in the context of broader SIWA adoption. +- **[WWDC 2022, Session 10122 -- Enhance Your Sign in with Apple Experience](https://developer.apple.com/videos/play/wwdc2022/10122/)** -- Follow-up with refined guidance on offering upgrades to existing users. + +## What It Does + +The extension provides two upgrade paths: + +1. **Upgrade to Sign in with Apple** -- Converts a password-based account to use Sign in with Apple. The system handles the SIWA authorization, your extension calls your backend to swap the credential, and the old password is deleted from iCloud Keychain. + +2. **Upgrade to a strong password** -- Replaces a weak/reused password with a system-generated strong password. The system generates the password (respecting your app's password rules), your extension calls your backend to change it, and iCloud Keychain stores the new credential. + +Users encounter these upgrades in three places: +- **Security Recommendations** in Settings > Passwords (flagged weak/breached passwords) +- **Password detail view** in the iCloud Keychain password manager +- **Automatic system prompts** when signing in with a weak password via AutoFill + +## Use Cases + +### Apps with password-based accounts +Any app that stores user credentials and wants to proactively upgrade users to stronger authentication. The system surfaces your extension when it detects the user's saved password is weak, reused, or appeared in a known data breach. + +### Sign in with Apple migration +Apps adopting Sign in with Apple for existing users. Instead of building a manual migration flow, the extension lets users upgrade with a single tap from the password manager. + +### Password manager integration +Third-party password managers can use this extension to offer automated credential upgrades when the system flags security issues. + +## Prerequisites + +Before the extension works, your app must be associated with your web domain: + +1. Add an `apple-app-site-association` file to your server with a `webcredentials` entry: + +```json +{ + "webcredentials": { + "apps": ["TEAMID.com.example.myapp"] + } +} +``` + +2. Add the Associated Domains capability to your app with `webcredentials:example.com`. + +This association is how the system knows which saved passwords belong to your app. + +## Key Classes + +| Class | Role | +|-------|------| +| `ASAccountAuthenticationModificationViewController` | Principal view controller. Subclass this. The system calls your override methods to perform upgrades. Shows UI only when additional authentication is needed. | +| `ASAccountAuthenticationModificationExtensionContext` | The extension context. Use it to complete upgrades, request SIWA authorization, or cancel with errors. | +| `ASCredentialServiceIdentifier` | Identifies the service (domain) the credential belongs to. | +| `ASPasswordCredential` | The existing username + password being upgraded. | + +## Implementation + +### Sign in with Apple Upgrade (Non-Interactive) + +The most common flow. The system calls this when it believes the upgrade can happen silently (user is already authenticated via biometrics to access the password manager). + +```swift +import AuthenticationServices + +class AccountAuthViewController: ASAccountAuthenticationModificationViewController { + + override func convertAccountToSignInWithAppleWithoutUserInteraction( + for serviceIdentifier: ASCredentialServiceIdentifier, + existingCredential: ASPasswordCredential, + userInfo: [AnyHashable: Any]? + ) { + // 1. Validate the existing credential with your backend + guard validateCredential(existingCredential) else { + // If you need to show UI for 2FA or re-auth, request it: + self.extensionContext.cancelRequest( + withError: ASExtensionError(.userInteractionRequired) + ) + return + } + + // 2. Request Sign in with Apple authorization + self.extensionContext.getSignInWithAppleUpgradeAuthorization( + state: generateState(), + nonce: generateNonce() + ) { authorization, error in + guard let authorization = authorization else { + self.extensionContext.cancelRequest( + withError: ASExtensionError(.failed) + ) + return + } + + // 3. Send the SIWA credential to your backend to link accounts + let appleIDCredential = authorization.credential + as! ASAuthorizationAppleIDCredential + + self.upgradeAccountOnServer( + existingCredential: existingCredential, + appleIDCredential: appleIDCredential + ) { success in + if success { + // 4. Tell the system the upgrade succeeded. + // The old password is deleted from iCloud Keychain. + self.extensionContext.completeUpgradeToSignInWithApple() + } else { + self.extensionContext.cancelRequest( + withError: ASExtensionError(.failed) + ) + } + } + } + } +} +``` + +### Strong Password Upgrade (Non-Interactive) + +```swift +override func changePasswordWithoutUserInteraction( + for serviceIdentifier: ASCredentialServiceIdentifier, + existingCredential: ASPasswordCredential, + newPassword: String, + userInfo: [AnyHashable: Any]? +) { + // 1. Validate existing credential + guard validateCredential(existingCredential) else { + self.extensionContext.cancelRequest( + withError: ASExtensionError(.userInteractionRequired) + ) + return + } + + // 2. Change the password on your backend + changePasswordOnServer( + username: existingCredential.user, + oldPassword: existingCredential.password, + newPassword: newPassword + ) { success in + if success { + // 3. Complete with the updated credential. + // iCloud Keychain stores the new strong password. + let newCredential = ASPasswordCredential( + user: existingCredential.user, + password: newPassword + ) + self.extensionContext.completeChangePasswordRequest( + updatedCredential: newCredential + ) + } else { + self.extensionContext.cancelRequest( + withError: ASExtensionError(.failed) + ) + } + } +} +``` + +### Interactive Flow (When Additional Auth Is Needed) + +If you cancel with `.userInteractionRequired`, the system presents your view controller's UI. Override the `prepareInterface` methods to show a step-up authentication screen (e.g., SMS 2FA code entry): + +```swift +override func prepareInterfaceToConvertAccountToSignInWithApple( + for serviceIdentifier: ASCredentialServiceIdentifier, + existingCredential: ASPasswordCredential, + userInfo: [AnyHashable: Any]? = nil +) { + // Show your 2FA or re-authentication UI here. + // When the user completes it, call the same + // getSignInWithAppleUpgradeAuthorization / completeUpgradeToSignInWithApple + // flow from your action handler. +} + +override func prepareInterfaceToChangePassword( + for serviceIdentifier: ASCredentialServiceIdentifier, + existingCredential: ASPasswordCredential, + newPassword: String, + userInfo: [AnyHashable: Any]? = nil +) { + // Show your re-authentication UI here. + // When complete, call completeChangePasswordRequest. +} +``` + +### In-App Upgrade Trigger + +You can also trigger upgrades from within your app (not just from the password manager): + +```swift +import AuthenticationServices + +let controller = ASAccountAuthenticationModificationController() +controller.delegate = self +controller.presentationContextProvider = self + +// Request a Sign in with Apple upgrade +let request = ASAccountAuthenticationModificationUpgradePasswordToStrongPasswordRequest( + user: "username", + serviceIdentifier: ASCredentialServiceIdentifier( + identifier: "example.com", + type: .domain + ) +) +controller.perform(request) +``` + +## Info.plist Configuration + +The extension declares which upgrade types it supports: + +| Key | Type | Description | +|-----|------|-------------| +| `ASAccountAuthenticationModificationSupportsUpgradeToSignInWithApple` | Boolean | Set `true` to offer Sign in with Apple upgrades. | +| `ASAccountAuthenticationModificationSupportsStrongPasswordChange` | Boolean | Set `true` to offer strong password upgrades. | + +Both are enabled by default in the `@bacons/apple-targets` config plugin. + +## Usage + +- Create: `bun create target account-auth` +- Edit: `targets/account-auth/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 14.0+ | Full support. | +| iPadOS | 14.0+ | Full support. | +| macOS | 11.0+ | Full support. | +| visionOS | 1.0+ | Supported. | +| watchOS | -- | Not supported. | +| tvOS | -- | Not supported. | + +## Gotchas + +- **Associated Domains are required.** The extension only activates for passwords saved against domains you own. Without a valid `apple-app-site-association` file and the `webcredentials` associated domain, the system will never surface your extension. +- **Prefer non-interactive flows.** Apple recommends completing upgrades without UI whenever possible. Only request `.userInteractionRequired` when you genuinely need a step-up authentication (2FA, re-auth). Unnecessary UI friction defeats the purpose of one-tap upgrades. +- **iOS 18 Passwords app limitation.** As of iOS 18, these extensions are not invoked from the new standalone Passwords app. They still work from AutoFill prompts and from within your own app. Apple has acknowledged this gap. +- **The old password is deleted on SIWA upgrade.** After `completeUpgradeToSignInWithApple()`, the system removes the password from iCloud Keychain. Make sure your backend has fully linked the Apple ID before completing. +- **No third-party tutorials exist.** This is a niche API. The WWDC 2020 session and Apple's documentation are the only substantive references available. diff --git a/plugins/expo/skills/apple-targets/action.md b/plugins/expo/skills/apple-targets/action.md new file mode 100644 index 0000000..c3d9950 --- /dev/null +++ b/plugins/expo/skills/apple-targets/action.md @@ -0,0 +1,197 @@ +--- +title: Action Extension +description: Transforms or processes content in-place from the system share sheet, optionally with a UI or as a headless background operation. +version: iOS 8.0+ +--- + +# Action Extension (`action`) + +An Action Extension appears alongside Share Extensions in the system share sheet but serves a different purpose: instead of sending content to another app, it transforms or processes content and optionally returns modified data back to the host app. Action Extensions come in two flavors -- UI-based (presents a view controller for user interaction) and headless (no UI, runs entirely in the background via `NSExtensionRequestHandling`). A powerful pattern pairs the headless Swift handler with a JavaScript preprocessing file that can read and modify the current webpage in Safari, creating a bridge between native code and web content. Content flows through `NSExtensionContext` using `NSExtensionItem` and `NSItemProvider`, the same mechanism as Share Extensions, but Action Extensions can return results via `completeRequest(returningItems:)`. + +## Apple Documentation + +- [App Extension Programming Guide: Action](https://developer.apple.com/library/archive/documentation/General/Conceptual/ExtensibilityPG/Action.html) +- [NSExtensionContext](https://developer.apple.com/documentation/foundation/nsextensioncontext) +- [NSItemProvider](https://developer.apple.com/documentation/foundation/nsitemprovider) +- [NSExtensionItem](https://developer.apple.com/documentation/foundation/nsextensionitem) +- [NSExtensionActivationRule](https://developer.apple.com/documentation/bundleresources/information-property-list/nsextension/nsextensionattributes/nsextensionactivationrule) +- [NSExtensionRequestHandling](https://developer.apple.com/documentation/foundation/nsextensionrequesthandling) +- [App Extension Keys Reference](https://developer.apple.com/library/archive/documentation/General/Reference/InfoPlistKeyReference/Articles/AppExtensionKeys.html) + +## WWDC History + +- **[WWDC 2014, Session 205 -- Creating Extensions for iOS and OS X, Part 1](https://developer.apple.com/videos/play/wwdc2014/205/)** -- Introduced the app extension model including Action Extensions. Covered the extension lifecycle, `NSExtensionContext`, and data flow between host app and extension. +- **[WWDC 2014, Session 217 -- Creating Extensions for iOS and OS X, Part 2](https://developer.apple.com/videos/play/wwdc2014/217/)** -- Deep dive into Action Extensions specifically. Demonstrated returning modified data back to the host app, Safari JavaScript preprocessing, and the `run()`/`finalize()` pattern for web page manipulation. +- **[WWDC 2015, Session 224 -- App Extension Best Practices](https://developer.apple.com/videos/play/wwdc2015/224/)** -- Covered activation rules, memory management, and the distinction between Action Extensions ("change content in place") and Share Extensions ("move content to another app"). + +## What It Does + +1. **User selects content and opens the share sheet.** The system evaluates your `NSExtensionActivationRule` against the available content. If it matches, your Action Extension appears in the action row of the activity view controller. +2. **For headless extensions:** The system calls `beginRequest(with:)` on your `NSExtensionRequestHandling` conforming class. No UI is presented. +3. **For UI-based extensions:** The system instantiates your principal view controller and presents it modally. +4. **Content arrives via `NSExtensionContext`.** Input items are `NSExtensionItem` objects with `NSItemProvider` attachments, identical to Share Extensions. +5. **JavaScript preprocessing (Safari).** If the extension declares an `NSExtensionJavaScriptPreprocessingFile`, Safari runs the file's `run()` function before launching the extension. The JavaScript can extract page data (title, URL, DOM content) and pass it to the native side as a property list dictionary via the `completionFunction` callback. +6. **Extension processes content.** Your native code receives the preprocessed data, performs transformations (translate text, apply image filter, extract metadata), and prepares results. +7. **Results flow back.** The extension creates a new `NSExtensionItem` with `NSItemProvider` attachments containing the modified data and calls `extensionContext.completeRequest(returningItems:)`. For Safari extensions, the results are passed to the JavaScript file's `finalize()` function, which can apply changes to the webpage DOM. + +## Use Cases + +### Webpage background color changer +A headless Action Extension paired with a JavaScript file. The JS `run()` reads the current `document.body.style.backgroundColor` and passes it to Swift. The Swift handler decides on a new color and returns it. The JS `finalize()` applies the new background color to the page. No UI required. + +### Text translation +A UI-based Action Extension that receives selected text from any app. The extension presents a view showing the original text and a translated version using a local ML model or API call. The user confirms, and the translated text is returned to the host app to replace the selection. + +### Image markup and annotation +A UI-based extension that receives an image attachment. The user draws annotations, adds text overlays, or applies filters. The modified image is returned to the host app via `completeRequest(returningItems:)`. + +### Save to reading list +A headless Action Extension that extracts the current page URL from Safari via JavaScript preprocessing and saves it to the app's shared data store. A brief system notification confirms the save without interrupting the user's browsing. + +## Key Classes + +| Class / Protocol | Role | +|------------------|------| +| `NSExtensionRequestHandling` | Protocol for headless (no-UI) Action Extensions. Implement `beginRequest(with:)` to receive the extension context. | +| `NSExtensionContext` | Provides `inputItems` from the host app and the `completeRequest(returningItems:completionHandler:)` method to return results. | +| `NSExtensionItem` | Wraps one logical item from the host. Contains `attachments` (array of `NSItemProvider`), plus optional `attributedTitle` and `attributedContentText`. | +| `NSItemProvider` | Represents a single attachment. Use `hasItemConformingToTypeIdentifier(_:)` to probe, then `loadItem(forTypeIdentifier:options:)` to retrieve data asynchronously. | +| `UIViewController` | For UI-based Action Extensions. Set as `NSExtensionPrincipalClass` and present your custom interface. Access input via `extensionContext`. | +| `NSExtensionJavaScriptPreprocessingResultsKey` | Dictionary key used to extract the results from the JavaScript `run()` function in the loaded property list. | +| `NSExtensionJavaScriptFinalizeArgumentKey` | Dictionary key for the data you pass back to the JavaScript `finalize()` function from your native handler. | + +## Implementation + +```swift +import UIKit +import UniformTypeIdentifiers + +// 1. Conform to NSExtensionRequestHandling for a headless (no-UI) Action Extension. +class ActionRequestHandler: NSObject, NSExtensionRequestHandling { + + var extensionContext: NSExtensionContext? + + func beginRequest(with context: NSExtensionContext) { + // 2. Store the context; do not call super for headless extensions. + self.extensionContext = context + + var found = false + + // 3. Iterate input items to find JavaScript preprocessing results. + outer: + for item in context.inputItems as! [NSExtensionItem] { + if let attachments = item.attachments { + for provider in attachments { + if provider.hasItemConformingToTypeIdentifier(UTType.propertyList.identifier) { + provider.loadItem( + forTypeIdentifier: UTType.propertyList.identifier, + options: nil + ) { (item, error) in + let dictionary = item as! [String: Any] + let results = dictionary[ + NSExtensionJavaScriptPreprocessingResultsKey + ] as? [String: Any] ?? [:] + // 4. Process the JavaScript results on the main queue. + OperationQueue.main.addOperation { + self.processResults(results) + } + } + found = true + break outer + } + } + } + } + + if !found { + // 5. No JavaScript results found; complete immediately. + done(with: nil) + } + } + + func processResults(_ jsResults: [String: Any]) { + // 6. Read the page title extracted by the JavaScript run() function. + let pageTitle = jsResults["pageTitle"] as? String ?? "Untitled" + + // 7. Prepare data to send back to the JavaScript finalize() function. + let responseData: [String: Any] = [ + "processedTitle": pageTitle.uppercased(), + "timestamp": ISO8601DateFormatter().string(from: Date()) + ] + done(with: responseData) + } + + func done(with resultsForJS: [String: Any]?) { + if let results = resultsForJS { + // 8. Wrap the results in an NSExtensionItem for the JS finalize() callback. + let resultsDictionary = [NSExtensionJavaScriptFinalizeArgumentKey: results] + let provider = NSItemProvider( + item: resultsDictionary as NSDictionary, + typeIdentifier: UTType.propertyList.identifier + ) + let outputItem = NSExtensionItem() + outputItem.attachments = [provider] + extensionContext?.completeRequest(returningItems: [outputItem], completionHandler: nil) + } else { + // 9. Nothing to return; still must signal completion. + extensionContext?.completeRequest(returningItems: [], completionHandler: nil) + } + extensionContext = nil + } +} +``` + +The companion JavaScript file (`assets/index.js`) bridges the native handler and the webpage: + +```javascript +// assets/index.js -- loaded by Safari before the native extension runs. +class Action { + // 10. run() executes in the webpage context before the native handler launches. + run({ extensionName, completionFunction }) { + completionFunction({ + pageTitle: document.title, + currentURL: window.location.href, + selectedText: window.getSelection()?.toString() || "", + }); + } + + // 11. finalize() executes after the native handler returns results. + finalize(args) { + if (args.processedTitle) { + document.title = args.processedTitle; + } + } +} + +// 12. Must assign to window.ExtensionPreprocessingJS for Safari to find it. +window.ExtensionPreprocessingJS = new Action(); +``` + +## Usage + +- Create: `bun create target action` +- Edit: `targets/action/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 8.0+ | Full support. Appears in the share sheet action row. | +| iPadOS | 8.0+ | Full support. Share sheet appears as a popover. | +| macOS | 10.10+ | Supported via `NSViewController`. Finder integration via `NSExtensionServiceAllowsFinderPreviewItem`. | +| Mac Catalyst | 13.1+ | Uses iOS extension APIs. | +| watchOS | -- | Not supported. | +| tvOS | -- | Not supported. | +| visionOS | 1.0+ | Share sheet supported. | + +## Gotchas + +- **`TRUEPREDICATE` will get your app rejected.** The default `NSExtensionActivationRule` is `TRUEPREDICATE`, which makes the extension appear for all content. Apple rejects App Store submissions containing this value. Replace it with specific keys (`NSExtensionActivationSupportsWebURLWithMaxCount`, `NSExtensionActivationSupportsText`, etc.) or a custom predicate string. +- **Activation rule dictionary version matters.** `NSExtensionActivationDictionaryVersion` defaults to 1, which requires the extension to handle ALL offered content types. Set it to 2 if you want "at least one matching type" semantics. Getting this wrong means your extension either never appears or appears when it should not. +- **Headless extensions have no UI lifecycle.** There is no `viewDidLoad` or `viewDidAppear`. All work happens in `beginRequest(with:)`. You must call `completeRequest(returningItems:)` when finished or the system will eventually kill the extension. +- **JavaScript file must use `window.ExtensionPreprocessingJS`.** Safari looks for this specific global variable. If the class is not assigned to `window.ExtensionPreprocessingJS`, the preprocessing step silently fails and the native handler receives no data. +- **Same 120 MB memory limit as other extensions.** Action Extensions are subject to the same ~120 MB memory cap. Image processing or heavy computation should be done incrementally and with downsampled inputs. +- **Return data type must match what the host expects.** Not all host apps handle returned items. Safari expects property list data keyed with `NSExtensionJavaScriptFinalizeArgumentKey`. Other host apps may ignore returned items entirely. Test with multiple host apps. +- **`NSExtensionServiceAllowsFinderPreviewItem` is macOS-only.** The Finder-specific and Touch Bar-specific keys in the default template Info.plist have no effect on iOS. They are included for cross-platform extensions that also target macOS. +- **Action and Share share the same sheet but different rows.** Action Extensions appear in the bottom row of the share sheet (the "action" row), while Share Extensions appear in the top row. Users can reorder and hide individual extensions, so do not assume your extension is visible by default. +- **No background execution after completion.** Unlike Share Extensions which can use background URL sessions for uploads, headless Action Extensions have no mechanism for continued background work. All processing must complete before calling `completeRequest`. diff --git a/plugins/expo/skills/apple-targets/app-intent.md b/plugins/expo/skills/apple-targets/app-intent.md new file mode 100644 index 0000000..ab0d78f --- /dev/null +++ b/plugins/expo/skills/apple-targets/app-intent.md @@ -0,0 +1,175 @@ +--- +title: App Intents Extension +description: Provides app actions to Siri, Shortcuts, Spotlight, interactive widgets, Focus filters, and Control Center controls using the modern pure-Swift App Intents framework via an ExtensionKit-based extension. +version: iOS 16.0+ +--- + +# App Intents Extension (`app-intent`) + +An App Intents extension is a lightweight ExtensionKit-based process that exposes your app's actions and entities to the system without launching your main app. Unlike legacy SiriKit Intents that required IntentDefinition files and Objective-C code generation, App Intents are declared entirely in Swift using structs conforming to the `AppIntent` protocol with `@Parameter` property wrappers. The framework powers Siri voice commands, Shortcuts automations, Spotlight suggestions, interactive widget buttons, Focus filters, the Action Button, Apple Pencil Pro squeeze, and -- starting with iOS 18 -- Control Center controls. Running intents in a dedicated extension keeps your app's startup cost out of the critical path, so Siri and Shortcuts respond faster. + +## Apple Documentation + +- [App Intents Framework](https://developer.apple.com/documentation/appintents) +- [AppIntent Protocol](https://developer.apple.com/documentation/appintents/appintent) +- [Creating Your First App Intent](https://developer.apple.com/documentation/appintents/creating-your-first-app-intent) +- [AppShortcutsProvider](https://developer.apple.com/documentation/appintents/appshortcutsprovider) +- [EntityQuery](https://developer.apple.com/documentation/appintents/entityquery) +- [AppIntentsExtension](https://developer.apple.com/documentation/appintents/appintentsextension) +- [Accelerating App Interactions with App Intents](https://developer.apple.com/documentation/appintents/acceleratingappinteractionswithappintents) +- [ExtensionKit Framework](https://developer.apple.com/documentation/extensionkit) + +## WWDC History + +- **[WWDC 2022, Session 10032 -- Dive into App Intents](https://developer.apple.com/videos/play/wwdc2022/10032/)** -- Introduced the App Intents framework as a pure-Swift replacement for SiriKit custom intents. Covered `AppIntent`, `AppEntity`, `EntityQuery`, and `AppShortcutsProvider` for zero-setup Siri phrases. +- **[WWDC 2022, Session 10170 -- Implement App Shortcuts with App Intents](https://developer.apple.com/videos/play/wwdc2022/10170/)** -- Detailed walkthrough of building App Shortcuts with parameterized phrases, including entity resolution and Siri dialog. +- **[WWDC 2022, Session 10169 -- Design App Shortcuts](https://developer.apple.com/videos/play/wwdc2022/10169/)** -- Design guidance for creating discoverable, natural-sounding App Shortcuts. +- **[WWDC 2023, Session 10103 -- Explore Enhancements to App Intents](https://developer.apple.com/videos/play/wwdc2023/10103/)** -- iOS 17 updates: App Intents in frameworks via `AppIntentsPackage`, Shortcuts integration improvements, and Spotlight entity indexing. +- **[WWDC 2024, Session 10134 -- What's New in App Intents](https://developer.apple.com/videos/play/wwdc2024/10134/)** -- Universal Links for entities, Transferable support, IntentFile APIs, union values, and the guidance shift to "every feature should be an App Intent." +- **[WWDC 2024, Session 10210 -- Bring Your App's Core Features to Users with App Intents](https://developer.apple.com/videos/play/wwdc2024/10210/)** -- End-to-end guide for surfacing app features through Siri, Spotlight, widgets, the Action Button, and Control Center controls in iOS 18. +- **[WWDC 2025, Session 244 -- Get to Know App Intents](https://developer.apple.com/videos/play/wwdc2025/244/)** -- The new `@AppIntent` macro, `ComputedProperty` and `DeferredProperty` macros, `UnionValue` for mixed entity queries, and Swift Package support for App Intents code. + +## What It Does + +1. You declare Swift structs conforming to `AppIntent` with a `title`, `@Parameter` properties, and a `perform()` method. +2. The system's metadata extractor discovers your intents at build time (no runtime registration needed). +3. When a user triggers the intent -- through Siri, Shortcuts, Spotlight, a widget button, or Control Center -- the system launches your App Intents extension process. +4. The extension calls your `perform() async throws -> some IntentResult` method. +5. The method executes the action, optionally returning a dialog string, a SwiftUI snippet view, or an `IntentResultValue` for downstream Shortcuts actions. +6. For Siri voice activation without user setup, you register phrases in an `AppShortcutsProvider` so the system knows which spoken commands map to which intents. + +## Use Cases + +### Voice-activated app actions +A task manager lets users say "Add a task in TaskApp" to create a new item. The `AppShortcutsProvider` registers the phrase, and the intent's `perform()` method creates the task in the shared data store and returns a confirmation dialog. + +### Shortcuts automations +A home automation app exposes "Set thermostat to X degrees" as a parameterized intent. Users build Shortcuts automations that trigger the intent at specific times or locations without opening the app. + +### Spotlight quick actions +A recipe app conforms its recipes to `AppEntity` with a `StringSearchableEntityQuery`. When the user searches in Spotlight, matching recipes appear as actionable results that open directly to the recipe detail view. + +### Interactive widget buttons +A media player widget includes play/pause buttons backed by App Intents. Tapping the button executes the intent in the extension, toggling playback without launching the full app. + +### Control Center controls (iOS 18+) +A smart-home app registers a `ControlWidget` backed by an App Intent to toggle lights. The control appears in the customizable Control Center grid and executes entirely in the extension. + +### Focus filters +A communication app uses `SetFocusFilterIntent` to configure which accounts are visible during a Focus mode, filtering notifications and content to match the user's current context. + +## Key Classes + +| Class / Protocol | Role | +|-----------------|------| +| `AppIntent` | Core protocol. Declare a `static var title`, `@Parameter` properties, and `perform() async throws -> some IntentResult`. | +| `AppIntentsExtension` | Marker protocol for the `@main` entry point struct of an App Intents extension. | +| `IntentResult` | Return type from `perform()`. Compose with `ProvidesDialog`, `ShowsSnippetView`, or `ReturnsValue` for richer results. | +| `AppEntity` | Protocol for exposing your data model to the system. Requires `id`, `displayRepresentation`, and a `defaultQuery`. | +| `EntityQuery` | Retrieves entities by identifier (`entities(for:)`) and provides suggestions (`suggestedEntities()`). Subtypes include `EnumerableEntityQuery` and `EntityStringQuery`. | +| `AppShortcutsProvider` | Declares `appShortcuts` array of `AppShortcut` with trigger phrases containing `.applicationName`. Enables Siri voice activation with zero user setup. | +| `@Parameter` | Property wrapper that declares a typed intent parameter. Supports default values, `title`, `description`, and `requestValueDialog`. | +| `AppIntentsPackage` | Protocol for re-exporting intents from frameworks and Swift packages. Use `includedPackages` to compose modular intent libraries. | +| `AppEnum` | Protocol for exposing a Swift enum as an intent parameter with localized display names for each case. | + +## Implementation + +```swift +import AppIntents + +// 1. Mark the entry point with @main. AppIntentsExtension is the +// required conformance for an App Intents extension target. +@main +struct MyAppIntentsExtension: AppIntentsExtension { +} + +// 2. Define an entity that represents your app's data model. +// The system uses the defaultQuery to look up entities by ID. +struct TaskEntity: AppEntity { + static var typeDisplayRepresentation: TypeDisplayRepresentation = "Task" + static var defaultQuery = TaskEntityQuery() + + var id: UUID + var name: String + var isComplete: Bool + + var displayRepresentation: DisplayRepresentation { + DisplayRepresentation(title: "\(name)") + } +} + +// 3. Implement an EntityQuery so Siri and Shortcuts can resolve +// task parameters by identifier or by search string. +struct TaskEntityQuery: EntityQuery { + func entities(for identifiers: [UUID]) async throws -> [TaskEntity] { + TaskStore.shared.tasks.filter { identifiers.contains($0.id) } + } + + func suggestedEntities() async throws -> [TaskEntity] { + TaskStore.shared.tasks.filter { !$0.isComplete } + } +} + +// 4. Declare the intent with @Parameter properties and a perform method. +// The title appears in Shortcuts and Spotlight as the action name. +struct CompleteTaskIntent: AppIntent { + static var title: LocalizedStringResource = "Complete Task" + static var description: IntentDescription = "Marks a task as done." + + // 5. @Parameter exposes typed inputs. The system uses the entity's + // defaultQuery to present a picker when the user taps this parameter. + @Parameter(title: "Task") + var task: TaskEntity + + // 6. perform() is called when the intent executes. Return a dialog + // string so Siri speaks confirmation, and a value for Shortcuts chaining. + func perform() async throws -> some IntentResult & ProvidesDialog & ReturnsValue { + TaskStore.shared.markComplete(task.id) + return .result(value: true, dialog: "Done! \(task.name) is complete.") + } +} + +// 7. Register App Shortcuts for voice activation. Each phrase must +// include .applicationName so Siri knows which app to route to. +struct MyAppShortcuts: AppShortcutsProvider { + static var appShortcuts: [AppShortcut] { + AppShortcut( + intent: CompleteTaskIntent(), + phrases: [ + "Mark a task done in \(.applicationName)", + "Complete a task in \(.applicationName)" + ], + shortTitle: "Complete Task", + systemImageName: "checkmark.circle" + ) + } +} +``` + +## Usage + +- Create: `bun create target app-intent` +- Edit: `targets/app-intent/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 16.0+ | Full support. Control Center controls require iOS 18+. | +| iPadOS | 16.0+ | Full support. | +| macOS | 13.0+ | Full support. App Intents power Shortcuts and Spotlight on Mac. | +| watchOS | 9.0+ | Supported. Intents can run in a watchOS App Intents extension. | +| tvOS | -- | Not supported. | +| visionOS | 1.0+ | Full support via compatible iOS intents. | + +## Gotchas + +- **Extension uses ExtensionKit, not NSExtension.** The App Intents extension uses the `EXAppExtensionAttributes` Info.plist key with `EXExtensionPointIdentifier` set to `com.apple.appintents-extension`, and its product type is `com.apple.product-type.extensionkit-extension`. It installs into the `Extensions/` directory, not `PlugIns/`. Mixing these up causes silent validation failures. +- **Metadata is extracted at build time.** The system discovers your intents by scanning compiled metadata, not by runtime registration. If your intent struct is not compiled into the extension target (or a framework linked to it), it will not appear in Shortcuts or Siri. There is no runtime error -- the intent simply does not exist. +- **AppShortcutsProvider phrases must include `.applicationName`.** Every phrase string in your `AppShortcut` must contain the `.applicationName` interpolation token. Without it, the phrase will not compile, and Siri cannot route the voice command to your app. +- **Extension and main app share intents via frameworks.** If you need the same intents available in both the extension and the main app, compile them into a shared framework and use `AppIntentsPackage` with `includedPackages` to re-export the metadata. Duplicating intent code across targets causes conflicts. +- **No UI by default.** Unlike legacy Intents UI extensions, App Intents do not show custom UI unless you explicitly return `ShowsSnippetView` from `perform()` with an attached SwiftUI view. The default result is a text dialog. +- **perform() runs on an arbitrary queue.** If your intent needs main-thread access (e.g., UIKit operations), annotate `perform()` with `@MainActor`. Accessing main-thread-only APIs without this annotation causes undefined behavior. +- **Entity IDs must be stable.** The system persists entity identifiers across Shortcuts runs and Siri sessions. If your entity IDs change between app launches (e.g., using array indices), saved Shortcuts will break silently with "entity not found" errors. +- **Older OS versions ignore newer intent features.** Features like Control Center controls (iOS 18), `@AppIntent` macro (iOS 26), and `UnionValue` (iOS 26) are not available on earlier OS versions. Use `@available` checks and provide fallback intents for broader compatibility. +- **App Intents extensions cannot show UI on launch.** The extension runs headlessly. If your action requires user interaction (login, confirmation), return `.result()` with a `needsContinueInForegroundError` or use `openAppWhenRun = true` to hand off to the main app. diff --git a/plugins/expo/skills/apple-targets/authentication-services.md b/plugins/expo/skills/apple-targets/authentication-services.md new file mode 100644 index 0000000..3f0d712 --- /dev/null +++ b/plugins/expo/skills/apple-targets/authentication-services.md @@ -0,0 +1,230 @@ +--- +title: Single Sign-On Extension (Authentication Services) +description: Enterprise SSO extension that intercepts web authentication requests to your identity provider and handles them natively, eliminating browser-based login flows on managed devices. +version: iOS 13.0+, macOS 10.15+ +--- + +# Single Sign-On Extension (`authentication-services`) + +An Authentication Services SSO extension intercepts HTTP authentication requests destined for your identity provider (IdP) and handles them directly in the extension process rather than loading a web login page. When any app or Safari on a managed device makes a request to a URL matching your IdP's domain, the system routes the authentication challenge to your extension. The extension can authenticate silently using device credentials, present a native login UI, use Secure Enclave keys, or perform multi-factor authentication -- all without a browser redirect. This is the foundation of enterprise SSO on Apple platforms, used by Okta, Microsoft Entra ID (Azure AD), Jamf Connect, and Apple's own internal SSO infrastructure. + +## Apple Documentation + +- [ASAuthorizationProviderExtensionAuthorizationRequestHandler](https://developer.apple.com/documentation/authenticationservices/asauthorizationproviderextensionauthorizationrequesthandler) -- The protocol your extension implements to handle authorization requests. +- [ASAuthorizationProviderExtensionAuthorizationRequest](https://developer.apple.com/documentation/authenticationservices/asauthorizationproviderextensionauthorizationrequest) -- The request object passed to your handler, containing the URL, headers, and body. +- [Creating Extensions That Support Platform SSO](https://developer.apple.com/documentation/authenticationservices/creating-extensions-that-support-platform-sso) -- Guide for building extensions that integrate with macOS login. +- [ASAuthorizationProviderExtensionLoginManager](https://developer.apple.com/documentation/authenticationservices/asauthorizationproviderextensionloginmanager) -- Platform SSO login manager for macOS (macOS 13+). +- [AuthenticationServices Framework](https://developer.apple.com/documentation/authenticationservices) +- [Platform Single Sign-On for macOS (Deployment Guide)](https://support.apple.com/guide/deployment/platform-sso-for-macos-dep7bbb05313/web) + +## WWDC History + +- **[WWDC 2019 -- What's New in Managing Apple Devices](https://developer.apple.com/videos/play/wwdc2019/303/)** -- Introduced Extensible Enterprise SSO. Announced the `ASAuthorizationProviderExtensionAuthorizationRequestHandler` protocol and the requirement for MDM-managed configuration. +- **[Tech Talk 301 -- Introducing Extensible Enterprise SSO](https://developer.apple.com/videos/play/tech-talks/301/)** -- Dedicated deep-dive into the SSO extension architecture, including how the system intercepts HTTP requests, the difference between Redirect and Credential extension types, and the MDM profile keys. +- **[WWDC 2020 -- What's New in Managing Apple Devices](https://developer.apple.com/videos/play/wwdc2020/10639/)** -- Updates to SSO extension capabilities, including associated domains for extension discovery. +- **[WWDC 2022, Session 10045 -- What's New in Managing Apple Devices](https://developer.apple.com/videos/play/wwdc2022/10045/)** -- Introduced **Platform SSO** for macOS 13 Ventura. Ties the local user account to the IdP, enabling sign-in at the macOS login window with IdP credentials or Secure Enclave keys. Replaces Active Directory binding. +- **[WWDC 2023 -- Manage Devices with Apple Frameworks](https://developer.apple.com/videos/play/wwdc2023/10040/)** -- Platform SSO enhancements: device compliance integration, improved token handling. + +## What It Does + +1. **MDM deploys a configuration profile.** The enterprise MDM server pushes an Extensible SSO profile to the device. This profile specifies your extension's bundle identifier, the IdP URLs it handles, and whether it operates in **Redirect** mode (intercepts HTTP redirects to your IdP) or **Credential** mode (intercepts HTTP 401 challenges). +2. **App or Safari makes an HTTP request.** When any app on the device makes a request that matches the configured IdP URLs, the system intercepts it before the response reaches the app. +3. **System loads your extension.** The system calls `beginAuthorization(with:)` on your `ASAuthorizationProviderExtensionAuthorizationRequestHandler`, passing an `ASAuthorizationProviderExtensionAuthorizationRequest` containing the original URL, HTTP headers, and body. +4. **Extension authenticates.** Your extension can: + - Complete silently by returning HTTP authorization headers (e.g., a cached token). + - Present a native login UI via `request.presentAuthorizationViewController(completion:)`. + - Perform device-bound authentication using Secure Enclave keys. + - Call your IdP's token endpoint and return the result. +5. **Extension completes the request.** Call one of: + - `request.complete(httpAuthorizationHeaders:)` -- inject auth headers into the original request. + - `request.complete(httpResponse:httpBody:)` -- return a full HTTP response. + - `request.complete(error:)` -- fail the authentication. + - `request.doNotHandle()` -- pass through to the default web flow. +6. **App receives authenticated response.** The requesting app is unaware that an extension handled the auth. It receives the response as if the server returned it directly. + +### Platform SSO (macOS 13+) + +Platform SSO extends this mechanism to the macOS login window: + +1. At the login screen, the user enters their IdP password (or uses Touch ID backed by a Secure Enclave key). +2. The system calls your extension's `ASAuthorizationProviderExtensionLoginManager` to authenticate with the IdP. +3. On success, the local macOS account is unlocked and the user is simultaneously authenticated to the IdP for all subsequent SSO requests. +4. This replaces Active Directory binding and mobile accounts. + +## Use Cases + +### Enterprise Identity Providers +Okta, Microsoft Entra ID (formerly Azure AD), Ping Identity, and OneLogin build SSO extensions so their customers' managed devices get native, seamless authentication across all apps and Safari. Employees sign in once and are silently authenticated everywhere. + +### Government and Regulated Industries +Organizations in healthcare, finance, and government use SSO extensions combined with certificate-based authentication or smart cards. The extension can enforce compliance policies (device attestation, OS version checks) before issuing tokens. + +### Education (Higher Ed) +Universities deploy SSO extensions via MDM so students and staff get seamless SAML/OIDC authentication across learning management systems, email, and internal portals without repeated browser-based login prompts. + +### macOS Login with Cloud Identity (Platform SSO) +IT departments use Platform SSO to eliminate Active Directory binding. Employees log in to their Mac at the login window with their cloud IdP credentials. Jamf Connect and Microsoft Entra ID are the primary Platform SSO providers as of 2025. + +## Key Classes + +| Class / Protocol | Role | +|------------------|------| +| `ASAuthorizationProviderExtensionAuthorizationRequestHandler` | Protocol your extension's principal class conforms to. Implement `beginAuthorization(with:)` to handle incoming auth requests. | +| `ASAuthorizationProviderExtensionAuthorizationRequest` | The request object. Contains `url`, `httpHeaders`, `httpBody`, `realm`, `requestedOperation`, `callerBundleIdentifier`. Call `complete(...)` or `doNotHandle()` when finished. | +| `ASAuthorizationProviderExtensionLoginManager` | macOS 13+ Platform SSO manager. Handles login window authentication, user registration, device registration, and token management. | +| `ASAuthorizationSingleSignOnProvider` | Used by client apps to explicitly trigger the SSO extension (less common -- most usage is automatic via HTTP interception). | +| `ASAuthorizationSingleSignOnCredential` | The credential returned to client apps after successful SSO authentication. | + +## Implementation + +### Redirect-Type SSO Extension + +```swift +import AuthenticationServices + +// 1. Conform to the SSO request handler protocol. +// This class is declared as NSExtensionPrincipalClass in Info.plist. +class AuthenticationExtension: NSObject, + ASAuthorizationProviderExtensionAuthorizationRequestHandler { + + private var currentRequest: ASAuthorizationProviderExtensionAuthorizationRequest? + + // 2. Called when the system intercepts an HTTP request matching + // your IdP's configured URLs. + func beginAuthorization( + with request: ASAuthorizationProviderExtensionAuthorizationRequest + ) { + self.currentRequest = request + + // 3. Check if we have a cached token that is still valid. + if let cachedToken = TokenCache.shared.validToken(for: request.url) { + // 4. Complete silently by injecting the auth header. + // The requesting app never sees a login prompt. + request.complete( + httpAuthorizationHeaders: [ + "Authorization": "Bearer \(cachedToken)" + ] + ) + return + } + + // 5. No valid cached token -- authenticate with the IdP. + // For a Redirect-type extension, exchange the intercepted + // URL for tokens using your IdP's token endpoint. + authenticateWithIdP(request: request) + } + + private func authenticateWithIdP( + request: ASAuthorizationProviderExtensionAuthorizationRequest + ) { + // 6. Build a token request to your IdP. + let tokenURL = URL(string: "https://idp.example.com/oauth2/token")! + var tokenRequest = URLRequest(url: tokenURL) + tokenRequest.httpMethod = "POST" + tokenRequest.setValue( + "application/x-www-form-urlencoded", + forHTTPHeaderField: "Content-Type" + ) + + // 7. Include device attestation or client certificate for + // strong device identity. + let body = "grant_type=device_code&client_id=myapp&device_id=\(deviceId())" + tokenRequest.httpBody = body.data(using: .utf8) + + URLSession.shared.dataTask(with: tokenRequest) { data, response, error in + guard let data = data, + let httpResponse = response as? HTTPURLResponse, + error == nil else { + // 8. If IdP is unreachable, fall back to web login. + request.doNotHandle() + return + } + + if httpResponse.statusCode == 200 { + // 9. Parse the token and cache it. + if let json = try? JSONSerialization.jsonObject(with: data) as? [String: Any], + let accessToken = json["access_token"] as? String { + TokenCache.shared.store(token: accessToken, for: request.url) + + // 10. Complete the request with the authorization header. + request.complete( + httpAuthorizationHeaders: [ + "Authorization": "Bearer \(accessToken)" + ] + ) + } else { + request.complete(error: SSOError.invalidResponse) + } + } else { + // 11. Token request failed -- present native login UI. + request.presentAuthorizationViewController(completion: { success, error in + if !success { + request.complete( + error: error ?? SSOError.userCancelled + ) + } + // The view controller handles completion via + // request.complete(httpAuthorizationHeaders:). + }) + } + }.resume() + } + + private func deviceId() -> String { + // Return a stable device identifier for your IdP. + return UIDevice.current.identifierForVendor?.uuidString ?? "unknown" + } +} + +enum SSOError: Error { + case invalidResponse + case userCancelled +} +``` + +### Minimal Extension (Template) + +The `create-target` scaffold produces this minimal starting point: + +```swift +import AuthenticationServices + +class AuthenticationExtension: NSObject, + ASAuthorizationProviderExtensionAuthorizationRequestHandler { + + func beginAuthorization( + with request: ASAuthorizationProviderExtensionAuthorizationRequest + ) { + // Pass through to default web login until implementation is ready. + request.doNotHandle() + } +} +``` + +## Usage + +- Create: `bun create target authentication-services` +- Edit: `targets/authentication-services/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 13.0+ | Redirect and Credential SSO extension types. Requires MDM configuration profile. | +| iPadOS | 13.0+ | Same as iOS. Shared iPad also supported. | +| macOS | 10.15+ | SSO extensions. Platform SSO (login window integration) requires macOS 13.0+. Simplified Setup in macOS 26+. | +| watchOS | -- | Not supported. | +| tvOS | -- | Not supported. tvOS has no SSO extension mechanism. | +| visionOS | 1.0+ | SSO extensions are supported on managed devices. | + +## Gotchas + +- **MDM is absolutely required.** The SSO extension does nothing without a configuration profile deployed via MDM (Jamf, Intune, Kandji, Mosyle, etc.). The profile maps your IdP's URLs to your extension's bundle identifier. Without it, the system never loads your extension. There is no way for a user to manually enable an SSO extension. +- **Two extension types: Redirect vs. Credential.** A **Redirect** extension intercepts HTTP 302 redirects to your IdP and handles the login flow. A **Credential** extension intercepts HTTP 401 challenges and provides credentials directly. Choose based on your IdP's flow. If you need to provide headers, use a Credential extension -- Redirect extensions are for handling full request/response flows. +- **Associated Domains are required for URL matching.** Your app must declare associated domains (`authsrv:` prefix) matching your IdP's URLs, or the URL list must be provided via the MDM profile. Without this, the system cannot determine which requests to route to your extension. +- **Testing requires a managed device.** You cannot test SSO extensions on the iOS Simulator or on unmanaged devices. You need either a real device enrolled in MDM or Apple Configurator 2 to push a configuration profile. +- **`doNotHandle()` is your escape hatch.** If your extension encounters a request it cannot handle, call `request.doNotHandle()` to let the system fall back to the default web-based login. Never leave a request hanging without calling a completion method. +- **Platform SSO requires macOS 13+ and IdP support.** Platform SSO is not a drop-in feature. Your IdP must implement the protocol, including device registration, token exchange, and key attestation endpoints. As of 2025, only Microsoft Entra ID, Okta, and Jamf Connect have production Platform SSO implementations. +- **Extension runs in a separate process with limited memory.** Keep your extension lightweight. Heavy operations (large token caches, complex UI) can cause the system to terminate your extension. +- **Apple uses this internally.** Apple has its own SSO extension for corporate authentication, which is how Apple employees get single sign-on across internal tools. This is evidence the API is production-hardened but also means Apple prioritizes enterprise use cases. +- **The Kerberos SSO extension is built in.** Apple ships a built-in Kerberos SSO extension (`com.apple.AppSSOKerberos.KerberosExtension`) that handles Active Directory/Kerberos authentication automatically when configured via MDM. You only need a custom extension for SAML, OIDC, or proprietary auth protocols. diff --git a/plugins/expo/skills/apple-targets/bg-download.md b/plugins/expo/skills/apple-targets/bg-download.md new file mode 100644 index 0000000..1609a90 --- /dev/null +++ b/plugins/expo/skills/apple-targets/bg-download.md @@ -0,0 +1,168 @@ +--- +title: Background Asset Download Extension +description: Schedules large asset downloads that run in the background, including immediately after App Store install before the user first launches the app. +version: iOS 16.1+, macOS 13.0+ +--- + +# Background Asset Download Extension (`bg-download`) + +A background asset download extension lets your app schedule large file downloads that execute outside the app's lifecycle -- during installation, after updates, or periodically in the background. The system invokes your extension to determine what needs downloading, then manages the transfers automatically. Starting with iOS 16.4, downloads marked as "essential" block the app's first launch until they complete, integrating into the App Store installation progress bar. The extension communicates with the main app through App Groups shared storage. + +## Apple Documentation + +- [BackgroundAssets Framework Overview](https://developer.apple.com/documentation/backgroundassets) +- [BADownloaderExtension Protocol](https://developer.apple.com/documentation/backgroundassets/badownloaderextension-qwaw) +- [Downloading Essential Assets in the Background](https://developer.apple.com/documentation/backgroundassets/downloading-essential-assets-in-the-background) +- [BADownloadManager](https://developer.apple.com/documentation/backgroundassets/badownloadmanager) +- [BAURLDownload](https://developer.apple.com/documentation/backgroundassets/baurldownload) +- [BAContentRequest](https://developer.apple.com/documentation/backgroundassets/bacontentrequest) + +## WWDC History + +- **[WWDC 2022, Session 110403 -- Meet Background Assets](https://developer.apple.com/videos/play/wwdc2022/110403/)** -- Introduced the BackgroundAssets framework for scheduling large downloads outside the app lifecycle, covering the extension architecture, BADownloadManager, essential vs. non-essential downloads, and App Group requirements. +- **[WWDC 2023, Session 10108 -- What's New in Background Assets](https://developer.apple.com/videos/play/wwdc2023/10108/)** -- Added essential downloads that block app launch during installation, manifest URL pre-fetching, and improved extension lifecycle management. +- **[WWDC 2025, Session 325 -- Discover Apple-Hosted Background Assets](https://developer.apple.com/videos/play/wwdc2025/325/)** -- Introduced managed background assets with Apple-hosted CDN support, automatic compression, and versioning via App Store Connect. + +## What It Does + +1. **System triggers the extension.** After an app install, update, or on a periodic schedule, the system launches your extension and calls `downloads(for:manifestURL:extensionInfo:)`. +2. **Extension reads the manifest.** The system pre-downloads a manifest file (specified by `BAManifestURL` in your app's Info.plist) and passes its local path to the extension. Your code parses it to determine which assets are needed. +3. **Extension returns download descriptors.** You create `BAURLDownload` objects specifying the URL, file size, whether the download is essential, and the App Group identifier for storage. Return them as a `Set`. +4. **System manages transfers.** The framework downloads files in the background, handling retries, cellular vs. Wi-Fi policies, and power management. +5. **Extension receives completion callbacks.** When a download finishes, `backgroundDownload(_:finishedWithFileURL:)` is called. Move the file to a permanent location in the App Group container. On failure, `backgroundDownload(_:failedWithError:)` fires. +6. **Main app reads downloaded assets.** On launch, the app checks the shared App Group container for downloaded files and uses them directly. + +## Use Cases + +### Games with large asset packs +A mobile game ships a small binary through the App Store and uses essential background downloads to fetch texture packs, level data, and audio assets during installation. The player sees a single progress bar and can start playing immediately after install completes. + +### Machine learning model delivery +An ML-powered app downloads updated CoreML models periodically in the background. When the extension fires on a periodic schedule, it checks the manifest for newer model versions and schedules non-essential downloads that complete while the device is idle. + +### Media and podcast pre-caching +A streaming app schedules background downloads of new episodes or content catalogs. The extension checks the manifest during periodic runs and queues downloads for content the user is likely to want, reducing wait times on next launch. + +### Map and navigation data +A navigation app downloads regional map tiles and routing databases as essential assets during install, ensuring the app is functional for offline use from the very first launch. + +## Key Classes + +| Class / Protocol | Role | +|------------------|------| +| `BADownloaderExtension` | Protocol your extension struct conforms to. Defines entry points for download scheduling and completion. | +| `BAContentRequest` | Enum indicating why the extension was invoked: `.install`, `.update`, or `.periodic`. | +| `BAAppExtensionInfo` | Metadata about the app including `applicationIdentifier`, `lastPeriodicCheckTime`, and `downloadSizeRestricted`. | +| `BAURLDownload` | A download descriptor specifying URL, file size, essential flag, App Group ID, and priority. | +| `BADownload` | Base class for download objects. `BAURLDownload` is the concrete subclass for URL-based downloads. | +| `BADownloadManager` | Singleton used by the main app (or extension) to schedule, cancel, and promote downloads. Foreground promotion is app-only. | + +## Implementation + +```swift +import BackgroundAssets + +// 1. Declare the extension entry point with @main. +@main +struct DownloadHandler: BADownloaderExtension { + + // 2. System calls this to ask what downloads are needed. + func downloads( + for request: BAContentRequest, + manifestURL: URL, + extensionInfo: BAAppExtensionInfo + ) -> Set { + let appGroup = "group.com.example.myapp" + + // 3. Parse the pre-downloaded manifest to determine available assets. + guard let manifestData = try? Data(contentsOf: manifestURL), + let manifest = try? JSONDecoder().decode(AssetManifest.self, from: manifestData) + else { + return [] + } + + // 4. Compare manifest against already-downloaded files in the App Group. + let downloaded = AlreadyDownloaded.load(appGroup: appGroup) + let needed = manifest.assets.filter { !downloaded.contains($0.identifier) } + + var downloads: Set = [] + for asset in needed { + let isEssential = (request == .install || request == .update) && asset.required + // 5. Create BAURLDownload for each needed asset. + let download = BAURLDownload( + identifier: asset.identifier, + request: URLRequest(url: asset.url), + essential: isEssential, + fileSize: asset.fileSize, + applicationGroupIdentifier: appGroup, + priority: isEssential ? .max : .default + ) + downloads.insert(download) + } + return downloads + } + + // 6. Called when a download completes successfully. + func backgroundDownload( + _ finishedDownload: BADownload, + finishedWithFileURL fileURL: URL + ) { + // Move the file to Library/Caches in the App Group container + // so the system can reclaim space if storage is critically low. + guard let container = FileManager.default.containerURL( + forSecurityApplicationGroupIdentifier: "group.com.example.myapp" + ) else { return } + + let destination = container + .appendingPathComponent("Library/Caches", isDirectory: true) + .appendingPathComponent(finishedDownload.identifier) + + try? FileManager.default.createDirectory( + at: destination.deletingLastPathComponent(), + withIntermediateDirectories: true + ) + try? FileManager.default.moveItem(at: fileURL, to: destination) + } + + // 7. Called when a download fails. Optionally reschedule. + func backgroundDownload( + _ failedDownload: BADownload, + failedWithError error: Error + ) { + // Log the failure. The system may retry automatically depending + // on the error type. You can also reschedule via BADownloadManager. + } + + // 8. Called shortly before the extension is terminated. + func extensionWillTerminate() { + // Serialize any in-memory state to disk immediately. + } +} +``` + +## Usage + +- Create: `bun create target bg-download` +- Edit: `targets/bg-download/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 16.1+ | Full support. Essential downloads added in 16.4. | +| iPadOS | 16.1+ | Full support. Essential downloads added in 16.4. | +| macOS | 13.0+ (Ventura) | Extension runs even when the app is terminated. | +| watchOS | -- | Not supported. | +| tvOS | -- | Not supported (managed assets coming in tvOS 26). | +| visionOS | -- | Not supported (managed assets coming in visionOS 26). | + +## Gotchas + +- **App Groups are mandatory.** The extension runs in a separate process from your app. All downloaded files must be stored in a shared App Group container. Without a correctly configured App Group, the extension cannot write files the app can read. +- **Downloaded files are marked purgeable.** The system may delete files downloaded by the framework when the device is critically low on storage. Store files in `Library/Caches` and always verify their existence before use. +- **Low Power Mode and Background App Refresh disable the extension.** If the user has enabled Low Power Mode or disabled Background App Refresh (globally or per-app), the extension will never run for periodic requests. Essential downloads during install/update are not affected. +- **On iOS, the user must not have force-quit the app.** If the user swipes the app away from the app switcher, the extension will not receive periodic runtime. On macOS, the extension runs regardless of app termination state. +- **Extension runtime is strictly limited.** The system measures runtime from when a function is invoked to when it returns. Once your function exits scope, the extension may be suspended and terminated immediately. Do not rely on instance variables persisting between calls. +- **Essential downloads block app launch.** If a download marked `essential: true` fails or takes too long, the user cannot launch your app until the system times out. Keep essential downloads small and ensure your CDN is reliable. +- **The `BAManifestURL` Info.plist key is required.** The system downloads this URL before invoking your extension. If the key is missing or the URL is unreachable, the `manifestURL` parameter will point to an empty or missing file. +- **`BADownloadManager.delegate` in the app overrides the extension.** If your main app registers a delegate on `BADownloadManager`, the app receives download callbacks instead of the extension while the app is running. Design your code to handle both paths. diff --git a/plugins/expo/skills/apple-targets/broadcast-setup-ui.md b/plugins/expo/skills/apple-targets/broadcast-setup-ui.md new file mode 100644 index 0000000..ec537d6 --- /dev/null +++ b/plugins/expo/skills/apple-targets/broadcast-setup-ui.md @@ -0,0 +1,187 @@ +--- +title: Broadcast Setup UI Extension +description: Presents a configuration interface before a ReplayKit live broadcast begins, allowing users to log in, select a channel, and configure stream settings. +version: iOS 10.0+, tvOS 10.0+, macOS 11.0+ +--- + +# Broadcast Setup UI Extension (`broadcast-setup-ui`) + +An extension that displays a modal view controller when the user starts a live broadcast through ReplayKit. The setup UI appears after the user picks your broadcast service from `RPBroadcastActivityViewController` or the system broadcast picker, and before any audio or video data starts flowing. Your view controller collects whatever configuration is needed -- account credentials, stream title, channel selection, quality preset -- then calls `extensionContext?.completeRequest(withBroadcast:setupInfo:)` to hand a `setupInfo` dictionary to the paired broadcast-upload extension and begin the broadcast. This is the companion to the `broadcast-upload` extension type. + +## Apple Documentation + +- [ReplayKit Framework](https://developer.apple.com/documentation/replaykit) +- [RPBroadcastActivityViewController](https://developer.apple.com/documentation/replaykit/rpbroadcastactivityviewcontroller) +- [RPSystemBroadcastPickerView](https://developer.apple.com/documentation/replaykit/rpsystembroadcastpickerview) +- [NSExtensionContext.completeRequest(withBroadcast:setupInfo:)](https://developer.apple.com/documentation/replaykit/nsextensioncontext) +- [App Extension Programming Guide -- Broadcast](https://developer.apple.com/library/archive/documentation/General/Conceptual/ExtensibilityPG/ExtensionOverview.html) + +## WWDC History + +- **[WWDC 2016, Session 601 -- Go Live with ReplayKit](https://developer.apple.com/videos/play/wwdc2016/601/)** -- Introduced broadcast extensions as a pair: a setup UI extension to collect user preferences and an upload extension to stream the data. Demonstrated live game streaming through a Mobcrush extension. +- **[WWDC 2017, Session 606 -- What's New with Screen Recording and Live Broadcast](https://developer.apple.com/videos/play/wwdc2017/606/)** -- Introduced ReplayKit 2 with system-wide broadcast from Control Center. Noted that Control Center broadcasts bypass the setup UI extension entirely, calling `broadcastStarted(withSetupInfo: nil)` on the upload extension. +- **[WWDC 2018, Session 601 -- Live Screen Broadcast with ReplayKit](https://developer.apple.com/videos/play/wwdc2018/601/)** -- Introduced `RPSystemBroadcastPickerView` (iOS 12+), which lets apps embed a broadcast button that can target a specific upload extension by bundle ID, potentially bypassing the setup UI for simpler flows. Covered best practices for handling sign-in inside broadcast extensions. +- **[WWDC 2020, Session 10633 -- Capture and Stream Apps on the Mac with ReplayKit](https://developer.apple.com/videos/play/wwdc2020/10633/)** -- Brought ReplayKit broadcast extensions to macOS, including Mac Catalyst support. + +## What It Does + +1. The user taps a broadcast button in an app (via `RPBroadcastActivityViewController` or `RPSystemBroadcastPickerView`) or selects a broadcast service from Control Center. +2. If the broadcast is started from within an app using `RPBroadcastActivityViewController`, the system presents a list of available broadcast services. The user picks yours. +3. The system launches your setup UI extension and displays your `BroadcastSetupViewController` as a modal sheet. +4. Your view controller presents login fields, channel pickers, quality selectors, or any other configuration UI. +5. When the user confirms, you call `extensionContext?.completeRequest(withBroadcast: broadcastURL, setupInfo: setupInfo)`. The `broadcastURL` is a URL where viewers can watch the stream. The `setupInfo` dictionary is forwarded to the paired broadcast-upload extension's `broadcastStarted(withSetupInfo:)` method. +6. If the user cancels, you call `extensionContext?.cancelRequest(withError:)` and the broadcast is aborted. +7. After the setup completes successfully, the system launches the broadcast-upload extension and begins delivering `CMSampleBuffer` objects. + +## Use Cases + +### Live Game Streaming Platforms + +A game streaming service (Twitch, YouTube Gaming) presents a setup UI where the user authenticates, selects a stream title, and picks a quality preset before going live. The `setupInfo` dictionary passes the server URL and stream key to the upload extension. + +### Video Conferencing Screen Sharing + +An enterprise conferencing app shows a setup screen to select which meeting room to broadcast into, confirm permissions, and choose whether to include microphone audio. The setup info includes the room ID and auth token. + +### Educational Broadcasting + +An instructor selects which class section to stream to and whether to record a local copy. The setup UI reads class rosters from shared `UserDefaults` (via App Groups) and passes the selected section ID to the upload extension. + +## Key Classes + +| Class | Role | +|-------|------| +| `UIViewController` | Subclass this as your `BroadcastSetupViewController`. Presents the configuration interface to the user. | +| `NSExtensionContext` | Provides `completeRequest(withBroadcast:setupInfo:)` to start the broadcast and `cancelRequest(withError:)` to abort it. | +| `RPBroadcastActivityViewController` | System-provided view controller (used by the host app) that lists available broadcast services and launches your setup UI. | +| `RPSystemBroadcastPickerView` | UIView (iOS 12+) that displays a system picker button. Can set `preferredExtension` to target a specific upload extension, bypassing setup UI for simpler flows. | + +## Implementation + +### Stream Configuration Setup UI + +A realistic broadcast setup UI extension that presents a login form and channel picker, then passes stream configuration to the paired upload extension: + +```swift +import ReplayKit +import UIKit + +class BroadcastSetupViewController: UIViewController { + + private let serverURLField = UITextField() + private let streamKeyField = UITextField() + private let channelPicker = UISegmentedControl(items: ["Main", "Gaming", "IRL"]) + private let startButton = UIButton(type: .system) + private let cancelButton = UIButton(type: .system) + + // 1. Build the setup UI programmatically. Xcode's template provides no + // default UI -- init(nibName:bundle:) is called with nil for both. + override func viewDidLoad() { + super.viewDidLoad() + view.backgroundColor = .systemBackground + + serverURLField.placeholder = "rtmp://live.example.com/stream" + serverURLField.borderStyle = .roundedRect + serverURLField.autocapitalizationType = .none + + streamKeyField.placeholder = "Stream key" + streamKeyField.borderStyle = .roundedRect + streamKeyField.isSecureTextEntry = true + + channelPicker.selectedSegmentIndex = 0 + + startButton.setTitle("Go Live", for: .normal) + startButton.addTarget(self, action: #selector(startTapped), for: .touchUpInside) + + cancelButton.setTitle("Cancel", for: .normal) + cancelButton.addTarget(self, action: #selector(cancelTapped), for: .touchUpInside) + + // 2. Lay out controls in a vertical stack + let stack = UIStackView(arrangedSubviews: [ + serverURLField, streamKeyField, channelPicker, startButton, cancelButton + ]) + stack.axis = .vertical + stack.spacing = 16 + stack.translatesAutoresizingMaskIntoConstraints = false + view.addSubview(stack) + + NSLayoutConstraint.activate([ + stack.centerYAnchor.constraint(equalTo: view.centerYAnchor), + stack.leadingAnchor.constraint(equalTo: view.leadingAnchor, constant: 24), + stack.trailingAnchor.constraint(equalTo: view.trailingAnchor, constant: -24), + ]) + + // 3. Pre-fill from shared UserDefaults if the user has broadcast before. + // Requires the App Groups entitlement. + if let defaults = UserDefaults(suiteName: "group.com.example.broadcast") { + serverURLField.text = defaults.string(forKey: "lastServerURL") + streamKeyField.text = defaults.string(forKey: "lastStreamKey") + } + } + + // 4. User confirmed -- build the setupInfo dictionary and hand it off. + @objc private func startTapped() { + let serverURL = serverURLField.text ?? "rtmp://live.example.com/stream" + let streamKey = streamKeyField.text ?? "" + let channel = channelPicker.titleForSegment(at: channelPicker.selectedSegmentIndex) ?? "Main" + + // 5. Persist the configuration for next time + if let defaults = UserDefaults(suiteName: "group.com.example.broadcast") { + defaults.set(serverURL, forKey: "lastServerURL") + defaults.set(streamKey, forKey: "lastStreamKey") + } + + // 6. The broadcastURL is where viewers can watch the stream. + // This URL is surfaced by RPBroadcastActivityViewController in the host app. + let broadcastURL = URL(string: "https://example.com/live/\(channel.lowercased())")! + + // 7. setupInfo is forwarded to the upload extension's broadcastStarted(withSetupInfo:). + // Keys and values must be NSCoding & NSObjectProtocol. + let setupInfo: [String: NSCoding & NSObjectProtocol] = [ + "serverURL": serverURL as NSString, + "streamKey": streamKey as NSString, + "channel": channel as NSString, + ] + + // 8. Tell ReplayKit the setup is complete. This dismisses the setup UI + // and launches the broadcast-upload extension. + extensionContext?.completeRequest(withBroadcast: broadcastURL, setupInfo: setupInfo) + } + + // 9. User cancelled -- dismiss without starting a broadcast. + @objc private func cancelTapped() { + let error = NSError( + domain: "com.example.broadcast", + code: -1, + userInfo: [NSLocalizedDescriptionKey: "User cancelled broadcast setup."] + ) + extensionContext?.cancelRequest(withError: error) + } +} +``` + +## Usage + +- Create: `bun create target broadcast-setup-ui` +- Edit: `targets/broadcast-setup-ui/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 10.0+ | Full support. System-wide broadcast from Control Center added in iOS 11, but Control Center bypasses the setup UI. | +| iPadOS | 10.0+ | Full support. | +| macOS | 11.0+ | Supported via Mac Catalyst. | +| tvOS | 10.0+ | Supported for in-app broadcasts only. | +| visionOS | -- | Not supported. | +| watchOS | -- | Not supported. | + +## Gotchas + +- **Control Center broadcasts bypass the setup UI entirely.** When a user starts a broadcast from Control Center (iOS 11+), the system launches only the upload extension and calls `broadcastStarted(withSetupInfo: nil)`. Your upload extension must handle a `nil` `setupInfo` gracefully -- fall back to configuration stored in shared `UserDefaults` via App Groups. +- **`RPSystemBroadcastPickerView` (iOS 12+) can also bypass setup UI.** When the host app sets `preferredExtension` to a specific upload extension bundle ID, the system may skip your setup UI and go straight to broadcasting. For simpler flows where no login is needed, this is intentional. +- **Xcode's template provides no visible UI.** The generated `BroadcastSetupViewController` stub has an empty `viewDidLoad`. You must build the entire view hierarchy programmatically or via a nib. The `init(nibName:bundle:)` initializer is called with `nil` for both parameters by default, so if you use Interface Builder, override `init` to provide your nib name explicitly. +- **The setupInfo dictionary values must conform to `NSCoding & NSObjectProtocol`.** You cannot pass arbitrary Swift types. Stick to `NSString`, `NSNumber`, `NSData`, `NSURL`, and `NSArray`/`NSDictionary` containing those types. +- **50MB memory limit applies to the setup UI extension process too.** Although the setup UI typically uses far less memory than the upload extension, loading large images or complex storyboards can push you close to the limit on older devices. +- **The extension cannot present additional view controllers.** Your `BroadcastSetupViewController` is presented modally by the system. Pushing or presenting additional view controllers on top of it may not work reliably. Keep your entire setup flow in a single view controller, or use child view controllers and container views. +- **iOS 13.0 had a bug with `RPSystemBroadcastPickerView`.** Tapping the picker button could throw an exception. This was fixed in iOS 13.1. If you support iOS 13.0, guard against this or use `RPBroadcastActivityViewController` as a fallback. diff --git a/plugins/expo/skills/apple-targets/broadcast-upload.md b/plugins/expo/skills/apple-targets/broadcast-upload.md new file mode 100644 index 0000000..a25af5b --- /dev/null +++ b/plugins/expo/skills/apple-targets/broadcast-upload.md @@ -0,0 +1,234 @@ +--- +title: Broadcast Upload Extension +description: Receives screen recording video and audio sample buffers from ReplayKit and uploads them to a live streaming service. +version: iOS 10.0+, tvOS 10.0+, macOS 11.0+ +--- + +# Broadcast Upload Extension (`broadcast-upload`) + +An extension that receives live screen capture data -- video frames, app audio, and microphone audio -- from ReplayKit and streams it to a remote broadcast service. The system delivers `CMSampleBuffer` objects to your extension process in real time, and your code encodes and uploads them. This is the engine behind live streaming features in apps like Twitch, YouTube Live, and game streaming platforms. + +## Apple Documentation + +- [ReplayKit Framework](https://developer.apple.com/documentation/replaykit) +- [RPBroadcastSampleHandler](https://developer.apple.com/documentation/replaykit/rpbroadcastsamplehandler) +- [processSampleBuffer(_:with:)](https://developer.apple.com/documentation/replaykit/rpbroadcastsamplehandler/processsamplebuffer(_:with:)) +- [broadcastStarted(withSetupInfo:)](https://developer.apple.com/documentation/replaykit/rpbroadcastsamplehandler/broadcaststarted(withsetupinfo:)) +- [broadcastFinished()](https://developer.apple.com/documentation/replaykit/rpbroadcastsamplehandler/broadcastfinished()) +- [finishBroadcastWithError(_:)](https://developer.apple.com/documentation/replaykit/rpbroadcastsamplehandler/finishbroadcastwitherror(_:)) +- [RPSampleBufferType](https://developer.apple.com/documentation/replaykit/rpsamplebuffertype) + +## WWDC History + +- **[WWDC 2016, Session 601 -- Go Live with ReplayKit](https://developer.apple.com/videos/play/wwdc2016/601/)** -- Introduced broadcast extensions (setup UI + upload). Demonstrated live game streaming with a Mobcrush extension. +- **[WWDC 2017, Session 606 -- What's New with Screen Recording and Live Broadcast](https://developer.apple.com/videos/play/wwdc2017/606/)** -- Added in-app camera overlay support and refined the broadcast upload API. +- **[WWDC 2018, Session 601 -- Live Screen Broadcast with ReplayKit](https://developer.apple.com/videos/play/wwdc2018/601/)** -- Introduced system-wide broadcast from Control Center (ReplayKit 2), `RPSystemBroadcastPickerView`, and best practices for handling account sign-in in broadcast extensions. +- **[WWDC 2020, Session 10633 -- Capture and Stream Apps on the Mac with ReplayKit](https://developer.apple.com/videos/play/wwdc2020/10633/)** -- Brought ReplayKit broadcast extensions to macOS, including Mac Catalyst support. + +## What It Does + +1. The user initiates a broadcast via Control Center, `RPSystemBroadcastPickerView`, or `RPBroadcastActivityViewController`. +2. The system launches the broadcast upload extension process and calls `broadcastStarted(withSetupInfo:)` with any configuration from the paired setup UI extension. +3. ReplayKit continuously delivers `CMSampleBuffer` objects to `processSampleBuffer(_:with:)` with one of three types: `.video` (screen frames), `.audioApp` (system/app audio), and `.audioMic` (microphone input). +4. Your extension encodes the buffers and uploads them to your streaming backend over a socket or HTTP connection. +5. When the user stops the broadcast, the system calls `broadcastFinished()`. Your extension flushes remaining data and tears down the connection. +6. If a fatal error occurs, your extension calls `finishBroadcastWithError(_:)` to surface a message to the user and terminate the session. + +## Use Cases + +### Game Live Streaming + +A gaming platform (Twitch, YouTube Gaming, Facebook Gaming) provides a broadcast upload extension so players can stream gameplay directly from any iOS game without third-party capture hardware. + +### Video Conferencing Screen Sharing + +Apps like Zoom, Teams, or WebEx use a broadcast upload extension to share the entire screen during a call. The extension captures screen content and sends it to the conferencing server via WebRTC or a proprietary protocol. + +### Remote Support and Collaboration + +Enterprise remote-assistance apps stream the user's screen to a support agent. The broadcast upload extension captures the display while the main app handles the bidirectional communication channel. + +### Educational Broadcasting + +An instructor streams their iPad screen to students via a custom LMS. The extension uploads the screen capture to a media server, and students view it through a companion app or web player. + +## Key Classes + +| Class | Role | +|-------|------| +| `RPBroadcastSampleHandler` | Subclass this. Receives lifecycle callbacks and sample buffers for the broadcast session. | +| `CMSampleBuffer` | Core Media buffer containing a video frame or audio samples delivered by ReplayKit. | +| `RPSampleBufferType` | Enum distinguishing `.video`, `.audioApp`, and `.audioMic` buffers. | +| `RPSystemBroadcastPickerView` | UIView (iOS 12+) that displays a system picker button to start a broadcast from within your app. | +| `RPBroadcastActivityViewController` | View controller that presents a list of available broadcast services (pre-iOS 12 flow). | + +## Implementation + +### RTMP-Style Streaming Handler + +A realistic broadcast upload extension that connects to an RTMP-style streaming server, encodes video frames, mixes audio, and handles lifecycle events: + +```swift +import ReplayKit +import VideoToolbox + +class SampleHandler: RPBroadcastSampleHandler { + + private var socketConnection: StreamSocketConnection? + private var videoEncoder: H264Encoder? + private var audioMixer: AudioSampleMixer? + private var isBroadcasting = false + + // 1. Called when the user starts the broadcast. Set up your upload pipeline. + override func broadcastStarted(withSetupInfo setupInfo: [String: NSObject]?) { + // Extract server URL and stream key from setup info (provided by the setup UI extension) + let serverURL = setupInfo?["serverURL"] as? String ?? "rtmp://live.example.com/stream" + let streamKey = setupInfo?["streamKey"] as? String ?? "" + + // 2. Initialize the socket connection to the streaming server + socketConnection = StreamSocketConnection(url: serverURL, key: streamKey) + socketConnection?.onError = { [weak self] error in + // Surface connection errors to the user + self?.finishBroadcastWithError(NSError( + domain: "com.example.broadcast", + code: -1, + userInfo: [NSLocalizedFailureReasonErrorKey: "Lost connection to server: \(error.localizedDescription)"] + )) + } + socketConnection?.connect() + + // 3. Set up hardware-accelerated H.264 encoding. + // Downscale to 720p to stay within the 50MB memory limit. + videoEncoder = H264Encoder( + width: 1280, + height: 720, + bitrate: 3_000_000, + keyFrameInterval: 2 + ) + videoEncoder?.onEncodedFrame = { [weak self] nalUnits, presentationTime in + self?.socketConnection?.sendVideo(nalUnits, timestamp: presentationTime) + } + + // 4. Set up audio mixing (app audio + mic audio into a single AAC stream) + audioMixer = AudioSampleMixer(sampleRate: 44100, channels: 2) + audioMixer?.onMixedSamples = { [weak self] aacData, presentationTime in + self?.socketConnection?.sendAudio(aacData, timestamp: presentationTime) + } + + isBroadcasting = true + } + + // 5. Called for every video frame and audio chunk. This is the hot path. + override func processSampleBuffer(_ sampleBuffer: CMSampleBuffer, with sampleBufferType: RPSampleBufferType) { + guard isBroadcasting else { return } + + switch sampleBufferType { + case .video: + // 6. Encode the video frame. The encoder downscales and compresses + // using VideoToolbox hardware acceleration. + videoEncoder?.encode(sampleBuffer) + + case .audioApp: + // 7. Feed app audio into the mixer + audioMixer?.appendAppAudio(sampleBuffer) + + case .audioMic: + // 8. Feed microphone audio into the mixer + audioMixer?.appendMicAudio(sampleBuffer) + + @unknown default: + break + } + } + + // 9. Called when the broadcast is paused (e.g., phone call interruption) + override func broadcastPaused() { + audioMixer?.pause() + videoEncoder?.pause() + } + + // 10. Called when the broadcast resumes after a pause + override func broadcastResumed() { + audioMixer?.resume() + videoEncoder?.resume() + } + + // 11. Called when the user stops the broadcast. Flush buffers and disconnect. + override func broadcastFinished() { + isBroadcasting = false + videoEncoder?.flush() + audioMixer?.flush() + socketConnection?.disconnect() + + videoEncoder = nil + audioMixer = nil + socketConnection = nil + } +} +``` + +### Minimal Sample Buffer Logger (Debugging) + +Useful for verifying your extension receives data before building the full upload pipeline: + +```swift +import ReplayKit + +class SampleHandler: RPBroadcastSampleHandler { + + private var frameCount = 0 + + override func broadcastStarted(withSetupInfo setupInfo: [String: NSObject]?) { + frameCount = 0 + NSLog("[BroadcastUpload] Broadcast started. Setup info: \(setupInfo ?? [:])") + } + + override func processSampleBuffer(_ sampleBuffer: CMSampleBuffer, with sampleBufferType: RPSampleBufferType) { + switch sampleBufferType { + case .video: + frameCount += 1 + if frameCount % 60 == 0 { + // 1. Log every 60th frame to avoid flooding the console + let timestamp = CMSampleBufferGetPresentationTimeStamp(sampleBuffer) + NSLog("[BroadcastUpload] Video frame #\(frameCount), PTS: \(timestamp.seconds)") + } + case .audioApp: + break // App audio received + case .audioMic: + break // Mic audio received + @unknown default: + break + } + } + + override func broadcastFinished() { + NSLog("[BroadcastUpload] Broadcast finished. Total video frames: \(frameCount)") + } +} +``` + +## Usage + +- Create: `bun create target broadcast-upload` +- Edit: `targets/broadcast-upload/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 10.0+ | Full support. System-wide broadcast added in iOS 11. | +| iPadOS | 10.0+ | Full support. Larger screen sizes increase memory pressure. | +| macOS | 11.0+ | Supported via Mac Catalyst and native macOS apps. | +| tvOS | 10.0+ | Supported for in-app broadcasts only (no Control Center picker). | +| visionOS | -- | Not supported. | +| watchOS | -- | Not supported. | + +## Gotchas + +- **50MB memory limit.** The extension process is killed if it exceeds 50MB of RAM. ReplayKit itself can consume 20-25MB of that budget for internal buffers, leaving your code with roughly 25MB. Downscale video frames and use hardware-accelerated H.264 (not VP8) to stay under the limit. +- **iPad screen sizes spike memory.** ReplayKit delivers pixel buffers at the native screen resolution. On iPad Pro (2732x2048), uncompressed frames are significantly larger than iPhone frames. Always downscale before encoding. +- **App switching causes memory spikes.** When the user presses the home button or switches apps during a system broadcast, ReplayKit may queue up to 6-8 unprocessed buffers simultaneously, causing a temporary memory spike that can push you over the 50MB limit even if your steady-state usage is well under it. +- **No direct access to the broadcast state.** The extension process is separate from your main app. Use App Groups and shared `UserDefaults` or a shared file to communicate state (stream key, authentication tokens) between your app and the extension. +- **`finishBroadcastWithError` is your only error UI.** The extension has no visible interface. The only way to communicate errors to the user is through the localized failure reason in the `NSError` you pass to `finishBroadcastWithError(_:)`. +- **Audio buffers arrive on a different cadence than video.** Do not assume audio and video buffers are interleaved 1:1. Audio buffers arrive more frequently and in smaller chunks. Your encoder/muxer must handle independent timing. +- **`RPBroadcastProcessMode` must be set in Info.plist.** The extension's Info.plist must include `RPBroadcastProcessMode` set to `RPBroadcastProcessModeSampleBuffer`. Without this, the system may not deliver sample buffers to your handler. The `@bacons/apple-targets` plugin sets this automatically. diff --git a/plugins/expo/skills/apple-targets/call-directory.md b/plugins/expo/skills/apple-targets/call-directory.md new file mode 100644 index 0000000..1962319 --- /dev/null +++ b/plugins/expo/skills/apple-targets/call-directory.md @@ -0,0 +1,167 @@ +--- +title: CallKit Call Directory Extension +description: Provides caller identification labels and call-blocking entries to the native Phone app via a database of phone numbers. +version: iOS 10.0+ +--- + +# CallKit Call Directory Extension (`call-directory`) + +A Call Directory extension lets your app supply the system with a database of phone numbers to block or identify. When an incoming call arrives, iOS looks up the number against all enabled Call Directory extensions and displays your label (e.g. "Likely Spam") on the incoming call screen or silently blocks the call. The extension itself never sees live call data -- it simply pre-populates a system-managed database. Users must explicitly enable the extension in Settings > Phone > Call Blocking & Identification before it takes effect. + +## Apple Documentation + +- [CXCallDirectoryProvider](https://developer.apple.com/documentation/callkit/cxcalldirectoryprovider) +- [CXCallDirectoryExtensionContext](https://developer.apple.com/documentation/callkit/cxcalldirectoryextensioncontext) +- [CXCallDirectoryManager](https://developer.apple.com/documentation/callkit/cxcalldirectorymanager) +- [beginRequest(with:)](https://developer.apple.com/documentation/callkit/cxcalldirectoryprovider/1779582-beginrequest) +- [CallKit Framework Overview](https://developer.apple.com/documentation/callkit) + +## WWDC History + +- **[WWDC 2016, Session 230 -- Enhancing VoIP Apps with CallKit](https://developer.apple.com/videos/play/wwdc2016/230/)** -- Introduced CallKit, including the Call Directory extension for caller identification and call blocking. + +## What It Does + +1. **System triggers a reload.** When the user enables the extension, or your app calls `CXCallDirectoryManager.sharedInstance.reloadExtension(withIdentifier:)`, the system launches the extension in the background. +2. **Extension provides phone numbers.** Your `CXCallDirectoryProvider` subclass receives a `beginRequest(with:)` callback. You add blocking entries via `context.addBlockingEntry(withNextSequentialPhoneNumber:)` and identification entries via `context.addIdentificationEntry(withNextSequentialPhoneNumber:label:)`. +3. **Numbers must be strictly ascending.** Every phone number is an `Int64` in E.164 format (country code + number, no punctuation). Each call to `addBlockingEntry` or `addIdentificationEntry` must supply a number greater than the previous one. Violating this causes the entire load to fail silently. +4. **Database is consulted at call time.** When a call arrives from an unknown number, iOS checks the blocking database first (blocking takes priority) and then the identification database. If a match is found, the label is shown on the call screen. +5. **Incremental loading (iOS 11+).** On subsequent reloads, `context.isIncremental` may be `true`. In that case, provide only additions and removals since the last full load, dramatically reducing reload time for large databases. + +## Use Cases + +### Spam and robocall blocking + +A crowd-sourced spam database app downloads known spam numbers from a server and loads them as blocking entries. The main app periodically triggers `reloadExtension` after syncing new data via App Groups shared storage. + +### Business caller identification + +An enterprise communication app labels incoming calls from known company contacts (e.g. "IT Help Desk", "HR Department") so employees can see who is calling before answering, even if the number is not in their personal contacts. + +### Regional telemarketer identification + +A consumer protection app provides caller ID labels for known telemarketing companies in the user's region, showing labels like "Telemarketer" or "Survey" on the incoming call screen. + +## Key Classes + +| Class / Protocol | Role | +|------------------|------| +| `CXCallDirectoryProvider` | Abstract base class for the extension. Subclass this and override `beginRequest(with:)`. | +| `CXCallDirectoryExtensionContext` | Passed to `beginRequest`. Use it to add blocking and identification entries, check `isIncremental`, and call `completeRequest()`. | +| `CXCallDirectoryExtensionContextDelegate` | Delegate protocol for handling errors during the request (e.g. out-of-order numbers). | +| `CXCallDirectoryManager` | Used from the main app to reload the extension, check its enabled status, and open Settings. | +| `CXCallDirectoryPhoneNumber` | Type alias for `Int64`. Represents a phone number in E.164 format. | + +## Implementation + +```swift +import Foundation +import CallKit + +// 1. Subclass CXCallDirectoryProvider -- the system instantiates this class +// when a reload is triggered. +class CallDirectoryHandler: CXCallDirectoryProvider { + + override func beginRequest(with context: CXCallDirectoryExtensionContext) { + context.delegate = self + + // 2. Check whether the system is asking for a full load or an + // incremental update (iOS 11+). + if context.isIncremental { + addOrRemoveIncrementalBlockingPhoneNumbers(to: context) + addOrRemoveIncrementalIdentificationPhoneNumbers(to: context) + } else { + addAllBlockingPhoneNumbers(to: context) + addAllIdentificationPhoneNumbers(to: context) + } + + // 3. Always call completeRequest() when finished. Failing to call + // this will cause the extension to be terminated. + context.completeRequest() + } + + private func addAllBlockingPhoneNumbers(to context: CXCallDirectoryExtensionContext) { + // 4. Load phone numbers from your shared data store (App Group). + // Numbers MUST be sorted in strictly ascending order (E.164 Int64). + let blockedNumbers: [CXCallDirectoryPhoneNumber] = [ + 1_408_555_1234, + 1_800_555_5678, + 1_888_555_9012, + ] + + // 5. For large datasets, use autoreleasepool batches to keep memory + // within the extension's ~6 MB limit. + for number in blockedNumbers { + context.addBlockingEntry(withNextSequentialPhoneNumber: number) + } + } + + private func addAllIdentificationPhoneNumbers(to context: CXCallDirectoryExtensionContext) { + // 6. Identification entries associate a label with a phone number. + // Again, numbers must be in ascending order. + let phoneNumbers: [CXCallDirectoryPhoneNumber] = [ + 1_877_555_1111, + 1_888_555_2222, + ] + let labels = ["Known Telemarketer", "Local Business"] + + for (number, label) in zip(phoneNumbers, labels) { + context.addIdentificationEntry( + withNextSequentialPhoneNumber: number, + label: label + ) + } + } + + private func addOrRemoveIncrementalBlockingPhoneNumbers(to context: CXCallDirectoryExtensionContext) { + // 7. For incremental updates, add new entries and remove stale ones. + // Removals: context.removeBlockingEntry(withPhoneNumber:) + // Additions: context.addBlockingEntry(withNextSequentialPhoneNumber:) + // Track changes using a timestamp in UserDefaults (App Group). + } + + private func addOrRemoveIncrementalIdentificationPhoneNumbers(to context: CXCallDirectoryExtensionContext) { + // 8. Same pattern for identification entries. + // context.removeIdentificationEntry(withPhoneNumber:) + // context.addIdentificationEntry(withNextSequentialPhoneNumber:label:) + } +} + +// 9. Implement the delegate to handle errors during loading. +extension CallDirectoryHandler: CXCallDirectoryExtensionContextDelegate { + func requestFailed(for extensionContext: CXCallDirectoryExtensionContext, withError error: Error) { + // 10. Check CXErrorCodeCallDirectoryManagerError for specific codes: + // .entriesOutOfOrder, .duplicateEntries, .unexpectedIncrementalRemoval, etc. + if let cxError = error as? CXErrorCodeCallDirectoryManagerError { + print("Call Directory request failed: \(cxError.code)") + } + } +} +``` + +## Usage + +- Create: `bun create target call-directory` +- Edit: `targets/call-directory/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 10.0+ | Full support. Incremental loading added in iOS 11. | +| iPadOS | -- | Not applicable (no Phone app on iPad). | +| macOS | -- | Not supported. | +| watchOS | -- | Not supported. | +| visionOS | -- | Not supported. | +| tvOS | -- | Not supported. | + +## Gotchas + +- **Phone numbers must be in strictly ascending order.** Every call to `addBlockingEntry` or `addIdentificationEntry` must supply a number numerically greater than the previous one. Out-of-order entries cause the entire reload to fail with `CXErrorCodeCallDirectoryManagerError.entriesOutOfOrder`, and no error UI is shown to the user. +- **Extension memory limit is approximately 6 MB.** For large databases (hundreds of thousands of numbers), load entries in batches wrapped in `autoreleasepool` blocks. Memory-mapped files are recommended over loading everything into an array. +- **User must manually enable the extension.** Installing the app does not activate the Call Directory extension. The user must go to Settings > Phone > Call Blocking & Identification and toggle your extension on. Use `CXCallDirectoryManager.sharedInstance.getEnabledStatusForExtension` to check and prompt the user. +- **Contacts always take priority over your labels.** If the incoming number matches a contact in the user's address book, the contact name is displayed instead of your identification label. Your blocking entries still apply even if the number is in contacts. +- **Killing the app during reload can cause a stuck state.** If the containing app is terminated while the extension is reloading, the extension can get stuck in a perpetual "loading" state. Subsequent calls to `reloadExtension` return `CXErrorCodeCallDirectoryManagerError.currentlyLoading`. The only recovery is waiting or, in severe cases, the user resetting all settings. +- **Incremental removal requires `isIncremental` check.** Calling `removeBlockingEntry(withPhoneNumber:)` or `removeIdentificationEntry(withPhoneNumber:)` when `context.isIncremental` is `false` causes an `unexpectedIncrementalRemoval` error. Always check the flag before removing entries. +- **App Group is required for data sharing.** The extension runs in a separate process and cannot access the main app's sandbox. Use an App Group container (`UserDefaults(suiteName:)` or shared file container) to pass phone number data from the app to the extension. +- **No real-time call interception.** The extension never receives the actual incoming phone number at call time. It pre-populates a database that the system consults. You cannot implement dynamic, per-call logic. diff --git a/plugins/expo/skills/apple-targets/classkit-context.md b/plugins/expo/skills/apple-targets/classkit-context.md new file mode 100644 index 0000000..c1df6dc --- /dev/null +++ b/plugins/expo/skills/apple-targets/classkit-context.md @@ -0,0 +1,192 @@ +--- +title: ClassKit Context Provider Extension +description: Provides your educational app's activity hierarchy to Apple's Schoolwork app so teachers can assign activities and track student progress. +version: iOS 12.2+ +--- + +# ClassKit Context Provider Extension (`classkit-context`) + +A ClassKit Context Provider extension tells the Schoolwork app about the educational activities inside your app without requiring your app to be launched. Teachers browse your activity tree in Schoolwork, assign specific activities to students, and then students open your app to complete them. Progress (time spent, scores, completion) is automatically reported back through ClassKit's `CLSDataStore`. This extension is used exclusively in Apple's K-12 education ecosystem. + +## Apple Documentation + +- [ClassKit Framework Overview](https://developer.apple.com/documentation/classkit) +- [CLSContextProvider Protocol](https://developer.apple.com/documentation/classkit/clscontextprovider) +- [CLSContext](https://developer.apple.com/documentation/classkit/clscontext) +- [CLSDataStore](https://developer.apple.com/documentation/classkit/clsdatastore) +- [ClassKit Environment Entitlement](https://developer.apple.com/documentation/bundleresources/entitlements/com.apple.developer.classkit-environment) +- [About ClassKit and User Roles](https://developer.apple.com/documentation/classkit/enabling_classkit_in_your_app/about_classkit_and_user_roles) + +## WWDC History + +- **[WWDC 2018, Session 215 -- Introducing ClassKit](https://developer.apple.com/videos/play/wwdc2018/215/)** -- ClassKit framework introduced with iOS 11.4. Covered `CLSContext` trees, `CLSActivity`, and `CLSDataStore` for reporting student progress. The Context Provider extension did not exist yet -- apps had to be launched to declare their context trees. +- **[WWDC 2019, Session 247 -- What's New in ClassKit](https://developer.apple.com/videos/play/wwdc2019/247/)** -- Introduced the **ClassKit Context Provider extension** (iOS 12.2). This session walked through creating the extension target in Xcode, implementing `updateDescendants(of:completion:)`, and marking activities as complete. +- **[WWDC 2020, Session 10672 -- What's New in ClassKit](https://developer.apple.com/videos/play/wwdc2020/10672/)** -- Added `identifierPath` read-only property, new context types (`course`, `custom`), and the ability to mark contexts as non-assignable. +- **[WWDC 2021, Session 10257 -- Meet ClassKit for File-Based Apps](https://developer.apple.com/videos/play/wwdc2021/10257/)** -- Extended ClassKit to document-based apps (Pages, Keynote, etc.) with file-based progress reporting. Also covered testing in developer mode. + +## What It Does + +1. **Teacher opens Schoolwork.** Schoolwork queries installed educational apps for their activity hierarchy. +2. **System loads your extension.** The system calls `updateDescendants(of:completion:)` on your `CLSContextProvider` subclass, passing in a `CLSContext`. The first call passes the root (main app) context, which has no parent. +3. **Extension populates children.** Your extension creates or updates `CLSContext` objects representing content tiers (courses, chapters, quizzes, etc.) and adds them as descendants of the passed-in context. +4. **Teacher assigns an activity.** In Schoolwork, the teacher selects one of your contexts and assigns it to students as a "handout." +5. **Student launches your app.** When the student taps the assignment, your app opens via a deep link. Your app navigates to the matching `CLSContext` using its identifier path. +6. **App reports progress.** As the student works, your app creates a `CLSActivity`, starts it, adds `CLSProgressReportItem` or `CLSScoreItem` objects, stops the activity, and saves via `CLSDataStore.shared.save()`. +7. **Teacher sees results.** Schoolwork aggregates progress data across students and displays completion status, time spent, and scores. + +## Use Cases + +### K-12 Math and Reading Apps +A math practice app exposes a hierarchy like "Grade 5 > Fractions > Adding Fractions > Quiz 1". Teachers assign specific quizzes, and Schoolwork shows each student's score and time spent. + +### Language Learning Apps +A language app publishes lessons organized by level and topic ("Spanish > Beginner > Greetings > Lesson 3"). Teachers can assign specific lessons to different groups of students and track who has completed them. + +### Science and STEM Interactive Labs +A virtual lab app exposes experiments as assignable contexts. Students complete the lab, and the app reports results (pass/fail, completion percentage) back to Schoolwork. + +## Key Classes + +| Class / Protocol | Role | +|------------------|------| +| `CLSContextProvider` | Conform to this protocol (via `NSObject` subclass) in your extension. Override `updateDescendants(of:completion:)` to populate the context tree. | +| `CLSContext` | Represents a single node in your content hierarchy (a course, chapter, quiz, etc.). Has a `title`, `type`, `identifier`, and optional `topic`. | +| `CLSDataStore` | Singleton that manages saving and fetching contexts and activities. Use `CLSDataStore.shared` in both your app and extension. | +| `CLSActivity` | Represents a student's engagement with a context. Start/stop it to track duration. Attach progress or score items. | +| `CLSProgressReportItem` | Reports a 0.0-1.0 progress value for a context. | +| `CLSScoreItem` | Reports a numeric score (e.g., 8 out of 10) for a context. | +| `CLSBinaryItem` | Reports a true/false outcome (e.g., pass/fail). | + +## Implementation + +```swift +import ClassKit + +// 1. Conform to CLSContextProvider. This is the only class in the extension. +class ContextProvider: NSObject, CLSContextProvider { + + // 2. Called by the system when Schoolwork needs your content tree. + // `context` is the parent whose children you should populate. + func updateDescendants( + of context: CLSContext, + completion: @escaping (Error?) -> Void + ) { + // 3. Check if this is the root (main app) context. + // The root context has no parent. + if context.parent == nil { + populateTopLevel(context: context, completion: completion) + } else { + populateChildren(of: context, completion: completion) + } + } + + private func populateTopLevel( + context: CLSContext, + completion: @escaping (Error?) -> Void + ) { + let dataStore = CLSDataStore.shared + + // 4. Create top-level course contexts. + let courses = [ + ("algebra", "Algebra", CLSContextTopic.math), + ("geometry", "Geometry", CLSContextTopic.math), + ("reading", "Reading Comprehension", CLSContextTopic.literacyAndWriting), + ] + + for (id, title, topic) in courses { + // 5. Check if the context already exists to avoid duplicates. + dataStore.mainAppContext.descendant( + matchingIdentifierPath: [id] + ) { existingContext in + if let existing = existingContext { + existing.title = title + existing.topic = topic + } else { + // 6. Create a new child context. + let child = CLSContext( + type: .course, + identifier: id, + title: title + ) + child.topic = topic + context.addChildContext(child) + } + } + } + + // 7. Save and signal completion. + dataStore.save { error in + completion(error) + } + } + + private func populateChildren( + of context: CLSContext, + completion: @escaping (Error?) -> Void + ) { + let dataStore = CLSDataStore.shared + + // 8. Build children based on parent identifier. + // In a real app, fetch this structure from your content database. + let lessons: [(String, String)] = { + switch context.identifier { + case "algebra": + return [ + ("linear-equations", "Linear Equations"), + ("quadratics", "Quadratic Equations"), + ] + case "geometry": + return [ + ("triangles", "Triangles"), + ("circles", "Circles"), + ] + default: + return [] + } + }() + + for (id, title) in lessons { + // 9. Create assignable activity contexts. + let child = CLSContext( + type: .exercise, + identifier: id, + title: title + ) + // 10. Mark as assignable so teachers can hand it out. + child.isAssignable = true + context.addChildContext(child) + } + + dataStore.save { error in + completion(error) + } + } +} +``` + +## Usage + +- Create: `bun create target classkit-context` +- Edit: `targets/classkit-context/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 12.2+ | Context Provider extension introduced in 12.2. ClassKit framework available since 11.4. | +| iPadOS | 12.2+ | Primary platform for Schoolwork. iPad is the dominant device in K-12 Apple deployments. | +| macOS | 11.0+ | ClassKit available via Catalyst. Schoolwork for Mac introduced with macOS 11. | +| watchOS | -- | Not supported. | +| tvOS | -- | Not supported. | +| visionOS | -- | Not supported. | + +## Gotchas + +- **ClassKit entitlement is required.** You must enable the ClassKit capability in Xcode, which adds the `com.apple.developer.ClassKit-environment` entitlement. Set it to `development` for testing and `production` for App Store builds. Without this entitlement, the extension will not load. +- **Testing requires Schoolwork in developer mode.** You need a managed Apple ID (Apple School Manager) and the Schoolwork app to test the full flow. For local development, set the entitlement value to `development` and use `CLSDataStore.shared.completeAllAssignedActivities(matching:)` to simulate completion. +- **The extension only provides the tree structure, not progress.** The context provider extension populates the hierarchy of assignable content. Actual progress reporting (`CLSActivity`, `CLSScoreItem`, etc.) must happen from your main app when the student interacts with the content. +- **`updateDescendants` may be called multiple times.** The system calls your extension whenever Schoolwork needs fresh data. Do not assume it is called only once. Guard against creating duplicate contexts by checking for existing descendants before adding new ones. +- **Context identifiers must be stable.** The `identifier` you give each `CLSContext` is used to match assignments to content. If you change identifiers between app updates, existing assignments will break and teachers will see errors in Schoolwork. +- **Schoolwork adoption is limited to managed education environments.** ClassKit only works with Apple School Manager-managed devices. Consumer users will never see your ClassKit integration. This limits the audience to institutional K-12 deployments. +- **The root context is created automatically.** You do not create the main app context yourself. It is provided by the system as the parentless `CLSContext` passed to `updateDescendants`. Attempting to create a root context manually will cause errors. +- **Context types affect Schoolwork UI.** Use appropriate `CLSContextType` values (`.course`, `.chapter`, `.exercise`, `.quiz`, etc.) because Schoolwork uses these to display different icons and group activities logically for teachers. diff --git a/plugins/expo/skills/apple-targets/clip.md b/plugins/expo/skills/apple-targets/clip.md new file mode 100644 index 0000000..5ea352f --- /dev/null +++ b/plugins/expo/skills/apple-targets/clip.md @@ -0,0 +1,299 @@ +--- +title: App Clip +description: A lightweight, instantly launchable subset of your app triggered by NFC tags, QR codes, App Clip Codes, Safari banners, Maps, and Messages links. +version: iOS 14.0+ +--- + +# App Clip (`clip`) + +An App Clip is a small, focused part of your full app that users can discover and launch instantly without installing anything from the App Store. It uses the product type `com.apple.product-type.application.on-demand-install-capable` and is NOT an extension -- it is a fully functional application target with size constraints. App Clips are triggered by real-world invocations (NFC tags, QR codes, App Clip Codes) and digital invocations (Safari App Clip banners, Messages links, Maps place cards). The system downloads the App Clip on demand, runs it, and automatically removes it after a period of inactivity. App Clips share code and assets with the full app via shared targets, and transfer user data to the full app through App Groups and shared Keychain access groups. + +## Apple Documentation + +- [App Clips Framework Overview](https://developer.apple.com/documentation/appclip) -- top-level framework reference covering the App Clip lifecycle, invocation handling, and available APIs. +- [Creating an App Clip with Xcode](https://developer.apple.com/documentation/appclip/creating-an-app-clip-with-xcode) -- step-by-step guide for adding an App Clip target, sharing code with the full app, and configuring build settings. +- [Configuring App Clip Experiences](https://developer.apple.com/documentation/appclip/configuring-the-launch-experience-of-your-app-clip) -- how to register invocation URLs in App Store Connect, set up App Clip cards, and configure advanced and default experiences. +- [Choosing the Right Functionality for Your App Clip](https://developer.apple.com/documentation/appclip/choosing-the-right-functionality-for-your-app-clip) -- guidance on which frameworks are available, size budgets, and what makes a great App Clip experience. +- [Testing the Launch Experience of Your App Clip](https://developer.apple.com/documentation/appclip/testing-the-launch-experience-of-your-app-clip) -- how to use `_XCAppClipURL`, local experiences in Developer Settings, and TestFlight for testing invocations. +- [Enabling Notifications in App Clips](https://developer.apple.com/documentation/appclip/enabling-notifications-in-app-clips) -- ephemeral notification permissions and how to request them. + +## WWDC History + +- **[WWDC 2020, Session 10174 -- Explore App Clips](https://developer.apple.com/videos/play/wwdc2020/10174/)** -- introduced the App Clip concept, design principles for short and fast interactions, and discovery surfaces (NFC, QR, Safari banners, Maps). +- **[WWDC 2020, Session 10146 -- Configure and Link Your App Clips](https://developer.apple.com/videos/play/wwdc2020/10146/)** -- covered associated domains, invocation URL configuration in App Store Connect, App Clip Codes, and web banner integration. +- **[WWDC 2020, Session 10120 -- Streamline Your App Clip](https://developer.apple.com/videos/play/wwdc2020/10120/)** -- best practices for transaction flows, ephemeral notifications, one-time location confirmation, and migrating data to the full app. +- **[WWDC 2020, Session 10118 -- Create App Clips for Other Businesses](https://developer.apple.com/videos/play/wwdc2020/10118/)** -- advanced App Clip experiences on behalf of third-party businesses, multi-experience configuration, and per-business App Clip cards. +- **[WWDC 2021, Session 10012 -- What's New in App Clips](https://developer.apple.com/videos/play/wwdc2021/10012/)** -- App Clip Codes as AR anchors, local testing improvements, and Safari/SafariViewController App Clip card support. +- **[WWDC 2022, Session 10097 -- What's New in App Clips](https://developer.apple.com/videos/play/wwdc2022/10097/)** -- diagnostics tools, CloudKit public database read access, and size limit increase to 15 MB for physical invocations. +- **[WWDC 2023, Session 10178 -- What's New in App Clips](https://developer.apple.com/videos/play/wwdc2023/10178/)** -- default App Clip links (iOS 16.4+), 50 MB size limit for digital invocations (iOS 17+), and cross-app invocation. + +## What It Does + +1. **User encounters an invocation.** A person taps an NFC tag, scans a QR code or App Clip Code, taps a Safari App Clip banner, opens a Maps place card, or taps a link in Messages. +2. **System shows the App Clip card.** iOS displays a system-provided card with the App Clip name, icon, subtitle, and action button. The card is configured in App Store Connect under App Clip Experiences. +3. **App Clip downloads and launches.** The system downloads the App Clip binary (subject to size limits) and launches it, passing the invocation URL as an `NSUserActivity` with activity type `NSUserActivityTypeBrowsingWeb`. +4. **App Clip parses the URL.** The App Clip extracts path components and query parameters from the invocation URL to determine what experience to show (e.g., which restaurant, which parking meter). +5. **User completes a focused task.** The App Clip presents a streamlined flow -- ordering food, renting a scooter, paying for parking -- using Apple Pay, Sign in with Apple, and other quick-interaction APIs. +6. **Ephemeral notifications fire (optional).** If `NSAppClipRequestEphemeralUserNotification` is enabled and the user granted permission on the App Clip card, the App Clip can send push notifications for up to 8 hours after each launch without prompting. +7. **Location confirmation completes (optional).** If `NSAppClipRequestLocationConfirmation` is enabled, the system shows a one-time confirmation dialog verifying the user is at the expected physical location. +8. **Data migrates to the full app.** When the user installs the full app, shared App Group containers and shared Keychain items make all saved data (credentials, preferences, order history) immediately available. +9. **System removes the App Clip.** After a period of inactivity, iOS automatically deletes the App Clip and its non-shared data. + +## Use Cases + +### Restaurant Ordering +A diner scans an NFC tag embedded in the table. The App Clip launches showing that table's menu, allows the user to customize an order, and completes payment with Apple Pay -- all without installing the full app. + +### Bike and Scooter Rental +A user scans a QR code on a rental scooter. The App Clip authenticates via Sign in with Apple, unlocks the scooter, and starts a ride timer. Ephemeral notifications alert the user about ride duration and cost. + +### Parking Meter Payment +A driver taps an App Clip Code on a parking meter. The App Clip uses one-time location confirmation to verify the correct meter, then processes payment. A notification reminds the driver before time expires. + +### Game Demo +A user taps a link in Messages. The App Clip downloads as a playable demo (up to 50 MB for digital invocations on iOS 17+). Progress and achievements transfer to the full game via shared App Group storage. + +## Key Classes + +| Class | Role | +|-------|------| +| `NSUserActivity` | Carries the invocation URL to the App Clip at launch. Activity type is `NSUserActivityTypeBrowsingWeb`. | +| `CLLocationManager` | Used for one-time location verification when `NSAppClipRequestLocationConfirmation` is enabled. | +| `ASAuthorizationController` | Provides Sign in with Apple for quick, privacy-preserving authentication. | +| `PKPaymentAuthorizationController` | Handles Apple Pay transactions for fast checkout flows. | +| `UNUserNotificationCenter` | Manages ephemeral and standard notification registration. | +| `SKOverlay` | Presents a system-provided banner recommending the full app for download. | +| `ASWebAuthenticationSession` | Handles OAuth/federated sign-in callbacks without registering custom URL schemes. | + +## Implementation + +### SwiftUI App Clip with Invocation URL Handling + +```swift +import SwiftUI +import AppClip +import StoreKit + +// 1. Define the main App Clip entry point using the SwiftUI App lifecycle. +@main +struct OrderingClipApp: App { + @StateObject private var orderManager = OrderManager() + + var body: some Scene { + WindowGroup { + ContentView() + .environmentObject(orderManager) + // 2. Handle the invocation URL when the App Clip launches or + // returns to foreground via a new invocation. + .onContinueUserActivity( + NSUserActivityTypeBrowsingWeb, + perform: handleUserActivity + ) + } + } + + func handleUserActivity(_ userActivity: NSUserActivity) { + // 3. Extract the invocation URL. In Xcode testing, this comes from + // the _XCAppClipURL environment variable. + guard let incomingURL = userActivity.webpageURL, + let components = URLComponents(url: incomingURL, resolvingAgainstBaseURL: true) + else { return } + + // 4. Parse path and query parameters to determine the experience. + // e.g., https://example.com/order?restaurant=42&table=7 + if let restaurantId = components.queryItems?.first(where: { + $0.name == "restaurant" + })?.value { + orderManager.loadRestaurant(id: restaurantId) + } + + if let tableId = components.queryItems?.first(where: { + $0.name == "table" + })?.value { + orderManager.setTable(id: tableId) + } + + // 5. Optionally verify the user's location against the expected + // business location (requires NSAppClipRequestLocationConfirmation). + verifyLocation(for: incomingURL) + } + + func verifyLocation(for url: URL) { + // 6. The system handles the location confirmation dialog automatically + // when the key is set in Info.plist. You can check the result: + guard let payload = try? AppClipActivationPayload(url: url) else { return } + + let region = CLCircularRegion( + center: CLLocationCoordinate2D(latitude: 37.7749, longitude: -122.4194), + radius: 100, + identifier: "restaurant-location" + ) + + Task { + do { + try await payload.confirmAcquired(in: region) + orderManager.locationVerified = true + } catch { + orderManager.locationVerified = false + } + } + } +} + +// 7. The order manager stores state in an App Group so the full app +// can access it after installation. +class OrderManager: ObservableObject { + @Published var restaurantName: String = "" + @Published var tableId: String = "" + @Published var locationVerified = false + @Published var items: [MenuItem] = [] + + private let defaults = UserDefaults( + suiteName: "group.com.example.ordering" + ) + + func loadRestaurant(id: String) { + // Fetch restaurant details from your API. + restaurantName = "Restaurant \(id)" + } + + func setTable(id: String) { + tableId = id + // 8. Persist to shared App Group so the full app can read it. + defaults?.set(id, forKey: "lastTableId") + } + + func saveOrderHistory(_ order: Order) { + // 9. Write order data to the shared container for migration. + guard let url = FileManager.default + .containerURL(forSecurityApplicationGroupIdentifier: "group.com.example.ordering")? + .appendingPathComponent("orders.json") + else { return } + + let encoder = JSONEncoder() + if let data = try? encoder.encode(order) { + try? data.write(to: url, options: .atomic) + } + } +} + +struct MenuItem: Identifiable, Codable { + let id: String + let name: String + let price: Decimal +} + +struct Order: Codable { + let items: [MenuItem] + let total: Decimal + let date: Date +} + +// 10. Main content view with menu, cart, and a banner prompting full app install. +struct ContentView: View { + @EnvironmentObject var orderManager: OrderManager + + var body: some View { + NavigationStack { + VStack { + if orderManager.restaurantName.isEmpty { + ProgressView("Loading...") + } else { + Text(orderManager.restaurantName) + .font(.largeTitle.bold()) + Text("Table \(orderManager.tableId)") + .font(.headline) + .foregroundStyle(.secondary) + + List(orderManager.items) { item in + HStack { + Text(item.name) + Spacer() + Text(item.price, format: .currency(code: "USD")) + } + } + + Button("Pay with Apple Pay") { + // 11. Integrate PKPaymentAuthorizationController here + // for a fast, focused checkout. + } + .buttonStyle(.borderedProminent) + .padding() + } + } + .toolbar { + ToolbarItem(placement: .bottomBar) { + // 12. Show the App Store overlay to encourage full app install. + // The overlay appears as a banner at the bottom of the screen. + AppStoreOverlayButton() + } + } + } + } +} + +struct AppStoreOverlayButton: View { + @State private var showOverlay = false + + var body: some View { + Button("Get the Full App") { + showOverlay = true + } + .appStoreOverlay(isPresented: $showOverlay) { + // 13. Use your full app's Apple ID from App Store Connect. + SKOverlay.AppClipConfiguration(position: .bottom) + } + } +} + +// 14. Register for ephemeral notifications on launch. +// Call this from your App's init or onAppear. +func requestEphemeralNotifications() { + let center = UNUserNotificationCenter.current() + center.getNotificationSettings { settings in + // 15. If ephemeral is already granted (from the App Clip card), + // skip the prompt entirely. + if settings.authorizationStatus == .ephemeral { + print("Ephemeral notifications granted for 8 hours.") + return + } + // 16. Otherwise, request standard authorization as a fallback. + center.requestAuthorization(options: [.alert, .sound]) { granted, error in + if granted { + DispatchQueue.main.async { + UIApplication.shared.registerForRemoteNotifications() + } + } + } + } +} +``` + +## Usage + +- Create: `bun create target clip` +- Edit: `targets/clip/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 14.0+ | Full App Clip support. Size limit: 10 MB (iOS 14-15), 15 MB for physical invocations (iOS 16+), 50 MB for digital invocations (iOS 17+). | +| iPadOS | 14.0+ | Supported. Same size limits as iOS. | +| macOS | -- | Not supported. App Clips are iOS/iPadOS only. | +| watchOS | -- | Not supported. | +| tvOS | -- | Not supported. | +| visionOS | -- | Not supported. | + +## Gotchas + +- **Size limits depend on invocation type and OS version.** Physical invocations (NFC, App Clip Codes) are capped at 15 MB (iOS 16+). Digital invocations (Safari, Messages, Maps) allow up to 50 MB but only on iOS 17+ with a deployment target of iOS 17. The size is measured as the uncompressed binary -- use `app-thinning-size-report.txt` from your App Store archive to verify. +- **App Clips cannot register URL schemes or Universal Links.** They receive invocation URLs through `NSUserActivity`, not through the standard `openURL` flow. Use `ASWebAuthenticationSession` for OAuth callbacks. +- **Ephemeral notifications expire after 8 hours per launch.** Each time the App Clip is opened, the 8-hour window resets. After that, notifications stop unless the user explicitly grants standard notification permission. +- **Data is deleted on inactivity.** The system removes the App Clip and its non-shared data after approximately 30 days of inactivity. Always store important data in a shared App Group container or Keychain access group so the full app can access it. +- **`_XCAppClipURL` does not work in UI tests.** The environment variable correctly passes the invocation URL when running the App Clip scheme normally, but `NSUserActivity` is not delivered during XCUITest execution. Use a separate test-only environment variable as a workaround. +- **Location confirmation is one-time per session.** The `confirmAcquired(in:)` call presents a system dialog once. If the user denies it or the location check fails, you cannot re-prompt in the same session. +- **CloudKit access is read-only.** Since iOS 16, App Clips can read from a CloudKit public database but cannot write to CloudKit or use cloud documents and key-value stores. +- **App Group container must match.** The App Clip's App Group identifier must exactly match the full app's identifier. Mismatches silently prevent data sharing. Configure this in the `expo-target.config.js` entitlements. +- **No background execution modes.** App Clips cannot register for most background modes (background fetch, background processing, etc.). They are designed for foreground-only, short-lived interactions. +- **Parent Application Identifiers entitlement is required.** The App Clip must include the `com.apple.developer.parent-application-identifiers` entitlement pointing to the full app's bundle identifier. The `@bacons/apple-targets` plugin configures this automatically. diff --git a/plugins/expo/skills/apple-targets/content-blocker.md b/plugins/expo/skills/apple-targets/content-blocker.md new file mode 100644 index 0000000..59fcd33 --- /dev/null +++ b/plugins/expo/skills/apple-targets/content-blocker.md @@ -0,0 +1,227 @@ +--- +title: Safari Content Blocker +description: Provides declarative JSON rules to Safari for blocking or hiding web content without runtime code execution. +version: iOS 9.0+ +--- + +# Safari Content Blocker (`content-blocker`) + +A content blocker extension supplies Safari with a JSON array of trigger/action rules that block network requests, hide page elements, or strip cookies -- all without executing any code at page-load time. Safari compiles the JSON rules into efficient bytecode at install time, so blocking decisions happen at native speed with zero information leaking back to the extension about the user's browsing activity. This privacy-by-design architecture means the extension has no knowledge of what sites the user visits. The containing app can update rules dynamically and call `SFContentBlockerManager.reloadContentBlocker(withIdentifier:)` to push changes to Safari. + +## Apple Documentation + +- [Creating a Content Blocker](https://developer.apple.com/documentation/safariservices/creating-a-content-blocker) +- [SFContentBlockerManager](https://developer.apple.com/documentation/safariservices/sfcontentblockermanager) +- [App Extension Programming Guide: Content Blocker](https://developer.apple.com/library/archive/documentation/General/Conceptual/ExtensibilityPG/ContentBlocker.html) +- [Safari Content-Blocking Rules Reference](https://developer.apple.com/documentation/safariservices/creating-a-content-blocker#overview) +- [Blocking Content with Your Safari Web Extension](https://developer.apple.com/documentation/safariservices/blocking-content-with-your-safari-web-extension) + +## WWDC History + +- **[WWDC 2015, Session 511 -- Safari Extensibility: Content Blocking and Shared Links](https://developer.apple.com/videos/play/wwdc2015/511/)** -- Introduced content blockers in iOS 9 and OS X El Capitan. Covered the JSON rule format, trigger/action model, and the privacy-first architecture where Safari compiles rules to bytecode with no runtime callbacks. + +## What It Does + +1. **Extension returns JSON rules.** The system calls your `NSExtensionRequestHandling` implementation (`beginRequest(with:)`), which returns one or more `NSItemProvider` attachments containing JSON rule arrays. +2. **Safari compiles rules to bytecode.** Safari parses the JSON and compiles it into an optimized bytecode representation. This compilation happens once at install time and after each `reloadContentBlocker` call. +3. **Rules are evaluated per resource load.** For every network request on a page, Safari checks the compiled rules against the URL and resource metadata. Matching happens entirely within the Safari/WebKit process -- no IPC to your extension occurs. +4. **Matched actions execute.** Depending on the action type, Safari blocks the network request, hides a DOM element via CSS, or ignores previously matched rules. +5. **No browsing data reaches the extension.** The extension never learns which URLs the user visits, which rules matched, or any page content. This is enforced architecturally, not by policy. +6. **App can update rules.** The containing app can regenerate the JSON (e.g., based on user preferences) and call `SFContentBlockerManager.reloadContentBlocker(withIdentifier:completionHandler:)` to tell Safari to re-fetch and recompile the rules. + +## Use Cases + +### Ad blocking +Block advertising network requests by matching known ad-serving domains in `url-filter` triggers. Use `resource-type: ["image", "script", "raw"]` to target ad-related resources. Combine with `css-display-none` actions to hide ad container elements that loaded before the block rule applied. + +### Privacy and tracker blocking +Block third-party tracking scripts and pixels by using `load-type: ["third-party"]` triggers combined with a list of known tracker domains in `if-domain` or `url-filter`. Strip cookies from specific domains using the `block-cookies` action type. + +### Parental controls and safe browsing +Block access to entire domains by matching broad URL patterns. Use `unless-domain` to allowlist specific sites while blocking everything else. The containing app can present a UI for parents to configure allowed/blocked domains and reload the rules. + +### Reader-friendly experience +Hide distracting page elements (pop-ups, newsletter overlays, sticky headers, autoplay video containers) using `css-display-none` actions with CSS selectors targeting common annoyance patterns. + +## Key Classes + +| Class / Protocol | Role | +|------------------|------| +| `NSExtensionRequestHandling` | Protocol your handler class conforms to. Implement `beginRequest(with:)` to return the JSON rules. | +| `NSExtensionContext` | The context object passed to `beginRequest(with:)`. Call `completeRequest(returningItems:completionHandler:)` with your JSON attachment. | +| `NSItemProvider` | Wraps the JSON file or data as an attachment on the `NSExtensionItem`. | +| `SFContentBlockerManager` | Class in SafariServices used by the containing app to reload rules. Call `reloadContentBlocker(withIdentifier:completionHandler:)`. | + +## Implementation + +```swift +import UIKit +import MobileCoreServices + +// 1. The request handler conforms to NSExtensionRequestHandling. +// Safari calls beginRequest(with:) to fetch the blocking rules. +class ContentBlockerRequestHandler: NSObject, NSExtensionRequestHandling { + + func beginRequest(with context: NSExtensionContext) { + // 2. Load a static JSON file bundled with the extension. + // You can also build the JSON dynamically based on user settings + // stored in a shared App Group container. + let attachment = NSItemProvider( + contentsOf: Bundle.main.url(forResource: "blockerList", withExtension: "json")! + )! + + let item = NSExtensionItem() + item.attachments = [attachment] + + // 3. Return the rules to Safari. Safari compiles them to bytecode. + context.completeRequest(returningItems: [item], completionHandler: nil) + } +} + +// 4. Example: dynamically building rules from user preferences. +class DynamicContentBlockerRequestHandler: NSObject, NSExtensionRequestHandling { + + func beginRequest(with context: NSExtensionContext) { + // 5. Read user preferences from a shared App Group. + let defaults = UserDefaults(suiteName: "group.com.example.myapp") + let blockedDomains = defaults?.stringArray(forKey: "blockedDomains") ?? [] + let hideAds = defaults?.bool(forKey: "hideAds") ?? true + + var rules: [[String: Any]] = [] + + // 6. Generate block rules for each user-specified domain. + for domain in blockedDomains { + rules.append([ + "trigger": ["url-filter": ".*", "if-domain": [domain]], + "action": ["type": "block"] + ]) + } + + // 7. Add a CSS-based rule to hide common ad containers. + if hideAds { + rules.append([ + "trigger": [ + "url-filter": ".*", + "resource-type": ["document"] + ], + "action": [ + "type": "css-display-none", + "selector": ".ad-banner, .ad-container, [id*='google_ads'], [class*='sponsored']" + ] + ]) + } + + // 8. Serialize to JSON and return via NSItemProvider. + guard let jsonData = try? JSONSerialization.data(withJSONObject: rules) else { + context.cancelRequest(withError: NSError( + domain: "ContentBlocker", code: 1, + userInfo: [NSLocalizedDescriptionKey: "Failed to serialize rules"] + )) + return + } + + let tempURL = FileManager.default.temporaryDirectory + .appendingPathComponent("rules.json") + try? jsonData.write(to: tempURL) + + let attachment = NSItemProvider(contentsOf: tempURL)! + let item = NSExtensionItem() + item.attachments = [attachment] + context.completeRequest(returningItems: [item], completionHandler: nil) + } +} +``` + +### JSON Rule Format Reference + +```json +[ + { + "trigger": { + "url-filter": "://ads\\.example\\.com/", + "resource-type": ["script", "image", "raw"], + "load-type": ["third-party"] + }, + "action": { + "type": "block" + } + }, + { + "trigger": { + "url-filter": ".*", + "if-domain": ["news.example.com"] + }, + "action": { + "type": "css-display-none", + "selector": ".newsletter-popup, .cookie-banner" + } + }, + { + "trigger": { + "url-filter": "://tracker\\.example\\.com/" + }, + "action": { + "type": "block-cookies" + } + }, + { + "trigger": { + "url-filter": "://cdn\\.trusted\\.com/", + "load-type": ["first-party"] + }, + "action": { + "type": "ignore-previous-rules" + } + } +] +``` + +### Reloading Rules from the Containing App + +```swift +import SafariServices + +// Call this from your containing app after updating the JSON rules. +func reloadBlockerRules() { + // The identifier must match the bundle identifier of the content blocker extension. + SFContentBlockerManager.reloadContentBlocker( + withIdentifier: "com.example.myapp.content-blocker" + ) { error in + if let error = error { + print("Failed to reload content blocker: \(error.localizedDescription)") + } else { + print("Content blocker rules reloaded successfully.") + } + } +} +``` + +## Usage + +- Create: `bun create target content-blocker` +- Edit: `targets/content-blocker/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 9.0+ | Safari only. Content blockers apply to all Safari tabs and SFSafariViewController instances. | +| iPadOS | 9.0+ | Same as iOS. | +| macOS | 10.11+ (Safari 9+) | Works in Safari. Also applies to other WebKit-based browsers on macOS 10.12+. | +| watchOS | -- | Not supported. | +| visionOS | -- | Not supported. | +| tvOS | -- | Not supported. | + +## Gotchas + +- **Hard limit of 50,000 rules per extension.** WebKit enforces a maximum of 50,000 rules per content blocker extension. If your JSON array exceeds this count, Safari silently drops the excess rules. To support more rules, register multiple content blocker extensions in the same app, each with its own 50,000-rule budget. +- **Rules are compiled once, not evaluated per-request.** Safari compiles JSON rules to bytecode at install/reload time. Changes to the JSON file have no effect until `SFContentBlockerManager.reloadContentBlocker(withIdentifier:)` is called from the containing app, or the user toggles the extension off and on in Settings. +- **No runtime code execution.** Unlike Safari Web Extensions, content blocker extensions cannot run JavaScript, inspect page content, or make decisions at load time. The entire blocking logic must be expressible as static JSON trigger/action pairs. +- **The extension never sees browsing data.** By design, the extension process receives zero information about which sites the user visits or which rules matched. You cannot collect analytics about blocked content. +- **url-filter uses a subset of regular expressions.** The `url-filter` field supports a limited regex syntax (no backreferences, no lookahead/lookbehind). Patterns are matched case-insensitively by default. Use `url-filter-is-case-sensitive: true` to override. +- **css-display-none only hides elements, it does not block network requests.** Using `"type": "css-display-none"` sets `display: none` on matching DOM elements but does not prevent the associated resources from loading. Combine with `block` rules for complete blocking. +- **ignore-previous-rules is order-dependent.** The `ignore-previous-rules` action type causes Safari to discard all previously matched rules for the current resource. Rule order in the JSON array matters. Place allowlist rules after block rules. +- **Large rule sets can crash on iOS 17.** Invoking `reloadContentBlocker` with very large JSON payloads (20 MB+) can cause the content blocker XPC service to crash on iOS 17. Keep individual JSON files reasonably sized and split across multiple extensions if needed. +- **User must explicitly enable the extension.** Content blockers are disabled by default. The user must go to Settings > Safari > Content Blockers (iOS) or Safari > Preferences > Extensions (macOS) to enable your blocker. Your containing app should guide the user to this setting. +- **Simulator requires toggling the extension off and on.** In the iOS Simulator, installing a new build or updating rules often does not take effect until you toggle the content blocker off and back on in Settings > Safari > Extensions. This forces Safari to reload, recompile the rules, and pick up changes. On a physical device `reloadContentBlocker` is usually sufficient, but in the Simulator the toggle is the most reliable way to force a refresh during development. +- **No effect in WKWebView.** Content blocker extensions only apply to Safari and `SFSafariViewController`. Arbitrary `WKWebView` instances in other apps are not affected, though `WKWebView` has its own `WKContentRuleListStore` API for programmatic content rules. diff --git a/plugins/expo/skills/apple-targets/credentials-provider.md b/plugins/expo/skills/apple-targets/credentials-provider.md new file mode 100644 index 0000000..b75d1c7 --- /dev/null +++ b/plugins/expo/skills/apple-targets/credentials-provider.md @@ -0,0 +1,217 @@ +--- +title: AutoFill Credential Provider Extension +description: Integrates a password manager with the system AutoFill UI to provide passwords, passkeys, and verification codes. +version: iOS 12.0+ +--- + +# AutoFill Credential Provider Extension (`credentials-provider`) + +A credential provider extension integrates your password manager or authentication app with the system-wide AutoFill UI, allowing users to fill passwords, passkeys, and one-time codes directly from the QuickType bar or the full credential list during sign-in flows in any app or Safari. The extension subclasses `ASCredentialProviderViewController` from the AuthenticationServices framework and implements methods to present a credential list, provide credentials without user interaction (for QuickType bar autofill), and handle user authentication before releasing sensitive data. Starting with iOS 17, extensions can also create and authenticate with passkeys and supply one-time verification codes, making this the primary integration point for full-featured credential managers on Apple platforms. + +## Apple Documentation + +- [ASCredentialProviderViewController](https://developer.apple.com/documentation/authenticationservices/ascredentialproviderviewcontroller) +- [AutoFill Credential Provider Entitlement](https://developer.apple.com/documentation/bundleresources/entitlements/com.apple.developer.authentication-services.autofill-credential-provider) +- [ASCredentialIdentityStore](https://developer.apple.com/documentation/authenticationservices/ascredentialidentitystore) +- [ASCredentialProviderExtensionContext](https://developer.apple.com/documentation/authenticationservices/ascredentialproviderextensioncontext) +- [ASPasswordCredential](https://developer.apple.com/documentation/authenticationservices/aspasswordcredential) +- [ASPasskeyCredentialRequest](https://developer.apple.com/documentation/authenticationservices/aspasskeycredentialrequest) +- [ASCredentialProviderExtensionCapabilities (Info.plist)](https://developer.apple.com/documentation/bundleresources/information_property_list/nsextension/nsextensionattributes/ascredentialproviderextensioncapabilities) +- [Credential Provider Extensions -- Apple Platform Security](https://support.apple.com/guide/security/credential-provider-extensions-sec6319ac7b9/web) + +## WWDC History + +- **[WWDC 2018, Session 721 -- Implementing AutoFill Credential Provider Extensions](https://developer.apple.com/videos/play/wwdc2018/721/)** -- Introduced the credential provider extension point in iOS 12. Covered `ASCredentialProviderViewController`, the `ASCredentialIdentityStore` for QuickType bar population, and the three-method lifecycle (credential list, no-UI provision, interactive provision). +- **[WWDC 2018, Session 204 -- Automatic Strong Passwords and Security Code AutoFill](https://developer.apple.com/videos/play/wwdc2018/204/)** -- Broader session on Password AutoFill in iOS 12, including how third-party credential providers integrate alongside iCloud Keychain. +- **[WWDC 2021, Session 10106 -- Move Beyond Passwords](https://developer.apple.com/videos/play/wwdc2021/10106/)** -- Introduced passkeys as a concept, laying groundwork for credential provider passkey support. +- **[WWDC 2023, Session 10263 -- Deploy Passkeys at Work](https://developer.apple.com/videos/play/wwdc2023/10263/)** -- Expanded the credential provider API to support passkeys for third-party password managers in iOS 17. Covered `ProvidesPasskeys` capability, managed Apple IDs with iCloud Keychain, and enterprise attestation. +- **[WWDC 2024, Session 10125 -- Streamline Sign-In with Passkey Upgrades and Credential Managers](https://developer.apple.com/videos/play/wwdc2024/10125/)** -- Added automatic passkey upgrades, one-time code filling, and the ability to fill credentials into any text field. + +## What It Does + +1. **User selects your provider in Settings.** The user goes to Settings > Passwords > Password Options (iOS 18+) or Settings > Passwords & Accounts > AutoFill Passwords (earlier iOS) and enables your app as a credential provider alongside or instead of iCloud Keychain. +2. **QuickType bar shows suggestions.** When the user taps a username or password field, iOS checks the `ASCredentialIdentityStore` for identities your extension previously registered. Matching credentials appear on the QuickType bar. +3. **No-UI provision for QuickType.** If the user taps a QuickType suggestion, iOS calls `provideCredentialWithoutUserInteraction(for:)`. If your credential store is unlocked, return the credential immediately. If it requires authentication, throw `ASExtensionError.userInteractionRequired`. +4. **Interactive credential list.** If the user taps the key icon or your extension requires authentication, iOS presents your view controller. Override `prepareCredentialList(for:)` to show a searchable list of credentials filtered by the requesting service identifiers. +5. **User selects a credential.** After optional authentication (biometrics, master password), call `extensionContext.completeRequest(withSelectedCredential:completionHandler:)` with the `ASPasswordCredential`. +6. **Passkey flows (iOS 17+).** For passkey authentication, override `provideCredentialWithoutUserInteraction(for:)` accepting `ASPasskeyCredentialRequest`, perform the WebAuthn assertion, and return an `ASPasskeyAssertionCredential`. For passkey registration, override `prepareInterface(forPasskeyRegistration:)`. +7. **Identity store keeps QuickType current.** Call `ASCredentialIdentityStore.shared.saveCredentialIdentities(_:)` whenever your credential database changes, so the QuickType bar always reflects the latest saved logins. + +## Use Cases + +### Third-party password manager +A password manager app (like 1Password or Bitwarden) registers all stored credentials in the `ASCredentialIdentityStore` so they appear in the QuickType bar. The extension authenticates the user with biometrics or a master password before releasing credentials. + +### Enterprise single sign-on +A corporate identity provider extension that integrates with an organization's identity service. When employees sign in to internal apps, the extension provides credentials from the enterprise directory, enforcing company security policies before filling. + +### Passkey manager +A cross-platform credential manager that stores passkeys alongside passwords. On iOS 17+, the extension creates passkeys during registration flows and performs WebAuthn assertions during sign-in, enabling passwordless authentication across all apps and Safari. + +### One-time code provider +Starting with iOS 18, a credential manager can provide time-based one-time passwords (TOTP codes) during two-factor authentication flows. The extension surfaces the current code in the QuickType bar without the user needing to open the authenticator app. + +## Key Classes + +| Class / Protocol | Role | +|------------------|------| +| `ASCredentialProviderViewController` | The view controller subclass your extension provides. Override its lifecycle methods to present UI and return credentials. | +| `ASCredentialIdentityStore` | Singleton store where you register `ASPasswordCredentialIdentity` and `ASPasskeyCredentialIdentity` objects. These populate the QuickType bar. | +| `ASPasswordCredentialIdentity` | Describes a stored password credential (service identifier, user, record identifier). Registered in the identity store. | +| `ASPasswordCredential` | The actual username + password pair returned to the system when the user selects a credential. | +| `ASPasskeyCredentialRequest` | (iOS 17+) Describes a passkey assertion request, including the relying party and challenge. | +| `ASPasskeyAssertionCredential` | (iOS 17+) The signed WebAuthn assertion response returned to the system for passkey authentication. | +| `ASCredentialServiceIdentifier` | Identifies the service (domain or bundle ID) the user is signing in to. Passed to your extension for filtering. | +| `ASCredentialProviderExtensionContext` | The extension context. Use `completeRequest(withSelectedCredential:)` to return credentials or `cancelRequest(withError:)` to abort. | + +## Implementation + +```swift +import AuthenticationServices + +// 1. Subclass ASCredentialProviderViewController -- the system instantiates this +// when the user interacts with AutoFill and selects your provider. +class CredentialProviderViewController: ASCredentialProviderViewController { + + // 2. Called when the user taps a QuickType bar suggestion. + // Provide the credential immediately if the store is unlocked, + // otherwise signal that user interaction is required. + override func provideCredentialWithoutUserInteraction( + for credentialIdentity: ASPasswordCredentialIdentity + ) { + guard let store = CredentialStore.shared.unlocked else { + // 3. Store is locked -- tell the system we need to show UI. + self.extensionContext.cancelRequest( + withError: NSError( + domain: ASExtensionErrorDomain, + code: ASExtensionError.userInteractionRequired.rawValue + ) + ) + return + } + + // 4. Look up the credential by its record identifier. + guard let entry = store.findEntry( + byRecordIdentifier: credentialIdentity.recordIdentifier ?? "" + ) else { + self.extensionContext.cancelRequest( + withError: NSError( + domain: ASExtensionErrorDomain, + code: ASExtensionError.credentialIdentityNotFound.rawValue + ) + ) + return + } + + // 5. Return the credential to AutoFill. + let credential = ASPasswordCredential(user: entry.username, password: entry.password) + self.extensionContext.completeRequest( + withSelectedCredential: credential, + completionHandler: nil + ) + } + + // 6. Called when the system presents your full credential list UI. + // The serviceIdentifiers describe the app or website the user is signing in to. + override func prepareCredentialList( + for serviceIdentifiers: [ASCredentialServiceIdentifier] + ) { + // 7. Build your credential list UI. Filter and sort credentials + // by matching against the service identifiers. + let matchingCredentials = CredentialStore.shared.credentials( + matching: serviceIdentifiers + ) + + // 8. Present a table/list view. When the user selects a credential + // (after optional biometric/master password auth), complete the request. + let listVC = CredentialListViewController( + credentials: matchingCredentials, + onSelect: { [weak self] entry in + let credential = ASPasswordCredential( + user: entry.username, + password: entry.password + ) + self?.extensionContext.completeRequest( + withSelectedCredential: credential, + completionHandler: nil + ) + }, + onCancel: { [weak self] in + self?.extensionContext.cancelRequest( + withError: NSError( + domain: ASExtensionErrorDomain, + code: ASExtensionError.userCanceled.rawValue + ) + ) + } + ) + // 9. Push or present your list view controller. + self.addChild(listVC) + self.view.addSubview(listVC.view) + listVC.view.frame = self.view.bounds + listVC.didMove(toParent: self) + } + + // 10. Called when provideCredentialWithoutUserInteraction fails with + // userInteractionRequired. Show authentication UI, then provide the credential. + override func prepareInterfaceToProvideCredential( + for credentialIdentity: ASPasswordCredentialIdentity + ) { + // Show biometric prompt or master password entry. + // On success, look up the credential and call completeRequest. + } +} + +// 11. Populate the identity store so credentials appear in the QuickType bar. +// Call this from your containing app whenever the credential database changes. +func updateCredentialIdentityStore(entries: [CredentialEntry]) { + let identities = entries.map { entry in + ASPasswordCredentialIdentity( + serviceIdentifier: ASCredentialServiceIdentifier( + identifier: entry.domain, + type: .domain + ), + user: entry.username, + recordIdentifier: entry.id + ) + } + + // 12. Replace all identities in the store with the current set. + ASCredentialIdentityStore.shared.replaceCredentialIdentities( + with: identities + ) { success, error in + if let error = error { + print("Failed to update identity store: \(error)") + } + } +} +``` + +## Usage + +- Create: `bun create target credentials-provider` +- Edit: `targets/credentials-provider/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 12.0+ | Password AutoFill. Passkeys in 17.0+. One-time codes in 18.0+. | +| iPadOS | 12.0+ | Same as iOS. | +| macOS | 11.0+ | Supported via AuthenticationServices. Some developers report code-signing entitlement issues on macOS. | +| watchOS | -- | Not supported. | +| visionOS | 1.0+ | Supported for password and passkey AutoFill. | +| tvOS | -- | Not supported. | + +## Gotchas + +- **AutoFill Credential Provider entitlement is required.** Both the extension and the containing app must include the `com.apple.developer.authentication-services.autofill-credential-provider` entitlement. Without it, the extension will not appear in Settings and AutoFill will not invoke it. +- **User must explicitly enable your provider in Settings.** Your extension is disabled by default. The user must go to Settings > Passwords > Password Options and enable your app. Guide the user to this screen from your containing app, as there is no API to deep-link directly to it. +- **Data sharing requires App Groups or Shared Keychain.** The extension runs in a separate process. Use App Groups (`UserDefaults(suiteName:)` or shared file containers) or Shared Keychain Access Groups to share the credential database between the app and extension. +- **provideCredentialWithoutUserInteraction must be fast.** This method is called inline during the QuickType bar flow. If it takes too long or shows UI, the system cancels the request. Return the credential immediately or throw `userInteractionRequired` to fall back to the interactive flow. +- **Identity store must be kept in sync.** The QuickType bar only shows credentials that have been registered in `ASCredentialIdentityStore`. If the user adds or deletes credentials in your app, you must update the store immediately. Stale identities lead to a broken user experience where tapping a QuickType suggestion fails to find the credential. +- **Passkey support requires Info.plist capabilities (iOS 17+).** To provide passkeys, add `ASCredentialProviderExtensionCapabilities` to your extension's Info.plist with `ProvidesPasskeys` set to `true`. Without this key, passkey requests are never routed to your extension. +- **One-time code support requires additional capability (iOS 18+).** To provide TOTP codes, add `ProvidesOneTimeCodes` to `ASCredentialProviderExtensionCapabilities`. This feature was introduced in iOS 18. +- **No direct biometric API in the extension.** While you can use `LAContext` for Local Authentication within the extension, the UI presentation context can be tricky. Test biometric prompts thoroughly on real devices, as Simulator behavior differs. +- **Extension has limited memory and runtime.** Like all app extensions, credential providers run with constrained resources. Avoid loading the entire credential database into memory. Use efficient queries against your encrypted store. +- **iCloud Keychain is always an option alongside your provider.** Users can enable multiple credential providers simultaneously. Your extension competes with iCloud Keychain and other providers for QuickType bar space. Ensure your identities have accurate service identifiers to rank well. diff --git a/plugins/expo/skills/apple-targets/device-activity-monitor.md b/plugins/expo/skills/apple-targets/device-activity-monitor.md new file mode 100644 index 0000000..39dc484 --- /dev/null +++ b/plugins/expo/skills/apple-targets/device-activity-monitor.md @@ -0,0 +1,156 @@ +--- +title: Device Activity Monitor Extension +description: Runs code in response to device activity schedule intervals starting or ending and usage thresholds being reached, enabling parental controls and digital wellbeing features without a visible UI. +version: iOS 15.0+ +--- + +# Device Activity Monitor Extension (`device-activity-monitor`) + +A Device Activity Monitor extension executes code when a `DeviceActivitySchedule` interval starts or ends, or when a monitored app or category reaches a usage threshold. It is one of three extension types in Apple's Screen Time API suite (alongside Shield Action and Shield Configuration) and requires the Family Controls entitlement. The extension runs entirely in the background with no user interface -- it typically responds to events by updating `ManagedSettingsStore` shields, writing to shared storage via App Groups, or triggering notifications. The host app schedules monitoring via `DeviceActivityCenter`, and the system invokes the extension's callbacks even when the app is not running. + +## Apple Documentation + +- [DeviceActivity Framework Overview](https://developer.apple.com/documentation/deviceactivity) +- [DeviceActivityMonitor](https://developer.apple.com/documentation/deviceactivity/deviceactivitymonitor) +- [DeviceActivityCenter](https://developer.apple.com/documentation/deviceactivity/deviceactivitycenter) +- [DeviceActivitySchedule](https://developer.apple.com/documentation/deviceactivity/deviceactivityschedule) +- [DeviceActivityEvent](https://developer.apple.com/documentation/deviceactivity/deviceactivityevent) +- [FamilyControls Framework](https://developer.apple.com/documentation/familycontrols) +- [ManagedSettings Framework](https://developer.apple.com/documentation/managedsettings) + +## WWDC History + +- **[WWDC 2021, Session 10123 -- Meet the Screen Time API](https://developer.apple.com/videos/play/wwdc2021/10123/)** -- Introduced the three Screen Time frameworks (FamilyControls, ManagedSettings, DeviceActivity) and the Device Activity Monitor extension for schedule-based and threshold-based callbacks. +- **[WWDC 2022, Session 110336 -- What's New in Screen Time API](https://developer.apple.com/videos/play/wwdc2022/110336/)** -- Added individual (non-child) authorization via `AuthorizationCenter.shared.requestAuthorization(for: .individual)`, enabling digital wellbeing apps for adults. Renamed `ShieldConfigurationProvider` to `ShieldConfigurationDataSource` and `ShieldActionHandler` to `ShieldActionDelegate`. + +## What It Does + +1. **Host app schedules monitoring.** Your app calls `DeviceActivityCenter().startMonitoring(_:during:events:)` with a named activity, a `DeviceActivitySchedule` defining the time window, and optional `DeviceActivityEvent` entries that specify usage thresholds for specific apps or categories. +2. **System launches the extension.** When the schedule interval begins, the system launches your extension process and calls `intervalDidStart(for:)`. +3. **Threshold callbacks fire.** If a monitored app or category accumulates usage that hits an event threshold, the system calls `eventDidReachThreshold(_:activity:)`. +4. **Interval ends.** When the schedule's end time arrives, the system calls `intervalDidEnd(for:)`. +5. **Warning callbacks (optional).** If the schedule includes a `warningTime`, the system calls `intervalWillStartWarning(for:)` or `intervalWillEndWarning(for:)` shortly before the boundary. Similarly, `eventWillReachThresholdWarning` fires before a threshold is hit. +6. **Extension responds.** The extension typically updates a `ManagedSettingsStore` to apply or remove shields, writes state to App Group shared storage, or posts a local notification. + +## Use Cases + +### Parental controls + +A guardian authorizes the app via `AuthorizationCenter.shared.requestAuthorization(for: .child)`. The app lets the guardian pick apps and time limits using `FamilyActivityPicker`. A schedule is set for school hours, and when the interval starts the extension shields the selected apps via `ManagedSettingsStore`. When the interval ends, the extension clears the shields. + +### Digital wellbeing for adults + +Using individual authorization (iOS 16+), a user sets their own focus schedule. The extension monitors social media usage and applies a shield when cumulative usage crosses a 30-minute threshold, encouraging the user to take a break. + +### Enterprise device management + +A corporate MDM companion app schedules work-hour monitoring. During work hours the extension shields entertainment apps. Outside work hours, shields are lifted automatically via `intervalDidEnd`. + +## Key Classes + +| Class / Protocol | Role | +|------------------|------| +| `DeviceActivityMonitor` | Base class for the extension's principal class. Override its callback methods to respond to schedule and threshold events. | +| `DeviceActivityCenter` | Used by the host app to start and stop monitoring named activities. | +| `DeviceActivitySchedule` | Defines a repeating time window (start `DateComponents`, end `DateComponents`, optional `warningTime`). | +| `DeviceActivityEvent` | Pairs an `FamilyActivitySelection` (apps/categories) with a usage threshold `DateComponents`. | +| `DeviceActivityName` | A `RawRepresentable` that uniquely identifies a monitored activity. | +| `DeviceActivityEvent.Name` | Identifies a specific threshold event within an activity. | +| `ManagedSettingsStore` | Reads and writes managed settings (shields, restrictions). Used inside the extension to apply or clear shields. | +| `AuthorizationCenter` | Requests Family Controls authorization (`.child` or `.individual`). | + +## Implementation + +```swift +import DeviceActivity +import ManagedSettings + +// 1. Subclass DeviceActivityMonitor. The class name must match the +// NSExtensionPrincipalClass value in Info.plist. +class DeviceActivityMonitorExtension: DeviceActivityMonitor { + + let store = ManagedSettingsStore() + + // 2. Called when the scheduled interval begins (e.g., "school hours start"). + // Apply shields to the apps the guardian selected. + override func intervalDidStart(for activity: DeviceActivityName) { + super.intervalDidStart(for: activity) + + let selection = SharedDataStore.loadSelection() + let appTokens = selection.applicationTokens + let categoryTokens = selection.categoryTokens + + store.shield.applications = appTokens.isEmpty ? nil : appTokens + store.shield.applicationCategories = categoryTokens.isEmpty + ? nil + : .specific(categoryTokens) + } + + // 3. Called when the scheduled interval ends. Remove all shields. + override func intervalDidEnd(for activity: DeviceActivityName) { + super.intervalDidEnd(for: activity) + + store.shield.applications = nil + store.shield.applicationCategories = nil + store.clearAllSettings() + } + + // 4. Called when cumulative usage for a monitored event crosses its threshold. + // For example, "30 minutes of social media" reached. + override func eventDidReachThreshold( + _ event: DeviceActivityEvent.Name, + activity: DeviceActivityName + ) { + super.eventDidReachThreshold(event, activity: activity) + + // Apply shields to all apps in the selection when the threshold hits. + let selection = SharedDataStore.loadSelection() + store.shield.applications = selection.applicationTokens.isEmpty + ? nil + : selection.applicationTokens + } + + // 5. Optional: warn the user a few minutes before the interval ends. + override func intervalWillEndWarning(for activity: DeviceActivityName) { + super.intervalWillEndWarning(for: activity) + // Post a local notification, update shared state, etc. + } + + // 6. Optional: warn before a threshold is reached. + override func eventWillReachThresholdWarning( + _ event: DeviceActivityEvent.Name, + activity: DeviceActivityName + ) { + super.eventWillReachThresholdWarning(event, activity: activity) + } +} +``` + +## Usage + +- Create: `bun create target device-activity-monitor` +- Edit: `targets/device-activity-monitor/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 15.0+ | Full support. Individual authorization added in 16.0+. | +| iPadOS | 15.0+ | Same as iOS. | +| macOS | -- | Not supported. Screen Time API is iOS/iPadOS only. | +| watchOS | -- | Not supported. | +| visionOS | -- | Not supported. | +| tvOS | -- | Not supported. | + +## Gotchas + +- **Hard 6 MB memory limit.** The extension process is capped at approximately 5-6 MB of memory. Exceeding this causes immediate termination (Jetsam crash). Avoid loading images, large data structures, or heavyweight frameworks. Keep work minimal and write results to disk. +- **No UI of any kind.** The extension cannot present views, alerts, or SwiftUI content. It runs headless. If you need to show UI in response to an event, update `ManagedSettingsStore` shields (which the system renders) or write data that a Shield Configuration extension reads. +- **Minimum 15-minute schedule intervals.** The `DeviceActivitySchedule` requires at least 15 minutes between start and end. Shorter intervals are silently ignored. This makes rapid iteration during development tedious. +- **Must test on a physical device.** The iOS Simulator does not track device activity or fire schedule callbacks. You need a real device with a valid Apple ID. +- **`print()` and breakpoints do not work by default.** The extension runs in a separate process. To debug, attach to the extension process explicitly in Xcode (Debug > Attach to Process) and check the Console app for log output. +- **App Group is required for shared data.** The extension and host app run in different sandboxes. Use `UserDefaults(suiteName:)` or a shared App Group container to exchange data such as `FamilyActivitySelection` tokens. +- **Family Controls entitlement requires Apple approval.** You must request the `com.apple.developer.family-controls` entitlement through Apple's developer portal. Without approval, the entitlement works only in development builds and cannot be distributed via TestFlight or the App Store. +- **`intervalDidEnd` is not guaranteed to fire on time.** If the device is locked, in standby, or powered off, the callback fires when the device wakes. Do not rely on exact timing for critical logic. +- **Tokens are opaque and process-bound.** `ApplicationToken` and `ActivityCategoryToken` values from `FamilyActivitySelection` are opaque. They cannot be serialized into human-readable app names. Share them between app and extension via `Codable` in App Group storage. +- **Only one schedule per `DeviceActivityName`.** Calling `startMonitoring` with the same name replaces the previous schedule. Use distinct names if you need concurrent schedules (e.g., `"school-hours"` and `"bedtime"`). diff --git a/plugins/expo/skills/apple-targets/entitlements/README.md b/plugins/expo/skills/apple-targets/entitlements/README.md new file mode 100644 index 0000000..8881be9 --- /dev/null +++ b/plugins/expo/skills/apple-targets/entitlements/README.md @@ -0,0 +1,50 @@ +# Entitlements & Capabilities + +This directory contains skill documents for iOS entitlements and capabilities that apply across multiple extension types. + +Unlike the extension-specific skills in the parent directory (which focus on a single target type like `widget` or `share`), these guides cover cross-cutting entitlements and frameworks that are used by many different extensions. + +## Available Guides + +### [app-groups.md](app-groups.md) +Comprehensive guide to the App Groups entitlement (`com.apple.security.application-groups`), which enables data sharing between your main app, extensions, and App Clips via shared containers and UserDefaults. + +**Read this when:** +- Setting up widgets that need to display data from the main app +- Building share extensions that save content for the main app to process +- Creating notification service extensions that decrypt messages using shared keychain items +- Implementing App Clips that migrate user data to the full app after installation +- Debugging "container URL returns nil" or "UserDefaults not syncing" issues +- Understanding how EAS Build automatically provisions App Group identifiers + +**Topics covered:** +- Standard naming conventions (`group.`) +- Using `UserDefaults(suiteName:)` for shared preferences +- Accessing shared file containers with `FileManager.containerURL(forSecurityApplicationGroupIdentifier:)` +- Sharing Keychain items with `kSecAttrAccessGroup` +- Automatic syncing for extensions with `appGroupsByDefault: true` +- EAS Build automatic code signing and provisioning +- Common gotchas and debugging techniques + +## Future Guides + +Additional entitlements that may be documented in this directory: + +- **Keychain Access Groups** — Sharing credentials across apps and extensions +- **Associated Domains** — Universal Links, App Clips, handoff, and web credentials +- **Push Notifications** — APNs entitlement and notification capabilities +- **HealthKit** — Accessing health and fitness data from extensions +- **iCloud** — CloudKit, Key-Value Storage, and Documents syncing +- **Network Extensions** — VPN and content filtering entitlements +- **Siri** — SiriKit intents and Shortcuts capabilities +- **Background Modes** — Background fetch, remote notifications, location updates + +## Contributing + +When adding a new entitlements guide: + +1. Use the extension skill template structure (frontmatter, Apple docs, WWDC history, implementation examples) +2. Focus on practical usage across multiple extension types rather than a single API +3. Include EAS-specific guidance for automatic provisioning where applicable +4. Add debugging sections for common entitlement validation failures +5. Update this README and the main `SKILL.md` with a reference to the new guide diff --git a/plugins/expo/skills/apple-targets/entitlements/app-groups.md b/plugins/expo/skills/apple-targets/entitlements/app-groups.md new file mode 100644 index 0000000..9370ec9 --- /dev/null +++ b/plugins/expo/skills/apple-targets/entitlements/app-groups.md @@ -0,0 +1,601 @@ +--- +title: App Groups Entitlement +description: Share data between your main app, extensions, and App Clips using shared containers and UserDefaults. +version: iOS 8.0+, macOS 10.10+ +--- + +# App Groups Entitlement + +App Groups enable multiple application targets within the same developer account to share data via a common container directory and shared UserDefaults. The entitlement grants read/write access to a shared file system location and keychain access group, allowing your main app, extensions (widgets, share extensions, notification service extensions, etc.), and App Clips to exchange files, preferences, databases, and credentials. + +## Apple Documentation + +- [App Groups Entitlement](https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_security_application-groups) -- official entitlement reference covering configuration and usage. +- [Sharing Data with Your Containing App](https://developer.apple.com/library/archive/documentation/General/Conceptual/ExtensibilityPG/ExtensionScenarios.html#//apple_ref/doc/uid/TP40014214-CH21-SW6) -- Apple's guide to using App Groups for extension-to-app communication. +- [FileManager.containerURL(forSecurityApplicationGroupIdentifier:)](https://developer.apple.com/documentation/foundation/filemanager/1412643-containerurl) -- returns the shared container directory for a given group identifier. +- [UserDefaults(suiteName:)](https://developer.apple.com/documentation/foundation/userdefaults/1409957-init) -- creates a UserDefaults instance backed by a shared App Group. +- [Keychain Services](https://developer.apple.com/documentation/security/keychain_services) -- with `kSecAttrAccessGroup`, allows keychain items to be shared across targets. + +## WWDC History + +- **[WWDC 2014, Session 205 -- Creating Extensions for iOS and OS X, Part 1](https://developer.apple.com/videos/play/wwdc2014/205/)** -- introduced app extensions and the App Groups entitlement for sharing data between the extension and containing app. +- **[WWDC 2014, Session 217 -- Creating Extensions for iOS and OS X, Part 2](https://developer.apple.com/videos/play/wwdc2014/217/)** -- covered data flow patterns using App Groups with `NSUserDefaults(suiteName:)` and shared file containers. +- **[WWDC 2015, Session 224 -- App Extension Best Practices](https://developer.apple.com/videos/play/wwdc2015/224/)** -- best practices for managing shared data, file coordination, and avoiding conflicts when multiple processes access the same App Group container. +- **[WWDC 2020, Session 10174 -- Explore App Clips](https://developer.apple.com/videos/play/wwdc2020/10174/)** -- demonstrated using App Groups to migrate data from an App Clip to the full app after installation. + +## Extension Storage in @bacons/apple-targets + +The `@bacons/apple-targets` plugin uses **Continuous Native Generation (CNG)** to keep extension source code **outside** the generated `ios/` directory. This allows you to version control your Swift files without checking in the entire Xcode project. + +**Directory structure:** +``` +your-expo-app/ + targets/ # Extension targets live here + widgets/ # Widget extension + expo-target.config.js # Configuration (type, entitlements, frameworks) + WidgetsBundle.swift # Your Swift code + widgets.entitlements # Generated entitlements file + share/ # Share extension + expo-target.config.js + ShareViewController.swift + share.entitlements + ios/ # Generated by expo prebuild --clean + YourApp.xcodeproj # Xcode project with linked targets + YourApp/ # Main app + YourApp.entitlements + app.json # Expo config with main app entitlements +``` + +**Key points:** +- **Extension source files** live in `targets//` and are version-controlled (never in `ios/`). +- **Xcode project** links to files in `targets/` directories — Swift code changes are automatically synchronized without needing to re-run prebuild. +- **Entitlements** are defined in `expo-target.config.js` and synced to `.entitlements` files when you run prebuild after config changes. +- **App Groups** are automatically synced from the main app (`app.json`) to extensions that have `appGroupsByDefault: true`. +- **Never edit files in `ios/`** — the directory is regenerated on prebuild. All customization happens in `app.json` and `targets/`. + +**How it works:** +1. You create extensions with `bunx create-target `, which scaffolds `targets//`. +2. You write Swift code and configure entitlements in `expo-target.config.js`. +3. You run `bunx expo prebuild --clean`, which generates the Xcode project and links your targets. +4. The plugin reads your `app.json` entitlements and syncs them to extensions that need them. +5. EAS Build (or `bunx expo run:ios`) uses the generated project and entitlements for signing. + +## What It Does + +1. **Developer configures the entitlement in app.json.** You add `com.apple.security.application-groups` to `expo.ios.entitlements` with group identifiers (e.g., `["group.com.example.myapp"]`). +2. **Prebuild generates entitlements files.** Running `bunx expo prebuild` creates `.entitlements` XML files for the main app and each extension in the `ios/` and `targets/` directories. +3. **System provisions the entitlement.** When EAS Build (or local build) signs the app, the provisioning profile validates that your developer account owns the App Group identifiers and grants access to the shared containers. +4. **Shared container is created.** On first launch, the system creates a directory at `~/Library/Group Containers//` (iOS/macOS) accessible to all targets sharing that group. +5. **Apps and extensions read/write.** Any target with the entitlement can use `FileManager.containerURL(forSecurityApplicationGroupIdentifier:)` to locate the directory and `UserDefaults(suiteName:)` to access shared preferences. +6. **Data persists across uninstall.** Unlike the main app's container, the App Group container is **not** deleted when the main app is uninstalled if another app on the device still uses the same group. When the last app using the group is removed, the system deletes the container. +7. **Keychain sharing.** Using `kSecAttrAccessGroup` in keychain queries, you can store credentials (passwords, tokens) in a shared keychain access group matching the App Group identifier, accessible from all targets. + +## Standard Naming Format + +Apple requires App Group identifiers to start with `group.` followed by a reverse-DNS-style identifier matching your team's conventions: + +``` +group. +``` + +### Examples + +| App Bundle ID | Recommended App Group ID | Notes | +|---------------|-------------------------|-------| +| `com.example.myapp` | `group.com.example.myapp` | Simplest convention: same as bundle ID with `group.` prefix | +| `com.example.myapp` | `group.com.example` | Shared across multiple apps from the same developer | +| `com.acme.fitness` | `group.com.acme.fitness.shared` | Explicitly named for multi-target sharing | +| `com.company.prod.ios` | `group.com.company.prod` | Production environment-specific group | + +**Best practice:** Use `group.` for single-app projects. For multi-app suites (e.g., a family of fitness apps), use a shared prefix like `group.com.company` so all apps can access the same data. + +## Common Use Cases + +### Widget Extension Sharing Data with Main App + +A weather app stores the current forecast in a shared UserDefaults so the widget can display it without making a separate network request. When the app fetches new data, it writes to the App Group and calls `WidgetCenter.shared.reloadTimelines(ofKind:)` to trigger a widget refresh. + +### Share Extension Saving Content + +A read-later app uses a share extension to receive URLs from Safari. The extension writes the URL and page title to a JSON file in the shared container, then enqueues a background upload task. The main app reads the file on next launch and imports the saved articles. + +### Notification Service Extension Decrypting Messages + +A messaging app encrypts push notification payloads end-to-end. The encryption keys are stored in the shared keychain using `kSecAttrAccessGroup`. The notification service extension retrieves the key from the keychain, decrypts the payload, and replaces the notification content before display. + +### App Clip Migrating User Data + +A restaurant ordering App Clip saves the user's favorite items and authentication token to the shared App Group container. When the user installs the full app, the app reads the container on first launch and imports the saved data, providing a seamless transition. + +## How to Use App Groups + +### 1. Configure the Entitlement in Expo Config + +Add the App Groups entitlement to your Expo app configuration. This applies to the main app target. + +```json +{ + "expo": { + "ios": { + "bundleIdentifier": "com.example.myapp", + "entitlements": { + "com.apple.security.application-groups": [ + "group.com.example.myapp" + ] + } + }, + "plugins": [ + ["@bacons/apple-targets"] + ] + } +} +``` + +### 2. Automatic Syncing for Extensions (Default Behavior) + +For extension types that commonly need App Groups (`widget`, `share`, `clip`, `bg-download`), the `@bacons/apple-targets` plugin **automatically syncs** the App Group identifiers from the main app to the extension **unless you override them manually**. + +**Automatic syncing applies to these types:** +- `widget` (WidgetKit extensions) +- `share` (Share extensions) +- `clip` (App Clips) +- `bg-download` (Background Download extensions) + +The plugin checks the `appGroupsByDefault` flag in the target registry (see `packages/apple-targets/src/target.ts`). If `true` and you haven't defined `entitlements["com.apple.security.application-groups"]` in your `expo-target.config.js`, the extension inherits the main app's groups. + +**Example console output:** +``` +[widgets] Syncing app groups with main app. Define entitlements["com.apple.security.application-groups"] in the expo-target.config file to override. +``` + +### 3. Create Extensions with bunx create-target + +Use the `create-target` CLI to scaffold a new extension in the `targets/` directory: + +```sh +bunx create-target widget +``` + +This creates: +``` +targets/widgets/ + expo-target.config.js # Configuration + WidgetsBundle.swift # Template Swift code +``` + +The `expo-target.config.js` defines the extension configuration: + +```js +// targets/widgets/expo-target.config.js +module.exports = { + type: "widget", // Extension type (required) + name: "Widgets", // Display name for the target + bundleIdentifier: "...", // Optional: override default bundle ID + deploymentTarget: "16.0", // Optional: minimum iOS version + frameworks: ["WidgetKit"], // Optional: additional frameworks to link + entitlements: { // Optional: custom entitlements + // If omitted, App Groups auto-sync from main app for widget/share/clip/bg-download + } +}; +``` + +**For extensions with `appGroupsByDefault: true` (widget, share, clip, bg-download):** +- Omit `entitlements["com.apple.security.application-groups"]` → auto-syncs from `app.json` +- Define it → overrides the main app's groups + +### 4. Manual Override in Extension Config (Optional) + +To use a **different** App Group for a specific extension (or to add App Groups to an extension type that doesn't sync by default), define the entitlement in `expo-target.config.js`: + +```js +// targets/widgets/expo-target.config.js +module.exports = { + type: "widget", + name: "Widgets", + entitlements: { + "com.apple.security.application-groups": [ + "group.com.example.widgets-only" // Override: different group + ] + } +}; +``` + +This overrides the automatic sync behavior for that specific extension target. + +### 5. Run expo prebuild + +Generate the native Xcode project with extension targets linked: + +```sh +bunx expo prebuild --clean +``` + +This: +- Creates `ios/YourApp.xcodeproj` with the main app target +- Links each extension from `targets/*/` as a separate Xcode target +- Generates `.entitlements` files with App Groups synced +- Updates `extra.eas.build.experimental.ios.appExtensions` for EAS Build + +### 6. Access Shared UserDefaults + +Use `UserDefaults(suiteName:)` to read/write preferences shared between the app and extension: + +```swift +import Foundation + +// In both your main app and extension: +let sharedDefaults = UserDefaults(suiteName: "group.com.example.myapp") + +// Write from the main app +sharedDefaults?.set("Sample Value", forKey: "shared_key") +sharedDefaults?.synchronize() // Not required on modern iOS, but harmless + +// Read from the widget extension +if let value = sharedDefaults?.string(forKey: "shared_key") { + print("Shared value: \(value)") +} +``` + +**Important:** The suite name **must exactly match** one of the App Group identifiers in your entitlements. Case-sensitive. + +### 7. Access Shared File Container + +Use `FileManager.containerURL(forSecurityApplicationGroupIdentifier:)` to get the shared directory: + +```swift +import Foundation + +guard let containerURL = FileManager.default.containerURL( + forSecurityApplicationGroupIdentifier: "group.com.example.myapp" +) else { + fatalError("App Group container not accessible. Check entitlements.") +} + +let sharedFileURL = containerURL.appendingPathComponent("data.json") + +// Write a file from the main app +let data = try JSONEncoder().encode(myModel) +try data.write(to: sharedFileURL, options: .atomic) + +// Read the file from the extension +let readData = try Data(contentsOf: sharedFileURL) +let decodedModel = try JSONDecoder().decode(MyModel.self, from: readData) +``` + +**Directory structure:** +``` +~/Library/Group Containers/group.com.example.myapp/ + Library/ + Preferences/ + group.com.example.myapp.plist (UserDefaults suite) + Caches/ + Application Support/ + Documents/ + tmp/ +``` + +### 8. Share Keychain Items + +To share passwords, tokens, or certificates between targets, use the `kSecAttrAccessGroup` attribute: + +```swift +import Security + +let accessGroup = "group.com.example.myapp" + +// Save a keychain item (from the main app) +let query: [String: Any] = [ + kSecClass as String: kSecClassGenericPassword, + kSecAttrAccount as String: "user@example.com", + kSecAttrAccessGroup as String: accessGroup, + kSecValueData as String: "my-secret-token".data(using: .utf8)! +] +let status = SecItemAdd(query as CFDictionary, nil) + +// Retrieve from the extension +let searchQuery: [String: Any] = [ + kSecClass as String: kSecClassGenericPassword, + kSecAttrAccount as String: "user@example.com", + kSecAttrAccessGroup as String: accessGroup, + kSecReturnData as String: true +] +var result: AnyObject? +let searchStatus = SecItemCopyMatching(searchQuery as CFDictionary, &result) + +if searchStatus == errSecSuccess, + let data = result as? Data, + let token = String(data: data, encoding: .utf8) { + print("Retrieved token: \(token)") +} +``` + +**Note:** On macOS, keychain access groups require a slightly different format: `.`. On iOS, use the App Group identifier directly. + +## EAS Build and Automatic Code Signing + +When using EAS Build, the `@bacons/apple-targets` plugin automatically exports each extension's bundle identifier and entitlements to the `extra.eas.build.experimental.ios.appExtensions` array in the Expo config. EAS reads this metadata and provisions the necessary certificates, provisioning profiles, and App Group identifiers **without manual configuration**. + +### How It Works + +1. **Plugin scans the Xcode project** after `expo prebuild` and extracts all extension targets, their bundle identifiers, and entitlements (including App Groups). +2. **Config is updated** with the `withAutoEasExtensionCredentials` config plugin, which adds an entry like: + +```json +{ + "extra": { + "eas": { + "build": { + "experimental": { + "ios": { + "appExtensions": [ + { + "targetName": "widgets", + "bundleIdentifier": "com.example.myapp.widgets", + "entitlements": { + "com.apple.security.application-groups": [ + "group.com.example.myapp" + ] + } + } + ] + } + } + } + } + } +} +``` + +3. **EAS Build provisions App Groups.** When you run `eas build`, the build service: + - Registers the App Group identifier (`group.com.example.myapp`) in your Apple Developer account if it doesn't already exist. + - Creates or updates provisioning profiles for the main app and each extension, including the App Group entitlement. + - Signs each target with the correct profile. + +4. **No manual Apple Developer Portal work required.** You do not need to manually create App Group identifiers, add them to App IDs, or regenerate provisioning profiles. EAS handles everything automatically based on the entitlements extracted during `expo prebuild`. + +The plugin automatically populates `extra.eas.build.experimental.ios.appExtensions` in your Expo config, so EAS Build knows exactly which entitlements each extension target needs. + +## Verifying App Groups are Configured + +After running `bunx expo prebuild --clean`, check the generated entitlements files: + +```sh +# Main app entitlements +cat ios/YourApp/YourApp.entitlements + +# Extension entitlements +cat targets/widgets/widgets.entitlements +``` + +Both should contain: + +```xml +com.apple.security.application-groups + + group.com.example.myapp + +``` + +You can also inspect the EAS metadata: + +```sh +bunx expo config --type introspect | jq '.extra.eas.build.experimental.ios.appExtensions' +``` + +## Debugging App Groups + +### Common Issues + +**"App Group container returns nil"** +- The entitlement is missing from the target's `.entitlements` file (verify in `targets//.entitlements`). +- The provisioning profile does not include the App Group entitlement (run `bunx expo prebuild --clean` and rebuild with EAS). +- The App Group identifier in code does not match the one in the entitlements (case-sensitive). + +**"UserDefaults writes from the app are not visible in the extension"** +- Both targets must use the **exact same** suite name. +- Call `.synchronize()` after writing (though not strictly required on iOS 14+, it forces an immediate flush). +- The extension process may be cached; kill the extension process using Xcode's Debug > Attach to Process by PID or Name, or restart the device. + +**"Shared file writes from one target cause crashes in another"** +- Use `NSFileCoordinator` for concurrent file access when multiple processes might read/write the same file simultaneously. +- Prefer SQLite with WAL mode (`PRAGMA journal_mode=WAL;`) for databases accessed by multiple processes. + +**"Data is not migrating from App Clip to full app"** +- Verify the App Clip and full app have identical App Group identifiers in their entitlements. +- The App Clip's bundle ID must start with the full app's bundle ID (e.g., `com.example.myapp.clip` is a child of `com.example.myapp`). + +### Logging and Inspection + +**Print the container path at runtime:** +```swift +if let url = FileManager.default.containerURL( + forSecurityApplicationGroupIdentifier: "group.com.example.myapp" +) { + print("App Group container: \(url.path)") +} else { + print("App Group container is nil! Check entitlements.") +} +``` + +**On device (iOS Simulator or macOS):** +Navigate to the logged path in Terminal: +```sh +open ~/Library/Developer/CoreSimulator/Devices//data/Containers/Shared/AppGroup// +``` + +**Inspect UserDefaults plist:** +```sh +plutil -p ~/Library/Group\ Containers/group.com.example.myapp/Library/Preferences/group.com.example.myapp.plist +``` + +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 8.0+ | Full support for app extensions and App Clips. | +| iPadOS | 8.0+ | Same as iOS. | +| macOS | 10.10+ | Supported. Keychain access groups require `.` format. | +| watchOS | 2.0+ | Supported. Use for Watch Connectivity data sync or standalone watch apps. | +| tvOS | 9.0+ | Supported (rare use case -- extensions are limited on tvOS). | +| visionOS | 1.0+ | Supported. | + +## Gotchas + +- **App Group IDs are global to your team.** Once you register `group.com.example.myapp` in the Apple Developer Portal under your team, you cannot delete it. You can disable it on specific App IDs, but the identifier remains reserved. Choose carefully. + +- **Case sensitivity matters.** `group.com.example.MyApp` and `group.com.example.myapp` are **different** identifiers. Always use lowercase to avoid mismatches. + +- **The `group.` prefix is mandatory.** Attempting to use `com.example.myapp` without the `group.` prefix will cause entitlement validation to fail during code signing. + +- **UserDefaults suite name must match exactly.** `UserDefaults(suiteName: "group.com.example.myapp")` only works if `"group.com.example.myapp"` is in the entitlements array. Typographical errors silently create a separate, unshared defaults store. + +- **File protection levels affect background access.** Files created in the App Group container inherit the default protection level (`NSFileProtectionCompleteUntilFirstUserAuthentication`). Background extensions (notification service, widget timeline providers) run while the device is locked, so files with `NSFileProtectionComplete` are inaccessible. Use `.completeUntilFirstUserAuthentication` or `.noFileProtection` for shared data accessed from background contexts. + +- **Data is NOT automatically migrated.** Adding an App Group entitlement to an existing app does not move data from the app's private container to the shared container. You must write migration code on first launch that copies files from the old location to the new one. + +- **Multiple apps can share the same group.** If you publish multiple apps under the same developer account, they can all use `group.com.example.shared` to exchange data. This is intentional but can create privacy issues if sensitive data is exposed to unintended apps. Scope your group identifiers to prevent accidental sharing. + +- **App Group containers are not deleted when one app is removed.** If you have two apps using the same group and the user uninstalls one, the container remains. This can lead to unexpectedly large storage usage. The container is only deleted when **all** apps using it are uninstalled. + +- **Core Data shared stores require NSPersistentContainer configuration.** When using Core Data across multiple processes, initialize `NSPersistentContainer` with a custom store URL pointing to the App Group container, and enable history tracking (`NSPersistentHistoryTrackingKey`) to merge changes between processes. + +- **Simulator vs. device paths differ.** On the simulator, the App Group container is at `~/Library/Developer/CoreSimulator/...`. On a physical device, it is in a sandboxed location invisible from macOS Finder. Use Xcode's Devices and Simulators window > Download Container to inspect device data. + +- **TestFlight and App Store builds use different provisioning profiles.** If your development build works but TestFlight does not, ensure you've run `bunx expo prebuild --clean` before the EAS build so the entitlements metadata is up-to-date. EAS will provision the correct profiles based on the `appExtensions` array. + +- **Entitlements mismatch between targets breaks EAS Build.** If your main app has `["group.A", "group.B"]` but your widget only has `["group.A"]`, EAS provisions both groups for the main app profile and only `group.A` for the widget. This is correct behavior, but if you later add `group.B` to the widget's `expo-target.config.js`, you must run `bunx expo prebuild --clean` before building with EAS to re-sync the metadata. + +- **Version control `targets/`, ignore `ios/`.** The `targets/` directory contains your source code and should be committed to git. The `ios/` directory is generated by `expo prebuild` and should be in `.gitignore`. Never manually edit files in `ios/` — they will be overwritten when prebuild runs. Swift files in `targets/` are linked into the Xcode project, so code changes don't require re-running prebuild. + +- **Generated `.entitlements` files live in `targets/`.** After running `bunx expo prebuild`, each extension will have a `.entitlements` file in its `targets//` directory. These are generated from your `expo-target.config.js` and should **not** be manually edited (they will be regenerated the next time you run prebuild after config changes). + +## Example: Full Integration + +**app.json:** +```json +{ + "expo": { + "slug": "myapp", + "ios": { + "bundleIdentifier": "com.example.myapp", + "entitlements": { + "com.apple.security.application-groups": [ + "group.com.example.myapp" + ] + } + }, + "plugins": [ + ["@bacons/apple-targets"] + ] + } +} +``` + +**targets/widgets/expo-target.config.js:** +```js +module.exports = { + type: "widget", + name: "Widgets" + // App Groups will be auto-synced from the main app +}; +``` + +**Main app writes data:** +```swift +// ContentView.swift (main app) +import SwiftUI + +struct ContentView: View { + var body: some View { + Button("Save Data") { + let sharedDefaults = UserDefaults(suiteName: "group.com.example.myapp") + sharedDefaults?.set(Date().timeIntervalSince1970, forKey: "last_update") + sharedDefaults?.set("Hello from main app!", forKey: "message") + + // Trigger widget reload + #if canImport(WidgetKit) + import WidgetKit + WidgetCenter.shared.reloadAllTimelines() + #endif + } + } +} +``` + +**Widget reads data:** +```swift +// WidgetBundle.swift (widget extension) +import WidgetKit +import SwiftUI + +struct SimpleEntry: TimelineEntry { + let date: Date + let message: String +} + +struct Provider: TimelineProvider { + func placeholder(in context: Context) -> SimpleEntry { + SimpleEntry(date: .now, message: "Placeholder") + } + + func getSnapshot(in context: Context, completion: @escaping (SimpleEntry) -> ()) { + completion(placeholder(in: context)) + } + + func getTimeline(in context: Context, completion: @escaping (Timeline) -> ()) { + let sharedDefaults = UserDefaults(suiteName: "group.com.example.myapp") + let message = sharedDefaults?.string(forKey: "message") ?? "No data" + let entry = SimpleEntry(date: .now, message: message) + let timeline = Timeline(entries: [entry], policy: .never) + completion(timeline) + } +} + +struct WidgetEntryView: View { + var entry: SimpleEntry + + var body: some View { + Text(entry.message) + .containerBackground(.fill.tertiary, for: .widget) + } +} + +@main +struct MyWidget: Widget { + let kind = "MyWidget" + + var body: some WidgetConfiguration { + StaticConfiguration(kind: kind, provider: Provider()) { entry in + WidgetEntryView(entry: entry) + } + .configurationDisplayName("My Widget") + .description("Shows data from the main app") + .supportedFamilies([.systemSmall]) + } +} +``` + +**Setup and build:** +```sh +# 1. Create the widget extension +bunx create-target widget + +# 2. Add the Swift code above to targets/widgets/WidgetsBundle.swift + +# 3. Generate the Xcode project with extension linked +bunx expo prebuild --clean + +# 4. Build and run +bunx expo run:ios + +# 5. In the app, tap "Save Data", then add the widget to the home screen +``` + +The widget displays "Hello from main app!" read from the shared UserDefaults. Both targets access the same App Group container because the plugin automatically synced the entitlements from `app.json`. + +## Additional Resources + +- **Official Apple Sample Code:** [Today Extension (Legacy)](https://developer.apple.com/library/archive/samplecode/Today/Introduction/Intro.html) -- demonstrates sharing `NSUserDefaults` via App Groups. +- **WWDC 2020 Lab Notes:** [App Clips and Data Sharing](https://developer.apple.com/videos/play/wwdc2020/10120/) -- covers App Group best practices for App Clip-to-app migration. +- **Expo Documentation:** [iOS Entitlements](https://docs.expo.dev/versions/latest/config/app/#entitlements) -- how to configure entitlements in `app.json`. +- **EAS Build Documentation:** [App Extensions and EAS Build](https://docs.expo.dev/build-reference/app-extensions/) -- experimental support for app extensions and automatic credential provisioning. diff --git a/plugins/expo/skills/apple-targets/file-provider-ui.md b/plugins/expo/skills/apple-targets/file-provider-ui.md new file mode 100644 index 0000000..4994a0c --- /dev/null +++ b/plugins/expo/skills/apple-targets/file-provider-ui.md @@ -0,0 +1,196 @@ +--- +title: File Provider UI Extension +description: Presents custom user interface for file provider actions and authentication errors within the Files app. +version: iOS 11.0+ +--- + +# File Provider UI Extension (`file-provider-ui`) + +A File Provider UI extension displays custom view controllers inside the Files app when a user triggers a provider-specific action or when the file provider reports an authentication error. It is a companion to the File Provider extension (`file-provider`), which handles file operations but has no UI. The UI extension subclasses `FPUIActionExtensionViewController` from the FileProviderUI framework, and the system presents it modally. Typical uses include sign-in flows, two-factor authentication prompts, server-specific sharing dialogs, and conflict resolution interfaces. + +## Apple Documentation + +- [FileProviderUI Framework](https://developer.apple.com/documentation/fileproviderui) +- [FPUIActionExtensionViewController](https://developer.apple.com/documentation/fileproviderui/fpuiactionextensionviewcontroller) +- [FPUIActionExtensionContext](https://developer.apple.com/documentation/fileproviderui/fpuiactionextensioncontext) +- [FPUIExtensionErrorCode](https://developer.apple.com/documentation/fileproviderui/fpuiextensionerrorcode) +- [NSFileProviderItem](https://developer.apple.com/documentation/fileprovider/nsfileprovideritem) + +## WWDC History + +- **[WWDC 2017, Session 243 -- File Provider Enhancements](https://developer.apple.com/videos/play/wwdc2017/243/)** -- Introduced the File Provider UI extension alongside the non-UI file provider. Covered how to declare custom actions in Info.plist and present authentication flows. +- **[WWDC 2021, Session 10182 -- Sync Files to the Cloud with FileProvider on macOS](https://developer.apple.com/videos/play/wwdc2021/10182/)** -- Updated guidance on pairing the UI extension with the replicated file provider model for authentication and error handling. + +## What It Does + +1. **Custom actions.** You declare named actions in your file provider's Info.plist under `NSExtensionFileProviderActions`. Each action has an identifier and a localized title. When the user selects items and picks your action from the context menu, the system launches your UI extension and calls `prepare(forAction:itemIdentifiers:)`. +2. **Authentication errors.** When your file provider extension throws an `NSFileProviderError` with code `.notAuthenticated`, the Files app shows a system "Sign In" button. Tapping it launches your UI extension and calls `prepare(forError:)`, giving you the opportunity to present a login screen. +3. **Completion.** When the user finishes the action (or cancels), you call `extensionContext.completeRequest()` or `extensionContext.cancelRequest(withError:)`. The system dismisses the view controller and, if needed, re-enumerates the affected items. + +## Use Cases + +### Sign-in and re-authentication + +When a user's OAuth token expires, the file provider throws `.notAuthenticated`. The Files app presents a sign-in prompt, and your UI extension shows a web view or native login form. On success, you store the new token in the shared App Group keychain and call `completeRequest()`. + +### Two-factor authentication + +Some enterprise storage requires a second factor when accessing sensitive folders. The file provider throws a custom authentication error, and the UI extension presents a code-entry screen. Once verified, it writes the session to shared storage and completes. + +### Server-specific sharing + +A cloud provider offers advanced sharing options (password-protected links, expiration dates, permissions). The UI extension presents a sharing configuration sheet when the user invokes the custom "Share Link" action on selected files. + +### Conflict resolution + +When the file provider detects a version conflict during sync, the UI extension can present a side-by-side diff or a "keep local / keep remote / keep both" dialog, then signal which resolution path to take. + +## Key Classes + +| Class / Protocol | Role | +|------------------|------| +| `FPUIActionExtensionViewController` | Subclass this. Your principal class for the extension. Receives action and error preparation callbacks. | +| `FPUIActionExtensionContext` | The extension context. Call `completeRequest()` to finish or `cancelRequest(withError:)` to abort. Accessed via `extensionContext`. | +| `FPUIExtensionErrorCode` | Error codes for cancellation (`.userCancelled`) and failure (`.failed`). | +| `NSFileProviderItemIdentifier` | Identifiers of the items the user selected when invoking the action. Passed to `prepare(forAction:itemIdentifiers:)`. | + +## Implementation + +### Authentication and Custom Action Handler + +```swift +import UIKit +import FileProviderUI +import FileProvider + +// 1. Subclass FPUIActionExtensionViewController. This is the principal class +// declared in your extension's Info.plist. +class FileProviderUIController: FPUIActionExtensionViewController { + + private var currentAction: String? + private var selectedItems: [NSFileProviderItemIdentifier] = [] + + // 2. Called when the user triggers a custom action declared in Info.plist. + // The actionIdentifier matches your NSExtensionFileProviderActions entry. + override func prepare(forAction actionIdentifier: String, + itemIdentifiers: [NSFileProviderItemIdentifier]) { + currentAction = actionIdentifier + selectedItems = itemIdentifiers + + switch actionIdentifier { + case "com.example.myapp.share-link": + // 3. Present a sharing configuration UI for the selected items. + let shareVC = ShareLinkViewController(itemIdentifiers: itemIdentifiers) + shareVC.onComplete = { [weak self] in + self?.extensionContext.completeRequest() + } + addChild(shareVC) + view.addSubview(shareVC.view) + shareVC.view.frame = view.bounds + shareVC.didMove(toParent: self) + + case "com.example.myapp.move-to-team": + // 4. Handle a "Move to Team Folder" action. + let moveVC = MoveToTeamViewController(itemIdentifiers: itemIdentifiers) + moveVC.onComplete = { [weak self] in + self?.extensionContext.completeRequest() + } + moveVC.onCancel = { [weak self] in + self?.cancelWithUserCancelled() + } + present(moveVC, animated: true) + + default: + // 5. Unknown action -- cancel gracefully. + cancelWithUserCancelled() + } + } + + // 6. Called when the file provider throws a .notAuthenticated error. + // Present your sign-in flow here. + override func prepare(forError error: Error) { + let loginVC = LoginViewController() + loginVC.onAuthenticated = { [weak self] token in + // 7. Store the new token in the shared App Group keychain + // so the file provider extension can read it. + SharedKeychain.store(token: token) + self?.extensionContext.completeRequest() + } + loginVC.onCancel = { [weak self] in + self?.cancelWithUserCancelled() + } + present(loginVC, animated: true) + } + + // 8. Helper to cancel with the standard user-cancelled error code. + private func cancelWithUserCancelled() { + extensionContext.cancelRequest( + withError: NSError( + domain: FPUIErrorDomain, + code: Int(FPUIExtensionErrorCode.userCancelled.rawValue) + ) + ) + } + + // 9. Wire up navigation bar buttons for simple cases. + @IBAction func doneButtonTapped(_ sender: Any) { + extensionContext.completeRequest() + } + + @IBAction func cancelButtonTapped(_ sender: Any) { + cancelWithUserCancelled() + } +} +``` + +### Info.plist Action Declaration + +Custom actions must be declared in the **file provider extension's** (not the UI extension's) Info.plist. The system reads these to build context menu entries: + +```xml + +NSExtension + + NSExtensionPointIdentifier + com.apple.fileprovider-nonui + NSExtensionFileProviderActions + + + + NSExtensionFileProviderActionIdentifier + com.example.myapp.share-link + NSExtensionFileProviderActionName + Create Share Link + NSExtensionFileProviderActionActivationRule + TRUEPREDICATE + + + +``` + +## Usage + +- Create: `bun create target file-provider-ui` +- Edit: `targets/file-provider-ui/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 11.0+ | Presented modally inside the Files app. | +| iPadOS | 11.0+ | Full support. Displayed as a sheet in the Files app. | +| macOS | 11.0+ | Presented as a sheet in Finder when a File Provider action is invoked. | +| visionOS | 1.0+ | Supported alongside the Files app. | +| tvOS | -- | Not supported. No Files app on tvOS. | +| watchOS | -- | Not supported. | + +## Gotchas + +- **This extension only provides UI -- it cannot perform file operations.** All file manipulation (download, upload, delete) must happen in the companion `file-provider` extension. The UI extension's sole job is to collect user input and communicate it back via shared storage (App Group UserDefaults, keychain, or a database). +- **Actions are declared in the file provider's Info.plist, not the UI extension's.** A common mistake is adding `NSExtensionFileProviderActions` to the wrong target. The system reads action declarations from the non-UI file provider extension and routes them to the UI extension by convention (same app bundle). +- **`prepare(forError:)` is only called for `.notAuthenticated` errors.** Other file provider errors are handled by the system's default UI. You cannot present custom UI for arbitrary errors -- only authentication failures trigger the UI extension. +- **The extension context must be completed or cancelled.** If you forget to call `completeRequest()` or `cancelRequest(withError:)`, the extension hangs indefinitely and the Files app becomes unresponsive for that provider. Always call one of these in every code path, including error handlers. +- **No access to the file provider's database directly.** The UI extension runs in a separate process. Use a shared App Group container or keychain to pass authentication tokens and action results between the UI extension and the file provider extension. +- **View controller lifecycle is standard UIKit.** Despite running inside Files, the extension view controller follows normal UIKit lifecycle rules. You can use Auto Layout, child view controllers, and navigation controllers. However, the view size is constrained by the modal presentation style the system chooses. +- **Pair with a file-provider target.** This extension type is useless without a companion `file-provider` extension in the same app. Always create both targets together. diff --git a/plugins/expo/skills/apple-targets/file-provider.md b/plugins/expo/skills/apple-targets/file-provider.md new file mode 100644 index 0000000..32c30bb --- /dev/null +++ b/plugins/expo/skills/apple-targets/file-provider.md @@ -0,0 +1,365 @@ +--- +title: File Provider Extension +description: Integrates a cloud storage backend into the system Files app, enabling transparent file browsing, downloading, uploading, and syncing via NSFileProviderReplicatedExtension. +version: iOS 11.0+ (legacy), iOS 16.0+ (replicated) +--- + +# File Provider Extension (`file-provider`) + +A File Provider extension surfaces your cloud storage service as a first-class location inside the Files app (iOS) and Finder (macOS). Users can browse, open, move, rename, and delete remote files as if they were local. The modern replicated extension model (iOS 16+/macOS 12.3+) uses `NSFileProviderReplicatedExtension` to maintain a bidirectional sync between a local replica and your remote server. The system manages the on-disk cache, evicts unused files, and calls your extension to materialize content on demand. This is one of the most complex extension types -- production implementations (Dropbox, Google Drive, OneDrive) typically require thousands of lines of sync logic, conflict resolution, and database management. + +## Apple Documentation + +- [File Provider Framework](https://developer.apple.com/documentation/fileprovider) +- [Replicated File Provider Extension](https://developer.apple.com/documentation/fileprovider/replicated-file-provider-extension) +- [NSFileProviderReplicatedExtension](https://developer.apple.com/documentation/fileprovider/nsfileproviderreplicatedextension) +- [NSFileProviderItem](https://developer.apple.com/documentation/fileprovider/nsfileprovideritem) +- [NSFileProviderEnumerator](https://developer.apple.com/documentation/fileprovider/nsfileproviderenumerator) +- [NSFileProviderManager](https://developer.apple.com/documentation/fileprovider/nsfileprovidermanager) +- [NSFileProviderDomain](https://developer.apple.com/documentation/fileprovider/nsfileproviderdomain) +- [item(for:request:completionHandler:)](https://developer.apple.com/documentation/fileprovider/nsfileproviderreplicatedextension/item(for:request:completionhandler:)) +- [fetchContents(for:version:request:completionHandler:)](https://developer.apple.com/documentation/fileprovider/nsfileproviderreplicatedextension/3553303-fetchcontentsforitemwithidentifi) +- [createItem(basedOn:fields:contents:options:request:completionHandler:)](https://developer.apple.com/documentation/fileprovider/nsfileproviderreplicatedextension/createitem(basedon:fields:contents:options:request:completionhandler:)) +- [deleteItem(identifier:baseVersion:options:request:completionHandler:)](https://developer.apple.com/documentation/fileprovider/nsfileproviderreplicatedextension/3656550-deleteitemwithidentifier) + +## WWDC History + +- **[WWDC 2017, Session 243 -- File Provider Enhancements](https://developer.apple.com/videos/play/wwdc2017/243/)** -- Introduced the File Provider framework alongside the Files app in iOS 11. Covered enumeration, thumbnails, and push-based change signaling. +- **[WWDC 2021, Session 10182 -- Sync Files to the Cloud with FileProvider on macOS](https://developer.apple.com/videos/play/wwdc2021/10182/)** -- Major rewrite: introduced `NSFileProviderReplicatedExtension` for macOS with bidirectional sync, icon decorations, Finder integration, and published sample code. +- **[WWDC 2022, Session 10045 -- What's New in File Provider](https://developer.apple.com/videos/play/wwdc2022/10045/)** -- Brought the replicated extension model to iOS 16. Covered progress reporting, eviction policies, and materialization. + +## What It Does + +1. **Domain registration.** Your main app registers an `NSFileProviderDomain` with `NSFileProviderManager.add(_:completionHandler:)`. This creates a visible location in the Files app sidebar. +2. **Enumeration.** When the user browses a folder, the system asks your extension for an `NSFileProviderEnumerator`. Your enumerator returns pages of `NSFileProviderItem` objects describing files and folders. +3. **Materialization (download).** When the user opens a file, the system calls `fetchContents(for:version:request:completionHandler:)`. Your extension downloads the file from your server and returns a local URL. +4. **Upload.** When the user creates or modifies a file, the system calls `createItem(basedOn:fields:contents:options:request:completionHandler:)` or `modifyItem(...)`. Your extension uploads the content to your server. +5. **Deletion.** The system calls `deleteItem(identifier:baseVersion:options:request:completionHandler:)` when a file is trashed or permanently removed. +6. **Change signaling.** Your app detects remote changes (via polling, push notifications, or websockets) and calls `NSFileProviderManager.signalEnumerator(for:)` to tell the system to re-enumerate and pick up new items. +7. **Eviction.** The system automatically evicts (removes local copies of) files that have not been accessed recently, keeping only metadata. Your extension re-downloads them on demand. + +## Use Cases + +### Cloud storage integration + +Services like Dropbox, Google Drive, and OneDrive use File Provider extensions to appear as native locations in Files and Finder. Users drag and drop files, use Quick Look, and share directly from the cloud provider without opening a separate app. + +### Enterprise document management + +Corporate DMS platforms (SharePoint, Box) provide File Provider extensions so employees can access company documents directly from Pages, Keynote, or any document picker. IT admins benefit from the system-managed cache and on-demand downloads that minimize local storage use. + +### Version-controlled repositories + +A developer tool could expose a Git repository as a File Provider domain, letting users browse branches and files from Finder without cloning the entire repo locally. + +### Encrypted vault access + +Password-manager and encrypted-storage apps (Cryptomator, Boxcryptor) use File Provider extensions to decrypt files transparently when opened and re-encrypt on save, surfacing vaults as normal folders in Files. + +## Key Classes + +| Class / Protocol | Role | +|------------------|------| +| `NSFileProviderReplicatedExtension` | The main protocol you implement. Handles item metadata, downloads, uploads, deletions, and enumeration. | +| `NSFileProviderItem` | Protocol describing a single file or folder: identifier, parent, filename, content type, size, version, capabilities. | +| `NSFileProviderEnumerator` | Protocol for listing items in a container. Implements `enumerateItems(for:startingAt:)` and `enumerateChanges(for:from:)`. | +| `NSFileProviderManager` | System manager for a domain. Used to signal changes, get container URLs, add/remove domains, and evict items. | +| `NSFileProviderDomain` | Represents a distinct storage location (e.g., one per account). Registered from the main app. | +| `NSFileProviderItemIdentifier` | Opaque identifier for an item. Special values: `.rootContainer`, `.workingSet`, `.trashContainer`. | +| `NSFileProviderItemVersion` | Tracks the content version and metadata version of an item for conflict detection. | +| `NSFileProviderSyncAnchor` | Opaque token used by change enumeration to track the last known sync point. | + +## Implementation + +### Replicated File Provider with Remote Sync + +```swift +import FileProvider +import UniformTypeIdentifiers + +// 1. Implement the replicated extension protocol. The system instantiates one +// per domain, potentially in separate processes for multi-account setups. +class FileProviderExtension: NSObject, NSFileProviderReplicatedExtension { + + let domain: NSFileProviderDomain + let manager: NSFileProviderManager + private let remoteClient: RemoteStorageClient + + // 2. The system passes the domain on init. Set up your database/network client here. + required init(domain: NSFileProviderDomain) { + self.domain = domain + self.manager = NSFileProviderManager(for: domain)! + self.remoteClient = RemoteStorageClient(accountID: domain.identifier.rawValue) + super.init() + } + + func invalidate() { + // 3. Called before the extension process is terminated. + // Close database connections, cancel pending network requests. + remoteClient.cancelAll() + } + + // 4. Return metadata for a single item. The system calls this frequently + // to validate items, populate Finder columns, and resolve conflicts. + func item(for identifier: NSFileProviderItemIdentifier, + request: NSFileProviderRequest, + completionHandler: @escaping (NSFileProviderItem?, Error?) -> Void) -> Progress { + Task { + do { + let metadata = try await remoteClient.fetchMetadata(for: identifier.rawValue) + completionHandler(FileProviderItem(remoteItem: metadata), nil) + } catch { + completionHandler(nil, error) + } + } + return Progress() + } + + // 5. Download file contents. Write the data to a temporary file and return its URL. + func fetchContents(for itemIdentifier: NSFileProviderItemIdentifier, + version requestedVersion: NSFileProviderItemVersion?, + request: NSFileProviderRequest, + completionHandler: @escaping (URL?, NSFileProviderItem?, Error?) -> Void) -> Progress { + let progress = Progress(totalUnitCount: 100) + + Task { + do { + let tempURL = FileManager.default.temporaryDirectory + .appendingPathComponent(UUID().uuidString) + let metadata = try await remoteClient.download( + itemID: itemIdentifier.rawValue, + to: tempURL, + progress: progress + ) + completionHandler(tempURL, FileProviderItem(remoteItem: metadata), nil) + } catch { + completionHandler(nil, nil, error) + } + } + return progress + } + + // 6. Upload a new item to the server. + func createItem(basedOn itemTemplate: NSFileProviderItem, + fields: NSFileProviderItemFields, + contents url: URL?, + options: NSFileProviderCreateItemOptions = [], + request: NSFileProviderRequest, + completionHandler: @escaping (NSFileProviderItem?, NSFileProviderItemFields, Bool, Error?) -> Void) -> Progress { + Task { + do { + let metadata = try await remoteClient.upload( + name: itemTemplate.filename, + parentID: itemTemplate.parentItemIdentifier.rawValue, + contentURL: url, + contentType: itemTemplate.contentType ?? .data + ) + // 7. Return the server-created item. The third Bool indicates + // whether the item already existed (deduplication). + completionHandler(FileProviderItem(remoteItem: metadata), [], false, nil) + } catch { + completionHandler(nil, [], false, error) + } + } + return Progress() + } + + // 8. Modify an existing item (rename, move, content change). + func modifyItem(_ item: NSFileProviderItem, + baseVersion version: NSFileProviderItemVersion, + changedFields: NSFileProviderItemFields, + contents newContents: URL?, + options: NSFileProviderModifyItemOptions = [], + request: NSFileProviderRequest, + completionHandler: @escaping (NSFileProviderItem?, NSFileProviderItemFields, Bool, Error?) -> Void) -> Progress { + Task { + do { + let metadata = try await remoteClient.modify( + itemID: item.itemIdentifier.rawValue, + fields: changedFields, + contentURL: newContents + ) + completionHandler(FileProviderItem(remoteItem: metadata), [], false, nil) + } catch { + completionHandler(nil, [], false, error) + } + } + return Progress() + } + + // 9. Delete an item from the server. + func deleteItem(identifier: NSFileProviderItemIdentifier, + baseVersion version: NSFileProviderItemVersion, + options: NSFileProviderDeleteItemOptions = [], + request: NSFileProviderRequest, + completionHandler: @escaping (Error?) -> Void) -> Progress { + Task { + do { + try await remoteClient.delete(itemID: identifier.rawValue) + completionHandler(nil) + } catch { + completionHandler(error) + } + } + return Progress() + } + + // 10. Return an enumerator for the given container (folder, root, or working set). + func enumerator(for containerItemIdentifier: NSFileProviderItemIdentifier, + request: NSFileProviderRequest) throws -> NSFileProviderEnumerator { + return FileProviderEnumerator( + containerID: containerItemIdentifier, + remoteClient: remoteClient + ) + } +} +``` + +### Enumerator with Change Tracking + +```swift +import FileProvider + +// 11. The enumerator lists items in a container and reports incremental changes. +class FileProviderEnumerator: NSObject, NSFileProviderEnumerator { + + private let containerID: NSFileProviderItemIdentifier + private let remoteClient: RemoteStorageClient + + init(containerID: NSFileProviderItemIdentifier, remoteClient: RemoteStorageClient) { + self.containerID = containerID + self.remoteClient = remoteClient + super.init() + } + + func invalidate() {} + + // 12. Full enumeration: return all items in the container, paginated. + func enumerateItems(for observer: NSFileProviderEnumerationObserver, + startingAt page: NSFileProviderPage) { + Task { + do { + let (items, nextPage) = try await remoteClient.listItems( + in: containerID.rawValue, + page: page + ) + let providerItems = items.map { FileProviderItem(remoteItem: $0) } + observer.didEnumerate(providerItems) + observer.finishEnumerating(upTo: nextPage) + } catch { + observer.finishEnumeratingWithError(error) + } + } + } + + // 13. Incremental change enumeration: return only items changed since the anchor. + func enumerateChanges(for observer: NSFileProviderChangeObserver, + from anchor: NSFileProviderSyncAnchor) { + Task { + do { + let (changed, deleted, newAnchor) = try await remoteClient.listChanges( + in: containerID.rawValue, + since: anchor + ) + observer.didUpdate(changed.map { FileProviderItem(remoteItem: $0) }) + observer.didDeleteItems(withIdentifiers: deleted.map { + NSFileProviderItemIdentifier($0) + }) + observer.finishEnumeratingChanges(upTo: newAnchor, moreComing: false) + } catch { + observer.finishEnumeratingWithError(error) + } + } + } + + // 14. Return the current sync anchor so the system can track change state. + func currentSyncAnchor(completionHandler: @escaping (NSFileProviderSyncAnchor?) -> Void) { + Task { + let anchor = try? await remoteClient.currentAnchor(for: containerID.rawValue) + completionHandler(anchor) + } + } +} +``` + +### File Provider Item Model + +```swift +import FileProvider +import UniformTypeIdentifiers + +// 15. Map your remote data model to the NSFileProviderItem protocol. +class FileProviderItem: NSObject, NSFileProviderItem { + + private let remoteItem: RemoteItemMetadata + + init(remoteItem: RemoteItemMetadata) { + self.remoteItem = remoteItem + } + + var itemIdentifier: NSFileProviderItemIdentifier { + NSFileProviderItemIdentifier(remoteItem.id) + } + + var parentItemIdentifier: NSFileProviderItemIdentifier { + NSFileProviderItemIdentifier(remoteItem.parentID) + } + + var filename: String { remoteItem.name } + + var contentType: UTType { + remoteItem.isFolder ? .folder : (UTType(filenameExtension: remoteItem.fileExtension) ?? .data) + } + + var documentSize: NSNumber? { + NSNumber(value: remoteItem.sizeBytes) + } + + // 16. The item version is critical for conflict detection. Return separate + // content and metadata version hashes from your server. + var itemVersion: NSFileProviderItemVersion { + NSFileProviderItemVersion( + contentVersion: remoteItem.contentHash.data(using: .utf8)!, + metadataVersion: remoteItem.metadataHash.data(using: .utf8)! + ) + } + + var capabilities: NSFileProviderItemCapabilities { + [.allowsReading, .allowsWriting, .allowsRenaming, + .allowsReparenting, .allowsTrashing, .allowsDeleting] + } + + // 17. Return dates so Finder/Files can display them in list view. + var creationDate: Date? { remoteItem.createdAt } + var contentModificationDate: Date? { remoteItem.modifiedAt } +} +``` + +## Usage + +- Create: `bun create target file-provider` +- Edit: `targets/file-provider/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 11.0+ (legacy), 16.0+ (replicated) | Legacy `NSFileProviderExtension` API in iOS 11. Modern replicated model requires iOS 16. | +| iPadOS | 11.0+ (legacy), 16.0+ (replicated) | Same as iOS. Files app is the primary interaction surface. | +| macOS | 11.0+ (legacy), 12.3+ (replicated) | Replicated model introduced in macOS 12.3. Finder integration with icon decorations. | +| visionOS | 1.0+ | Supported. Files app available in visionOS. | +| tvOS | -- | Not supported. No Files app on tvOS. | +| watchOS | -- | Not supported. | + +## Gotchas + +- **Three-state reconciliation is mandatory.** You must track three independent states: the remote server, your local metadata database, and the system's on-disk replica. Conflicts arise when the user edits a file offline while another client modifies it on the server. Your extension must detect version mismatches and resolve them. +- **Each domain runs in a separate process.** If you support multiple accounts, each `NSFileProviderDomain` gets its own extension process. Your database or data store must handle concurrent access from multiple processes -- use SQLite with WAL mode or a shared App Group container with proper locking. +- **The extension has no UI.** File Provider extensions use the `com.apple.fileprovider-nonui` extension point. If you need to show authentication prompts or error dialogs, you must pair it with a File Provider UI extension (`file-provider-ui`). +- **`signalEnumerator` is your only way to push remote changes.** The system does not poll your extension. You must call `NSFileProviderManager.signalEnumerator(for:)` whenever remote changes occur. Set up push notifications, a websocket, or periodic polling in your main app or extension to detect changes. +- **Completion handlers, not async/await.** The core `NSFileProviderReplicatedExtension` protocol methods use old-style completion handlers, not Swift concurrency. You can bridge to async/await internally with `Task`, but the completion handler must be called exactly once. +- **App Group entitlement is required.** The extension and main app must share an App Group to exchange authentication tokens, database files, and configuration. The `@bacons/apple-targets` plugin configures this automatically when `appGroupsByDefault` is true. +- **50MB memory limit for extensions.** Like all app extensions, File Provider extensions run under a strict memory cap. Avoid loading entire files into memory -- stream large downloads and uploads using temporary files. +- **`NSFileProviderItemVersion` must be accurate.** If you return stale or incorrect version data, the system will skip uploading changes or will re-download unchanged files. Always derive versions from your server's actual content and metadata hashes. +- **The legacy API (`NSFileProviderExtension`) is effectively deprecated.** Apple strongly encourages migrating to `NSFileProviderReplicatedExtension`. The legacy API does not support eviction, progress reporting, or bidirectional change enumeration. +- **Debugging requires `qlmanage` or Finder/Files restarts.** Changes to your extension's enumeration logic may not appear immediately because the system caches metadata aggressively. Use `killall Finder` on macOS or restart the Files app on iOS to force re-enumeration during development. diff --git a/plugins/expo/skills/apple-targets/intent-ui.md b/plugins/expo/skills/apple-targets/intent-ui.md new file mode 100644 index 0000000..974e4f7 --- /dev/null +++ b/plugins/expo/skills/apple-targets/intent-ui.md @@ -0,0 +1,180 @@ +--- +title: Siri Intent UI Extension +description: Displays custom UI inline within Siri, Shortcuts, or Maps when a SiriKit intent is being confirmed or handled, replacing the default system-provided snippet with your own branded view controller. +version: iOS 10.0+ +--- + +# Siri Intent UI Extension (`intent-ui`) + +An Intents UI extension lets you inject a custom view controller into the Siri transcript, the Shortcuts results card, or the Maps action sheet whenever the system processes one of your supported intents. Instead of the generic Siri snippet, users see your app's branded interface -- showing richer data, custom layouts, or confirmation details -- while the intent is being resolved, confirmed, or handled. The extension is a UIViewController subclass conforming to INUIHostedViewControlling, and it runs in a separate process alongside your Intents extension. It receives the same INInteraction object that your intent handler works with, so it can display contextual information about the ongoing request. + +## Apple Documentation + +- [Creating an Intents UI Extension](https://developer.apple.com/documentation/sirikit/creating-an-intents-ui-extension) +- [INUIHostedViewControlling](https://developer.apple.com/documentation/intentsui/inuihostedviewcontrolling) +- [Configuring Your Intents UI App Extension Target](https://developer.apple.com/documentation/sirikit/configuring-your-intents-ui-app-extension-target) +- [configureView(for:of:interactiveBehavior:context:completion:)](https://developer.apple.com/documentation/intentsui/inuihostedviewcontrolling/configureview(for:of:interactivebehavior:context:completion:)) +- [INUIHostedViewContext](https://developer.apple.com/documentation/intentsui/inuihostedviewcontext) +- [INUIInteractiveBehavior](https://developer.apple.com/documentation/intentsui/inuiinteractivebehavior) +- [IntentsUI Framework](https://developer.apple.com/documentation/intentsui) + +## WWDC History + +- **[WWDC 2016, Session 217 -- Introducing SiriKit](https://developer.apple.com/videos/play/wwdc2016/217/)** -- Launched SiriKit with two extension points: Intents and Intents UI. Demonstrated how the UI extension presents a "Siri snippet" -- a custom view controller embedded directly in the Siri conversation transcript. +- **[WWDC 2016, Session 225 -- Extending Your Apps with SiriKit](https://developer.apple.com/videos/play/wwdc2016/225/)** -- Walkthrough of building both an Intents extension and an Intents UI extension, including storyboard layout and the `configure(with:context:completion:)` method. +- **[WWDC 2017, Session 214 -- What's New in SiriKit](https://developer.apple.com/videos/play/wwdc2017/214/)** -- Introduced the more granular `configureView(for:of:interactiveBehavior:context:completion:)` API in iOS 11, enabling per-parameter UI replacement in the Siri snippet stack. +- **[WWDC 2018, Session 211 -- Introduction to Siri Shortcuts](https://developer.apple.com/videos/play/wwdc2018/211/)** -- Custom intents via IntentDefinition files can now surface in Shortcuts, where the Intents UI extension provides the result card. +- **[WWDC 2020, Session 10073 -- Empower Your Intents](https://developer.apple.com/videos/play/wwdc2020/10073/)** -- Added interactive behavior support, richer disambiguation with images, and in-app intent handling as an alternative to extension-based handling. + +## What It Does + +1. The user triggers an intent through Siri, the Shortcuts app, or Maps (e.g., "Send a message using MyApp"). +2. The system launches the paired Intents extension to resolve, confirm, and handle the request. +3. At each phase where Siri would display a snippet (confirmation, handling result), the system also launches the Intents UI extension. +4. The system calls `configureView(for:of:interactiveBehavior:context:completion:)` on your `IntentViewController`, passing the current `INInteraction`, the set of `INParameter` values for the current row in the snippet stack, and an `INUIHostedViewContext` indicating whether the view appears in Siri, Maps, or Shortcuts. +5. Your view controller lays out its custom UI, calculates a `desiredSize`, and calls the completion handler with `(true, parameters, size)` to tell the system it will handle those parameters. +6. The system embeds your view controller's view into the Siri transcript, Maps card, or Shortcuts result in place of the default system UI. + +## Use Cases + +### Branded ride-booking confirmations +A ride-hailing app replaces the default Siri snippet with a map view showing driver location, estimated arrival time, and vehicle details in the app's visual style, giving users confidence they booked the right ride. + +### Rich messaging previews +A messaging app displays the sent message in a styled chat bubble with the recipient's avatar and read receipts, matching the look of the main app rather than relying on Siri's plain text summary. + +### Payment confirmation cards +A banking app shows a branded transaction summary with account balance, recipient info, and a confirmation checkmark, providing a more trustworthy visual than the generic Siri payment UI. + +### Workout summary display +A fitness app presents a workout completion card with a chart of heart rate data, calories burned, and elapsed time when the user ends a workout through Siri. + +## Key Classes + +| Class / Protocol | Role | +|-----------------|------| +| `INUIHostedViewControlling` | Protocol your view controller conforms to. Provides `configureView(for:of:interactiveBehavior:context:completion:)` for per-parameter UI and the simpler `configure(with:context:completion:)` for full-snippet replacement. | +| `INUIHostedViewContext` | Enum passed to `configureView` indicating where the UI is displayed: `.siri`, `.maps`, or `.hostedInApp` (Shortcuts). Lets you adapt layout for each surface. | +| `INUIInteractiveBehavior` | Enum indicating whether the view can receive interactions: `.none`, `.launch`, `.genericAction`, `.nextView`. Most Siri contexts are `.none` (read-only). | +| `INInteraction` | Contains the `INIntent` and optional `INIntentResponse` for the current request. Extract parameters and results from this to populate your UI. | +| `INParameter` | Identifies a specific parameter in the intent (e.g., recipients, amount). Used with `configureView` to replace individual rows in the Siri snippet stack. | +| `INUIHostedViewSiriProviding` | Optional protocol to tell Siri your extension will handle displaying specific content (maps, messages, payment transactions), preventing duplicate information. | + +## Implementation + +```swift +import IntentsUI + +// 1. Subclass UIViewController and conform to INUIHostedViewControlling. +// This is the principal class declared in the extension's Info.plist. +class IntentViewController: UIViewController, INUIHostedViewControlling { + + // 2. Create outlets or subviews for your custom content. + private let titleLabel = UILabel() + private let detailLabel = UILabel() + private let iconView = UIImageView() + + override func viewDidLoad() { + super.viewDidLoad() + + // 3. Set up your view hierarchy. Use Auto Layout to handle + // varying widths across Siri, Maps, and Shortcuts surfaces. + titleLabel.font = .boldSystemFont(ofSize: 17) + detailLabel.font = .systemFont(ofSize: 14) + detailLabel.textColor = .secondaryLabel + detailLabel.numberOfLines = 0 + + iconView.contentMode = .scaleAspectFit + iconView.tintColor = .systemBlue + + let stack = UIStackView(arrangedSubviews: [iconView, titleLabel, detailLabel]) + stack.axis = .vertical + stack.spacing = 8 + stack.alignment = .center + stack.translatesAutoresizingMaskIntoConstraints = false + view.addSubview(stack) + + NSLayoutConstraint.activate([ + iconView.widthAnchor.constraint(equalToConstant: 40), + iconView.heightAnchor.constraint(equalToConstant: 40), + stack.topAnchor.constraint(equalTo: view.topAnchor, constant: 16), + stack.leadingAnchor.constraint(equalTo: view.leadingAnchor, constant: 20), + stack.trailingAnchor.constraint(equalTo: view.trailingAnchor, constant: -20), + stack.bottomAnchor.constraint(lessThanOrEqualTo: view.bottomAnchor, constant: -16), + ]) + } + + // 4. Implement configureView to handle per-parameter UI replacement. + // The system calls this once per parameter row in the Siri snippet stack. + func configureView( + for parameters: Set, + of interaction: INInteraction, + interactiveBehavior: INUIInteractiveBehavior, + context: INUIHostedViewContext, + completion: @escaping (Bool, Set, CGSize) -> Void + ) { + // 5. Check the context to adapt layout for Siri vs. Maps vs. Shortcuts. + switch context { + case .siri: + view.backgroundColor = .systemBackground + case .mapsCard: + view.backgroundColor = .secondarySystemBackground + default: + view.backgroundColor = .systemBackground + } + + // 6. Extract intent data from the interaction to populate the UI. + if let sendIntent = interaction.intent as? INSendMessageIntent { + titleLabel.text = "Message Sent" + detailLabel.text = sendIntent.content ?? "No content" + iconView.image = UIImage(systemName: "checkmark.circle.fill") + } + + // 7. Calculate the desired size. Use the maximum allowed width + // and compute the height your content needs. + let maxSize = self.extensionContext!.hostedViewMaximumAllowedSize + let fittingSize = view.systemLayoutSizeFitting( + CGSize(width: maxSize.width, height: UIView.layoutFittingCompressedSize.height), + withHorizontalFittingPriority: .required, + verticalFittingPriority: .fittingSizeLevel + ) + + // 8. Call completion with true to indicate you are handling these + // parameters, pass back the parameters you consumed, and your size. + completion(true, parameters, CGSize(width: maxSize.width, height: fittingSize.height)) + } + + // 9. Fallback: the desiredSize property is used if configureView is not + // implemented. Return the maximum allowed size as a reasonable default. + var desiredSize: CGSize { + return self.extensionContext!.hostedViewMaximumAllowedSize + } +} +``` + +## Usage + +- Create: `bun create target intent-ui` +- Edit: `targets/intent-ui/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 10.0+ | Full support. `configureView` per-parameter API available from iOS 11+. | +| iPadOS | 10.0+ | Full support. Wider snippet area gives more layout room. | +| macOS | 12.0+ | Supported for Shortcuts results. Siri on Mac uses the same extension. | +| watchOS | -- | Not supported. watchOS Siri uses a different UI mechanism (WKInterfaceController). | +| tvOS | -- | Not supported. | +| visionOS | 1.0+ | Supported via compatible iOS intents. | + +## Gotchas + +- **Must be paired with an Intents extension.** The Intents UI extension only provides the visual layer. You must also have a separate Intents extension (or in-app intent handling on iOS 14+) that actually resolves, confirms, and handles the intent. The UI extension alone does nothing. +- **INUIHostedViewContext tells you where you are.** Your view appears in Siri (`.siri`), Maps (`.mapsCard`), or Shortcuts (`.hostedInApp`), and each surface has different size constraints and visual expectations. Always check the context parameter and adapt your layout accordingly. +- **Views do not receive touch events in Siri.** When `interactiveBehavior` is `.none` (the default in the Siri transcript), your view is read-only. Buttons and gesture recognizers will not fire. Only when the behavior is `.launch` or `.genericAction` can the user interact, and even then the interaction typically just opens your app. +- **Limited vertical space.** The system enforces a maximum height via `hostedViewMaximumAllowedSize`. If your view exceeds this, it will be clipped without warning. Always calculate your `desiredSize` within bounds and test on smaller devices. +- **IntentsSupported must be declared in both extensions.** The Info.plist for the Intents UI extension must list every intent class name it handles in the `IntentsSupported` array, just like the Intents extension. If an intent is missing from the UI extension's list, the system silently falls back to the default Siri snippet. +- **The extension is legacy for new projects.** Apple recommends the App Intents framework (iOS 16+) with `ShowsSnippetView` for new custom intent UI work. The Intents UI extension remains necessary only for built-in SiriKit domains (messaging, payments, ride booking) that have not migrated to App Intents. +- **Siri override control is limited to three content types.** If you implement `INUIHostedViewSiriProviding` to suppress duplicate system UI, only maps, message content, and payment transactions can be overridden. Other content types will still show the system default alongside your custom view. +- **iOS 13 Shortcuts regression.** On iOS 13, Siri sometimes redirects custom intent invocations to the Shortcuts app instead of displaying the intent UI inline. This was largely resolved in iOS 14 but may still affect users on older OS versions. diff --git a/plugins/expo/skills/apple-targets/intent.md b/plugins/expo/skills/apple-targets/intent.md new file mode 100644 index 0000000..f3b6ab1 --- /dev/null +++ b/plugins/expo/skills/apple-targets/intent.md @@ -0,0 +1,250 @@ +--- +title: Siri Intents Extension +description: Handles Siri voice commands and system intents using the legacy SiriKit framework, allowing your app to respond to predefined intent domains like messaging, payments, and ride booking. +version: iOS 10.0+ +--- + +# Siri Intents Extension (`intent`) + +A background extension that receives and processes Siri voice requests on behalf of your app. When a user speaks a command that matches one of Apple's predefined intent domains (messaging, payments, workouts, etc.), the system routes the structured intent to your extension, which resolves parameters, confirms feasibility, and handles the action -- all without launching your main app. + +## Apple Documentation + +- [SiriKit Overview](https://developer.apple.com/documentation/sirikit) +- [Creating an Intents App Extension](https://developer.apple.com/documentation/sirikit/intent_handling_infrastructure/creating_an_intents_app_extension) +- [INExtension](https://developer.apple.com/documentation/intents/inextension) +- [INSendMessageIntentHandling](https://developer.apple.com/documentation/intents/insendmessageintenthandling) +- [Intents Framework](https://developer.apple.com/documentation/intents) +- [NSExtensionPointIdentifier](https://developer.apple.com/documentation/bundleresources/information-property-list/nsextension/nsextensionpointidentifier) + +## WWDC History + +- **[WWDC 2016, Session 217 -- Introducing SiriKit](https://developer.apple.com/videos/play/wwdc2016/217/)** -- Introduced the Intents framework with support for messaging, VoIP calling, payments, photo search, workouts, and ride booking domains. +- **[WWDC 2016, Session 225 -- Extending Your Apps with SiriKit](https://developer.apple.com/videos/play/wwdc2016/225/)** -- Deep dive into building your first Intents extension and bringing custom UI into Siri. +- **[WWDC 2017, Session 214 -- What's New in SiriKit](https://developer.apple.com/videos/play/wwdc2017/214/)** -- Added lists and notes domain, visual codes, and new payment intents for iOS 11. +- **[WWDC 2018, Session 211 -- Introduction to Siri Shortcuts](https://developer.apple.com/videos/play/wwdc2018/211/)** -- Introduced custom intents via IntentDefinition files and Siri Shortcuts for iOS 12. +- **[WWDC 2019, Session 207 -- Introducing SiriKit Media Intents](https://developer.apple.com/videos/play/wwdc2019/207/)** -- Added INPlayMediaIntent for audio apps, enabling "Play X on MyApp" commands. +- **[WWDC 2020, Session 10073 -- Empower Your Intents](https://developer.apple.com/videos/play/wwdc2020/10073/)** -- In-app intent handling, rich disambiguation with images and subtitles, improved parameters. +- **[WWDC 2020, Session 10068 -- What's New in SiriKit and Shortcuts](https://developer.apple.com/videos/play/wwdc2020/10068/)** -- New shortcut running experience, folders, and event triggers in iOS 14. +- **[WWDC 2022, Session 10032 -- Dive into App Intents](https://developer.apple.com/videos/play/wwdc2022/10032/)** -- Introduced the modern App Intents framework as the successor for custom intents. Legacy SiriKit Intents remain necessary for messaging, media, and other built-in domains. + +## What It Does + +1. The user speaks a command to Siri that maps to a supported intent domain (e.g., "Send a message to John using MyApp"). +2. Siri parses the voice input into a structured `INIntent` object with typed parameters. +3. The system launches your Intents extension in the background and calls `handler(for:)` on your `INExtension` subclass. +4. Your handler implements a three-phase protocol: + - **Resolve** -- Validate and disambiguate each parameter (e.g., which "John" did they mean?). + - **Confirm** -- Verify the action is feasible (e.g., is the user authenticated? Is there network connectivity?). + - **Handle** -- Execute the action and return a response with a success or failure code. +5. Siri presents the result to the user, optionally showing your custom Intent UI extension if one is configured. + +## Use Cases + +### Messaging apps +A chat application that lets users send messages, search conversations, and mark messages as read entirely through Siri voice commands, without opening the app. + +### Ride booking services +A transportation app that allows users to request rides, check ride status, and get ETAs through Siri, integrating with the ride-booking intent domain. + +### Payment and banking apps +A banking app that handles "Send $50 to Sarah" commands through the payments domain, resolving contacts, confirming amounts, and executing transfers via Siri. + +### Fitness and workout apps +A workout tracker that starts, pauses, and ends workout sessions through Siri voice commands using the workouts intent domain. + +## Key Classes + +| Class / Protocol | Role | +|-----------------|------| +| `INExtension` | Principal class of the extension. Subclass it and override `handler(for:)` to route intents to the correct handler object. | +| `INIntent` | Base class for all system intents. Each domain has concrete subclasses like `INSendMessageIntent`. | +| `INIntentHandling` | Base protocol. Domain-specific protocols like `INSendMessageIntentHandling` add resolve/confirm/handle methods. | +| `INIntentResponse` | Base class for intent responses. Carry a response code (`.success`, `.failure`, etc.) and an optional `NSUserActivity` for handoff. | +| `INInteraction` | Wraps an intent and its response. Used for donating completed interactions to the system for Siri Suggestions. | +| `INPerson` | Represents a contact in messaging and calling intents. Contains handle, display name, and optional image. | + +## Implementation + +### Messaging Intent Handler (Resolve / Confirm / Handle) + +```swift +import Intents + +// 1. Subclass INExtension as the principal class declared in Info.plist. +// Route each intent type to the appropriate handler. +class IntentHandler: INExtension { + override func handler(for intent: INIntent) -> Any { + switch intent { + case is INSendMessageIntent: + return SendMessageHandler() + case is INSearchForMessagesIntent: + return SearchMessagesHandler() + default: + return self + } + } +} + +// 2. Implement the full resolve/confirm/handle cycle for sending messages. +class SendMessageHandler: NSObject, INSendMessageIntentHandling { + + // 3. RESOLVE: Disambiguate recipients. Siri calls this before confirmation. + func resolveRecipients( + for intent: INSendMessageIntent, + with completion: @escaping ([INSendMessageRecipientResolutionResult]) -> Void + ) { + guard let recipients = intent.recipients, !recipients.isEmpty else { + completion([.needsValue()]) + return + } + + var results = [INSendMessageRecipientResolutionResult]() + for recipient in recipients { + // 4. Look up matching contacts in your app's data store. + let matches = ContactStore.shared.findContacts(matching: recipient) + + switch matches.count { + case 0: + results.append(.unsupported()) + case 1: + results.append(.success(with: matches[0])) + default: + // 5. Multiple matches -- ask Siri to let the user pick one. + results.append(.disambiguation(with: matches)) + } + } + completion(results) + } + + // 6. RESOLVE: Validate the message content. + func resolveContent( + for intent: INSendMessageIntent, + with completion: @escaping (INStringResolutionResult) -> Void + ) { + if let text = intent.content, !text.isEmpty { + completion(.success(with: text)) + } else { + completion(.needsValue()) + } + } + + // 7. CONFIRM: Verify the app is ready to send (e.g., user is logged in). + func confirm( + intent: INSendMessageIntent, + completion: @escaping (INSendMessageIntentResponse) -> Void + ) { + guard AuthManager.shared.isAuthenticated else { + let response = INSendMessageIntentResponse(code: .failureRequiringAppLaunch, userActivity: nil) + completion(response) + return + } + completion(INSendMessageIntentResponse(code: .ready, userActivity: nil)) + } + + // 8. HANDLE: Actually send the message and return the result. + func handle( + intent: INSendMessageIntent, + completion: @escaping (INSendMessageIntentResponse) -> Void + ) { + guard let recipients = intent.recipients, + let content = intent.content else { + completion(INSendMessageIntentResponse(code: .failure, userActivity: nil)) + return + } + + MessageService.shared.send( + content: content, + to: recipients.compactMap { $0.personHandle?.value } + ) { success in + let code: INSendMessageIntentResponseCode = success ? .success : .failure + let activity = NSUserActivity(activityType: NSStringFromClass(INSendMessageIntent.self)) + completion(INSendMessageIntentResponse(code: code, userActivity: activity)) + } + } +} + +// 9. Donate completed interactions so the system can offer Siri Suggestions. +func donateInteraction(for intent: INSendMessageIntent, response: INSendMessageIntentResponse) { + let interaction = INInteraction(intent: intent, response: response) + interaction.donate { error in + if let error = error { + print("Donation failed: \(error.localizedDescription)") + } + } +} +``` + +### Custom Intent via IntentDefinition File + +For domains not covered by built-in intents, you can define custom intents using an `.intentdefinition` file in Xcode: + +```swift +import Intents + +// 1. Xcode auto-generates OrderCoffeeIntent from your IntentDefinition file. +// Implement the generated handling protocol. +class OrderCoffeeHandler: NSObject, OrderCoffeeIntentHandling { + + // 2. Resolve the coffee size parameter. + func resolveCoffeeSize( + for intent: OrderCoffeeIntent, + with completion: @escaping (CoffeeSizeResolutionResult) -> Void + ) { + if intent.coffeeSize == .unknown { + completion(.needsValue()) + } else { + completion(.success(with: intent.coffeeSize)) + } + } + + // 3. Confirm the order is valid. + func confirm( + intent: OrderCoffeeIntent, + completion: @escaping (OrderCoffeeIntentResponse) -> Void + ) { + completion(OrderCoffeeIntentResponse(code: .ready, userActivity: nil)) + } + + // 4. Place the order. + func handle( + intent: OrderCoffeeIntent, + completion: @escaping (OrderCoffeeIntentResponse) -> Void + ) { + CoffeeOrderService.shared.place( + size: intent.coffeeSize, + flavor: intent.flavor + ) { orderId in + let response = OrderCoffeeIntentResponse(code: .success, userActivity: nil) + response.orderNumber = NSNumber(value: orderId) + completion(response) + } + } +} +``` + +## Usage + +- Create: `bun create target intent` +- Edit: `targets/intent/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 10.0+ | Full support for all intent domains. | +| iPadOS | 10.0+ | Full support. | +| macOS | 11.0+ | Limited domain support (primarily messaging and media). | +| watchOS | 3.2+ | Supports intents via a separate watchOS extension target. | +| tvOS | -- | Not supported. | +| visionOS | 1.0+ | Supported via compatible iOS intents. | + +## Gotchas + +- **IntentDefinition file must be in both targets.** If you define custom intents, the `.intentdefinition` file must be included in both the Intents extension and your main app target. Missing it from the app target causes Siri Shortcuts to fail silently. +- **Extension memory limit is ~30 MB.** Intents extensions run in a constrained environment. Loading large frameworks, images, or data stores will cause the system to kill your extension with no crash log visible to the user. +- **The Intents framework is legacy for custom intents.** Apple recommends App Intents (iOS 16+) for all new custom intent work. SiriKit Intents remain required only for built-in domains like messaging (`INSendMessageIntent`) and media (`INPlayMediaIntent`) that have not yet migrated to App Intents. +- **`handler(for:)` must return synchronously.** You cannot perform async work in this method. Return the handler object immediately and do all async work in the resolve/confirm/handle methods. +- **IntentsSupported in Info.plist must match.** The `IntentsSupported` array in your extension's Info.plist must list every intent class name you handle. If an intent is missing from this list, Siri will not route it to your extension, with no error or diagnostic message. +- **Siri Shortcuts donation requires the main app.** Donating `INInteraction` objects to improve Siri Suggestions only works from the main app process, not from the extension. Donate after the user completes the action in your app. +- **In-app intent handling (iOS 14+) can replace the extension.** For simpler use cases, you can handle intents directly in your main app by registering handlers on `INInteraction`. This avoids the extension's memory constraints but requires your app to launch. diff --git a/plugins/expo/skills/apple-targets/keyboard.md b/plugins/expo/skills/apple-targets/keyboard.md new file mode 100644 index 0000000..08631a6 --- /dev/null +++ b/plugins/expo/skills/apple-targets/keyboard.md @@ -0,0 +1,213 @@ +--- +title: Custom Keyboard Extension +description: Replaces the system keyboard with a fully custom input interface available across all apps. +version: iOS 8.0+ +--- + +# Custom Keyboard Extension (`keyboard`) + +A custom keyboard extension replaces the standard system keyboard with your own input interface, available system-wide across all apps once the user enables it in Settings. The extension subclasses `UIInputViewController` and communicates with the active text field through a `textDocumentProxy` object, which allows inserting and deleting text without direct access to the text field itself. Keyboards run in a restrictive sandbox by default -- network access, pasteboard reading, and shared container access all require the user to grant "Allow Full Access" in Settings. Data sharing with the containing app uses App Groups. + +## Apple Documentation + +- [Creating a Custom Keyboard](https://developer.apple.com/documentation/uikit/creating-a-custom-keyboard) +- [UIInputViewController](https://developer.apple.com/documentation/uikit/uiinputviewcontroller) +- [Configuring a Custom Keyboard Interface](https://developer.apple.com/documentation/uikit/configuring-a-custom-keyboard-interface) +- [UITextDocumentProxy](https://developer.apple.com/documentation/uikit/uitextdocumentproxy) +- [App Extension Programming Guide: Custom Keyboard](https://developer.apple.com/library/archive/documentation/General/Conceptual/ExtensibilityPG/CustomKeyboard.html) +- [UILexicon](https://developer.apple.com/documentation/uikit/uilexicon) + +## WWDC History + +- **[WWDC 2014, Session 205 -- Creating Extensions for iOS and OS X, Part 1](https://developer.apple.com/videos/play/wwdc2014/205/)** -- Introduced the extension model in iOS 8, including custom keyboard extensions as a new extension point. +- **[WWDC 2014, Session 217 -- Creating Extensions for iOS and OS X, Part 2](https://developer.apple.com/videos/play/wwdc2014/217/)** -- Deeper implementation details for building extensions, covering keyboard lifecycle and sandboxing. +- **[WWDC 2017, Session 242 -- The Keys to a Better Text Input Experience](https://developer.apple.com/videos/play/wwdc2017/242/)** -- Best practices for text input, QuickType support, and upgrading a custom keyboard to a system-wide extension. + +## What It Does + +1. **User enables the keyboard.** The user goes to Settings > General > Keyboards > Add New Keyboard and selects your keyboard from the list of third-party keyboards. +2. **System instantiates UIInputViewController.** When the user switches to your keyboard (via the globe key), iOS loads your extension and calls `viewDidLoad()` on your `UIInputViewController` subclass. +3. **You build the UI.** Add buttons, labels, or host a SwiftUI view inside the controller's `inputView`. The keyboard draws within the bounds of this view. +4. **Text input goes through textDocumentProxy.** Call `textDocumentProxy.insertText(_:)` to type characters and `textDocumentProxy.deleteBackward()` to delete. You can read limited surrounding context via `documentContextBeforeInput` and `documentContextAfterInput`. +5. **Globe key switches keyboards.** Call `advanceToNextInputMode()` or use `handleInputModeList(from:with:)` to let the user switch to the next keyboard. The system manages keyboard order. +6. **Height is controlled via constraints.** Set a height constraint on `inputView` to change the keyboard height from the default. +7. **Full Access unlocks advanced features.** If `RequestsOpenAccess` is set to `true` in Info.plist and the user enables Full Access in Settings, the keyboard gains network access, pasteboard reading, shared container access via App Groups, and location services. + +## Use Cases + +### Custom layouts and languages +A keyboard that provides a layout not available on the standard system keyboard, such as Dvorak, Colemak, or a layout for a language Apple does not natively support. The extension renders the key grid in `viewDidLoad` and inserts characters via `textDocumentProxy.insertText(_:)`. + +### GIF and sticker keyboards +A media keyboard that searches a GIF API and inserts images into the text field. Requires Full Access for network requests. Uses `textDocumentProxy.insertText(_:)` with a special pasteboard workaround, or relies on the host app supporting `UIPasteboard` for image input. + +### Specialized input methods +A keyboard for entering mathematical notation, music symbols, or code snippets. The extension presents a domain-specific set of buttons and inserts the corresponding Unicode characters or formatted strings. + +### Accessibility keyboards +A keyboard with enlarged keys, high-contrast colors, or simplified layouts designed for users with motor or visual impairments. Can also add haptic feedback via `UIFeedbackGenerator`. + +## Key Classes + +| Class / Protocol | Role | +|------------------|------| +| `UIInputViewController` | Primary view controller for the keyboard extension. Subclass this to build your keyboard UI. | +| `UITextDocumentProxy` | Proxy object for interacting with the current text field. Provides `insertText(_:)`, `deleteBackward()`, and context properties. | +| `UILexicon` | System-provided lexicon containing unpaired first names from Contacts and text shortcuts from Settings. Use for autocorrect suggestions. | +| `UIInputView` | The root view of the keyboard. Set `inputView` to a custom instance to control appearance and height. | +| `NSExtensionContext` | The extension context. Rarely used directly in keyboard extensions. | + +## Implementation + +```swift +import UIKit + +// 1. Subclass UIInputViewController -- this is the entry point for any custom keyboard. +class KeyboardViewController: UIInputViewController { + + private var nextKeyboardButton: UIButton! + private var keyboardStackView: UIStackView! + + override func viewDidLoad() { + super.viewDidLoad() + + // 2. Build the keyboard layout using a vertical stack of horizontal rows. + keyboardStackView = UIStackView() + keyboardStackView.axis = .vertical + keyboardStackView.spacing = 6 + keyboardStackView.translatesAutoresizingMaskIntoConstraints = false + view.addSubview(keyboardStackView) + + NSLayoutConstraint.activate([ + keyboardStackView.topAnchor.constraint(equalTo: view.topAnchor, constant: 8), + keyboardStackView.leadingAnchor.constraint(equalTo: view.leadingAnchor, constant: 4), + keyboardStackView.trailingAnchor.constraint(equalTo: view.trailingAnchor, constant: -4), + ]) + + let rows = ["QWERTYUIOP", "ASDFGHJKL", "ZXCVBNM"] + for row in rows { + let rowStack = UIStackView() + rowStack.axis = .horizontal + rowStack.spacing = 4 + rowStack.distribution = .fillEqually + for char in row { + let key = UIButton(type: .system) + key.setTitle(String(char), for: .normal) + key.titleLabel?.font = .systemFont(ofSize: 22) + key.backgroundColor = UIColor.systemBackground + key.layer.cornerRadius = 5 + // 3. Each key inserts its character into the active text field via textDocumentProxy. + key.addTarget(self, action: #selector(keyTapped(_:)), for: .touchUpInside) + rowStack.addArrangedSubview(key) + } + keyboardStackView.addArrangedSubview(rowStack) + } + + // 4. Build the bottom row with globe, space, backspace, and return keys. + let bottomRow = UIStackView() + bottomRow.axis = .horizontal + bottomRow.spacing = 4 + bottomRow.distribution = .fill + + // 5. The globe button is REQUIRED when needsInputModeSwitchKey is true. + // Use handleInputModeList(from:with:) for the long-press keyboard picker. + nextKeyboardButton = UIButton(type: .system) + nextKeyboardButton.setImage(UIImage(systemName: "globe"), for: .normal) + nextKeyboardButton.addTarget(self, action: #selector(handleInputModeList(from:with:)), for: .allTouchEvents) + nextKeyboardButton.widthAnchor.constraint(equalToConstant: 44).isActive = true + bottomRow.addArrangedSubview(nextKeyboardButton) + + let spaceButton = UIButton(type: .system) + spaceButton.setTitle("space", for: .normal) + spaceButton.backgroundColor = UIColor.systemBackground + spaceButton.layer.cornerRadius = 5 + spaceButton.addTarget(self, action: #selector(spaceTapped), for: .touchUpInside) + bottomRow.addArrangedSubview(spaceButton) + + let deleteButton = UIButton(type: .system) + deleteButton.setImage(UIImage(systemName: "delete.left"), for: .normal) + deleteButton.addTarget(self, action: #selector(deleteTapped), for: .touchUpInside) + deleteButton.widthAnchor.constraint(equalToConstant: 44).isActive = true + bottomRow.addArrangedSubview(deleteButton) + + let returnButton = UIButton(type: .system) + returnButton.setTitle("return", for: .normal) + returnButton.addTarget(self, action: #selector(returnTapped), for: .touchUpInside) + returnButton.widthAnchor.constraint(equalToConstant: 72).isActive = true + bottomRow.addArrangedSubview(returnButton) + + keyboardStackView.addArrangedSubview(bottomRow) + + // 6. Set a custom height for the keyboard. The default is ~216pt on iPhone. + let heightConstraint = NSLayoutConstraint( + item: view!, attribute: .height, + relatedBy: .equal, + toItem: nil, attribute: .notAnAttribute, + multiplier: 1.0, constant: 260 + ) + heightConstraint.priority = .defaultHigh + view.addConstraint(heightConstraint) + } + + override func viewWillLayoutSubviews() { + super.viewWillLayoutSubviews() + // 7. Hide the globe button on devices that provide it natively (e.g. iPhone X+). + nextKeyboardButton.isHidden = !needsInputModeSwitchKey + } + + @objc func keyTapped(_ sender: UIButton) { + guard let title = sender.title(for: .normal) else { return } + // 8. Insert the character into the current text field. + textDocumentProxy.insertText(title) + } + + @objc func spaceTapped() { + textDocumentProxy.insertText(" ") + } + + @objc func deleteTapped() { + // 9. Delete one character backward. + textDocumentProxy.deleteBackward() + } + + @objc func returnTapped() { + textDocumentProxy.insertText("\n") + } + + override func textDidChange(_ textInput: UITextInput?) { + // 10. Adapt key colors to match the host app's keyboard appearance (light/dark). + let isDark = textDocumentProxy.keyboardAppearance == .dark + let textColor: UIColor = isDark ? .white : .black + nextKeyboardButton.tintColor = textColor + } +} +``` + +## Usage + +- Create: `bun create target keyboard` +- Edit: `targets/keyboard/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 8.0+ | Full custom keyboard support. Globe key provided by system on iPhone X+ (iOS 11+). | +| iPadOS | 8.0+ | Supports split-screen and floating keyboard contexts. | +| macOS | -- | Not supported. macOS uses Input Methods (IMKit), not keyboard extensions. | +| watchOS | -- | Not supported. | +| visionOS | -- | Not supported. | +| tvOS | -- | Not supported. | + +## Gotchas + +- **You must implement a globe/next-keyboard button or Apple will reject your app.** Check `needsInputModeSwitchKey` in `viewWillLayoutSubviews` and show or hide the button accordingly. On devices with a system-level globe key (iPhone X and later), `needsInputModeSwitchKey` returns `false` and your button should be hidden. On all other devices, omitting this button makes it impossible to switch keyboards and triggers an App Store rejection. +- **Keyboards are heavily sandboxed by default.** Without Full Access enabled, the extension cannot access the network, read the pasteboard, use shared containers, or access location services. Your keyboard must provide useful basic functionality without Full Access. +- **No camera or microphone access.** Keyboard extensions are explicitly prohibited from accessing the camera and microphone APIs, even with Full Access enabled. Dictation input is not available to third-party keyboards. +- **Secure text fields always use the system keyboard.** When the user taps a password field (`isSecureTextEntry = true`), iOS automatically switches to the system keyboard. Your extension has no control over this behavior. +- **Host apps can block custom keyboards entirely.** An app developer can return `false` from `application(_:shouldAllowExtensionPointIdentifier:)` to force the system keyboard in their app. Banking and healthcare apps commonly do this. +- **Height must be set via a constraint on inputView.** There is no `preferredContentSize` API for keyboards. Set a height constraint with `.defaultHigh` priority on `self.view` (the `inputView`). The default height is approximately 216 points on iPhone. +- **Autocorrect and predictive text must be self-implemented.** The system does not provide autocorrect, spell-checking, or predictive text to third-party keyboards. You can use `requestSupplementaryLexicon(completion:)` to get a `UILexicon` containing the user's text shortcuts and contact names, but building a full autocorrect engine is your responsibility. +- **Memory limit is strict (~40-50 MB).** iOS aggressively terminates keyboard extensions that exceed the memory budget. Avoid loading large assets, dictionaries, or ML models directly into the extension process. Consider lazy loading and memory-mapped files. +- **textDocumentProxy context is limited.** `documentContextBeforeInput` and `documentContextAfterInput` only return a limited window of surrounding text (often just the current paragraph or less). You cannot read the entire document or access text outside the visible context. +- **App Groups are required for sharing data with the containing app.** The keyboard extension runs in a separate process with its own container. Use `UserDefaults(suiteName:)` or `FileManager.containerURL(forSecurityApplicationGroupIdentifier:)` to share settings, learned words, or user preferences. The `@bacons/apple-targets` plugin syncs app groups automatically when `appGroupsByDefault` is set. diff --git a/plugins/expo/skills/apple-targets/location-push.md b/plugins/expo/skills/apple-targets/location-push.md new file mode 100644 index 0000000..2d09241 --- /dev/null +++ b/plugins/expo/skills/apple-targets/location-push.md @@ -0,0 +1,177 @@ +--- +title: Location Push Service Extension +description: Receives a special APNs push notification and requests a one-shot device location for location-sharing apps without requiring the app to be running. +version: iOS 15.0+ +--- + +# Location Push Service Extension (`location-push`) + +A Location Push Service Extension gives location-sharing apps a power-efficient way to retrieve a user's location on demand. When your server sends a specially typed APNs push (`"location"` push type), the system wakes the extension in the background -- even if the app is not running. The extension creates a `CLLocationManager`, calls `requestLocation()`, and sends the result back to your server. This avoids the need for continuous background location tracking, dramatically reducing battery impact. The extension is gated behind a managed entitlement (`com.apple.developer.location.push`) that requires explicit approval from Apple. + +## Apple Documentation + +- [CLLocationPushServiceExtension](https://developer.apple.com/documentation/corelocation/cllocationpushserviceextension) +- [Creating a Location Push Service Extension](https://developer.apple.com/documentation/corelocation/creating-a-location-push-service-extension) +- [Location Push Service Extension Entitlement](https://developer.apple.com/documentation/bundleresources/entitlements/com.apple.developer.location.push) +- [CLLocationManager](https://developer.apple.com/documentation/corelocation/cllocationmanager) +- [Core Location Framework](https://developer.apple.com/documentation/corelocation) + +## WWDC History + +- **[WWDC 2021, Session 10102 -- Meet the Location Button](https://developer.apple.com/videos/play/wwdc2021/10102/)** -- Introduced the Location Push Service Extension alongside the CLLocationButton, covering the push-based location retrieval architecture and entitlement requirements. + +## What It Does + +1. **App registers for location pushes.** Your main app calls `CLLocationManager.startMonitoringLocationPushes(completion:)` to receive an APNs device token specific to location pushes. You send this token to your server. +2. **Server sends a location push.** Your server sends an APNs notification with push type `"location"` and topic `.location-query` to the device token. The payload can contain arbitrary data (e.g. a request ID). +3. **System wakes the extension.** iOS launches your `CLLocationPushServiceExtension` and calls `didReceiveLocationPushPayload(_:completion:)` with the push payload dictionary. +4. **Extension requests location.** You create a `CLLocationManager`, set its delegate, and call `requestLocation()`. The system delivers a single location fix to your delegate's `locationManager(_:didUpdateLocations:)` method. +5. **Extension sends location to server.** You process the location (ideally end-to-end encrypting it for the requesting user), send it to your server via a network request, and call the provided `completion()` handler. +6. **System terminates the extension.** After you call `completion()`, or after approximately 30 seconds, the system terminates the extension. If you fail to call `completion()`, `serviceExtensionWillTerminate()` is called as a last chance. + +## Use Cases + +### Find My-style device location sharing + +A family safety app lets users request each other's location. When User A taps "Find User B," the server sends a location push to User B's device. The extension retrieves the location, encrypts it, and posts it back. User A sees the updated location on a map -- all without User B needing to have the app open. + +### Fleet and asset tracking + +A logistics company needs periodic location updates from delivery drivers' phones. Instead of maintaining constant GPS tracking (which drains battery), the dispatch system sends location pushes at defined intervals (e.g. every 15 minutes) to collect position reports on demand. + +### Emergency check-in systems + +A workplace safety app allows a control room to ping field workers' devices for their current location during an emergency roll call. The location push fires even if the worker's app is backgrounded or terminated. + +## Key Classes + +| Class / Protocol | Role | +|------------------|------| +| `CLLocationPushServiceExtension` | Protocol your extension's principal class adopts. Provides `didReceiveLocationPushPayload(_:completion:)` and `serviceExtensionWillTerminate()`. | +| `CLLocationManager` | Used within the extension to call `requestLocation()` for a single location fix. Also used in the main app to call `startMonitoringLocationPushes(completion:)` for token registration. | +| `CLLocationManagerDelegate` | Delegate protocol that receives the location result via `locationManager(_:didUpdateLocations:)` or errors via `locationManager(_:didFailWithError:)`. | +| `CLLocation` | The location object containing latitude, longitude, altitude, accuracy, and timestamp. | + +## Implementation + +```swift +import CoreLocation + +// 1. Adopt CLLocationPushServiceExtension on your principal class. Also +// conform to CLLocationManagerDelegate to receive the location result. +class LocationPushService: NSObject, + CLLocationPushServiceExtension, + CLLocationManagerDelegate +{ + var completion: (() -> Void)? + var locationManager: CLLocationManager? + + // 2. Called when a location push arrives. The payload dictionary contains + // whatever your server included in the APNs notification body. + func didReceiveLocationPushPayload( + _ payload: [String: Any], + completion: @escaping () -> Void + ) { + self.completion = completion + + // 3. Create a location manager and request a single location fix. + // The user must have granted "Always" location permission to the + // main app for this to succeed. + self.locationManager = CLLocationManager() + self.locationManager!.delegate = self + self.locationManager!.requestLocation() + } + + // 4. Called if the system is about to terminate the extension before + // completion() was called. Use this as a last-chance cleanup. + func serviceExtensionWillTerminate() { + self.completion?() + } + + // MARK: - CLLocationManagerDelegate + + func locationManager( + _ manager: CLLocationManager, + didUpdateLocations locations: [CLLocation] + ) { + guard let location = locations.first else { + self.completion?() + return + } + + // 5. Process the location. In a real app, you would end-to-end + // encrypt the coordinates and send them to your server. + let latitude = location.coordinate.latitude + let longitude = location.coordinate.longitude + print("Location: \(latitude), \(longitude)") + + // 6. Send location to your server (simplified example). + sendLocationToServer(latitude: latitude, longitude: longitude) { + // 7. ALWAYS call completion() when done. Failing to call it + // will cause the system to terminate the extension after ~30s. + self.completion?() + } + } + + func locationManager( + _ manager: CLLocationManager, + didFailWithError error: Error + ) { + // 8. Location request failed (e.g. location services disabled, + // no GPS fix available). Still must call completion(). + print("Location error: \(error.localizedDescription)") + self.completion?() + } + + private func sendLocationToServer( + latitude: Double, + longitude: Double, + done: @escaping () -> Void + ) { + // 9. In production, create a URLSession data task to POST the + // encrypted location to your server. Call done() in the + // completion handler. + var request = URLRequest( + url: URL(string: "https://api.example.com/location")! + ) + request.httpMethod = "POST" + request.httpBody = try? JSONSerialization.data(withJSONObject: [ + "lat": latitude, + "lng": longitude, + "timestamp": Date().timeIntervalSince1970, + ]) + + URLSession.shared.dataTask(with: request) { _, _, _ in + done() + }.resume() + } +} +``` + +## Usage + +- Create: `bun create target location-push` +- Edit: `targets/location-push/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 15.0+ | Full support. Requires managed entitlement approval from Apple. | +| iPadOS | 15.0+ | Supported on iPads with location hardware. | +| macOS | -- | Not supported. | +| watchOS | -- | Not supported. | +| visionOS | -- | Not supported. | +| tvOS | -- | Not supported. | + +## Gotchas + +- **Requires Apple entitlement approval.** The `com.apple.developer.location.push` entitlement is a managed capability. You must apply through your Apple Developer account and be approved before you can distribute the app. Development provisioning profiles may work without approval, but distribution profiles require it. Approval can take weeks to months. +- **User must grant "Always" location permission.** The extension only fires if the user has granted your app "Always" location authorization. "When In Use" is not sufficient because the extension runs when the app is not in the foreground. +- **APNs push type must be "location".** The push notification sent by your server must use `apns-push-type: location` in the HTTP/2 headers and the topic must be `.location-query`. Using the wrong push type or topic will silently fail. +- **Extension has approximately 30 seconds to complete.** If you do not call the `completion()` handler within about 30 seconds, the system calls `serviceExtensionWillTerminate()` and kills the extension. Plan your network requests accordingly. +- **Single location fix only.** `requestLocation()` delivers one location and then stops. You cannot start continuous location updates or use `startUpdatingLocation()` within the extension. The fix accuracy depends on available hardware and conditions. +- **Deployment target mismatch causes silent failure.** Xcode may set the extension's deployment target to the latest iOS version by default. If the device runs an older iOS version, the extension silently fails with a `missingPushExtension` error. Ensure the deployment target is set to iOS 15.0. +- **No background processing beyond the location request.** The extension cannot schedule background tasks, play audio, or perform other background activities. It exists solely to handle the location push and respond. +- **End-to-end encryption is strongly recommended.** Apple designed this API for peer-to-peer location sharing. Transmitting raw coordinates to a server without encryption is a privacy concern and may affect App Store review. +- **Token is separate from regular push tokens.** The token returned by `startMonitoringLocationPushes(completion:)` is distinct from your regular APNs push token. You must manage and send this token separately to your server. diff --git a/plugins/expo/skills/apple-targets/matter.md b/plugins/expo/skills/apple-targets/matter.md new file mode 100644 index 0000000..d13cffc --- /dev/null +++ b/plugins/expo/skills/apple-targets/matter.md @@ -0,0 +1,160 @@ +--- +title: Matter Device Setup Extension +description: Provides a custom commissioning flow for pairing Matter smart home devices into your ecosystem, letting you customize network selection, credential validation, and room assignment. +version: iOS 16.1+ +--- + +# Matter Device Setup Extension (`matter`) + +A Matter device setup extension lets your smart home app customize the commissioning experience when users add Matter-compatible accessories. The system handles the Matter protocol handshake, QR code scanning, and network credential management, while your extension customizes the setup UX by validating device credentials, selecting Wi-Fi or Thread networks, commissioning the device onto your fabric, and configuring room assignments. The extension launches in response to `MatterAddDeviceRequest.perform()` called from your main app and runs as a separate process. + +## Apple Documentation + +- [MatterSupport Framework Overview](https://developer.apple.com/documentation/mattersupport/) +- [MatterAddDeviceExtensionRequestHandler](https://developer.apple.com/documentation/mattersupport/matteradddeviceextensionrequesthandler) +- [Adding Matter Support to Your Ecosystem](https://developer.apple.com/documentation/mattersupport/adding-matter-support-to-your-ecosystem) +- [MatterAddDeviceRequest](https://developer.apple.com/documentation/mattersupport/matteradddevicerequest) +- [Matter Framework (low-level)](https://developer.apple.com/documentation/matter) +- [Apple Home -- Matter](https://developer.apple.com/apple-home/matter/) + +## WWDC History + +- **[WWDC 2021, Session 10298 -- Add Support for Matter in Your Smart Home App](https://developer.apple.com/videos/play/wwdc2021/10298/)** -- Introduced Matter as a new industry standard for smart home accessories, covered the initial HomeKit + Matter integration, and explained how iOS handles accessory pairing, network provisioning, and multi-ecosystem support. + +## What It Does + +1. **Your app initiates pairing.** The main app creates a `MatterAddDeviceRequest` and calls `perform()`. iOS takes over and presents the pairing UI, including QR code scanning. +2. **System performs device discovery.** iOS scans the Matter setup code, discovers the device over BLE, and begins the PASE (Passcode Authenticated Session Establishment) handshake. +3. **Extension validates credentials.** The system calls `validateDeviceCredential(_:)` so your extension can perform additional attestation checks against your ecosystem's requirements. +4. **Extension selects network.** Depending on the device's capabilities, the system calls `selectWiFiNetwork(from:)` or `selectThreadNetwork(from:)`. Your extension can pick a specific network or return `.defaultSystemNetwork` to use the device's current network. +5. **Extension commissions the device.** The system calls `commissionDevice(in:onboardingPayload:commissioningID:)` with a payload your extension uses to commission the device onto your fabric via your Matter stack. +6. **Extension configures rooms.** The system calls `rooms(in:)` to get available rooms, then `configureDevice(named:in:)` once the user has named the device and assigned it to a room. + +## Use Cases + +### Smart home hub manufacturers +A company building a smart home ecosystem app uses the extension to commission Matter devices onto their proprietary fabric, ensuring the device is registered with their cloud backend and assigned to the correct home and room in their system. + +### Lighting and appliance manufacturers +A lighting company ships a companion app that uses the Matter extension to pair their bulbs and switches. The extension validates that the device is a genuine product via `validateDeviceCredential(_:)` before commissioning, and selects the optimal Thread network based on signal strength. + +### Multi-ecosystem pairing +A smart home platform uses the extension to add devices that are already paired with Apple Home or another ecosystem. Matter's multi-admin feature allows the same physical device to exist on multiple fabrics, and the extension handles the second-fabric commissioning flow. + +## Key Classes + +| Class / Protocol | Role | +|------------------|------| +| `MatterAddDeviceExtensionRequestHandler` | Abstract base class you subclass. Provides overridable methods for each step of the commissioning flow. | +| `MatterAddDeviceRequest` | Created by your main app to initiate pairing. Call `perform()` to launch the system UI and your extension. | +| `MatterAddDeviceRequest.Home` | Represents a home in your ecosystem. Passed to `commissionDevice(in:...)` and `rooms(in:)`. | +| `MatterAddDeviceRequest.Room` | Represents a room within a home. Return these from `rooms(in:)` for the user to choose from. | +| `MatterAddDeviceExtensionRequestHandler.DeviceCredential` | Contains attestation data for the device. Passed to `validateDeviceCredential(_:)`. | +| `MatterAddDeviceExtensionRequestHandler.WiFiScanResult` | A discovered Wi-Fi network with SSID and signal info. | +| `MatterAddDeviceExtensionRequestHandler.ThreadScanResult` | A discovered Thread network with network name and extended PAN ID. | +| `MatterAddDeviceExtensionRequestHandler.WiFiNetworkAssociation` | Return type for `selectWiFiNetwork(from:)`. Use `.defaultSystemNetwork` or `.network(wifi:)`. | +| `MatterAddDeviceExtensionRequestHandler.ThreadNetworkAssociation` | Return type for `selectThreadNetwork(from:)`. Use `.defaultSystemNetwork` or `.network(thread:)`. | + +## Implementation + +```swift +import MatterSupport + +// 1. Subclass MatterAddDeviceExtensionRequestHandler as the principal class. +class RequestHandler: MatterAddDeviceExtensionRequestHandler { + + // 2. Validate the device's attestation certificate against your ecosystem. + override func validateDeviceCredential( + _ deviceCredential: MatterAddDeviceExtensionRequestHandler.DeviceCredential + ) async throws { + // Check the device's DAC (Device Attestation Certificate) against + // your ecosystem's list of approved vendors. + let isApproved = try await EcosystemAPI.shared.validateAttestation( + dacCertificate: deviceCredential.certificationDeclaration + ) + guard isApproved else { + throw MatterAddDeviceError.attestationFailed + } + } + + // 3. Choose a Wi-Fi network for the device. + override func selectWiFiNetwork( + from wifiScanResults: [MatterAddDeviceExtensionRequestHandler.WiFiScanResult] + ) async throws -> MatterAddDeviceExtensionRequestHandler.WiFiNetworkAssociation { + // Return .defaultSystemNetwork to use the iPhone's current Wi-Fi, + // or pick a specific network from the scan results. + return .defaultSystemNetwork + } + + // 4. Choose a Thread network for Thread-capable devices. + override func selectThreadNetwork( + from threadScanResults: [MatterAddDeviceExtensionRequestHandler.ThreadScanResult] + ) async throws -> MatterAddDeviceExtensionRequestHandler.ThreadNetworkAssociation { + // Use the system default Thread network unless your ecosystem + // manages its own border routers. + return .defaultSystemNetwork + } + + // 5. Commission the device onto your fabric using the onboarding payload. + override func commissionDevice( + in home: MatterAddDeviceRequest.Home?, + onboardingPayload: String, + commissioningID: UUID + ) async throws { + // The system has already opened a commissioning window on the device. + // Use the onboardingPayload to establish a CASE session and write + // your fabric credentials. + try await MatterStack.shared.commission( + payload: onboardingPayload, + commissioningID: commissioningID, + homeID: home?.displayName + ) + } + + // 6. Return the rooms in the selected home for the user to pick from. + override func rooms( + in home: MatterAddDeviceRequest.Home? + ) async -> [MatterAddDeviceRequest.Room] { + guard let home else { return [] } + let rooms = await EcosystemAPI.shared.fetchRooms(for: home.displayName) + return rooms.map { MatterAddDeviceRequest.Room(displayName: $0.name) } + } + + // 7. Finalize configuration after the user names the device and picks a room. + override func configureDevice( + named name: String, + in room: MatterAddDeviceRequest.Room? + ) async { + await EcosystemAPI.shared.registerDevice( + name: name, + room: room?.displayName + ) + } +} +``` + +## Usage + +- Create: `bun create target matter` +- Edit: `targets/matter/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 16.1+ | Full MatterSupport framework with device setup extension. | +| iPadOS | 16.1+ | Supported. | +| macOS | -- | Not supported. Matter commissioning is iOS/iPadOS only. | +| watchOS | -- | Not supported. | +| tvOS | 16.1+ | HomePod and Apple TV act as Matter controllers but do not run extensions. | +| visionOS | -- | Not supported. | + +## Gotchas + +- **Entitlement required for distribution.** Development builds can use the "MatterSupport -- Development Only" entitlement enabled in Xcode. For App Store distribution, you must request the managed `com.apple.developer.matter.allow-setup-payload` entitlement from Apple. +- **Extension and app cannot share data bidirectionally.** You can read App Group data written by the main app inside the extension, but writing data from the extension back to the main app via App Groups is unreliable. Use `os_log` and Console.app for debugging the extension. +- **The system may abort the flow after first-fabric commissioning.** A known issue causes `HomeUIService` to stop the setup flow after commissioning to the first fabric, displaying `HMErrorDomain Code=2`. This can happen intermittently and is outside your control. +- **Thread device commissioning can fail with custom border routers.** If your ecosystem manages its own Thread border router rather than using the Apple Home infrastructure, selecting a custom Thread network in `selectThreadNetwork(from:)` may fail silently. Test thoroughly with your specific hardware. +- **`commissionDevice` is not called on re-pairing.** If a device was previously removed from a third-party ecosystem but its fabric entry remains in the iOS keychain, the system may skip calling `commissionDevice` entirely. The user may need to factory-reset the device. +- **The extension has no UI of its own.** The system manages the entire visual flow (QR scanning, home/room selection, progress). Your extension only provides data and logic via the handler overrides. You cannot present custom view controllers. +- **Matter protocol version matters.** Ensure your Matter stack supports the same specification version as the device. Mismatched versions can cause silent commissioning failures that surface as generic errors in the extension. diff --git a/plugins/expo/skills/apple-targets/message-filter.md b/plugins/expo/skills/apple-targets/message-filter.md new file mode 100644 index 0000000..f4e7f47 --- /dev/null +++ b/plugins/expo/skills/apple-targets/message-filter.md @@ -0,0 +1,183 @@ +--- +title: SMS/MMS Message Filter Extension +description: Filters SMS and MMS messages from unknown senders into Junk, Promotions, or Transactions folders in the Messages app. +version: iOS 11.0+ +--- + +# SMS/MMS Message Filter Extension (`message-filter`) + +A Message Filter extension lets your app classify incoming SMS and MMS messages from unknown senders (people not in the recipient's contacts). When a message arrives from an unknown number, iOS passes the sender and message body to your extension, which returns a filtering action -- allow, junk, promotion, or transaction. The system then routes the message to the appropriate folder in the Messages app. The extension operates entirely on-device by default, but can optionally defer to a network server for more sophisticated classification. Only messages from unknown senders are ever passed to the extension; messages from known contacts are never filtered. + +## Apple Documentation + +- [Creating a Message Filter App Extension](https://developer.apple.com/documentation/identitylookup/creating-a-message-filter-app-extension) +- [ILMessageFilterExtension](https://developer.apple.com/documentation/identitylookup/ilmessagefilterextension) +- [ILMessageFilterQueryHandling](https://developer.apple.com/documentation/identitylookup/ilmessagefilterqueryhandling) +- [ILMessageFilterAction](https://developer.apple.com/documentation/identitylookup/ilmessagefilteraction) +- [ILMessageFilterQueryResponse (subAction)](https://developer.apple.com/documentation/identitylookup/ilmessagefilterqueryresponse/subaction) +- [SMS and MMS Message Filtering](https://developer.apple.com/documentation/identitylookup/sms-and-mms-message-filtering) +- [IdentityLookup Framework](https://developer.apple.com/documentation/identitylookup) + +## WWDC History + +- **[WWDC 2017, Session 249 -- Filtering Unwanted Messages with Identity Lookup](https://developer.apple.com/videos/play/wwdc2017/249/)** -- Introduced the IdentityLookup framework and the Message Filter extension point, covering on-device classification and network deferral. +- **[WWDC 2022, Session 110341 -- Explore SMS Message Filters](https://developer.apple.com/videos/play/wwdc2022/110341/)** -- Added 12 sub-categories under Promotions and Transactions (iOS 16), plus the `ILMessageFilterCapabilitiesQueryHandling` protocol for declaring supported sub-actions. + +## What It Does + +1. **Unknown sender message arrives.** When an SMS or MMS is received from a number not in the user's contacts, iOS invokes the enabled Message Filter extension. +2. **Extension receives the query.** Your `ILMessageFilterQueryHandling` implementation receives an `ILMessageFilterQueryRequest` containing the sender and message body. +3. **On-device classification.** Your extension inspects the message content and returns an `ILMessageFilterQueryResponse` with an action: `.allow`, `.junk`, `.promotion`, or `.transaction`. +4. **Optional network deferral.** If your extension returns `.none`, the system forwards the query to your server (URL configured in Info.plist under `ILMessageFilterExtensionNetworkURL`). The server response is passed back through your extension for final classification. +5. **Sub-categories (iOS 16+).** Your extension can declare supported sub-actions (e.g. Finance, Orders, Health under Transactions; Coupons, Offers under Promotions) via `ILMessageFilterCapabilitiesQueryHandling`. Messages are then sorted into sub-folders in the Messages app. +6. **User enables in Settings.** The extension is inactive until the user goes to Settings > Messages > Unknown & Spam and selects your filter. + +## Use Cases + +### Spam and phishing protection + +A security app scans message content for known phishing patterns (shortened URLs, urgency language, OTP interception attempts) and marks matching messages as junk. The user never sees them in their primary inbox. + +### Promotional message organization + +An e-commerce companion app classifies delivery notifications as Transactions and marketing offers as Promotions, letting the user keep their primary SMS inbox clean while still seeing important order updates. + +### Regional SMS filtering + +In countries with high SMS spam volume (India, Brazil), a locally popular filter app uses a combination of on-device keyword matching and server-side crowd-sourced databases to classify messages into granular sub-categories like Finance, Health, and Public Services. + +## Key Classes + +| Class / Protocol | Role | +|------------------|------| +| `ILMessageFilterExtension` | Abstract base class for the extension's principal class. Subclass this to create your filter. | +| `ILMessageFilterQueryHandling` | Protocol your extension adopts. Requires `handle(_:context:completion:)` to classify each message. | +| `ILMessageFilterQueryRequest` | Contains the sender's address and the message body. Passed to your handler. | +| `ILMessageFilterQueryResponse` | Your reply. Set `.action` to `.allow`, `.junk`, `.promotion`, or `.transaction`. Optionally set `.subAction` (iOS 16+). | +| `ILMessageFilterAction` | Enum of top-level filter actions: `.none`, `.allow`, `.junk`, `.promotion`, `.transaction`. | +| `ILMessageFilterSubAction` | Enum of sub-categories (iOS 16+): `.transactionalFinance`, `.transactionalOrders`, `.transactionalHealth`, `.promotionalOffers`, `.promotionalCoupons`, and more. | +| `ILMessageFilterCapabilitiesQueryHandling` | Protocol (iOS 16+) for declaring which sub-actions your filter supports. The system calls this to configure folder UI. | +| `ILMessageFilterExtensionContext` | Extension context providing `deferQueryRequestToNetwork()` for server-side classification. | + +## Implementation + +```swift +import IdentityLookup + +// 1. Subclass ILMessageFilterExtension -- the system loads this as the +// extension's principal class. +final class MessageFilterExtension: ILMessageFilterExtension {} + +// 2. Adopt ILMessageFilterQueryHandling to receive and classify messages. +extension MessageFilterExtension: ILMessageFilterQueryHandling { + + func handle( + _ queryRequest: ILMessageFilterQueryRequest, + context: ILMessageFilterExtensionContext, + completion: @escaping (ILMessageFilterQueryResponse) -> Void + ) { + // 3. Attempt to classify the message using on-device logic first. + let (action, subAction) = classifyLocally(queryRequest) + + if action != .none { + // 4. We have a confident local classification -- respond immediately. + let response = ILMessageFilterQueryResponse() + response.action = action + response.subAction = subAction + completion(response) + } else { + // 5. Uncertain -- defer to the network server configured in + // Info.plist under ILMessageFilterExtensionNetworkURL. + // The system sends the query to your server and calls back + // with the server's JSON response. + context.deferQueryRequestToNetwork { networkResponse, error in + let response = ILMessageFilterQueryResponse() + + if let networkResponse = networkResponse { + // 6. Parse the server response and set the action. + response.action = self.action( + for: networkResponse + ) + } else { + // 7. Network failed -- allow the message through rather + // than silently dropping legitimate messages. + response.action = .allow + } + + completion(response) + } + } + } + + // 8. Local classification using keyword matching, regex, or a + // local database of known spam senders. + private func classifyLocally( + _ request: ILMessageFilterQueryRequest + ) -> (ILMessageFilterAction, ILMessageFilterSubAction) { + guard let body = request.messageBody?.lowercased() else { + return (.none, .none) + } + + // 9. Example: classify OTP and banking messages as transactions. + let transactionKeywords = ["otp", "verification code", "one-time password", "account ending"] + if transactionKeywords.contains(where: { body.contains($0) }) { + return (.transaction, .transactionalFinance) + } + + // 10. Example: classify marketing messages as promotions. + let promoKeywords = ["unsubscribe", "limited time", "% off", "free shipping"] + if promoKeywords.contains(where: { body.contains($0) }) { + return (.promotion, .promotionalOffers) + } + + // 11. No confident match -- return .none to defer to network or allow. + return (.none, .none) + } + + private func action( + for networkResponse: ILNetworkResponse + ) -> ILMessageFilterAction { + // 12. Parse the server's JSON response. The format is up to you, but + // the response must map to an ILMessageFilterAction. + guard let data = networkResponse.urlResponse.url.flatMap({ _ in + try? JSONSerialization.jsonObject(with: Data()) as? [String: Any] + }) else { + return .allow + } + + switch data["action"] as? String { + case "junk": return .junk + case "promotion": return .promotion + case "transaction": return .transaction + default: return .allow + } + } +} +``` + +## Usage + +- Create: `bun create target message-filter` +- Edit: `targets/message-filter/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 11.0+ | Full support. Sub-categories added in iOS 16.0+. | +| iPadOS | 11.0+ | Supported on iPads with cellular capability. | +| macOS | -- | Not supported. | +| watchOS | -- | Not supported. | +| visionOS | -- | Not supported. | +| tvOS | -- | Not supported. | + +## Gotchas + +- **Only unknown senders are filtered.** Messages from contacts, known senders, and iMessage conversations are never passed to the extension. Your filter has zero visibility into messages from people in the user's address book. +- **The extension never sees iMessages.** Only SMS and MMS messages are filtered. iMessage (blue bubble) conversations are completely excluded from the IdentityLookup pipeline. +- **Network URL must be HTTPS and statically configured.** The `ILMessageFilterExtensionNetworkURL` in Info.plist must be a hardcoded HTTPS URL. It cannot vary per user or per request. The server must not require any ATS exceptions, and the URL must pass Associated Domains validation. +- **Sub-categories are region-dependent.** While the API is available globally, the Messages app folder UI for Transactions, Promotions, and sub-categories is only surfaced in certain regions (notably India and Brazil as of iOS 16). In other regions, messages are classified as Junk or allowed through without sub-folder sorting. +- **User must manually enable the filter.** Go to Settings > Messages > Unknown & Spam and select your filter. Only one third-party SMS filter can be active at a time across all installed apps. +- **Privacy constraints prevent data exfiltration.** The extension cannot make arbitrary network requests. The only network path is the system-managed deferral to your configured server URL. You cannot send message content to analytics or other endpoints. +- **No access to message metadata beyond sender and body.** The `ILMessageFilterQueryRequest` provides only the sender address and message body text. You do not receive timestamps, attachments, or thread context. +- **Sub-action support requires capabilities declaration (iOS 16+).** To use sub-categories, your extension must also adopt `ILMessageFilterCapabilitiesQueryHandling` and return the sub-actions you support in `handle(_:capabilitiesQueryRequest:context:completion:)`. Without this, the system only uses top-level actions. diff --git a/plugins/expo/skills/apple-targets/network-app-proxy.md b/plugins/expo/skills/apple-targets/network-app-proxy.md new file mode 100644 index 0000000..6ab4088 --- /dev/null +++ b/plugins/expo/skills/apple-targets/network-app-proxy.md @@ -0,0 +1,258 @@ +--- +title: App Proxy Provider +description: Implements a flow-oriented network proxy that routes specific app traffic through a custom proxy server, operating at the TCP/UDP flow level rather than raw IP packets. +version: iOS 9.0+, macOS 10.11+ +--- + +# App Proxy Provider (`network-app-proxy`) + +A network extension that implements the client side of a custom app-layer proxy protocol. Unlike the packet tunnel provider which captures all device traffic as raw IP packets, the app proxy provider operates at the flow level, receiving individual TCP and UDP connections (`NEAppProxyTCPFlow` and `NEAppProxyUDPFlow`) from specific apps matched by per-app VPN rules. This makes it ideal for enterprise per-app VPN solutions where only designated managed apps need to route traffic through a proxy. + +## Apple Documentation + +- [App Proxy Provider](https://developer.apple.com/documentation/networkextension/app-proxy-provider) -- overview guide for implementing a flow-oriented VPN client +- [NEAppProxyProvider](https://developer.apple.com/documentation/networkextension/neappproxyprovider) -- principal class to subclass +- [NEAppProxyProviderManager](https://developer.apple.com/documentation/networkextension/neappproxyprovidermanager) -- configure and control the proxy from the containing app +- [NEAppProxyTCPFlow](https://developer.apple.com/documentation/networkextension/neappproxytcpflow) -- represents a TCP connection from a matched app +- [NEAppProxyUDPFlow](https://developer.apple.com/documentation/networkextension/neappproxyudpflow) -- represents a UDP datagram session from a matched app +- [NEAppProxyFlow](https://developer.apple.com/documentation/networkextension/neappproxyflow) -- abstract base class for TCP and UDP flows + +## WWDC History + +- **[WWDC 2015, Session 717 -- What's New in Network Extension and VPN](https://developer.apple.com/videos/play/wwdc2015/717/)** -- Introduced NEAppProxyProvider as one of two tunnel provider types (alongside packet tunnel). Explained the flow-level vs. IP-level distinction and per-app VPN rules. +- **[WWDC 2017, Session 707 -- Advances in Networking, Part 1](https://developer.apple.com/videos/play/wwdc2017/707/)** -- Covered Network Extension framework updates and improvements to provider lifecycle handling. +- **[WWDC 2019, Session 714 -- Network Extensions for the Modern Mac](https://developer.apple.com/videos/play/wwdc2019/714/)** -- Introduced System Extensions on macOS Catalina, enabling app proxy providers to run as system extensions. Also introduced `NETransparentProxyProvider` as a macOS-only alternative. + +## What It Does + +1. An MDM profile or app configuration defines per-app VPN rules (`NEAppRule`) that specify which apps should route traffic through the proxy. +2. When a matched app opens a network connection, the system launches your extension and calls `startProxy(options:completionHandler:)`. +3. Your extension establishes a connection to the proxy server using your custom protocol. +4. For each new connection from a matched app, the system calls `handleNewFlow(_:)` with an `NEAppProxyTCPFlow` or `NEAppProxyUDPFlow`. +5. Your extension opens the flow, reads data from it, forwards it through the proxy server, and writes response data back to the flow. +6. When the proxy configuration is removed or the system decides to stop, it calls `stopProxy(with:completionHandler:)`. + +## Use Cases + +### Enterprise Per-App VPN + +Route traffic from specific managed apps (e.g., corporate email, internal wikis) through a secure proxy while leaving personal app traffic on the normal network path. This is the primary use case Apple designed the API for. + +### App-Specific Proxy Servers + +Proxy traffic from designated applications through a custom protocol that performs inspection, logging, or transformation at the application flow level. Because you receive typed TCP and UDP flows rather than raw packets, you can apply proxy logic without IP-level parsing. + +### Selective Traffic Routing + +Direct traffic from specific apps through different network paths based on business rules -- for example, routing a finance app through a dedicated secure channel while other corporate apps use a general VPN. + +## Key Classes + +| Class | Role | +|-------|------| +| `NEAppProxyProvider` | Subclass this. The system calls `startProxy`, `stopProxy`, and `handleNewFlow` on it. | +| `NEAppProxyProviderManager` | Used by the containing app to install, configure, and manage the proxy. Holds `NEAppRule` routing rules. | +| `NEAppProxyTCPFlow` | A single TCP connection from a matched app. Call `open` to begin, then `readData` / `write` to shuttle bytes. | +| `NEAppProxyUDPFlow` | A related stream of UDP datagrams. Call `open` then `readDatagrams` / `writeDatagrams`. | +| `NEAppProxyFlow` | Abstract base of TCP/UDP flows. Provides `metaData` (source app bundle ID, etc.) and `networkInterface`. | +| `NETunnelProviderProtocol` | Configuration object holding server address, provider bundle ID, and custom `providerConfiguration` dictionary. | + +## Implementation + +### App Proxy Provider (Per-App VPN Proxy) + +```swift +import NetworkExtension + +class AppProxyProvider: NEAppProxyProvider { + + private var serverConnection: NWTCPConnection? + + // 1. System calls this when a matched app triggers the proxy. + override func startProxy( + options: [String: Any]? = nil, + completionHandler: @escaping (Error?) -> Void + ) { + let serverAddress = protocolConfiguration.serverAddress ?? "proxy.example.com" + + // 2. Connect to the proxy server outside the tunnel. + let endpoint = NWHostEndpoint(hostname: serverAddress, port: "8443") + serverConnection = createTCPConnectionThroughTunnel( + to: endpoint, + enableTLS: true, + tlsParameters: nil, + delegate: nil + ) + + serverConnection?.addObserver(self, forKeyPath: "state", context: nil) + completionHandler(nil) + } + + // 3. System calls this when a matched app opens a new connection. + override func handleNewFlow(_ flow: NEAppProxyFlow) -> Bool { + if let tcpFlow = flow as? NEAppProxyTCPFlow { + handleTCPFlow(tcpFlow) + return true + } else if let udpFlow = flow as? NEAppProxyUDPFlow { + handleUDPFlow(udpFlow) + return true + } + // 4. Return false to reject flows you cannot handle. + return false + } + + // 5. Open the TCP flow and begin reading data from the app. + private func handleTCPFlow(_ flow: NEAppProxyTCPFlow) { + flow.open(withLocalEndpoint: nil) { error in + if let error = error { + flow.closeReadWithError(error) + flow.closeWriteWithError(error) + return + } + self.readFromTCPFlow(flow) + } + } + + // 6. Read loop: pull data from the app's TCP connection. + private func readFromTCPFlow(_ flow: NEAppProxyTCPFlow) { + flow.readData { data, error in + if let error = error { + flow.closeReadWithError(error) + return + } + guard let data = data, !data.isEmpty else { + // 7. Empty data means the flow has ended. + flow.closeReadWithError(nil) + return + } + // 8. Forward the data to the proxy server, then continue reading. + self.forwardToServer(data) { responseData in + flow.write(responseData) { writeError in + if writeError != nil { + flow.closeWriteWithError(writeError) + return + } + self.readFromTCPFlow(flow) + } + } + } + } + + // 9. Handle UDP flows similarly using readDatagrams/writeDatagrams. + private func handleUDPFlow(_ flow: NEAppProxyUDPFlow) { + flow.open(withLocalEndpoint: nil) { error in + guard error == nil else { return } + self.readFromUDPFlow(flow) + } + } + + private func readFromUDPFlow(_ flow: NEAppProxyUDPFlow) { + flow.readDatagrams { datagrams, endpoints, error in + guard let datagrams = datagrams, + let endpoints = endpoints, + error == nil else { return } + + // 10. Forward each datagram to the proxy server. + for (datagram, endpoint) in zip(datagrams, endpoints) { + self.forwardDatagramToServer(datagram, destination: endpoint) + } + self.readFromUDPFlow(flow) + } + } + + private func forwardToServer( + _ data: Data, + completion: @escaping (Data) -> Void + ) { + // Forward data through the proxy server connection and + // return the response. Implementation depends on your protocol. + } + + private func forwardDatagramToServer(_ data: Data, destination: NWEndpoint) { + // Forward UDP datagram through the proxy server. + } + + // 11. System calls this to tear down the proxy. + override func stopProxy( + with reason: NEProviderStopReason, + completionHandler: @escaping () -> Void + ) { + serverConnection?.cancel() + serverConnection = nil + completionHandler() + } + + // 12. Receive messages from the containing app. + override func handleAppMessage( + _ messageData: Data, + completionHandler: ((Data?) -> Void)? + ) { + completionHandler?(messageData) + } + + override func sleep(completionHandler: @escaping () -> Void) { + completionHandler() + } + + override func wake() { + // Re-establish server connection if needed. + } +} +``` + +### Containing App -- Installing the Proxy Configuration + +```swift +import NetworkExtension + +func installAppProxyProfile() { + let manager = NEAppProxyProviderManager() + + let proto = NETunnelProviderProtocol() + proto.providerBundleIdentifier = "com.example.app.AppProxy" + proto.serverAddress = "proxy.example.com" + proto.providerConfiguration = ["port": "8443"] + + manager.protocolConfiguration = proto + manager.localizedDescription = "Corporate App Proxy" + manager.isEnabled = true + + // Per-app VPN rules: only proxy traffic from these apps. + let mailRule = NEAppRule(signingIdentifier: "com.example.corporate-mail") + let wikiRule = NEAppRule(signingIdentifier: "com.example.internal-wiki") + manager.appRules = [mailRule, wikiRule] + + manager.saveToPreferences { error in + if let error = error { + print("Failed to save proxy profile: \(error)") + } + } +} +``` + +## Usage + +- Create: `bun create target network-app-proxy` +- Edit: `targets/network-app-proxy/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 9.0+ | Full support. Requires MDM or configuration profile for per-app VPN rules. | +| iPadOS | 9.0+ | Full support. | +| macOS | 10.11+ | Runs as app extension (10.11) or system extension (10.15+). Consider `NETransparentProxyProvider` on macOS 11+. | +| tvOS | 17.0+ | Added in tvOS 17. | +| visionOS | 1.0+ | Supported. | +| watchOS | -- | Not supported. | + +## Gotchas + +- **Network Extension entitlement required.** You must request the Network Extension entitlement from Apple via [the request form](https://developer.apple.com/contact/request/network-extension/). Your app will be rejected without it. This is a manual approval process that can take days or weeks. +- **Per-app VPN rules require MDM or configuration profile.** On iOS, `NEAppRule` matching only works when the proxy is configured via a managed app configuration or MDM profile. You cannot set up per-app VPN rules purely in code on unmanaged devices. +- **Returning `false` from `handleNewFlow` drops the connection.** Unlike `NETransparentProxyProvider` (macOS 11+), where returning `false` lets the OS handle the flow normally, `NEAppProxyProvider` will discard the flow entirely if you return `false`. You must proxy every flow you receive. +- **Flows are app-layer, not IP packets.** You receive typed `NEAppProxyTCPFlow` and `NEAppProxyUDPFlow` objects, not raw bytes. This is more convenient than packet tunnel but means you cannot inspect or modify IP headers. +- **`open` must complete before reading.** You must call `open(withLocalEndpoint:completionHandler:)` on a flow and wait for the completion handler before calling `readData` or `readDatagrams`. Reading before the flow is open will fail silently. +- **Memory limits are strict.** Like all network extensions, the app proxy provider runs in a constrained memory environment (~15 MB on iOS). Avoid buffering large amounts of data from multiple concurrent flows. +- **Consider `NETransparentProxyProvider` on macOS.** If you only target macOS 11+, the transparent proxy provider is a better choice: it supports selective proxying (return `false` to let the OS handle a flow) and does not require per-app VPN rules. `NEAppProxyProvider` remains necessary for iOS or older macOS support. +- **`handleNewUDPFlow(_:initialRemoteEndpoint:)` was added in iOS 13.** Before iOS 13, UDP flows arrived via the base `handleNewFlow(_:)` without an initial remote endpoint. Override the newer method if you need the destination address upfront and your minimum target is iOS 13+. diff --git a/plugins/expo/skills/apple-targets/network-dns-proxy.md b/plugins/expo/skills/apple-targets/network-dns-proxy.md new file mode 100644 index 0000000..40771d0 --- /dev/null +++ b/plugins/expo/skills/apple-targets/network-dns-proxy.md @@ -0,0 +1,252 @@ +--- +title: DNS Proxy Provider +description: Intercepts all system DNS queries and resolves them through a custom DNS resolver, enabling encrypted DNS (DoH/DoT) or DNS-based filtering. +version: iOS 11.0+, macOS 10.15+ +--- + +# DNS Proxy Provider (`network-dns-proxy`) + +A network extension that intercepts every DNS query on the device and resolves them through your custom DNS resolver. The system diverts all DNS traffic to your `NEDNSProxyProvider` subclass, which receives queries as `NEAppProxyUDPFlow` (and occasionally `NEAppProxyTCPFlow` for TCP-based DNS). You must parse raw DNS packets yourself, forward them to your resolver using any protocol you choose (DNS-over-HTTPS, DNS-over-TLS, or plain DNS), and write the response back to the flow. This gives you complete control over DNS resolution system-wide. + +## Apple Documentation + +- [DNS Proxy Provider](https://developer.apple.com/documentation/networkextension/dns-proxy-provider) -- overview guide for building a custom DNS proxy +- [NEDNSProxyProvider](https://developer.apple.com/documentation/networkextension/nednsproxyprovider) -- principal class to subclass +- [NEDNSProxyManager](https://developer.apple.com/documentation/networkextension/nednsproxymanager) -- configure and enable the DNS proxy from the containing app +- [NEDNSProxyProviderProtocol](https://developer.apple.com/documentation/networkextension/nednsproxyproviderprotocol) -- configuration parameters for the DNS proxy +- [NEAppProxyUDPFlow](https://developer.apple.com/documentation/networkextension/neappproxyudpflow) -- UDP flow carrying DNS query datagrams +- [NEAppProxyTCPFlow](https://developer.apple.com/documentation/networkextension/neappproxytcpflow) -- TCP flow for DNS-over-TCP queries + +## WWDC History + +- **[WWDC 2017, Session 707 -- Advances in Networking, Part 1](https://developer.apple.com/videos/play/wwdc2017/707/)** -- Introduced `NEDNSProxyProvider` in iOS 11 as a way to implement custom DNS resolution, including DNS-over-HTTPS and DNS-over-TLS, with full control over all DNS traffic. +- **[WWDC 2019, Session 714 -- Network Extensions for the Modern Mac](https://developer.apple.com/videos/play/wwdc2019/714/)** -- Brought DNS proxy providers to macOS via system extensions. Also introduced `NEDNSSettingsManager` as a simpler alternative for configuring system DNS to use DoH/DoT without writing a full proxy. +- **[WWDC 2025, Session 234 -- Filter and Tunnel Network Traffic with NetworkExtension](https://developer.apple.com/videos/play/wwdc2025/234/)** -- Updated guidance on network extension providers including DNS proxy lifecycle improvements. + +## What It Does + +1. The containing app uses `NEDNSProxyManager` to install and enable the DNS proxy configuration. +2. Once active, the system diverts all DNS queries to your extension instead of the system resolver. +3. For each DNS query, the system calls `handleNewFlow(_:)` with an `NEAppProxyUDPFlow` (for standard UDP DNS) or `NEAppProxyTCPFlow` (for DNS-over-TCP). +4. Your extension calls `open` on the flow, then reads raw DNS query datagrams from it. +5. You parse the DNS packet, forward the query to your resolver (e.g., via HTTPS for DoH or TLS for DoT), and receive the response. +6. You write the DNS response datagram back to the flow. +7. The system delivers the response to the requesting application. + +## Use Cases + +### Encrypted DNS (DoH / DoT) + +Encrypt all device DNS queries using DNS-over-HTTPS or DNS-over-TLS to prevent ISP snooping, DNS hijacking, and man-in-the-middle attacks. The DNS proxy intercepts plain DNS queries and forwards them over an encrypted channel to a trusted resolver like Cloudflare (1.1.1.1) or Google (8.8.8.8). + +### DNS-Based Ad and Tracker Blocking + +Intercept DNS queries and block resolution for known advertising and tracking domains by returning NXDOMAIN or 0.0.0.0 responses. This provides system-wide ad blocking at the DNS level without inspecting HTTP traffic. + +### Custom DNS Routing + +Route DNS queries to different resolvers based on the queried domain -- for example, sending internal corporate domain lookups to an on-premise DNS server while routing all other queries to a public resolver. + +## Key Classes + +| Class | Role | +|-------|------| +| `NEDNSProxyProvider` | Subclass this. The system calls `startProxy`, `stopProxy`, and `handleNewFlow` on it. Receives all DNS queries system-wide. | +| `NEDNSProxyManager` | Used by the containing app to install, configure, and enable the DNS proxy. | +| `NEDNSProxyProviderProtocol` | Configuration object holding server address and provider bundle identifier. | +| `NEAppProxyUDPFlow` | Carries DNS query/response datagrams for standard UDP DNS. Call `readDatagrams` / `writeDatagrams`. | +| `NEAppProxyTCPFlow` | Carries DNS query/response data for TCP-based DNS (less common). Call `readData` / `write`. | + +## Implementation + +### DNS Proxy Provider (DoH Resolver) + +```swift +import NetworkExtension + +class DNSProxyProvider: NEDNSProxyProvider { + + // 1. System calls this when the DNS proxy is enabled. + override func startProxy( + options: [String: Any]? = nil, + completionHandler: @escaping (Error?) -> Void + ) { + // Initialize your DNS resolver connection (e.g., URLSession + // for DoH or NWConnection for DoT). + completionHandler(nil) + } + + override func stopProxy( + with reason: NEProviderStopReason, + completionHandler: @escaping () -> Void + ) { + completionHandler() + } + + // 2. System calls this for every DNS query on the device. + override func handleNewFlow(_ flow: NEAppProxyFlow) -> Bool { + if let udpFlow = flow as? NEAppProxyUDPFlow { + handleDNSUDPFlow(udpFlow) + return true + } else if let tcpFlow = flow as? NEAppProxyTCPFlow { + handleDNSTCPFlow(tcpFlow) + return true + } + return false + } + + // 3. Open the UDP flow and read DNS query datagrams. + private func handleDNSUDPFlow(_ flow: NEAppProxyUDPFlow) { + flow.open(withLocalEndpoint: nil) { error in + guard error == nil else { return } + self.readDNSDatagrams(from: flow) + } + } + + // 4. Read loop for UDP DNS queries. + private func readDNSDatagrams(from flow: NEAppProxyUDPFlow) { + flow.readDatagrams { datagrams, endpoints, error in + guard let datagrams = datagrams, + let endpoints = endpoints, + error == nil, + !datagrams.isEmpty else { + return + } + + // 5. Process each DNS query datagram. + for (datagram, endpoint) in zip(datagrams, endpoints) { + self.resolveDNSQuery(datagram) { response in + // 6. Write the DNS response back to the flow. + flow.writeDatagrams([response], sentBy: [endpoint]) { writeError in + if writeError != nil { + flow.closeWriteWithError(writeError) + } + } + } + } + + // 7. Continue reading subsequent queries on this flow. + self.readDNSDatagrams(from: flow) + } + } + + // 8. Handle TCP DNS flows (DNS-over-TCP uses a 2-byte length prefix). + private func handleDNSTCPFlow(_ flow: NEAppProxyTCPFlow) { + flow.open(withLocalEndpoint: nil) { error in + guard error == nil else { return } + self.readDNSTCPData(from: flow) + } + } + + private func readDNSTCPData(from flow: NEAppProxyTCPFlow) { + flow.readData { data, error in + guard let data = data, !data.isEmpty, error == nil else { return } + + // 9. TCP DNS prepends a 2-byte length. Strip it, resolve, + // then write response with the length prefix back. + let queryData = data.dropFirst(2) + self.resolveDNSQuery(Data(queryData)) { response in + var responseWithLength = Data() + var length = UInt16(response.count).bigEndian + responseWithLength.append( + Data(bytes: &length, count: MemoryLayout.size) + ) + responseWithLength.append(response) + flow.write(responseWithLength) { _ in } + } + + self.readDNSTCPData(from: flow) + } + } + + // 10. Forward the DNS query to a DoH resolver (e.g., Cloudflare). + private func resolveDNSQuery( + _ queryData: Data, + completion: @escaping (Data) -> Void + ) { + var request = URLRequest( + url: URL(string: "https://cloudflare-dns.com/dns-query")! + ) + request.httpMethod = "POST" + request.setValue("application/dns-message", forHTTPHeaderField: "Content-Type") + request.setValue("application/dns-message", forHTTPHeaderField: "Accept") + request.httpBody = queryData + + // 11. Use URLSession to send the query over HTTPS. + // The DNS proxy extension CAN make network requests + // (unlike the filter data provider). + URLSession.shared.dataTask(with: request) { data, _, error in + if let data = data, error == nil { + completion(data) + } + }.resume() + } + + override func sleep(completionHandler: @escaping () -> Void) { + completionHandler() + } + + override func wake() { + // Re-initialize resolver connections if needed. + } +} +``` + +### Containing App -- Enabling the DNS Proxy + +```swift +import NetworkExtension + +func enableDNSProxy() { + let manager = NEDNSProxyManager.shared() + + manager.loadFromPreferences { error in + if let error = error { + print("Failed to load DNS proxy preferences: \(error)") + return + } + + let proto = NEDNSProxyProviderProtocol() + proto.providerBundleIdentifier = "com.example.app.DNSProxy" + proto.serverAddress = "cloudflare-dns.com" + + manager.providerProtocol = proto + manager.isEnabled = true + + manager.saveToPreferences { error in + if let error = error { + print("Failed to save DNS proxy: \(error)") + } + } + } +} +``` + +## Usage + +- Create: `bun create target network-dns-proxy` +- Edit: `targets/network-dns-proxy/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 11.0+ | Requires supervised device or MDM configuration profile for production. Development signing bypasses for testing. | +| iPadOS | 11.0+ | Same requirements as iOS. | +| macOS | 10.15+ | Runs as a system extension. No supervised requirement. | +| Mac Catalyst | 13.0+ | Supported via Catalyst. | +| tvOS | -- | Not supported. | +| visionOS | -- | Not supported. | +| watchOS | -- | Not supported. | + +## Gotchas + +- **Network Extension entitlement required.** You must request the Network Extension entitlement from Apple via [the request form](https://developer.apple.com/contact/request/network-extension/). Your app will be rejected without it. This is a manual approval process that can take days or weeks. +- **You must parse DNS packets yourself.** Apple does not provide a DNS parsing API. You receive raw DNS wire-format bytes in each datagram and must parse the query (extracting the domain name, record type, transaction ID, etc.) and construct valid DNS response packets. Use a third-party Swift DNS library or write your own parser. +- **Supervised device requirement on iOS.** Like other network extension providers, the DNS proxy requires a supervised device (managed via MDM) for production builds on iOS. Development-signed builds bypass this check. This limits consumer App Store distribution. +- **`NEDNSSettingsManager` is simpler for basic DoH/DoT.** If you only need to redirect DNS to a specific DoH or DoT resolver without inspecting or modifying queries, use `NEDNSSettingsManager` (iOS 14+) instead. It does not require a network extension, a supervised device, or DNS packet parsing. +- **System DoH/DoT bypasses the proxy.** If the system DNS is already configured to use a DoH or DoT resolver (e.g., via Settings or a configuration profile), those encrypted DNS transactions will not be routed to your `NEDNSProxyProvider`. Your `handleNewFlow` will not be called for those queries. This is by design. +- **You must proxy all DNS traffic.** Unlike transparent proxy providers, you cannot selectively handle some queries and let the OS handle others. Every DNS query arrives at your extension, and you must resolve and respond to all of them. Failing to respond will cause DNS timeouts for the querying app. +- **Multiple DNS queries can arrive on a single flow.** A `NEAppProxyUDPFlow` may carry multiple DNS query datagrams over its lifetime, not just one. Your read loop must continue reading after processing each datagram. +- **TCP DNS uses a 2-byte length prefix.** DNS queries over TCP prepend a 2-byte big-endian length before the DNS message. You must strip this prefix when reading queries and add it back when writing responses. Forgetting this will produce malformed responses. +- **Memory and performance constraints apply.** The DNS proxy handles every DNS query on the device, including system services, background refresh, push notifications, and all apps. Slow resolution or memory issues will degrade the entire device's network performance. Keep resolution fast and avoid caching large amounts of state. diff --git a/plugins/expo/skills/apple-targets/network-filter-data.md b/plugins/expo/skills/apple-targets/network-filter-data.md new file mode 100644 index 0000000..0076d65 --- /dev/null +++ b/plugins/expo/skills/apple-targets/network-filter-data.md @@ -0,0 +1,234 @@ +--- +title: Content Filter Data Provider +description: Inspects network traffic flows on-device and makes allow/block decisions for content filtering without sending user data off the device. +version: iOS 9.0+, macOS 10.15+ +--- + +# Content Filter Data Provider (`network-filter-data`) + +A network extension that examines TCP and UDP flows passing through the device and renders allow, block, or need-more-data verdicts for each one. The filter data provider runs in a heavily sandboxed process with read-only access to flow data -- it cannot make network requests or write to disk, ensuring user privacy. A companion filter control provider extension (with network access but no data access) can supply updated rules. This architecture was designed for parental controls, enterprise content filtering, and educational device management. + +## Apple Documentation + +- [Content Filter Providers](https://developer.apple.com/documentation/networkextension/content-filter-providers) -- overview guide for building on-device content filters +- [NEFilterDataProvider](https://developer.apple.com/documentation/networkextension/nefilterdataprovider) -- principal class to subclass for inspecting flow data +- [NEFilterControlProvider](https://developer.apple.com/documentation/networkextension/nefiltercontrolprovider) -- companion extension that fetches updated rules (has network access) +- [NEFilterManager](https://developer.apple.com/documentation/networkextension/nefiltermanager) -- configure and enable the filter from the containing app +- [NEFilterNewFlowVerdict](https://developer.apple.com/documentation/networkextension/nefilternewflowverdict) -- initial verdict for a new flow (allow, drop, filterData, needRules) +- [NEFilterDataVerdict](https://developer.apple.com/documentation/networkextension/nefilterdataverdict) -- verdict after inspecting flow data chunks (allow, drop, needRules) +- [NEFilterFlow](https://developer.apple.com/documentation/networkextension/nefilterflow) -- base class representing a network flow with metadata + +## WWDC History + +- **[WWDC 2015, Session 717 -- What's New in Network Extension and VPN](https://developer.apple.com/videos/play/wwdc2015/717/)** -- Introduced the content filter provider architecture with the split data/control provider model for privacy. +- **[WWDC 2019, Session 714 -- Network Extensions for the Modern Mac](https://developer.apple.com/videos/play/wwdc2019/714/)** -- Brought content filter providers to macOS via system extensions. Covered the macOS-specific content filter behavior including the `pause` verdict. +- **[WWDC 2025, Session 234 -- Filter and Tunnel Network Traffic with NetworkExtension](https://developer.apple.com/videos/play/wwdc2025/234/)** -- Introduced URL-based filtering on iOS 26 as a simpler alternative to data-level inspection for URL-based decisions. + +## What It Does + +1. The containing app uses `NEFilterManager` to install and enable a filter configuration. +2. On iOS, the device must be supervised (MDM-managed) for production builds. Development-signed builds bypass this restriction for testing. +3. Once active, the system passes every new TCP and UDP flow to `handleNewFlow(_:)` as an `NEFilterFlow`. +4. Your extension returns an `NEFilterNewFlowVerdict`: `.allow()`, `.drop()`, `.needRules()`, or `.filterDataVerdict(withFilterInbound:peekInboundBytes:filterOutbound:peekOutboundBytes:)` to request data inspection. +5. If you requested data inspection, the system calls `handleInboundData(from:readBytesStartOffset:readBytes:)` and `handleOutboundData(from:readBytesStartOffset:readBytes:)` as data chunks arrive. +6. You return `NEFilterDataVerdict` for each chunk: `.allow()`, `.drop()`, `.needRules()`, or request more data with a new byte range. +7. When the flow completes, `handleInboundDataComplete(for:)` and `handleOutboundDataComplete(for:)` are called for final decisions. + +## Use Cases + +### Parental Controls + +Filter inappropriate content on children's devices by inspecting flow hostnames and data patterns. Combine with the filter control provider to download updated blocklists without exposing user data. + +### Enterprise Content Filtering + +Enforce acceptable-use policies on corporate-managed devices by blocking access to unauthorized services, detecting data exfiltration patterns, or restricting specific protocols. + +### Educational Device Management + +Schools using supervised iPads can filter student network access during class hours, blocking social media and streaming while allowing educational resources. + +## Key Classes + +| Class | Role | +|-------|------| +| `NEFilterDataProvider` | Subclass this. The system calls `handleNewFlow`, `handleInboundData`, and `handleOutboundData` on it. Runs in a strict sandbox (no network, no disk writes). | +| `NEFilterControlProvider` | Companion extension that can fetch updated rules from a server. Has network access but no access to flow data. | +| `NEFilterManager` | Used by the containing app to install and enable the content filter configuration. | +| `NEFilterProviderConfiguration` | Holds the filter configuration: organization name, server address, filter browsers flag. | +| `NEFilterFlow` | Represents a network flow. Provides `identifier`, `direction`, and source app info via `sourceAppAuditToken`. | +| `NEFilterSocketFlow` | Subclass of `NEFilterFlow` for socket-based flows. Provides `remoteHostname`, `remoteEndpoint`, `localEndpoint`. | +| `NEFilterNewFlowVerdict` | The initial verdict: allow, drop, need rules, or request data inspection with byte ranges. | +| `NEFilterDataVerdict` | Verdict after inspecting data chunks: allow, drop, need rules, or request more bytes. | + +## Implementation + +### Content Filter Data Provider + +```swift +import NetworkExtension + +class FilterDataProvider: NEFilterDataProvider { + + // 1. System calls this when the filter is enabled. + override func startFilter(completionHandler: @escaping (Error?) -> Void) { + // Load any cached rules from the shared container. + // The filter control provider writes rules; we read them. + completionHandler(nil) + } + + override func stopFilter( + with reason: NEProviderStopReason, + completionHandler: @escaping () -> Void + ) { + completionHandler() + } + + // 2. Called for every new TCP/UDP flow on the device. + override func handleNewFlow(_ flow: NEFilterFlow) -> NEFilterNewFlowVerdict { + guard let socketFlow = flow as? NEFilterSocketFlow, + let hostname = socketFlow.remoteHostname else { + // 3. No hostname available -- allow by default. + return .allow() + } + + // 4. Check the hostname against a blocklist. + if isBlocked(hostname) { + return .drop() + } + + // 5. For suspicious domains, request to inspect the first 4 KB + // of inbound and outbound data before deciding. + if needsInspection(hostname) { + return .filterDataVerdict( + withFilterInbound: true, + peekInboundBytes: 4096, + filterOutbound: true, + peekOutboundBytes: 4096 + ) + } + + return .allow() + } + + // 6. Called with chunks of inbound data for flows being inspected. + override func handleInboundData( + from flow: NEFilterFlow, + readBytesStartOffset offset: Int, + readBytes: Data + ) -> NEFilterDataVerdict { + // 7. Inspect the data for blocked content patterns. + if containsBlockedContent(readBytes) { + return .drop() + } + + // 8. If we have seen enough data, allow the rest of the flow. + if offset + readBytes.count >= 4096 { + return .allow() + } + + // 9. Request more data before making a final decision. + return .needRules() + } + + // 10. Called with chunks of outbound data for flows being inspected. + override func handleOutboundData( + from flow: NEFilterFlow, + readBytesStartOffset offset: Int, + readBytes: Data + ) -> NEFilterDataVerdict { + if containsBlockedContent(readBytes) { + return .drop() + } + return .allow() + } + + // 11. Called when all inbound data has been delivered. + override func handleInboundDataComplete( + for flow: NEFilterFlow + ) -> NEFilterNewFlowVerdict { + return .allow() + } + + // 12. Called when all outbound data has been delivered. + override func handleOutboundDataComplete( + for flow: NEFilterFlow + ) -> NEFilterNewFlowVerdict { + return .allow() + } + + // MARK: - Private Helpers + + private func isBlocked(_ hostname: String) -> Bool { + let blocklist = ["blocked-site.example.com", "malware.example.com"] + return blocklist.contains(where: { hostname.hasSuffix($0) }) + } + + private func needsInspection(_ hostname: String) -> Bool { + let watchlist = ["suspicious.example.com"] + return watchlist.contains(where: { hostname.hasSuffix($0) }) + } + + private func containsBlockedContent(_ data: Data) -> Bool { + // Inspect data bytes for blocked patterns. + return false + } +} +``` + +### Containing App -- Enabling the Content Filter + +```swift +import NetworkExtension + +func enableContentFilter() { + NEFilterManager.shared().loadFromPreferences { error in + if let error = error { + print("Failed to load filter preferences: \(error)") + return + } + + let config = NEFilterProviderConfiguration() + config.filterBrowsers = true + config.filterSockets = true + config.organization = "Example Corp" + + NEFilterManager.shared().providerConfiguration = config + NEFilterManager.shared().isEnabled = true + + NEFilterManager.shared().saveToPreferences { error in + if let error = error { + print("Failed to save filter: \(error)") + } + } + } +} +``` + +## Usage + +- Create: `bun create target network-filter-data` +- Edit: `targets/network-filter-data/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 9.0+ | Requires supervised device for production builds. Development signing bypasses this for testing. | +| iPadOS | 9.0+ | Same supervised requirement as iOS. Primary deployment target for education. | +| macOS | 10.15+ | Runs as a system extension. No supervised requirement. No WebKit integration (browsers may not use WebKit). | +| tvOS | -- | Not supported. | +| visionOS | -- | Not supported. | +| watchOS | -- | Not supported. | + +## Gotchas + +- **Network Extension entitlement required.** You must request the Network Extension entitlement from Apple via [the request form](https://developer.apple.com/contact/request/network-extension/). Your app will be rejected without it. This is a manual approval process that can take days or weeks. +- **iOS requires supervised devices in production.** The content filter API enforces supervised-device-only on iOS for any build signed with distribution, Ad Hoc, or Enterprise certificates. Only development-signed builds (with `get-task-allow` entitlement) bypass this check. This effectively limits iOS deployment to MDM-managed enterprise or education devices. +- **The data provider cannot make network requests.** The filter data provider runs in a strict sandbox with no network access and no disk write access. If you need to download updated rules, implement a `NEFilterControlProvider` companion extension which has network access but cannot see flow data. +- **Two extensions are required on iOS.** On iOS, you need both a filter data provider and a filter control provider extension. On macOS, the control provider is optional. +- **`handleNewFlow` receives every flow.** The system sends all TCP and UDP flows to your extension. Returning verdicts quickly for known-good traffic is critical for performance. Slow verdicts will degrade the user's entire network experience. +- **HTTP/3 traffic arrives as UDP flows.** Modern browsers using QUIC/HTTP/3 will send UDP flows, not TCP. If you only check `NEFilterSocketFlow` for TCP connections, you will miss HTTP/3 traffic. Always handle both flow types. +- **`remoteHostname` may be nil.** If DNS resolution happened outside the filter's view or the app connected by IP address directly, `remoteHostname` on `NEFilterSocketFlow` will be nil. Do not rely on it being present for every flow. +- **Screen Time API is an alternative for consumer apps.** If you need content filtering in a consumer (non-supervised) iOS app, look at the Screen Time `ManagedSettings` framework (WWDC 2021 Session 10123) which provides web content filtering without the supervised device requirement. +- **`filterBrowsers` is deprecated on macOS.** Many macOS browsers do not use WebKit or NSURLSession, so the browser filtering path is unreliable. Use `filterSockets = true` on macOS and inspect `NEFilterSocketFlow` instead. diff --git a/plugins/expo/skills/apple-targets/network-packet-tunnel.md b/plugins/expo/skills/apple-targets/network-packet-tunnel.md new file mode 100644 index 0000000..6c5fd3e --- /dev/null +++ b/plugins/expo/skills/apple-targets/network-packet-tunnel.md @@ -0,0 +1,223 @@ +--- +title: Packet Tunnel Provider +description: Implements a custom VPN client that captures all device network traffic at the IP layer, tunneling it through your VPN server. +version: iOS 9.0+, macOS 10.11+ +--- + +# Packet Tunnel Provider (`network-packet-tunnel`) + +A network extension that implements the client side of a custom VPN tunneling protocol. The system routes all device IP traffic through your extension, which reads and writes raw IP packets via a virtual network interface and forwards them to your VPN server over an encrypted connection. + +## Apple Documentation + +- [Packet Tunnel Provider](https://developer.apple.com/documentation/networkextension/packet-tunnel-provider) -- overview guide for implementing a VPN client +- [NEPacketTunnelProvider](https://developer.apple.com/documentation/networkextension/nepackettunnelprovider) -- principal class to subclass +- [NEPacketTunnelNetworkSettings](https://developer.apple.com/documentation/networkextension/nepackettunnelnetworksettings) -- virtual interface configuration (DNS, routes, MTU) +- [NEPacketTunnelFlow](https://developer.apple.com/documentation/networkextension/nepackettunnelflow) -- read/write IP packets from the tunnel's virtual interface +- [NETunnelProviderManager](https://developer.apple.com/documentation/networkextension/netunnelprovidermanager) -- configure and control the tunnel from the containing app +- [TN3120: Expected use cases for Network Extension packet tunnel providers](https://developer.apple.com/documentation/technotes/tn3120-expected-use-cases-for-network-extension-packet-tunnel-providers) -- Apple's guidance on approved use cases + +## WWDC History + +- **[WWDC 2015, Session 717 -- What's New in Network Extension and VPN](https://developer.apple.com/videos/play/wwdc2015/717/)** -- Introduced NEPacketTunnelProvider alongside the NETunnelProvider family of APIs for custom VPN protocols on iOS 9 and OS X 10.11. +- **[WWDC 2017, Session 707 -- Advances in Networking, Part 1](https://developer.apple.com/videos/play/wwdc2017/707/)** -- Covered Network Extension framework updates including improvements to tunnel providers. +- **[WWDC 2019, Session 714 -- Network Extensions for the Modern Mac](https://developer.apple.com/videos/play/wwdc2019/714/)** -- Introduced System Extensions on macOS Catalina, allowing packet tunnel providers to run as system extensions instead of kernel extensions. + +## What It Does + +1. The user enables the VPN from Settings or your app calls `NETunnelProviderManager.loadAllFromPreferences()` and starts the connection. +2. The system launches your extension and calls `startTunnel(options:completionHandler:)`. +3. Your extension establishes a connection to the VPN server (TLS, UDP, custom protocol). +4. You configure the virtual interface by calling `setTunnelNetworkSettings(_:completionHandler:)` with DNS servers, IP routes, MTU, and proxy settings. +5. The system begins routing device traffic to the tunnel's virtual interface. +6. Your extension reads outbound IP packets from `packetFlow.readPackets(completionHandler:)`, encrypts them, and sends them to the server. +7. Your extension receives encrypted packets from the server, decrypts them, and writes them back via `packetFlow.writePackets(_:withProtocols:)`. +8. When the user disconnects, the system calls `stopTunnel(with:completionHandler:)`. + +## Use Cases + +### Custom VPN Client +Apps like WireGuard and OpenVPN that implement non-standard VPN protocols. The packet tunnel provider gives you raw IP packets so you can wrap them in any tunneling protocol your server supports. + +### Enterprise Network Security +Corporate VPN solutions that route all employee device traffic through a secure gateway for compliance, data loss prevention, and access to internal resources behind a firewall. + +### Privacy-Focused Networking +Consumer VPN apps that encrypt all device traffic to protect users on untrusted networks (public Wi-Fi, hotel networks) by tunneling everything to a trusted exit node. + +## Key Classes + +| Class | Role | +|-------|------| +| `NEPacketTunnelProvider` | Subclass this. The system calls `startTunnel` and `stopTunnel` on it. Read/write IP packets via `packetFlow`. | +| `NEPacketTunnelNetworkSettings` | Configures the virtual TUN interface -- DNS servers, IPv4/IPv6 routes, MTU, proxy settings. | +| `NEPacketTunnelFlow` | The packet I/O object. Call `readPackets` to get outbound IP packets and `writePackets` to inject inbound packets. | +| `NETunnelProviderManager` | Used by the containing app to install, configure, and start/stop the tunnel. | +| `NETunnelProviderProtocol` | Holds the VPN configuration: server address, provider bundle identifier, and custom `providerConfiguration` dictionary. | + +## Implementation + +### Packet Tunnel Provider (Realistic VPN Client) + +```swift +import NetworkExtension + +class PacketTunnelProvider: NEPacketTunnelProvider { + + private var connection: NWTCPConnection? + private var pendingStartCompletion: ((Error?) -> Void)? + + // 1. System calls this when the user enables the VPN. + override func startTunnel( + options: [String: NSObject]?, + completionHandler: @escaping (Error?) -> Void + ) { + let serverAddress = protocolConfiguration.serverAddress ?? "vpn.example.com" + let port = (protocolConfiguration as? NETunnelProviderProtocol)? + .providerConfiguration?["port"] as? String ?? "443" + + // 2. Establish a connection to the VPN server through the system's + // network stack (not the tunnel itself). + let endpoint = NWHostEndpoint(hostname: serverAddress, port: port) + connection = createTCPConnectionThroughTunnel( + to: endpoint, + enableTLS: true, + tlsParameters: nil, + delegate: nil + ) + + // 3. Configure the virtual network interface. + let settings = NEPacketTunnelNetworkSettings(tunnelRemoteAddress: serverAddress) + + // IPv4 settings -- route all traffic through the tunnel. + let ipv4 = NEIPv4Settings( + addresses: ["10.8.0.2"], + subnetMasks: ["255.255.255.0"] + ) + ipv4.includedRoutes = [NEIPv4Route.default()] + settings.ipv4Settings = ipv4 + + // DNS settings + settings.dnsSettings = NEDNSSettings(servers: ["10.8.0.1"]) + + // MTU + settings.mtu = 1400 as NSNumber + + // 4. Apply the settings. Once complete, the system starts routing + // traffic to the tunnel's virtual interface. + setTunnelNetworkSettings(settings) { error in + if let error = error { + completionHandler(error) + return + } + + // 5. Begin reading packets from the virtual interface. + self.readPacketsFromTUN() + completionHandler(nil) + } + } + + // 6. Continuously read outbound IP packets and forward to the VPN server. + private func readPacketsFromTUN() { + packetFlow.readPackets { [weak self] packets, protocols in + guard let self = self else { return } + + for (index, packet) in packets.enumerated() { + // In a real implementation, encrypt the packet and send + // it to the VPN server via self.connection. + self.sendPacketToServer(packet, protocolFamily: protocols[index]) + } + + // 7. Continue reading -- this is a recursive loop. + self.readPacketsFromTUN() + } + } + + private func sendPacketToServer(_ packet: Data, protocolFamily: NSNumber) { + // Encrypt and frame the packet for your VPN protocol, then write + // to self.connection. When encrypted packets arrive from the server, + // decrypt them and call: + // self.packetFlow.writePackets([decryptedData], withProtocols: [protocolFamily]) + } + + // 8. System calls this when the user disables the VPN. + override func stopTunnel( + with reason: NEProviderStopReason, + completionHandler: @escaping () -> Void + ) { + connection?.cancel() + connection = nil + completionHandler() + } + + // 9. Receive messages from the containing app (e.g., status queries). + override func handleAppMessage( + _ messageData: Data, + completionHandler: ((Data?) -> Void)? + ) { + completionHandler?(messageData) + } + + // 10. Power management hooks for sleep/wake. + override func sleep(completionHandler: @escaping () -> Void) { + completionHandler() + } + + override func wake() { + // Re-establish server connection if needed. + } +} +``` + +### Containing App -- Installing the VPN Configuration + +```swift +import NetworkExtension + +func installVPNProfile() { + let manager = NETunnelProviderManager() + + let proto = NETunnelProviderProtocol() + proto.providerBundleIdentifier = "com.example.app.PacketTunnel" + proto.serverAddress = "vpn.example.com" + proto.providerConfiguration = ["port": "443"] + + manager.protocolConfiguration = proto + manager.localizedDescription = "My VPN" + manager.isEnabled = true + + manager.saveToPreferences { error in + if let error = error { + print("Failed to save VPN profile: \(error)") + return + } + // Now the user can enable the VPN from Settings or programmatically. + } +} +``` + +## Usage + +- Create: `bun create target network-packet-tunnel` +- Edit: `targets/network-packet-tunnel/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 9.0+ | Full support. | +| iPadOS | 9.0+ | Full support. | +| macOS | 10.11+ | Runs as app extension (10.11) or system extension (10.15+). | +| tvOS | 17.0+ | Added in tvOS 17. | +| visionOS | 1.0+ | Supported. | +| watchOS | -- | Not supported. | + +## Gotchas + +- **Network Extension entitlement required.** You must request the Network Extension entitlement from Apple via [the request form](https://developer.apple.com/contact/request/network-extension/). Your app will be rejected without it. This is a manual approval process that can take days or weeks. +- **Memory limit is strict.** Packet tunnel providers run in a low-memory extension environment (typically ~15 MB on iOS). Exceeding the limit causes the system to kill your extension silently. Profile memory usage carefully and avoid buffering large amounts of packet data. +- **`readPackets` is a recursive loop, not a delegate.** You must call `readPackets(completionHandler:)` again inside its own completion handler to keep receiving packets. Forgetting to re-call it will silently stop all traffic flow. +- **Use `createTCPConnectionThroughTunnel` for server connections.** If you create a normal `URLSession` or `NWConnection` from within the extension, your traffic will loop through the tunnel itself. Use the provider's `createTCPConnection(to:enableTLS:)` or `createUDPSession(to:from:)` methods to reach your VPN server outside the tunnel. +- **TN3120 lists approved use cases.** Apple explicitly states that packet tunnel providers should only be used for VPN. Using them for content filtering, ad blocking, or local traffic inspection (loopback VPN tricks) may result in App Store rejection. See [TN3120](https://developer.apple.com/documentation/technotes/tn3120-expected-use-cases-for-network-extension-packet-tunnel-providers). +- **`setTunnelNetworkSettings` must complete before reading packets.** If you start calling `readPackets` before the settings completion handler fires, you will get no packets. Always chain the read loop inside the settings callback. +- **Sleep/wake handling is important.** On iOS, the system may suspend your extension when the device sleeps. Override `sleep(completionHandler:)` to save state and `wake()` to re-establish the server connection. Failing to handle this causes VPN disconnections after sleep. diff --git a/plugins/expo/skills/apple-targets/notification-content.md b/plugins/expo/skills/apple-targets/notification-content.md new file mode 100644 index 0000000..55933e3 --- /dev/null +++ b/plugins/expo/skills/apple-targets/notification-content.md @@ -0,0 +1,238 @@ +--- +title: Notification Content Extension +description: Displays a custom view controller when a notification is expanded, replacing the default notification UI with rich interactive content like images, maps, or custom controls. +version: iOS 10.0+ +--- + +# Notification Content Extension (`notification-content`) + +A notification content extension provides a custom user interface that appears when a user long-presses or 3D Touches a notification. Instead of the system's default layout, your extension renders a full `UIViewController` with any combination of images, video, maps, custom drawing, and -- starting with iOS 12 -- interactive controls like buttons and sliders. + +## Apple Documentation + +- [UNNotificationContentExtension](https://developer.apple.com/documentation/usernotificationsui/unnotificationcontentextension) -- The protocol your view controller adopts to receive and display notification content. +- [Customizing the Appearance of Notifications](https://developer.apple.com/documentation/usernotificationsui/customizing-the-appearance-of-notifications) -- Apple's guide to building notification content extensions. +- [UNNotificationContentExtensionResponseOption](https://developer.apple.com/documentation/usernotificationsui/unnotificationcontentextensionresponseoption) -- Return values for action handling in the extension. +- [Modifying and Presenting Notifications (Archive)](https://developer.apple.com/library/archive/documentation/NetworkingInternet/Conceptual/RemoteNotificationsPG/ModifyingNotifications.html) -- Legacy guide covering both service and content extensions. + +## WWDC History + +- **[WWDC 2016, Session 708 -- Advanced Notifications](https://developer.apple.com/videos/play/wwdc2016/708/)** -- Introduced notification content extensions alongside the new UserNotifications framework. Covers custom UI, media attachments, and action handling. +- **[WWDC 2017, Session 708 -- Best Practices and What's New in User Notifications](https://developer.apple.com/videos/play/wwdc2017/708/)** -- Refined guidance on content extension best practices and hidden default content. +- **[WWDC 2018, Session 710 -- What's New in User Notifications](https://developer.apple.com/videos/play/wwdc2018/710/)** -- Added interactive controls (buttons, sliders) via `UNNotificationExtensionUserInteractionEnabled`, dynamic action button updates, and `performNotificationDefaultAction` / `dismissNotificationContentExtension` methods. + +## What It Does + +1. A local or remote notification arrives with a `categoryIdentifier` matching the extension's `UNNotificationExtensionCategory` in Info.plist. +2. The user long-presses (or 3D Touches) the notification banner. +3. The system loads the extension's `UIViewController` and calls `didReceive(_:)` with the full `UNNotification`. +4. Your view controller populates its UI -- images, labels, maps, media players, custom views. +5. If the notification category defines actions, the system renders action buttons below your custom view. +6. When the user taps an action, the system calls `didReceive(_:completionHandler:)` on your view controller, giving you a chance to update the UI before dismissing. + +## Use Cases + +### Messaging apps with rich media +Display inline images, GIFs, or video thumbnails directly in the notification. A chat app can show the sender's avatar, the message bubble, and a preview of any attached media -- all without opening the app. + +### Delivery and ride-sharing apps with maps +Embed a `MKMapView` showing the driver's current location or the package's delivery route. The map updates each time a new notification arrives for the same conversation thread. + +### Calendar and event apps with quick RSVP +Show event details (time, location, attendee list) in a styled card layout with interactive "Accept" / "Decline" buttons (iOS 12+) that update the UI inline and call your server. + +### Sports and live score apps +Display a custom scoreboard layout with team logos, scores, and game clock. Each score update notification refreshes the same expanded view. + +## Key Classes + +| Class / Protocol | Role | +|-----------------|------| +| `UNNotificationContentExtension` | Protocol adopted by your `UIViewController`. Provides `didReceive(_:)` to populate UI and optionally `didReceive(_:completionHandler:)` to handle actions. | +| `UNNotification` | The delivered notification passed to your extension, containing the full `UNNotificationContent`. | +| `UNNotificationContent` | The notification payload -- title, subtitle, body, attachments, userInfo, categoryIdentifier. | +| `UNNotificationAction` | An action button registered on the notification category. The system renders these below your view. | +| `NSExtensionContext` | Provides `performNotificationDefaultAction()` to open the app and `dismissNotificationContentExtension()` to dismiss (iOS 12+). | + +## Implementation + +### Rich Media Notification with Action Handling + +A notification content extension that displays an image from the notification's attachments and handles a "Like" action with inline UI feedback. + +```swift +import UIKit +import UserNotifications +import UserNotificationsUI + +class NotificationViewController: UIViewController, UNNotificationContentExtension { + + // 1. Create UI elements programmatically (or use a storyboard) + private let imageView: UIImageView = { + let iv = UIImageView() + iv.contentMode = .scaleAspectFill + iv.clipsToBounds = true + iv.translatesAutoresizingMaskIntoConstraints = false + return iv + }() + + private let titleLabel: UILabel = { + let label = UILabel() + label.font = .boldSystemFont(ofSize: 17) + label.numberOfLines = 2 + label.translatesAutoresizingMaskIntoConstraints = false + return label + }() + + private let bodyLabel: UILabel = { + let label = UILabel() + label.font = .systemFont(ofSize: 15) + label.textColor = .secondaryLabel + label.numberOfLines = 0 + label.translatesAutoresizingMaskIntoConstraints = false + return label + }() + + override func viewDidLoad() { + super.viewDidLoad() + + // 2. Layout the custom UI + view.addSubview(imageView) + view.addSubview(titleLabel) + view.addSubview(bodyLabel) + + NSLayoutConstraint.activate([ + imageView.topAnchor.constraint(equalTo: view.topAnchor), + imageView.leadingAnchor.constraint(equalTo: view.leadingAnchor), + imageView.trailingAnchor.constraint(equalTo: view.trailingAnchor), + imageView.heightAnchor.constraint(equalTo: view.widthAnchor, multiplier: 0.5625), + + titleLabel.topAnchor.constraint(equalTo: imageView.bottomAnchor, constant: 12), + titleLabel.leadingAnchor.constraint(equalTo: view.leadingAnchor, constant: 16), + titleLabel.trailingAnchor.constraint(equalTo: view.trailingAnchor, constant: -16), + + bodyLabel.topAnchor.constraint(equalTo: titleLabel.bottomAnchor, constant: 4), + bodyLabel.leadingAnchor.constraint(equalTo: titleLabel.leadingAnchor), + bodyLabel.trailingAnchor.constraint(equalTo: titleLabel.trailingAnchor), + bodyLabel.bottomAnchor.constraint(lessThanOrEqualTo: view.bottomAnchor, constant: -12), + ]) + } + + // 3. Called when the notification is expanded -- populate your UI + func didReceive(_ notification: UNNotification) { + let content = notification.request.content + + titleLabel.text = content.title + bodyLabel.text = content.body + + // 4. Load the first attachment (e.g., an image downloaded by a service extension) + if let attachment = content.attachments.first, + attachment.url.startAccessingSecurityScopedResource() { + defer { attachment.url.stopAccessingSecurityScopedResource() } + if let data = try? Data(contentsOf: attachment.url) { + imageView.image = UIImage(data: data) + } + } + + // 5. Adjust preferred content size based on whether there's an image + if imageView.image == nil { + imageView.isHidden = true + preferredContentSize = CGSize(width: view.bounds.width, height: 80) + } + } + + // 6. Handle action button taps inline (optional) + func didReceive( + _ response: UNNotificationResponse, + completionHandler completion: @escaping (UNNotificationContentExtensionResponseOption) -> Void + ) { + switch response.actionIdentifier { + case "LIKE_ACTION": + // 7. Update UI to reflect the action + titleLabel.text = "Liked!" + titleLabel.textColor = .systemBlue + + // 8. Dismiss after a short delay so the user sees the feedback + DispatchQueue.main.asyncAfter(deadline: .now() + 0.8) { + completion(.dismiss) + } + + case "REPLY_ACTION": + // 9. Open the app for complex interactions + completion(.dismissAndForwardAction) + + default: + completion(.doNotDismiss) + } + } +} +``` + +### Registering the Notification Category (in your main app) + +Your main app must register the category that triggers the content extension: + +```swift +import UserNotifications + +func registerNotificationCategories() { + let likeAction = UNNotificationAction( + identifier: "LIKE_ACTION", + title: "Like", + options: [] + ) + let replyAction = UNTextInputNotificationAction( + identifier: "REPLY_ACTION", + title: "Reply", + options: [], + textInputButtonTitle: "Send", + textInputPlaceholder: "Type a reply..." + ) + + // The category identifier must match UNNotificationExtensionCategory in the extension's Info.plist + let category = UNNotificationCategory( + identifier: "myNotificationCategory", + actions: [likeAction, replyAction], + intentIdentifiers: [], + options: [] + ) + + UNUserNotificationCenter.current().setNotificationCategories([category]) +} +``` + +## Usage + +- Create: `bun create target notification-content` +- Edit: `targets/notification-content/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 10.0+ | Full support. Interactive controls from iOS 12+. | +| iPadOS | 10.0+ | Full support. Some layout inconsistencies with `UNNotificationExtensionInitialContentSizeRatio` on iPad. | +| macOS | 11.0+ | Listed in API docs but does not work reliably for native macOS apps. Works via Mac Catalyst. | +| visionOS | -- | Not supported. | +| watchOS | -- | Not supported. watchOS uses WatchKit notification interfaces instead. | +| tvOS | -- | Not supported. | + +## Gotchas + +- **Category identifier must match exactly.** The `UNNotificationExtensionCategory` value in your extension's Info.plist must be an exact, case-sensitive match to the `categoryIdentifier` on the notification and the category registered with `UNUserNotificationCenter`. A mismatch causes the extension to silently never appear. + +- **Touch events are blocked before iOS 12.** Prior to iOS 12, the system prevents delivery of all touch events to your view controller. Gesture recognizers and interactive controls will not work. On iOS 12+, set `UNNotificationExtensionUserInteractionEnabled` to `true` in Info.plist to enable interaction. + +- **`preferredContentSize` timing is tricky.** At `viewDidLoad` time, the view's `bounds.size.width` is not yet correct (it reports full screen size). Set `preferredContentSize` in `didReceive(_:)` after layout, or use `UNNotificationExtensionInitialContentSizeRatio` in Info.plist as the initial aspect ratio and let Auto Layout handle the rest. + +- **One extension per category set.** Each content extension must handle a unique set of categories. If two extensions claim the same category identifier, behavior is undefined. Use an array value for `UNNotificationExtensionCategory` if one extension handles multiple categories. + +- **iPad layout sizing is inconsistent.** Several developers have reported that iPad does not respect `UNNotificationExtensionInitialContentSizeRatio` correctly, and Auto Layout guides can report very large values during the first layout pass. Test on iPad hardware or simulator specifically. + +- **Action buttons are system-managed.** The system automatically renders action buttons defined on the `UNNotificationCategory` below your custom view. Do not create your own action buttons -- use `didReceive(_:completionHandler:)` to intercept taps and update your UI. + +- **Attachments require security-scoped access.** When reading notification attachment files, you must call `startAccessingSecurityScopedResource()` before and `stopAccessingSecurityScopedResource()` after. Without this, file reads will fail silently or throw permission errors. + +- **The extension has a separate memory budget.** Content extensions run in their own process with limited memory (roughly 120 MB on recent devices). Loading large images or video without downsampling can cause the system to kill the extension. + +- **Default content is shown unless hidden.** The system shows the standard notification title/body below your custom view by default. Set `UNNotificationExtensionDefaultContentHidden` to `true` in Info.plist if your custom UI already displays this information. diff --git a/plugins/expo/skills/apple-targets/notification-service.md b/plugins/expo/skills/apple-targets/notification-service.md new file mode 100644 index 0000000..b97ff8a --- /dev/null +++ b/plugins/expo/skills/apple-targets/notification-service.md @@ -0,0 +1,272 @@ +--- +title: Notification Service Extension +description: Intercepts and modifies remote notification content before display, enabling image downloads, payload decryption, and content enrichment in the background. +version: iOS 10.0+ +--- + +# Notification Service Extension (`notification-service`) + +A notification service extension runs in the background when a remote push notification arrives, giving you up to 30 seconds to modify the notification content before the system displays it. The most common use is downloading an image from a URL in the payload and attaching it so the notification displays a rich media preview. It also handles end-to-end encryption decryption, content localization, and analytics tracking. + +## Apple Documentation + +- [UNNotificationServiceExtension](https://developer.apple.com/documentation/usernotifications/unnotificationserviceextension) -- The class you subclass to modify remote notification content before delivery. +- [didReceive(_:withContentHandler:)](https://developer.apple.com/documentation/usernotifications/unnotificationserviceextension/1648229-didreceive) -- Called when a notification arrives. Modify content and call the handler. +- [serviceExtensionTimeWillExpire()](https://developer.apple.com/documentation/usernotifications/unnotificationserviceextension/1648227-serviceextensiontimewillexpire) -- Last-chance callback before the system delivers the original notification. +- [Modifying Content in Newly Delivered Notifications](https://developer.apple.com/documentation/usernotifications/modifying-content-in-newly-delivered-notifications) -- Apple's guide to implementing a notification service extension. + +## WWDC History + +- **[WWDC 2016, Session 708 -- Advanced Notifications](https://developer.apple.com/videos/play/wwdc2016/708/)** -- Introduced notification service extensions as part of the new UserNotifications framework. Covers the extension lifecycle, mutable-content flag, and the 30-second processing window. +- **[WWDC 2017, Session 708 -- Best Practices and What's New in User Notifications](https://developer.apple.com/videos/play/wwdc2017/708/)** -- Best practices for service extension performance and reliability. +- **[WWDC 2018, Session 710 -- What's New in User Notifications](https://developer.apple.com/videos/play/wwdc2018/710/)** -- Grouped notifications and how service extensions interact with notification threading. + +## What It Does + +1. Your server sends a remote push notification with `"mutable-content": 1` in the `aps` dictionary. +2. The system launches (or reuses) your service extension process before displaying the notification. +3. The system calls `didReceive(_:withContentHandler:)` with the original notification request. +4. Your code modifies the `UNMutableNotificationContent` -- downloading images, decrypting the body, changing the title, adding attachments. +5. You call the `contentHandler` with the modified content. +6. If you take too long (~30 seconds), the system calls `serviceExtensionTimeWillExpire()` as a last chance to deliver your best-effort modification. +7. If neither method calls the handler in time, the system displays the original unmodified notification. + +## Use Cases + +### Rich push notifications with images +The most common use case. Your server includes an image URL in the push payload's `userInfo`. The service extension downloads the image, saves it to a temporary file, and attaches it as a `UNNotificationAttachment`. The notification then shows a thumbnail on the right side and a full preview when expanded. + +### End-to-end encrypted messaging +Messaging apps that encrypt notification payloads on the server. The service extension decrypts the ciphertext using keys stored in the shared Keychain (via App Groups), then replaces the placeholder title and body with the actual message content. + +### Analytics and delivery tracking +Confirm that a push notification was actually delivered to the device. The service extension pings your analytics endpoint when the notification arrives, providing delivery confirmation independent of whether the user opens the notification. + +### Content localization and enrichment +Enrich a minimal push payload with locally-available data. For example, a payload containing only a conversation ID can be expanded by the service extension into a full message preview by querying a local database or shared Core Data store. + +## Key Classes + +| Class | Role | +|-------|------| +| `UNNotificationServiceExtension` | Abstract class to subclass. Override `didReceive(_:withContentHandler:)` and `serviceExtensionTimeWillExpire()`. | +| `UNNotificationRequest` | The incoming notification request containing the original `UNNotificationContent`. | +| `UNMutableNotificationContent` | Mutable copy of the notification content. Modify `title`, `body`, `subtitle`, `attachments`, `userInfo`, `badge`, `sound`, `threadIdentifier`. | +| `UNNotificationAttachment` | A media file (image, audio, video) attached to the notification. Created from a file URL on disk. | + +## Implementation + +### Downloading an Image for Rich Notifications + +The standard implementation: download an image from a URL in the push payload and attach it to the notification. + +```swift +import UserNotifications + +class NotificationService: UNNotificationServiceExtension { + + var contentHandler: ((UNNotificationContent) -> Void)? + var bestAttemptContent: UNMutableNotificationContent? + + override func didReceive( + _ request: UNNotificationRequest, + withContentHandler contentHandler: @escaping (UNNotificationContent) -> Void + ) { + self.contentHandler = contentHandler + bestAttemptContent = (request.content.mutableCopy() as? UNMutableNotificationContent) + + guard let bestAttemptContent = bestAttemptContent else { + contentHandler(request.content) + return + } + + // 1. Extract the image URL from the push payload's userInfo + guard let imageURLString = request.content.userInfo["image_url"] as? String, + let imageURL = URL(string: imageURLString) else { + // No image to download -- deliver the content as-is + contentHandler(bestAttemptContent) + return + } + + // 2. Download the image to a temporary file + let task = URLSession.shared.downloadTask(with: imageURL) { location, response, error in + defer { contentHandler(bestAttemptContent) } + + guard let location = location, error == nil else { return } + + // 3. Determine file extension from the response MIME type + let fileExtension: String + if let mimeType = response?.mimeType { + switch mimeType { + case "image/png": fileExtension = "png" + case "image/gif": fileExtension = "gif" + case "image/jpeg": fileExtension = "jpg" + default: fileExtension = "jpg" + } + } else { + fileExtension = "jpg" + } + + // 4. Move the file to a uniquely-named temporary path + // (UNNotificationAttachment requires the file extension) + let tempDir = FileManager.default.temporaryDirectory + let tempFile = tempDir + .appendingPathComponent(UUID().uuidString) + .appendingPathExtension(fileExtension) + + do { + try FileManager.default.moveItem(at: location, to: tempFile) + + // 5. Create the attachment and add it to the notification + let attachment = try UNNotificationAttachment( + identifier: "image", + url: tempFile, + options: nil + ) + bestAttemptContent.attachments = [attachment] + } catch { + // If attachment creation fails, deliver without the image + print("Notification attachment error: \(error)") + } + } + task.resume() + } + + override func serviceExtensionTimeWillExpire() { + // 6. Called when the system is about to kill the extension. + // Deliver whatever you have -- possibly without the image. + if let contentHandler = contentHandler, + let bestAttemptContent = bestAttemptContent { + contentHandler(bestAttemptContent) + } + } +} +``` + +### Decrypting Encrypted Notification Content + +A service extension that decrypts an encrypted payload using a key stored in the shared Keychain: + +```swift +import UserNotifications +import CryptoKit + +class NotificationService: UNNotificationServiceExtension { + + var contentHandler: ((UNNotificationContent) -> Void)? + var bestAttemptContent: UNMutableNotificationContent? + + override func didReceive( + _ request: UNNotificationRequest, + withContentHandler contentHandler: @escaping (UNNotificationContent) -> Void + ) { + self.contentHandler = contentHandler + bestAttemptContent = (request.content.mutableCopy() as? UNMutableNotificationContent) + + guard let bestAttemptContent = bestAttemptContent else { + contentHandler(request.content) + return + } + + // 1. Extract the encrypted payload from userInfo + guard let encryptedBase64 = request.content.userInfo["encrypted_body"] as? String, + let encryptedData = Data(base64Encoded: encryptedBase64) else { + contentHandler(bestAttemptContent) + return + } + + // 2. Retrieve the decryption key from the shared Keychain (via App Groups) + guard let symmetricKey = loadKeyFromSharedKeychain() else { + bestAttemptContent.body = "New encrypted message" + contentHandler(bestAttemptContent) + return + } + + // 3. Decrypt the content + do { + let sealedBox = try AES.GCM.SealedBox(combined: encryptedData) + let decryptedData = try AES.GCM.open(sealedBox, using: symmetricKey) + + if let decryptedString = String(data: decryptedData, encoding: .utf8) { + // 4. Replace the placeholder body with the decrypted message + bestAttemptContent.body = decryptedString + } + } catch { + bestAttemptContent.body = "New encrypted message" + } + + contentHandler(bestAttemptContent) + } + + override func serviceExtensionTimeWillExpire() { + if let contentHandler = contentHandler, + let bestAttemptContent = bestAttemptContent { + bestAttemptContent.body = "New encrypted message" + contentHandler(bestAttemptContent) + } + } + + private func loadKeyFromSharedKeychain() -> SymmetricKey? { + // Load from Keychain with kSecAttrAccessGroup set to your App Group + // Implementation depends on your key storage strategy + return nil + } +} +``` + +### Example Push Payload + +The push notification must include `mutable-content: 1` to trigger the service extension: + +```json +{ + "aps": { + "alert": { + "title": "New Photo", + "body": "Sarah shared a photo with you" + }, + "mutable-content": 1, + "sound": "default" + }, + "image_url": "https://example.com/photos/12345.jpg" +} +``` + +## Usage + +- Create: `bun create target notification-service` +- Edit: `targets/notification-service/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 10.0+ | Full support. | +| iPadOS | 10.0+ | Full support. | +| macOS | 10.14+ | Supported for native macOS apps receiving remote notifications. | +| Mac Catalyst | 13.0+ | Supported. | +| visionOS | -- | Not documented. | +| watchOS | -- | Not supported. watchOS notifications are processed by the paired iPhone. | +| tvOS | -- | Not supported. tvOS does not display notification banners. | + +## Gotchas + +- **Push must include `mutable-content: 1`.** Without this flag in the `aps` dictionary, the system will never launch your service extension. This is the single most common reason the extension appears to "not work." The payload must also include an `alert` dictionary (title, subtitle, or body) -- silent notifications do not trigger the extension. + +- **You have approximately 30 seconds.** The system enforces a hard time limit. If neither `didReceive` nor `serviceExtensionTimeWillExpire` calls the content handler in time, the original unmodified notification is displayed. Always implement `serviceExtensionTimeWillExpire()` as a fallback. + +- **Memory limit is approximately 24 MB.** The extension runs in a constrained process. Downloading or processing large images can push you over the limit, causing immediate termination with no callback. Resize images before creating attachments if the source is large. + +- **The extension does not run when the app is in the foreground.** If your app is active and implements `userNotificationCenter(_:willPresent:withCompletionHandler:)`, the delegate receives the original unmodified payload. The service extension is skipped entirely. + +- **The extension process is reused between notifications.** iOS keeps the extension process alive after processing a notification. A second notification arriving shortly after reuses the same process. This means stale state from a previous `didReceive` call can leak into the next one. Always reset `bestAttemptContent` and `contentHandler` at the start of each call. + +- **Attachments require files on disk with correct extensions.** `UNNotificationAttachment` reads the file extension to determine the media type. A file named `image` with no extension will fail. Always append `.jpg`, `.png`, `.gif`, or `.mp4` to your temporary files. + +- **Shared data requires App Groups.** The extension runs in a separate process and sandbox. To access shared preferences, databases, or Keychain items, configure an App Group on both the main app and the extension. Use `UserDefaults(suiteName:)` or `kSecAttrAccessGroup` for Keychain. + +- **Debugging is difficult.** You cannot launch the extension target directly. Attach the debugger to the extension process by selecting Debug > Attach to Process by PID or Name in Xcode after sending a push notification. Use `Console.app` filtering by your extension's bundle ID to see logs. + +- **`content-available: 1` is not `mutable-content: 1`.** These are different flags. `content-available` triggers background app refresh (silent push). `mutable-content` triggers the service extension. You can use both, but they serve different purposes. Confusing the two is a common mistake. + +- **Notification filtering requires a special entitlement.** Since iOS 13.3, you can suppress notification display entirely by not calling the content handler. However, this requires the `com.apple.developer.usernotifications.filtering` entitlement, which must be requested from Apple. Without it, suppressing delivery will show the original notification. diff --git a/plugins/expo/skills/apple-targets/photo-editing.md b/plugins/expo/skills/apple-targets/photo-editing.md new file mode 100644 index 0000000..d24888f --- /dev/null +++ b/plugins/expo/skills/apple-targets/photo-editing.md @@ -0,0 +1,285 @@ +--- +title: Photo Editing Extension +description: Adds custom editing tools directly inside the Photos app, allowing users to apply filters, adjustments, and markup without leaving their library. +version: iOS 8.0+, macOS 10.11+ +--- + +# Photo Editing Extension (`photo-editing`) + +An extension that integrates custom image and video editing tools into the system Photos app. When a user taps Edit on a photo and selects your extension from the action menu, the Photos app hands your view controller the asset as a `PHContentEditingInput` along with a placeholder image. Your UI applies edits -- filters, crops, overlays, markup -- and returns a `PHContentEditingOutput` containing the rendered result and a `PHAdjustmentData` blob that encodes the edit recipe. Because Photos always preserves the original asset, your edits are non-destructive: the user can reopen your extension later, and if your `canHandle(_:)` method recognizes the adjustment data, you can reconstruct the previous edit state and let them modify or revert changes. + +## Apple Documentation + +- [Creating Photo Editing Extensions](https://developer.apple.com/documentation/photokit/creating-photo-editing-extensions) +- [PHContentEditingController Protocol](https://developer.apple.com/documentation/photokit/phcontenteditingcontroller) +- [PHContentEditingInput](https://developer.apple.com/documentation/photokit/phcontenteditinginput) +- [PHContentEditingOutput](https://developer.apple.com/documentation/photokit/phcontenteditingoutput) +- [PHAdjustmentData](https://developer.apple.com/documentation/photokit/phadjustmentdata) +- [App Extension Programming Guide -- Photo Editing](https://developer.apple.com/library/archive/documentation/General/Conceptual/ExtensibilityPG/Photos.html) +- [Sample Photo Editing Extension (Apple Archive)](https://developer.apple.com/library/archive/samplecode/SamplePhotoEditingExtension/Introduction/Intro.html) + +## WWDC History + +- **[WWDC 2014, Session 511 -- Introducing the Photos Frameworks](https://developer.apple.com/videos/play/wwdc2014/511/)** -- Introduced PhotoKit and the Photo Editing extension point. Demonstrated how `PHContentEditingController` integrates third-party editing into the Photos app with non-destructive adjustment data. +- **[WWDC 2014, Session 514 -- Advances in Core Image](https://developer.apple.com/videos/play/wwdc2014/514/)** -- Showed how custom `CIKernel` and `CIFilter` subclasses can be used inside Photo Editing extensions for GPU-accelerated image processing. +- **[WWDC 2014, Session 217 -- Creating Extensions for iOS and OS X, Part 2](https://developer.apple.com/videos/play/wwdc2014/217/)** -- Covered the broader app extensions architecture that Photo Editing extensions are built on, including lifecycle and memory management. +- **[WWDC 2018, Session 505 -- Integrating with Photos on macOS](https://developer.apple.com/videos/play/wwdc2018/505/)** -- Covered macOS-specific Photo Editing extension integration and the unified Photos experience across platforms. + +## What It Does + +1. The user opens a photo or video in the Photos app and taps Edit. +2. The user taps the extensions button (the ellipsis icon) and selects your extension from the activity list. +3. Photos calls `canHandle(_:)` with the existing `PHAdjustmentData` (if any previous edit was made). If you return `true`, Photos provides the original, unmodified asset so you can reconstruct the prior edit state. If you return `false`, Photos provides the already-rendered version. +4. Photos calls `startContentEditing(with:placeholderImage:)`, passing a `PHContentEditingInput` (containing the full-size image URL or video AVAsset) and a `UIImage` placeholder for immediate display. +5. Your view controller presents editing controls. The user applies filters, adjustments, or markup. +6. When the user taps Done, Photos calls `finishContentEditing(completionHandler:)`. You render the final image, create a `PHContentEditingOutput` with the rendered JPEG written to `output.renderedContentURL`, and attach a `PHAdjustmentData` object encoding your edit parameters. +7. Photos stores the output alongside the original. The edit is non-destructive -- the user can revert to the original at any time. +8. If the user taps Cancel, Photos calls `cancelContentEditing()` and your extension cleans up any temporary files. + +## Use Cases + +### Filter and Effects Apps + +A photo filter app (VSCO, Snapseed-style) exposes its filter library directly inside Photos. Users apply a look, adjust intensity, and save -- all without leaving the Photos app. The `PHAdjustmentData` stores the filter name and parameter values so users can re-edit later. + +### Markup and Annotation Tools + +A markup tool adds text, arrows, shapes, and freehand drawings on top of photos. The adjustment data serializes each annotation's position, style, and content, enabling full round-trip editing. + +### Professional Retouching + +A portrait retouching extension provides skin smoothing, blemish removal, and lighting adjustments. Because it receives the full-resolution image via `PHContentEditingInput.fullSizeImageURL`, it can operate at the native camera resolution. + +### Batch Watermarking + +A business or photographer's extension applies a watermark overlay with customizable position, opacity, and text. The simple parameter set makes `PHAdjustmentData` round-trips straightforward. + +## Key Classes + +| Class / Protocol | Role | +|-------|------| +| `PHContentEditingController` | Protocol your `UIViewController` adopts. Provides the lifecycle methods: `canHandle`, `startContentEditing`, `finishContentEditing`, `cancelContentEditing`. | +| `PHContentEditingInput` | Describes the asset being edited. Provides `fullSizeImageURL`, `displaySizeImage`, `avAsset` (for video), and existing `adjustmentData`. | +| `PHContentEditingOutput` | Container for your edited result. Write the rendered JPEG/video to `renderedContentURL` and attach `adjustmentData`. | +| `PHAdjustmentData` | Encodes your edit recipe as a `Data` blob with a `formatIdentifier` (reverse-DNS) and `formatVersion` (semantic versioning recommended). Enables non-destructive round-trip editing. | +| `CIFilter` / `CIContext` | Core Image classes commonly used to apply GPU-accelerated filters to the full-size image inside the extension. | + +## Implementation + +### Core Image Filter Extension with Adjustment Data Round-Trip + +A realistic photo editing extension that applies a sepia + vignette filter chain, serializes the parameters into `PHAdjustmentData`, and supports re-editing previous sessions: + +```swift +import UIKit +import Photos +import PhotosUI + +class PhotoEditingViewController: UIViewController, PHContentEditingController { + + // 1. Define a stable format identifier and version for adjustment data. + // Use reverse-DNS and semantic versioning so you can evolve the format. + private static let formatIdentifier = "com.example.photo-editing" + private static let formatVersion = "1.0" + + private var input: PHContentEditingInput? + private let imageView = UIImageView() + private let intensitySlider = UISlider() + private let ciContext = CIContext() + + // 2. Track the current filter intensity (0.0 to 1.0) + private var filterIntensity: Float = 0.5 + + override func viewDidLoad() { + super.viewDidLoad() + view.backgroundColor = .black + + imageView.contentMode = .scaleAspectFit + imageView.translatesAutoresizingMaskIntoConstraints = false + view.addSubview(imageView) + + intensitySlider.minimumValue = 0.0 + intensitySlider.maximumValue = 1.0 + intensitySlider.value = filterIntensity + intensitySlider.addTarget(self, action: #selector(sliderChanged), for: .valueChanged) + intensitySlider.translatesAutoresizingMaskIntoConstraints = false + view.addSubview(intensitySlider) + + NSLayoutConstraint.activate([ + imageView.topAnchor.constraint(equalTo: view.safeAreaLayoutGuide.topAnchor), + imageView.leadingAnchor.constraint(equalTo: view.leadingAnchor), + imageView.trailingAnchor.constraint(equalTo: view.trailingAnchor), + imageView.bottomAnchor.constraint(equalTo: intensitySlider.topAnchor, constant: -16), + intensitySlider.leadingAnchor.constraint(equalTo: view.leadingAnchor, constant: 24), + intensitySlider.trailingAnchor.constraint(equalTo: view.trailingAnchor, constant: -24), + intensitySlider.bottomAnchor.constraint(equalTo: view.safeAreaLayoutGuide.bottomAnchor, constant: -16), + ]) + } + + // MARK: - PHContentEditingController + + // 3. Check whether we can resume editing from existing adjustment data. + // Return true only if the data was created by this extension and version. + func canHandle(_ adjustmentData: PHAdjustmentData) -> Bool { + return adjustmentData.formatIdentifier == Self.formatIdentifier + && adjustmentData.formatVersion == Self.formatVersion + } + + // 4. Called before the view appears. Receive the asset and restore prior edits. + func startContentEditing(with contentEditingInput: PHContentEditingInput, placeholderImage: UIImage) { + input = contentEditingInput + + // 5. If we have prior adjustment data from a previous session, decode it + // to restore the slider position. + if let adjustmentData = contentEditingInput.adjustmentData, + adjustmentData.formatIdentifier == Self.formatIdentifier, + let params = try? JSONDecoder().decode(EditParameters.self, from: adjustmentData.data) { + filterIntensity = params.intensity + intensitySlider.value = filterIntensity + } + + // 6. Show the placeholder immediately, then apply the filter + imageView.image = placeholderImage + applyFilter() + } + + // 7. Render the final output and return it via the completion handler. + func finishContentEditing(completionHandler: @escaping (PHContentEditingOutput?) -> Void) { + guard let input = input else { + completionHandler(nil) + return + } + + DispatchQueue.global(qos: .userInitiated).async { [self] in + let output = PHContentEditingOutput(contentEditingInput: input) + + // 8. Encode the current parameters as adjustment data so the edit + // can be reconstructed in a future session. + let params = EditParameters(intensity: filterIntensity) + guard let paramData = try? JSONEncoder().encode(params) else { + completionHandler(nil) + return + } + + output.adjustmentData = PHAdjustmentData( + formatIdentifier: Self.formatIdentifier, + formatVersion: Self.formatVersion, + data: paramData + ) + + // 9. Load the full-size image and apply the filter at full resolution. + guard let fullSizeURL = input.fullSizeImageURL, + let fullImage = CIImage(contentsOf: fullSizeURL)? + .oriented(forExifOrientation: Int32(input.fullSizeImageOrientation)) else { + completionHandler(nil) + return + } + + let filtered = self.applyFilterChain(to: fullImage, intensity: self.filterIntensity) + + // 10. Write the rendered JPEG to the output URL. + // Use a CIContext to avoid loading the full image into a UIImage. + guard let colorSpace = CGColorSpace(name: CGColorSpace.sRGB), + let jpegData = self.ciContext.jpegRepresentation( + of: filtered, + colorSpace: colorSpace, + options: [kCGImageDestinationLossyCompressionQuality as CIImageRepresentationOption: 0.9] + ) else { + completionHandler(nil) + return + } + + do { + try jpegData.write(to: output.renderedContentURL) + completionHandler(output) + } catch { + NSLog("[PhotoEditing] Failed to write output: \(error)") + completionHandler(nil) + } + } + } + + var shouldShowCancelConfirmation: Bool { + // 11. Return true if the user has made changes worth confirming + return filterIntensity != 0.5 + } + + func cancelContentEditing() { + // 12. Clean up any temporary files or state + input = nil + } + + // MARK: - Filter Logic + + @objc private func sliderChanged(_ sender: UISlider) { + filterIntensity = sender.value + applyFilter() + } + + private func applyFilter() { + guard let input = input, + let url = input.fullSizeImageURL, + let image = CIImage(contentsOf: url)? + .oriented(forExifOrientation: Int32(input.fullSizeImageOrientation)) else { return } + + let filtered = applyFilterChain(to: image, intensity: filterIntensity) + + // 13. Render a display-size preview. Use a smaller extent to save memory. + if let cgImage = ciContext.createCGImage(filtered, from: filtered.extent) { + DispatchQueue.main.async { + self.imageView.image = UIImage(cgImage: cgImage) + } + } + } + + // 14. Apply a sepia tone + vignette filter chain. The intensity parameter + // controls both the sepia amount and vignette radius. + private func applyFilterChain(to image: CIImage, intensity: Float) -> CIImage { + let sepia = CIFilter(name: "CISepiaTone")! + sepia.setValue(image, forKey: kCIInputImageKey) + sepia.setValue(NSNumber(value: intensity), forKey: kCIInputIntensityKey) + + let vignette = CIFilter(name: "CIVignette")! + vignette.setValue(sepia.outputImage!, forKey: kCIInputImageKey) + vignette.setValue(NSNumber(value: intensity * 2.0), forKey: kCIInputRadiusKey) + vignette.setValue(NSNumber(value: intensity), forKey: kCIInputIntensityKey) + + return vignette.outputImage! + } +} + +// 15. Codable struct for serializing edit parameters into PHAdjustmentData. +// Keep this lightweight -- store the recipe, not the rendered pixels. +struct EditParameters: Codable { + let intensity: Float +} +``` + +## Usage + +- Create: `bun create target photo-editing` +- Edit: `targets/photo-editing/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 8.0+ | Full support. Extension appears in the Photos app Edit screen. | +| iPadOS | 8.0+ | Full support. Must handle Slide Over and Split View presentations. | +| macOS | 10.11+ | Supported. Uses a combined title bar and toolbar instead of a navigation bar. | +| tvOS | -- | Not supported. Photos app on tvOS does not support editing extensions. | +| visionOS | -- | Not supported. | +| watchOS | -- | Not supported. | + +## Gotchas + +- **`PHAdjustmentData` round-trips are required for a good user experience.** If your `canHandle(_:)` returns `false` for your own previous edits, the user gets the flattened (already-rendered) image instead of the original, losing the ability to tweak prior adjustments. Always check `formatIdentifier` and `formatVersion` and decode your parameters. +- **Returning `true` from `canHandle` means you get the original image.** This is counterintuitive. When you claim you can handle the adjustment data, Photos gives you the unmodified original so you can reconstruct the edit from scratch. When you return `false`, you get the previously rendered output. Many developers expect the opposite. +- **Store the recipe, not the result, in adjustment data.** The `PHAdjustmentData.data` blob has a size limit enforced by the Photos framework. Serialize filter names and parameter values (a few hundred bytes), not pixel data or thumbnails. +- **Full-size images can be very large.** A 48MP iPhone 15 Pro photo is over 8000x6000 pixels. Loading it uncompressed into a `UIImage` can consume hundreds of megabytes. Use `CIImage` with a `CIContext` for lazy evaluation, and render only the regions you need. Extensions are killed under memory pressure with no warning. +- **Image orientation must be applied manually.** The `fullSizeImageOrientation` property on `PHContentEditingInput` returns the EXIF orientation as an `Int32`. You must apply it to your `CIImage` with `.oriented(forExifOrientation:)` or the image will appear rotated. This is a common source of bugs. +- **Video editing has a different workflow.** For video assets, `PHContentEditingInput` provides an `avAsset` (AVAsset) instead of an image URL. You must use AVFoundation (`AVAssetExportSession` or `AVComposition`) to produce the output video. The `PHAdjustmentData` round-trip requirement still applies. +- **The Photos app provides the navigation bar.** Do not add your own navigation bar or top toolbar. Photos displays a system navigation bar with Cancel and Done buttons. Adding your own creates a confusing double-bar UI. +- **Cross-app adjustment data compatibility is possible but rare.** Multiple apps can agree on a shared `formatIdentifier` and data schema so users can seamlessly switch between editing tools. In practice, almost no apps do this -- but be aware that your extension may receive adjustment data from an unknown source. +- **The Xcode template defaults `canHandle` to `false`.** The generated stub returns `false`, which means re-editing always starts from the flattened output. You must change this to support proper non-destructive editing. +- **Test with varied media formats.** Do not assume all inputs are JPEG from the device camera. HEIF, PNG, RAW (DNG), and Live Photos all flow through the same extension. Verify your `CIFilter` chain handles each format without crashing. diff --git a/plugins/expo/skills/apple-targets/print-service.md b/plugins/expo/skills/apple-targets/print-service.md new file mode 100644 index 0000000..bad2dcd --- /dev/null +++ b/plugins/expo/skills/apple-targets/print-service.md @@ -0,0 +1,142 @@ +--- +title: Print Service Discovery Extension +description: Discovers and configures custom printers that are not supported by AirPrint or standard IPP discovery, primarily used by printer manufacturers. +version: iOS 14.0+ +--- + +# Print Service Discovery Extension (`print-service`) + +A Print Service extension locates and sets up printers that cannot be discovered through standard AirPrint or Bonjour/DNS-SD mechanisms. When the user opens the system print dialog, iOS loads installed Print Service extensions to find additional printers beyond those visible via AirPrint. This is an extremely niche extension type used almost exclusively by printer manufacturers who need to support proprietary discovery protocols (e.g., vendor-specific Wi-Fi Direct, Bluetooth, or USB handshakes). Public documentation from Apple is minimal. + +## Apple Documentation + +- [UIPrintServiceExtension](https://developer.apple.com/documentation/uikit/uiprintserviceextension) -- The principal class your extension subclasses. Apple's documentation is a single page with no discussion or sample code. +- [UIPrinter](https://developer.apple.com/documentation/uikit/uiprinter) -- Represents a printer discovered by your extension. +- [UIPrintInteractionController](https://developer.apple.com/documentation/uikit/uiprintinteractioncontroller) -- The system print UI that triggers printer discovery. +- [Printing (UIKit)](https://developer.apple.com/documentation/uikit/printing) -- Overview of the iOS printing architecture. +- [AirPrint Overview](https://developer.apple.com/airprint/) -- AirPrint is the standard path; this extension exists for printers that cannot use it. +- [Bonjour Printing Specification](https://developer.apple.com/bonjour/printing-specification/) -- The standard discovery protocol that this extension type bypasses. + +## WWDC History + +There are no dedicated WWDC sessions covering the Print Service extension type. It was introduced quietly alongside other app extension points and has received no public stage time. + +- **[WWDC 2016, Session 725 -- Deploying AirPrint in Enterprise](https://developer.apple.com/videos/play/wwdc2016/725/)** -- Covered AirPrint deployment and Bonjour-based discovery in enterprise networks. Did not mention `UIPrintServiceExtension` but provides essential context for understanding the printer discovery landscape. + +## What It Does + +1. **User taps Print.** An app presents `UIPrintInteractionController`, which triggers the system print UI. +2. **System discovers printers.** iOS runs AirPrint/Bonjour discovery by default. Simultaneously, it loads any installed Print Service extensions. +3. **Extension discovers custom printers.** Your `UIPrintServiceExtension` subclass runs its proprietary discovery logic (e.g., scanning for Bluetooth devices, querying a vendor-specific network protocol, or connecting via USB accessory). +4. **Printers appear in the picker.** Printers found by your extension appear alongside AirPrint printers in the system printer list. +5. **User selects your printer.** The system routes the print job through your extension, which handles communication with the physical printer. + +**Note:** The internal mechanics of how print data flows through the extension are not publicly documented. The `UIPrintServiceExtension` class itself exposes no override points in the public headers -- it inherits from `NSObject` and acts as a container for your discovery and communication logic. + +## Use Cases + +### Printer Manufacturer Apps +HP, Canon, Epson, Brother, and other printer manufacturers build Print Service extensions into their companion iOS apps. When a user installs "HP Smart" or "Canon PRINT," the extension enables the system print dialog to find printers that use vendor-specific protocols beyond standard AirPrint. + +### Legacy Enterprise Printers +Organizations with older network printers that do not support AirPrint or IPP Everywhere can build a custom Print Service extension that discovers these printers via their proprietary protocol (e.g., legacy LPR, vendor APIs, or SNMP-based discovery). + +### Specialty / Industrial Printers +Label printers (Zebra, DYMO), receipt printers (Star Micronics), and other specialty hardware often use Bluetooth or vendor USB protocols. A Print Service extension can make them appear in the standard iOS print dialog rather than requiring a dedicated printing app. + +## Key Classes + +| Class / Protocol | Role | +|------------------|------| +| `UIPrintServiceExtension` | Subclass this as your extension's principal class. The base class provides the extension lifecycle. Discovery and printer setup logic is your responsibility. | +| `UIPrinter` | Represents a discovered printer. Create instances with a `URL` and `displayName`. | +| `UIPrintInteractionController` | System print UI that triggers extension loading. Not used inside the extension itself. | +| `UIPrintInfo` | Describes the print job settings (output type, orientation, duplex). | + +## Implementation + +**Important:** Apple provides almost no public documentation on the internal API of `UIPrintServiceExtension`. The class has no documented override points beyond the standard `NSObject` lifecycle. The implementation below reflects the minimal template pattern, annotated with what is known. + +```swift +import UIKit + +// 1. Subclass UIPrintServiceExtension. +// This is the extension's principal class, declared in Info.plist. +class PrintServiceExtension: UIPrintServiceExtension { + + // 2. There are no documented override points on UIPrintServiceExtension. + // The extension lifecycle and printer discovery protocol are not + // publicly documented by Apple. + // + // Printer manufacturers receive documentation under NDA through + // the MFi (Made for iPhone) program or direct partnership with + // Apple's AirPrint team (airprint@apple.com). + // + // 3. In practice, manufacturer implementations use private or + // semi-private APIs provided through partnership agreements. + // The discovery logic typically involves: + // - Scanning for Bluetooth LE peripherals (CoreBluetooth) + // - Querying vendor-specific mDNS service types + // - Communicating over vendor USB accessory protocols + // + // 4. If you are a printer manufacturer, contact airprint@apple.com + // to license AirPrint technology and receive the implementation + // documentation for UIPrintServiceExtension. +} +``` + +### What a Real Implementation Might Look Like (Conceptual) + +While the actual API is under NDA, the general pattern used by printer manufacturer extensions involves: + +```swift +import UIKit +import CoreBluetooth + +// Conceptual only -- actual API details require Apple partnership. +class PrintServiceExtension: UIPrintServiceExtension { + + // 1. The system triggers printer discovery when the print dialog opens. + // Your extension scans for printers using your vendor's protocol. + func discoverPrinters() { + // 2. Scan for Bluetooth LE peripherals matching your printer's + // service UUID, or probe a proprietary network protocol. + + // 3. For each discovered printer, create a UIPrinter and report + // it to the system so it appears in the printer picker. + } + + // 4. When the user selects your printer, the system asks your extension + // to handle the print job data and send it to the hardware. + func handlePrintJob(data: Data, printer: UIPrinter) { + // 5. Convert the print data to your printer's native format + // and transmit it over Bluetooth, USB, or network. + } +} +``` + +## Usage + +- Create: `bun create target print-service` +- Edit: `targets/print-service/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 14.0+ | Primary platform. Extensions appear in the system print dialog. | +| iPadOS | 14.0+ | Same as iOS. | +| macOS | -- | macOS uses CUPS drivers and Print Dialog Extensions (PDEs) instead, not `UIPrintServiceExtension`. | +| watchOS | -- | Not supported. No printing on watchOS. | +| tvOS | -- | Not supported. No printing on tvOS. | +| visionOS | -- | Not confirmed. Printing support on visionOS is limited. | + +## Gotchas + +- **Documentation is essentially nonexistent.** Apple's public documentation for `UIPrintServiceExtension` consists of a single page with a one-sentence description and no sample code. The full implementation details are available only to printer manufacturers under NDA or MFi partnership. If you are not a printer hardware vendor, this extension type is likely not what you need. +- **Contact Apple directly for implementation guidance.** Apple directs printer manufacturers to email `airprint@apple.com` to license AirPrint technology and receive the private API documentation needed to build a functional Print Service extension. +- **AirPrint is the preferred path.** Apple strongly encourages all printers to support AirPrint (IPP + Bonjour). The Print Service extension exists as a fallback for hardware that cannot implement the standard protocol. If your printer can speak IPP, use AirPrint instead. +- **No Simulator support.** Print Service extensions require real hardware (both the iOS device and the printer) for testing. The iOS Simulator does not load print service extensions. +- **The extension runs in a sandboxed process.** Like all app extensions, it has limited memory and no access to the main app's data unless you use App Groups. +- **Bluetooth requires background modes and permissions.** If your discovery logic uses CoreBluetooth, you need the `bluetooth-central` background mode and `NSBluetoothAlwaysUsageDescription` in your extension's Info.plist. +- **MFi program may be required.** If your printer uses a Lightning or USB accessory protocol, you likely need MFi certification, which involves a separate Apple partnership process. diff --git a/plugins/expo/skills/apple-targets/quicklook-preview.md b/plugins/expo/skills/apple-targets/quicklook-preview.md new file mode 100644 index 0000000..40008a8 --- /dev/null +++ b/plugins/expo/skills/apple-targets/quicklook-preview.md @@ -0,0 +1,261 @@ +--- +title: Quick Look Preview Provider +description: Renders full-size previews of custom file types for Quick Look, enabling space-bar preview in Finder, share sheet previews, and in-app QLPreviewController display. +version: iOS 13.0+, macOS 10.15+ +--- + +# Quick Look Preview Provider (`quicklook-preview`) + +A Quick Look Preview extension renders full-size previews for custom file types throughout the system. On macOS, pressing the space bar on a file in Finder triggers Quick Look; on iOS, previews appear in the Files app, share sheets, drag-and-drop, and any app using `QLPreviewController`. There are two approaches: a view-controller-based preview (subclass `UIViewController` conforming to `QLPreviewingController`) for rich interactive previews, or a data-based preview (subclass `QLPreviewProvider`) for returning static content like HTML, PDF, or images. The view-controller approach is the default template and works on iOS; the data-based `QLPreviewProvider` approach was added in iOS 15/macOS 12 and is preferred for cross-platform extensions. + +## Apple Documentation + +- [Quick Look Framework](https://developer.apple.com/documentation/quicklook) +- [Quick Look UI Framework](https://developer.apple.com/documentation/quicklookui) +- [QLPreviewingController](https://developer.apple.com/documentation/quicklook/qlpreviewingcontroller) +- [QLPreviewProvider](https://developer.apple.com/documentation/quicklookui/qlpreviewprovider) +- [QLPreviewReply](https://developer.apple.com/documentation/quicklookui/qlpreviewreply) +- [QLFilePreviewRequest](https://developer.apple.com/documentation/quicklook/qlfilepreviewrequest) +- [QLPreviewController](https://developer.apple.com/documentation/quicklook/qlpreviewcontroller) +- [Providing Previews of Your Custom File Types](https://developer.apple.com/documentation/quicklook/providing-previews-of-your-custom-file-types) + +## WWDC History + +- **[WWDC 2019, Session 719 -- What's New in File Management and Quick Look](https://developer.apple.com/videos/play/wwdc2019/719/)** -- Introduced the Quick Look Preview extension point for iOS alongside the thumbnail extension. Covered view-controller-based previews using `QLPreviewingController`. +- **[WWDC 2021, Session 10012 -- What's New in Quick Look](https://developer.apple.com/videos/play/wwdc2021/10012/)** -- Introduced the data-based `QLPreviewProvider` API for cross-platform support (iOS 15+/macOS 12+). Covered `QLPreviewReply` with PDF, data, and string initializers. + +## What It Does + +1. **Quick Look requests a preview.** When the user presses space bar (macOS), taps a file in Files (iOS), or any app invokes `QLPreviewController`, the system checks for a registered preview extension matching the file's UTI. +2. **View-controller approach (iOS 13+).** The system instantiates your `UIViewController` subclass conforming to `QLPreviewingController` and calls `preparePreviewOfFile(at:completionHandler:)`. You populate your view hierarchy and call the completion handler. +3. **Data-based approach (iOS 15+).** The system calls your `QLPreviewProvider` subclass's `providePreview(for:_:)`. You create a `QLPreviewReply` containing the preview data (HTML, PDF, image, or drawn content) and call the completion handler. +4. **The system displays the preview.** The rendered content is shown in the Quick Look panel (macOS) or the preview controller (iOS). The user can zoom, scroll, and interact with the content. + +## Use Cases + +### Custom document format + +A productivity app (Notion, Bear) exports files in a custom format. The preview extension parses the file and returns an HTML-based `QLPreviewReply` with styled content, so users can preview documents from Finder without opening the app. + +### 3D model viewer + +A 3D design tool registers custom model file UTIs. The preview extension uses SceneKit or a Metal-backed view controller to render a rotatable 3D preview of the model. + +### Code and configuration files + +A developer tool registers custom config formats. The preview extension renders syntax-highlighted source code as an HTML page with a monospaced font, making files readable in Quick Look. + +### Scientific data visualization + +A research application stores data in a custom binary format. The preview extension reads the data, generates a chart or graph using Core Graphics, and returns it as a PDF-based `QLPreviewReply`. + +## Key Classes + +| Class / Protocol | Role | +|------------------|------| +| `QLPreviewingController` | Protocol for view-controller-based previews. Implement `preparePreviewOfFile(at:completionHandler:)`. | +| `QLPreviewProvider` | Subclass for data-based previews (iOS 15+). Override `providePreview(for:_:)`. | +| `QLPreviewReply` | The reply object for data-based previews. Initializers for drawing, PDF, data (with content type), file URL, and string. | +| `QLFilePreviewRequest` | The request object for data-based previews. Contains `fileURL` and `contentType`. | +| `QLPreviewController` | Client-side view controller for displaying Quick Look previews. Not used in the extension itself. | +| `QLPreviewReplyAttachment` | Inline attachment (e.g., images) for HTML-based preview replies. | + +## Implementation + +### View-Controller-Based Preview (iOS 13+) + +```swift +import UIKit +import QuickLook + +// 1. Subclass UIViewController and conform to QLPreviewingController. +// This is the approach used by the default Xcode template. +class PreviewViewController: UIViewController, QLPreviewingController { + + private let textView = UITextView() + + override func viewDidLoad() { + super.viewDidLoad() + + // 2. Set up your preview UI. You have full UIKit available here, + // including scroll views, collection views, and custom drawing. + textView.isEditable = false + textView.font = .monospacedSystemFont(ofSize: 14, weight: .regular) + textView.translatesAutoresizingMaskIntoConstraints = false + view.addSubview(textView) + NSLayoutConstraint.activate([ + textView.topAnchor.constraint(equalTo: view.topAnchor), + textView.bottomAnchor.constraint(equalTo: view.bottomAnchor), + textView.leadingAnchor.constraint(equalTo: view.leadingAnchor), + textView.trailingAnchor.constraint(equalTo: view.trailingAnchor), + ]) + } + + // 3. Called when a file needs to be previewed. Load the file and + // populate your UI, then call the handler to signal readiness. + func preparePreviewOfFile(at url: URL, completionHandler handler: @escaping (Error?) -> Void) { + do { + let content = try String(contentsOf: url, encoding: .utf8) + let document = try CustomDocument.parse(content) + + // 4. Render the document into attributed text with syntax highlighting. + textView.attributedText = document.attributedRepresentation() + handler(nil) + } catch { + handler(error) + } + } + + // 5. Optional: preview searchable items from Spotlight. + func preparePreviewOfSearchableItem(identifier: String, + queryString: String?, + completionHandler handler: @escaping (Error?) -> Void) { + // Resolve the item from your app's Spotlight index and display it. + handler(nil) + } +} +``` + +### Data-Based Preview with HTML (iOS 15+) + +```swift +import QuickLook +import QuickLookUI +import UniformTypeIdentifiers + +// 6. Subclass QLPreviewProvider for a data-based (non-view-controller) preview. +// This approach works cross-platform on iOS and macOS. +class PreviewProvider: QLPreviewProvider { + + override func providePreview(for request: QLFilePreviewRequest, + _ handler: @escaping (QLPreviewReply?, Error?) -> Void) { + do { + let content = try String(contentsOf: request.fileURL, encoding: .utf8) + let document = try CustomDocument.parse(content) + + // 7. Generate an HTML representation of the document. + let html = """ + + + + + + + +

\(document.title)

+

Created: \(document.createdAt) | Items: \(document.items.count)

+ 
\(document.formattedContent)
+ + + """ + + // 8. Create a reply with HTML content. The system renders it in a web view. + let reply = QLPreviewReply( + dataOfContentType: .html, + contentSize: CGSize(width: 600, height: 800) + ) { _ in + return html.data(using: .utf8)! + } + + // 9. Optionally set a title for the preview window's title bar. + reply.title = document.title + + handler(reply, nil) + } catch { + handler(nil, error) + } + } +} +``` + +### PDF-Based Preview + +```swift +import QuickLook +import QuickLookUI +import PDFKit + +class PreviewProvider: QLPreviewProvider { + + override func providePreview(for request: QLFilePreviewRequest, + _ handler: @escaping (QLPreviewReply?, Error?) -> Void) { + do { + let data = try Data(contentsOf: request.fileURL) + let document = try CustomDocument(data: data) + + let pageSize = CGSize(width: 612, height: 792) // US Letter + + // 10. Create a PDF-based reply. The closure receives a CGContext + // configured for PDF drawing. + let reply = QLPreviewReply(forPDFWithPageSize: pageSize) { + (pdfContext: CGContext) -> Void in + + // 11. Begin a PDF page and draw document content. + pdfContext.beginPDFPage(nil) + + // Draw title + let titleAttributes: [NSAttributedString.Key: Any] = [ + .font: UIFont.boldSystemFont(ofSize: 24), + .foregroundColor: UIColor.black + ] + let title = document.title as NSString + title.draw(at: CGPoint(x: 36, y: pageSize.height - 60), + withAttributes: titleAttributes) + + // 12. Draw body content + let bodyAttributes: [NSAttributedString.Key: Any] = [ + .font: UIFont.systemFont(ofSize: 12), + .foregroundColor: UIColor.darkGray + ] + let body = document.bodyText as NSString + let bodyRect = CGRect(x: 36, y: 36, + width: pageSize.width - 72, + height: pageSize.height - 120) + body.draw(in: bodyRect, withAttributes: bodyAttributes) + + pdfContext.endPDFPage() + } + + handler(reply, nil) + } catch { + handler(nil, error) + } + } +} +``` + +## Usage + +- Create: `bun create target quicklook-preview` +- Edit: `targets/quicklook-preview/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 13.0+ (view controller), 15.0+ (data-based) | View-controller previews via `QLPreviewingController`. Data-based via `QLPreviewProvider` requires iOS 15. | +| iPadOS | 13.0+ (view controller), 15.0+ (data-based) | Same as iOS. | +| macOS | 10.15+ (view controller), 12.0+ (data-based) | Space-bar Quick Look in Finder. Data-based previews via `QLPreviewProvider` require macOS 12. | +| visionOS | 1.0+ | Supported in the Files app. | +| tvOS | -- | Not supported. | +| watchOS | -- | Not supported. | + +## Gotchas + +- **You must declare supported UTIs in Info.plist.** Add your UTIs to the `QLSupportedContentTypes` array in the extension's Info.plist. Parent UTIs do not automatically match child types -- list each specific UTI you support. +- **The default template uses the view-controller approach.** The `@bacons/apple-targets` template generates a `QLPreviewingController`-based view controller. If you want the newer data-based `QLPreviewProvider` API, replace the generated files with a `QLPreviewProvider` subclass and update the Info.plist `NSExtensionPrincipalClass` accordingly. +- **`QLPreviewProvider` and `QLPreviewingController` are mutually exclusive.** An extension uses one approach or the other, not both. The choice is determined by the `NSExtensionPrincipalClass` in Info.plist. +- **HTML previews have limited JavaScript support.** When using `QLPreviewReply(dataOfContentType: .html, ...)`, the system renders the HTML in a constrained web view. Complex JavaScript, external network requests, and some CSS features may not work. Keep the HTML self-contained. +- **Preview extensions have a short timeout.** If your extension takes too long to generate a preview, the system cancels the request and shows "No Preview Available." Avoid heavy computation or network I/O. Pre-process data if possible. +- **The `ALWAYS_EMBED_SWIFT_STANDARD_LIBRARIES` build setting is required.** This extension type needs embedded Swift libraries. The `@bacons/apple-targets` plugin sets `needsEmbeddedSwift: true` automatically. +- **Test with `qlmanage -p` on macOS.** Run `qlmanage -p /path/to/your/file.custom` from Terminal to trigger a Quick Look preview and debug your extension. Use Xcode's Debug > Attach to Process to set breakpoints. +- **On macOS, reset the Quick Look cache with `qlmanage -r`.** During development, cached previews may persist even after rebuilding your extension. Run `qlmanage -r` and `qlmanage -r cache` to clear both the generator and cache databases. +- **Spotlight integration is optional but valuable.** Implementing `preparePreviewOfSearchableItem(identifier:queryString:completionHandler:)` lets your extension provide previews for Spotlight search results, not just file browsing. +- **File access is read-only.** The extension receives a URL to the file but cannot modify it. If your preview requires transforming the file (e.g., decompressing), write temporary files and clean them up in your extension's dealloc or invalidation. diff --git a/plugins/expo/skills/apple-targets/quicklook-thumbnail.md b/plugins/expo/skills/apple-targets/quicklook-thumbnail.md new file mode 100644 index 0000000..433fa9b --- /dev/null +++ b/plugins/expo/skills/apple-targets/quicklook-thumbnail.md @@ -0,0 +1,242 @@ +--- +title: Quick Look Thumbnail Provider +description: Generates system-wide thumbnail images for custom file types, displayed in Files, Finder, Spotlight, and document pickers. +version: iOS 13.0+, macOS 10.15+ +--- + +# Quick Look Thumbnail Provider (`quicklook-thumbnail`) + +A Quick Look Thumbnail extension generates thumbnail images for custom file types so they appear with rich previews throughout the system -- in the Files app, Finder, Spotlight results, document pickers, and any app that uses `QLThumbnailGenerator`. You subclass `QLThumbnailProvider` from the QuickLookThumbnailing framework and override `provideThumbnail(for:_:)` to draw or supply an image for each request. The system provides the file URL, maximum size, and scale factor; you return a `QLThumbnailReply` containing a rendered image, a drawing block, or a file URL. + +## Apple Documentation + +- [QuickLookThumbnailing Framework](https://developer.apple.com/documentation/quicklookthumbnailing) +- [QLThumbnailProvider](https://developer.apple.com/documentation/quicklookthumbnailing/qlthumbnailprovider) +- [provideThumbnail(for:_:)](https://developer.apple.com/documentation/quicklookthumbnailing/qlthumbnailprovider/providethumbnail(for:_:)) +- [QLFileThumbnailRequest](https://developer.apple.com/documentation/quicklookthumbnailing/qlfilethumbnailrequest) +- [QLThumbnailReply](https://developer.apple.com/documentation/quicklookthumbnailing/qlthumbnailreply) +- [QLThumbnailGenerator](https://developer.apple.com/documentation/quicklookthumbnailing/qlthumbnailgenerator) +- [Creating Quick Look Thumbnails to Preview Files in Your App](https://developer.apple.com/documentation/quicklookthumbnailing/creating-quick-look-thumbnails-to-preview-files-in-your-app) +- [Providing Thumbnails of Your Custom File Types](https://developer.apple.com/documentation/quicklookthumbnailing/providing-thumbnails-of-your-custom-file-types) + +## WWDC History + +- **[WWDC 2019, Session 719 -- What's New in File Management and Quick Look](https://developer.apple.com/videos/play/wwdc2019/719/)** -- Introduced the QuickLookThumbnailing framework and the `QLThumbnailProvider` extension point. Covered the three reply modes (context drawing, Core Graphics drawing, image file URL) and cross-platform support for iOS and macOS. + +## What It Does + +1. **The system requests a thumbnail.** When a file with your declared UTI appears in Finder, Files, Spotlight, or a `QLThumbnailGenerator` request, the system launches your extension and calls `provideThumbnail(for:_:)`. +2. **You receive a `QLFileThumbnailRequest`.** This contains the file `url`, `maximumSize` (CGSize), `minimumDimension` (CGFloat), and `scale` (CGFloat). Your thumbnail should fit within these constraints. +3. **You create a `QLThumbnailReply`.** Three options: (a) draw into the current UIKit graphics context, (b) draw into a provided `CGContext`, or (c) return a URL to a pre-rendered image file. +4. **You call the completion handler.** Pass the reply and `nil` error on success, or `nil` reply and an error on failure. +5. **The system caches the thumbnail.** Thumbnails are cached by the system. Your extension is only called when the cache is cold or the file's modification date changes. + +## Use Cases + +### Custom document format + +A design application (Sketch, Figma) registers a custom `.sketch` or `.fig` UTI. The thumbnail extension reads the file, extracts the first artboard, and renders a scaled-down preview so users can identify documents at a glance in Finder. + +### 3D model files + +A CAD or 3D modeling app registers UTIs for `.obj`, `.stl`, or custom formats. The thumbnail extension renders a simple wireframe or shaded preview of the model using SceneKit or Metal offline rendering. + +### Source code and data files + +A developer tool registers UTIs for custom config files (`.yml`, `.toml`, custom DSLs). The thumbnail extension renders syntax-highlighted text into a small image so files are visually distinguishable in Finder. + +### Game save files + +A game registers a custom save-file UTI. The thumbnail extension reads the save data and renders the player's character portrait or a minimap screenshot as the file icon. + +## Key Classes + +| Class / Protocol | Role | +|------------------|------| +| `QLThumbnailProvider` | Abstract base class. Subclass this and override `provideThumbnail(for:_:)`. | +| `QLFileThumbnailRequest` | The request object. Properties: `fileURL`, `maximumSize`, `minimumDimension`, `scale`. | +| `QLThumbnailReply` | The reply object. Initialize with one of three modes: `currentContextDrawing`, `drawing` (CGContext), or `imageFileURL`. | +| `QLThumbnailGenerator` | Client-side class for requesting thumbnails from any app. Not used in the extension itself, but useful for testing. | +| `QLThumbnailRepresentation` | The result type from `QLThumbnailGenerator`. Contains the generated `CGImage` or `UIImage`. | + +## Implementation + +### Custom Document Thumbnail with Core Graphics + +```swift +import UIKit +import QuickLookThumbnailing + +class ThumbnailProvider: QLThumbnailProvider { + + // 1. Override this single method. The system calls it for each thumbnail request. + override func provideThumbnail(for request: QLFileThumbnailRequest, + _ handler: @escaping (QLThumbnailReply?, Error?) -> Void) { + + // 2. Read enough of the file to generate a meaningful preview. + // Keep this fast -- the system expects thumbnails quickly. + guard let documentData = try? Data(contentsOf: request.fileURL), + let document = try? CustomDocument(data: documentData) else { + handler(nil, NSError(domain: "com.example.thumbnail", + code: -1, + userInfo: [NSLocalizedDescriptionKey: "Unable to read document"])) + return + } + + // 3. Calculate the thumbnail size respecting the aspect ratio + // and the maximum size from the request. + let aspectRatio = document.canvasWidth / document.canvasHeight + let thumbnailSize: CGSize + if aspectRatio > 1 { + // Landscape + thumbnailSize = CGSize( + width: request.maximumSize.width, + height: request.maximumSize.width / aspectRatio + ) + } else { + // Portrait or square + thumbnailSize = CGSize( + width: request.maximumSize.height * aspectRatio, + height: request.maximumSize.height + ) + } + + // 4. Use the CGContext drawing mode for full control over rendering. + let reply = QLThumbnailReply(contextSize: thumbnailSize) { context -> Bool in + + // 5. Fill a white background. + context.setFillColor(UIColor.white.cgColor) + context.fill(CGRect(origin: .zero, size: thumbnailSize)) + + // 6. Draw document content scaled to fit the thumbnail. + // This is where your file-format-specific rendering happens. + let drawRect = CGRect(origin: .zero, size: thumbnailSize) + for element in document.elements { + element.draw(in: context, rect: drawRect) + } + + // 7. Draw a subtle border so the thumbnail is distinguishable + // against white backgrounds in Finder. + context.setStrokeColor(UIColor.separator.cgColor) + context.setLineWidth(1.0 / request.scale) + context.stroke(CGRect(origin: .zero, size: thumbnailSize)) + + // 8. Return true to indicate success. Returning false discards the thumbnail. + return true + } + + handler(reply, nil) + } +} +``` + +### Image-Based Thumbnail (Embedded Preview) + +For file formats that embed a preview image (like many document formats), you can extract it to a temporary file: + +```swift +import UIKit +import QuickLookThumbnailing + +class ThumbnailProvider: QLThumbnailProvider { + + override func provideThumbnail(for request: QLFileThumbnailRequest, + _ handler: @escaping (QLThumbnailReply?, Error?) -> Void) { + + // 1. Extract the embedded preview image from the file. + guard let previewData = extractEmbeddedPreview(from: request.fileURL) else { + handler(nil, NSError(domain: "com.example.thumbnail", code: -1)) + return + } + + // 2. Write the preview to a temporary file. + let tempURL = FileManager.default.temporaryDirectory + .appendingPathComponent(UUID().uuidString) + .appendingPathExtension("png") + + do { + try previewData.write(to: tempURL) + // 3. Return the file URL directly. The system reads and scales the image. + handler(QLThumbnailReply(imageFileURL: tempURL), nil) + } catch { + handler(nil, error) + } + } + + private func extractEmbeddedPreview(from url: URL) -> Data? { + // Parse your file format and extract the embedded preview image bytes. + guard let fileData = try? Data(contentsOf: url) else { return nil } + return CustomFileFormat.extractPreviewPNG(from: fileData) + } +} +``` + +### UIKit Context Drawing (Simple Text Preview) + +```swift +import UIKit +import QuickLookThumbnailing + +class ThumbnailProvider: QLThumbnailProvider { + + override func provideThumbnail(for request: QLFileThumbnailRequest, + _ handler: @escaping (QLThumbnailReply?, Error?) -> Void) { + + // 1. Read the first few lines of a text-based custom file. + guard let content = try? String(contentsOf: request.fileURL, encoding: .utf8) else { + handler(nil, NSError(domain: "com.example.thumbnail", code: -1)) + return + } + + let previewText = content.prefix(500) + let size = request.maximumSize + + // 2. Use the currentContextDrawing mode (UIKit coordinate system). + let reply = QLThumbnailReply(contextSize: size, currentContextDrawing: { () -> Bool in + + // 3. Fill background + UIColor.systemBackground.setFill() + UIRectFill(CGRect(origin: .zero, size: size)) + + // 4. Draw the text with a small monospaced font. + let attributes: [NSAttributedString.Key: Any] = [ + .font: UIFont.monospacedSystemFont(ofSize: 6 * request.scale, weight: .regular), + .foregroundColor: UIColor.label + ] + let textRect = CGRect(origin: .zero, size: size).insetBy(dx: 4, dy: 4) + (previewText as NSString).draw(in: textRect, withAttributes: attributes) + + return true + }) + + handler(reply, nil) + } +} +``` + +## Usage + +- Create: `bun create target quicklook-thumbnail` +- Edit: `targets/quicklook-thumbnail/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 13.0+ | Thumbnails appear in Files, document pickers, Spotlight, and any app using `QLThumbnailGenerator`. | +| iPadOS | 13.0+ | Same as iOS. | +| macOS | 10.15+ | Thumbnails appear in Finder, Spotlight, and Open/Save panels. Replaces the legacy QLThumbnailGenerator C API. | +| visionOS | 1.0+ | Supported in the Files app. | +| tvOS | -- | Not supported. | +| watchOS | -- | Not supported. | + +## Gotchas + +- **You must declare supported UTIs in Info.plist.** Add your custom UTIs to the `QLSupportedContentTypes` array in the extension's Info.plist. The system will only invoke your extension for file types listed here. Parent UTIs do not match -- you must list the exact UTI. +- **Thumbnails must be generated quickly.** The system imposes a short timeout (a few seconds). If your extension takes too long, the request is cancelled and the system falls back to a generic icon. Avoid heavy computation or network requests. +- **Memory limit applies.** Like all extensions, thumbnail providers run under a strict memory cap. For large files, read only the bytes needed for a preview -- do not load the entire file into memory. +- **The `scale` property matters.** On Retina displays, `request.scale` is 2.0 or 3.0. If you use the `imageFileURL` reply mode, provide an image at the appropriate pixel dimensions (`maximumSize * scale`). The context-based modes handle scaling automatically. +- **`currentContextDrawing` uses UIKit coordinates; `drawing` uses Core Graphics coordinates.** In the UIKit mode, the origin is at the top-left. In the Core Graphics mode, the origin is at the bottom-left. Mixing up coordinate systems produces upside-down thumbnails. +- **Thumbnails are cached aggressively.** During development, the system may continue showing a stale thumbnail even after you update your extension. On macOS, use `qlmanage -r` to reset the Quick Look cache. On iOS, deleting and reinstalling the app clears the cache. +- **Test with `qlmanage -t` on macOS.** Run `qlmanage -t -s 512 /path/to/your/file.custom` from Terminal to generate a thumbnail and verify your extension works. Attach the debugger in Xcode to your extension target for breakpoints. +- **`QLThumbnailMinimumDimension` is optional but recommended.** Set this key in your extension's Info.plist to specify the smallest useful thumbnail size. The system will not request thumbnails smaller than this dimension, avoiding wasted work for tiny icon views. diff --git a/plugins/expo/skills/apple-targets/safari.md b/plugins/expo/skills/apple-targets/safari.md new file mode 100644 index 0000000..340397a --- /dev/null +++ b/plugins/expo/skills/apple-targets/safari.md @@ -0,0 +1,217 @@ +--- +title: Safari Web Extension +description: Adds browser functionality to Safari using the cross-browser WebExtension API with manifest.json, content scripts, background scripts, and a popup UI. +version: iOS 15.0+, macOS 11.0+ +--- + +# Safari Web Extension (`safari`) + +A Safari Web Extension uses the standard cross-browser WebExtension API -- the same `manifest.json`, content scripts, background scripts, and popup HTML used by Chrome and Firefox extensions -- packaged inside a native iOS or macOS app. Unlike every other Apple extension type, the bulk of the logic is written in JavaScript/HTML/CSS rather than Swift. A thin native `SafariWebExtensionHandler` class (conforming to `NSExtensionRequestHandling`) acts as the bridge for native messaging via `browser.runtime.sendNativeMessage()`. The extension requires a containing app to be distributed through the App Store, and users must explicitly enable it in Safari's settings. This architecture is fundamentally different from other iOS extensions: there is no `NSExtensionContext.inputItems` flow, no `NSItemProvider` attachments, and no share sheet -- instead, the extension operates within Safari's tab and page lifecycle through the WebExtension API. + +## Apple Documentation + +- [Safari Web Extensions Overview](https://developer.apple.com/documentation/safariservices/safari-web-extensions) +- [Creating a Safari Web Extension](https://developer.apple.com/documentation/safariservices/creating-a-safari-web-extension) +- [Assessing Your Safari Web Extension's Browser Compatibility](https://developer.apple.com/documentation/safariservices/assessing-your-safari-web-extension-s-browser-compatibility) +- [Messaging a Web Extension's Native App](https://developer.apple.com/documentation/safariservices/messaging-a-web-extension-s-native-app) +- [SFSafariExtensionHandler (macOS)](https://developer.apple.com/documentation/safariservices/sfsafariextensionhandler) +- [Converting a Web Extension for Safari](https://developer.apple.com/documentation/safariservices/converting-a-web-extension-for-safari) +- [Running Your Safari Web Extension During Development](https://developer.apple.com/documentation/safariservices/running-your-safari-web-extension-during-development) + +## WWDC History + +- **[WWDC 2020, Session 10665 -- Meet Safari Web Extensions](https://developer.apple.com/videos/play/wwdc2020/10665/)** -- Introduced Safari Web Extensions on macOS. Covered the `manifest.json` format, `safari-web-extension-converter` tool, and the privacy-focused permissions model. +- **[WWDC 2021, Session 10104 -- Meet Safari Web Extensions on iOS](https://developer.apple.com/videos/play/wwdc2021/10104/)** -- Brought Safari Web Extensions to iOS 15 and iPadOS 15. Demonstrated building a single extension that works across iPhone, iPad, and Mac. +- **[WWDC 2021, Session 10027 -- Explore Safari Web Extension Improvements](https://developer.apple.com/videos/play/wwdc2021/10027/)** -- Introduced non-persistent background pages for better performance, the `declarativeNetRequest` API for content blocking, and tab-override pages. +- **[WWDC 2022, Session 10099 -- What's New in Safari Web Extensions](https://developer.apple.com/videos/play/wwdc2022/10099/)** -- Manifest v3 support, `declarativeNetRequest` improvements, and updated permissions UI. +- **[WWDC 2023, Session 10119 -- What's New in Safari Extensions](https://developer.apple.com/videos/play/wwdc2023/10119/)** -- Per-site permissions, profile-aware extensions, and additional WebExtension API coverage. + +## What It Does + +1. **User enables the extension.** In iOS Settings > Safari > Extensions (or Safari > Preferences > Extensions on macOS), the user toggles the extension on and grants per-site or all-sites permission. +2. **Content scripts inject into web pages.** Scripts listed in `manifest.json`'s `content_scripts` array run in the context of matching web pages. They can read and modify the DOM, intercept events, and communicate with the background script. +3. **Background script handles events.** The `background.service_worker` (manifest v3) or `background.scripts` (manifest v2) file runs in a separate context. It handles extension events (install, tab updates, messages from content scripts, alarms) and manages extension-wide state. +4. **Popup UI provides controls.** The `action.default_popup` HTML page appears when the user taps the extension icon in Safari's toolbar. It can communicate with the background script to read/write state and trigger actions. +5. **Native messaging bridges to Swift.** Calling `browser.runtime.sendNativeMessage("application.id", message)` from JavaScript sends a message to the native `SafariWebExtensionHandler.beginRequest(with:)` method. The native side can access Keychain, Core Data, file system, or any iOS API and return a response. +6. **Permissions control access.** The `permissions` and `host_permissions` arrays in `manifest.json` declare what APIs and domains the extension needs. Safari's privacy model requires explicit user consent for each site (or all sites). + +## Use Cases + +### Ad and tracker blocking +A `declarativeNetRequest`-based extension with rules defined in JSON files that block network requests matching specific URL patterns. No background script needed for basic blocking -- Safari evaluates the rules natively for better performance and privacy. + +### Password manager autofill +A content script detects login forms on web pages and offers to fill credentials. The content script sends a message to the background script, which calls `browser.runtime.sendNativeMessage()` to request credentials from the native app's Keychain storage via `SafariWebExtensionHandler`. + +### Page translation +A content script extracts the page text, sends it to the background script, which calls a translation API. The translated text is sent back to the content script, which replaces the DOM content. The popup provides language selection and toggle controls. + +### Reading mode and accessibility +A content script applies custom CSS and DOM transformations to simplify page layouts, adjust fonts, or add dark mode. User preferences are stored via `browser.storage.local` and synced across devices with `browser.storage.sync`. + +## Key Classes + +| Class / Protocol | Role | +|------------------|------| +| `SafariWebExtensionHandler` | Native Swift class conforming to `NSExtensionRequestHandling`. Receives messages from `browser.runtime.sendNativeMessage()` and returns responses. The bridge between WebExtension JavaScript and native iOS/macOS APIs. | +| `NSExtensionRequestHandling` | Protocol that `SafariWebExtensionHandler` conforms to. Provides `beginRequest(with:)` where native message handling occurs. | +| `NSExtensionContext` | Delivers the incoming message and provides `completeRequest(returningItems:)` to send the response back to JavaScript. | +| `SFExtensionMessageKey` | String constant used as the key in `NSExtensionItem.userInfo` to extract the message dictionary sent from JavaScript and to set the response dictionary. | +| `SFSafariApplication` (macOS) | On macOS, provides methods to interact with Safari (get active window, dispatch messages to extension). Not available on iOS. | +| `SFContentBlockerManager` | Used by both content blocker extensions and web extensions to reload content blocking rules. Call `reloadContentBlocker(withIdentifier:)` when rules change. | + +## Implementation + +```swift +import SafariServices +import os.log + +// 1. The native handler receives messages from browser.runtime.sendNativeMessage(). +class SafariWebExtensionHandler: NSObject, NSExtensionRequestHandling { + + func beginRequest(with context: NSExtensionContext) { + // 2. Extract the message sent from JavaScript. + let item = context.inputItems[0] as! NSExtensionItem + guard let message = item.userInfo?[SFExtensionMessageKey] as? [String: Any] else { + context.completeRequest(returningItems: [], completionHandler: nil) + return + } + + os_log(.default, "Received native message: %{public}@", String(describing: message)) + + let action = message["action"] as? String ?? "" + + // 3. Handle different message types from the JavaScript side. + var responseData: [String: Any] = [:] + + switch action { + case "getCredentials": + // 4. Access native-only APIs like Keychain. + let domain = message["domain"] as? String ?? "" + responseData = [ + "username": lookupUsername(for: domain), + "found": true + ] + + case "savePreference": + // 5. Write to UserDefaults in the shared App Group. + let key = message["key"] as? String ?? "" + let value = message["value"] as? String ?? "" + let defaults = UserDefaults(suiteName: "group.com.example.myapp") + defaults?.set(value, forKey: key) + responseData = ["saved": true] + + default: + responseData = ["error": "Unknown action: \(action)"] + } + + // 6. Send the response back to the JavaScript callback. + let response = NSExtensionItem() + response.userInfo = [SFExtensionMessageKey: responseData] + context.completeRequest(returningItems: [response], completionHandler: nil) + } + + private func lookupUsername(for domain: String) -> String { + // In a real app, query the Keychain or Core Data here. + return "user@example.com" + } +} +``` + +The `manifest.json` defines the extension structure, permissions, and scripts: + +```json +{ + "manifest_version": 3, + "name": "My Safari Extension", + "version": "1.0", + "description": "Example Safari Web Extension", + + "icons": { + "48": "images/icon-48.png", + "96": "images/icon-96.png", + "128": "images/icon-128.png" + }, + + "background": { + "service_worker": "background.js" + }, + + "content_scripts": [{ + "js": ["content.js"], + "matches": ["*://*.example.com/*"] + }], + + "action": { + "default_popup": "popup.html", + "default_icon": { + "16": "images/toolbar-icon-16.png", + "32": "images/toolbar-icon-32.png" + } + }, + + "permissions": [ + "storage", + "activeTab", + "nativeMessaging" + ] +} +``` + +The `background.js` handles extension lifecycle events and native messaging: + +```javascript +// 7. Listen for messages from content scripts. +browser.runtime.onMessage.addListener((message, sender, sendResponse) => { + if (message.action === "getCredentials") { + // 8. Forward to the native SafariWebExtensionHandler. + browser.runtime.sendNativeMessage( + "application.id", + { action: "getCredentials", domain: message.domain }, + (response) => { + // 9. Relay the native response back to the content script. + sendResponse(response); + } + ); + return true; // Keep the message channel open for async response. + } +}); + +// 10. Handle extension install and update events. +browser.runtime.onInstalled.addListener((details) => { + if (details.reason === "install") { + console.log("Extension installed"); + } +}); +``` + +## Usage + +- Create: `bun create target safari` +- Edit: `targets/safari/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 15.0+ | Safari Web Extensions introduced. Users enable in Settings > Safari > Extensions. | +| iPadOS | 15.0+ | Same as iOS. Extensions work in both compact and regular size classes. | +| macOS | 11.0+ | Safari Web Extensions introduced at WWDC 2020. Full desktop extension support. | +| Mac Catalyst | 15.0+ | Can share the same extension target between iOS and Mac Catalyst builds. | +| watchOS | -- | Not supported. | +| tvOS | -- | Not supported. | +| visionOS | 1.0+ | Safari available; extension support follows iOS model. | + +## Gotchas + +- **Service worker lifecycle is unreliable on iOS.** On iOS 17.4 through 17.5, the service worker would stop after approximately 30 seconds. On iOS 18+, the hard timeout was removed but the service worker can still die and fail to wake up for subsequent events. Content scripts and popup messages may silently fail when the service worker is dead. Consider using `background.scripts` with a non-persistent page instead of `service_worker` if you need more reliable background processing. +- **`sendNativeMessage` response handler may not fire.** Apple has acknowledged a bug where the response callback from `browser.runtime.sendNativeMessage()` never executes. As a workaround, use `browser.runtime.connectNative()` with port-based messaging instead. +- **Manifest v3 debugging is broken in Safari.** When using manifest v3 with `background.service_worker`, the Web Inspector shows "No Web Extension Background Content" and you cannot inspect or view `console.log` output from the service worker. Manifest v2 background pages can be inspected normally. +- **`preferred_environment` controls background mode.** When both `background.scripts` and `background.service_worker` are in `manifest.json`, Safari uses `scripts` unless `preferred_environment` is set to `"service_worker"`. On iOS, background content cannot be persistent -- set `"persistent": false` or the build will fail. +- **User must explicitly enable the extension.** Unlike other extension types that appear automatically, Safari Web Extensions require the user to navigate to Settings > Safari > Extensions and toggle the extension on. Your containing app should guide users through this with a deep link to the settings page. +- **Per-site permission model.** Safari asks for permission on each domain the extension wants to access. Extensions declared with broad host permissions (``) trigger a more prominent permission prompt. Users can revoke per-site access at any time. +- **`webRequest` API dropped in Safari 18.4 for manifest v3.** After the macOS 15.4 / Safari 18.4 update, non-persistent background content cannot listen to `webRequest` events. Use `declarativeNetRequest` instead for request interception and blocking. +- **Safari does not extend service worker lifetime for native messaging.** Unlike Chrome, which keeps the service worker alive while a native messaging connection is open, Safari will terminate the service worker on its normal schedule. Long-running native messaging sessions will be interrupted. +- **Cross-browser differences exist.** Safari uses the `browser.*` namespace (with Promise support) rather than Chrome's `chrome.*` namespace. Some APIs like `browser.webRequest` have restrictions. Always test in Safari separately. Apple provides a compatibility assessment tool to check your manifest and API usage. +- **The containing app is required but can be minimal.** Apple requires a native iOS/macOS app to wrap the extension for App Store distribution. The app can be a simple single-screen app that explains the extension and links to Safari settings, but it must exist. +- **Extension resources must be in the `assets/` directory.** The `manifest.json`, JavaScript files, HTML files, CSS, and images for the web extension must all be inside the `assets/` (or `Resources/`) directory of the extension target. The `SafariWebExtensionHandler.swift` lives at the target root alongside them. diff --git a/plugins/expo/skills/apple-targets/share.md b/plugins/expo/skills/apple-targets/share.md new file mode 100644 index 0000000..d5e2182 --- /dev/null +++ b/plugins/expo/skills/apple-targets/share.md @@ -0,0 +1,165 @@ +--- +title: Share Extension +description: Presents a compose UI in the system share sheet so users can share content from any app to your app or service. +version: iOS 8.0+ +--- + +# Share Extension (`share`) + +A Share Extension adds your app to the system share sheet, allowing users to send content -- URLs, images, text, files -- directly from any app into yours. The extension runs in a separate process with tight resource constraints and communicates with its containing app through App Groups. You can either subclass `SLComposeServiceViewController` for a standard compose-style UI with a preview and text field, or provide a fully custom `UIViewController` (or SwiftUI view via `UIHostingController`) for complete control over the presentation. Content arrives via `NSExtensionContext.inputItems` as an array of `NSExtensionItem` objects, each carrying `NSItemProvider` attachments that you load asynchronously by type identifier. + +## Apple Documentation + +- [App Extension Programming Guide: Share](https://developer.apple.com/library/archive/documentation/General/Conceptual/ExtensibilityPG/Share.html) +- [SLComposeServiceViewController](https://developer.apple.com/documentation/social/slcomposeserviceviewcontroller) +- [NSExtensionContext](https://developer.apple.com/documentation/foundation/nsextensioncontext) +- [NSItemProvider](https://developer.apple.com/documentation/foundation/nsitemprovider) +- [NSExtensionItem](https://developer.apple.com/documentation/foundation/nsextensionitem) +- [NSExtensionPointIdentifier](https://developer.apple.com/documentation/bundleresources/information-property-list/nsextension/nsextensionpointidentifier) +- [App Groups Entitlement](https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_security_application-groups) + +## WWDC History + +- **[WWDC 2014, Session 205 -- Creating Extensions for iOS and OS X, Part 1](https://developer.apple.com/videos/play/wwdc2014/205/)** -- Introduced app extensions including Share Extensions. Covered `SLComposeServiceViewController`, `NSExtensionContext`, and the share sheet lifecycle. +- **[WWDC 2014, Session 217 -- Creating Extensions for iOS and OS X, Part 2](https://developer.apple.com/videos/play/wwdc2014/217/)** -- Deep dive into data flow between host apps and extensions using `NSExtensionItem` and `NSItemProvider`. Covered returning modified data from Action Extensions (contrast with Share Extensions). +- **[WWDC 2015, Session 224 -- App Extension Best Practices](https://developer.apple.com/videos/play/wwdc2015/224/)** -- Guidance on memory management, activation rules, and sharing data between extension and containing app via App Groups. + +## What It Does + +1. **User taps Share.** The system presents the share sheet. If the extension's `NSExtensionActivationRule` matches the content being shared, your extension appears in the list. +2. **Extension launches in-process.** The system instantiates your principal class (typically a `SLComposeServiceViewController` subclass or a custom `UIViewController`) in a separate process. +3. **Content is delivered via `NSExtensionContext`.** The `extensionContext.inputItems` array contains `NSExtensionItem` objects. Each item's `attachments` property holds `NSItemProvider` instances representing the shared data. +4. **You load attachments asynchronously.** Call `NSItemProvider.loadItem(forTypeIdentifier:options:)` with the appropriate UTI (e.g., `public.url`, `public.image`, `public.plain-text`) to retrieve the actual data. +5. **User composes and posts.** If using `SLComposeServiceViewController`, the user sees a standard compose view with a text field and content preview. Tapping Post calls `didSelectPost()`. +6. **Extension completes the request.** Call `extensionContext?.completeRequest(returningItems:completionHandler:)` to dismiss the extension. Use `NSURLSession` with a background configuration for any uploads that might outlast the extension process. +7. **Data flows to the main app via App Groups.** Write shared data to an App Group container (`UserDefaults(suiteName:)` or a shared file) so the containing app can access it on next launch. + +## Use Cases + +### Social media posting +A user selects a photo in the Photos app, taps Share, and picks your social media app. The share extension shows a compose UI where they add a caption. On post, the extension writes the image and caption to the App Group container and enqueues an upload via a background URL session. + +### Save to reading list +A user shares a URL from Safari to your read-later app. The extension extracts the URL via `NSItemProvider.loadItem(forTypeIdentifier: "public.url")`, stores it in a shared Core Data store or JSON file in the App Group container, and confirms the save with a brief animation. + +### Send to messaging app +A user shares a document from Files to your messaging app. The extension presents a contact picker (custom UI), copies the file to the App Group container, and signals the main app to send the message on next launch or via a background task. + +### Quick note capture +A user highlights text in any app and shares it to your note-taking app. The extension grabs the text via `public.plain-text`, optionally lets the user choose a notebook from a configuration item, and writes the note to shared storage. + +## Key Classes + +| Class / Protocol | Role | +|------------------|------| +| `SLComposeServiceViewController` | Provides a standard compose UI with text field, character count, content preview, and configuration items. Subclass and override `didSelectPost()`, `isContentValid()`, and `configurationItems()`. | +| `UIViewController` | Use a plain view controller (with `UIHostingController` for SwiftUI) for a fully custom share UI when `SLComposeServiceViewController` is too restrictive. | +| `NSExtensionContext` | The bridge between the host app and your extension. Provides `inputItems` and the `completeRequest(returningItems:completionHandler:)` method. | +| `NSExtensionItem` | Represents one item from the host app. Contains `attachments` (array of `NSItemProvider`), `attributedTitle`, and `attributedContentText`. | +| `NSItemProvider` | Wraps a single attachment. Use `hasItemConformingToTypeIdentifier(_:)` to check availability, then `loadItem(forTypeIdentifier:options:)` to retrieve the data. | +| `SLComposeSheetConfigurationItem` | A table-cell-style row at the bottom of the compose sheet for additional options (e.g., choose an account or album). Returned from `configurationItems()`. | +| `UserDefaults(suiteName:)` | Reads/writes preferences in a shared App Group container so both the extension and main app can access them. | + +## Implementation + +```swift +import UIKit +import Social +import SwiftUI +import UniformTypeIdentifiers + +// 1. Subclass SLComposeServiceViewController for the standard compose UI. +class ShareViewController: SLComposeServiceViewController { + + private var sharedURL: URL? + private var sharedText: String? + + // 2. Called when the compose sheet appears. Begin loading attachments here. + override func presentationAnimationDidFinish() { + guard let item = extensionContext?.inputItems.first as? NSExtensionItem, + let attachments = item.attachments else { return } + + for provider in attachments { + // 3. Check each attachment for known types and load asynchronously. + if provider.hasItemConformingToTypeIdentifier(UTType.url.identifier) { + provider.loadItem(forTypeIdentifier: UTType.url.identifier, options: nil) { [weak self] item, _ in + if let url = item as? URL { + DispatchQueue.main.async { + self?.sharedURL = url + self?.validateContent() + } + } + } + } else if provider.hasItemConformingToTypeIdentifier(UTType.plainText.identifier) { + provider.loadItem(forTypeIdentifier: UTType.plainText.identifier, options: nil) { [weak self] item, _ in + if let text = item as? String { + DispatchQueue.main.async { + self?.sharedText = text + self?.validateContent() + } + } + } + } + } + } + + // 4. Validate that the user has entered enough content to post. + override func isContentValid() -> Bool { + return !contentText.isEmpty || sharedURL != nil + } + + // 5. Called when the user taps Post. Save data to the App Group container. + override func didSelectPost() { + let defaults = UserDefaults(suiteName: "group.com.example.myapp") + + var savedItems = defaults?.array(forKey: "sharedItems") as? [[String: String]] ?? [] + var entry: [String: String] = ["text": contentText] + if let url = sharedURL { entry["url"] = url.absoluteString } + savedItems.append(entry) + defaults?.set(savedItems, forKey: "sharedItems") + + // 6. Complete the request to dismiss the extension. + extensionContext?.completeRequest(returningItems: [], completionHandler: nil) + } + + // 7. Add configuration items for additional options (e.g., pick a folder). + override func configurationItems() -> [Any]! { + let folder = SLComposeSheetConfigurationItem()! + folder.title = "Save to" + folder.value = "Inbox" + folder.tapHandler = { + // Push a selection view controller + } + return [folder] + } +} +``` + +## Usage + +- Create: `bun create target share` +- Edit: `targets/share/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 8.0+ | Full share sheet support. | +| iPadOS | 8.0+ | Full share sheet support. Appears in popover on iPad. | +| macOS | 10.10+ | Share menu in Finder and apps. Uses `NSViewController` instead of `UIViewController`. | +| Mac Catalyst | 13.1+ | Uses iOS extension APIs. | +| watchOS | -- | Not supported. | +| tvOS | -- | Not supported. | +| visionOS | 1.0+ | Share sheet supported. | + +## Gotchas + +- **120 MB memory limit.** iOS enforces a hard ~120 MB memory cap on share extensions. Exceeding it triggers an immediate `EXC_RESOURCE` crash with no warning. Avoid loading large images at full resolution; use `CGImageSourceCreateThumbnailAtIndex` to downsample, and defer heavy work until `presentationAnimationDidFinish()`. +- **`SLComposeServiceViewController` silently loads a default UI.** Even if you override the view entirely, the base class still creates its standard compose popup underneath, consuming memory. On iOS 17 this became worse as the system eagerly loads shared URL content. If you need a fully custom UI, consider subclassing `UIViewController` directly instead. +- **Handle multiple attachment types.** Users can share mixed content (e.g., text + image + URL). Always iterate over all `NSItemProvider` attachments and check `hasItemConformingToTypeIdentifier` for each type you support. Do not assume there is only one attachment. +- **`TRUEPREDICATE` will get you rejected.** The default `NSExtensionActivationRule` is `TRUEPREDICATE`, which shows the extension for all content types. Apple rejects App Store submissions that ship with this value. Replace it with specific activation rule keys or a predicate string before submitting. +- **Uploads must use background URL sessions.** The extension process can be terminated at any time after `completeRequest` returns. Use `URLSession` with a `.background` configuration so uploads continue even after the extension is dismissed. +- **App Groups are essential.** The extension runs in a separate sandboxed process. The only way to pass data to your main app is through an App Group shared container (via `UserDefaults(suiteName:)`, shared files, or Core Data with a shared container). +- **`contentText` is only available with `SLComposeServiceViewController`.** If you use a custom `UIViewController`, you must build your own text input and manage the text yourself. +- **No access to the network in some contexts.** Some MDM configurations and content types can restrict network access in extensions. Always handle network failures gracefully and consider queuing uploads for the main app to retry. +- **Keyboard appearance can be delayed.** In the compose view, the keyboard may not appear immediately. Calling `becomeFirstResponder()` in `viewDidAppear` does not always work; use `presentationAnimationDidFinish()` as the safe trigger point. diff --git a/plugins/expo/skills/apple-targets/shield-action.md b/plugins/expo/skills/apple-targets/shield-action.md new file mode 100644 index 0000000..f553e01 --- /dev/null +++ b/plugins/expo/skills/apple-targets/shield-action.md @@ -0,0 +1,157 @@ +--- +title: Shield Action Extension +description: Handles button taps on the Screen Time shield overlay, allowing the app to close the shielded app, defer the action, or do nothing in response to user interaction. +version: iOS 15.0+ +--- + +# Shield Action Extension (`shield-action`) + +A Shield Action extension responds to user taps on the primary and secondary buttons of the Screen Time shield overlay. When `ManagedSettingsStore` applies a shield to an app, website, or activity category, the system displays a blocking overlay. This extension controls what happens when the user taps either button on that overlay. It is one of three extension types in Apple's Screen Time API suite (alongside Device Activity Monitor and Shield Configuration) and requires the Family Controls entitlement. The extension subclasses `ShieldActionDelegate` and returns a `ShieldActionResponse` (`.close`, `.defer`, or `.none`) to tell the system how to proceed. + +## Apple Documentation + +- [ShieldActionDelegate](https://developer.apple.com/documentation/managedsettings/shieldactiondelegate) +- [ShieldAction](https://developer.apple.com/documentation/managedsettings/shieldaction) +- [ShieldActionResponse](https://developer.apple.com/documentation/managedsettings/shieldactionresponse) +- [handle(action:for:completionHandler:) -- Application](https://developer.apple.com/documentation/managedsettings/shieldactiondelegate/handle(action:for:completionhandler:)-9hcqc) +- [handle(action:for:completionHandler:) -- WebDomain](https://developer.apple.com/documentation/managedsettings/shieldactiondelegate/handle(action:for:completionhandler:)-9o7ql) +- [handle(action:for:completionHandler:) -- ActivityCategory](https://developer.apple.com/documentation/managedsettings/shieldactiondelegate/handle(action:for:completionhandler:)-3akjf) +- [ManagedSettings Framework](https://developer.apple.com/documentation/managedsettings) + +## WWDC History + +- **[WWDC 2021, Session 10123 -- Meet the Screen Time API](https://developer.apple.com/videos/play/wwdc2021/10123/)** -- Introduced the Screen Time API with three frameworks. Demonstrated the shield action handler (originally `ShieldActionHandler`) for responding to button taps on the shield overlay. +- **[WWDC 2022, Session 110336 -- What's New in Screen Time API](https://developer.apple.com/videos/play/wwdc2022/110336/)** -- Renamed `ShieldActionHandler` to `ShieldActionDelegate`. Added individual authorization for non-child use cases. Clarified the extension lifecycle and response model. + +## What It Does + +1. **Shield is displayed.** When `ManagedSettingsStore.shield.applications` or `.shield.applicationCategories` contains tokens, the system overlays a shield on those apps or websites. +2. **User taps a button.** The shield presents up to two buttons (primary and secondary, whose labels are customized by the Shield Configuration extension). When tapped, the system launches the Shield Action extension. +3. **Extension receives the action.** The system calls `handle(action:for:completionHandler:)` with a `ShieldAction` (`.primaryButtonPressed` or `.secondaryButtonPressed`) and the token identifying the shielded application, web domain, or activity category. +4. **Extension returns a response.** The completion handler is called with one of three `ShieldActionResponse` values: + - `.close` -- Dismisses the shielded app (sends it to background). + - `.defer` -- Tells the system to let the user ask a guardian for permission (triggers the Screen Time approval flow for child authorization). + - `.none` -- Does nothing; the shield remains visible. +5. **Extension can perform side effects.** Before calling the completion handler, the extension can update `ManagedSettingsStore`, write to App Group shared storage, or log analytics. + +## Use Cases + +### Parental controls with guardian approval + +The primary button is labeled "Ask for More Time" (set by the Shield Configuration extension). When tapped, the Shield Action extension returns `.defer`, which triggers the system's built-in guardian approval prompt. If the guardian approves, the shield is temporarily lifted. + +### Hard block with dismiss + +A digital wellbeing app shields social media apps after a daily limit. The primary button is labeled "Close App" and returns `.close`. The secondary button is hidden (no label set in Shield Configuration). The user can only dismiss the app. + +### Logging and analytics + +An enterprise management app wants to track how often employees attempt to bypass work-hour restrictions. The extension logs each tap to App Group shared storage before returning `.close`, allowing the management dashboard to display compliance metrics. + +## Key Classes + +| Class / Protocol | Role | +|------------------|------| +| `ShieldActionDelegate` | Base class for the extension's principal class. Override `handle(action:for:completionHandler:)` for applications, web domains, and activity categories. | +| `ShieldAction` | Enum with cases `.primaryButtonPressed` and `.secondaryButtonPressed`. | +| `ShieldActionResponse` | Enum with cases `.close`, `.defer`, and `.none`. Returned via the completion handler. | +| `ApplicationToken` | Opaque token identifying a shielded application. | +| `WebDomainToken` | Opaque token identifying a shielded web domain. | +| `ActivityCategoryToken` | Opaque token identifying a shielded activity category. | +| `ManagedSettingsStore` | Can be used inside the extension to modify shields or restrictions in response to a button tap. | + +## Implementation + +```swift +import ManagedSettings + +// 1. Subclass ShieldActionDelegate. The class name must match the +// NSExtensionPrincipalClass value in Info.plist. +class ShieldActionExtension: ShieldActionDelegate { + + // 2. Handle button taps on shields covering individual applications. + override func handle( + action: ShieldAction, + for application: ApplicationToken, + completionHandler: @escaping (ShieldActionResponse) -> Void + ) { + switch action { + case .primaryButtonPressed: + // 3. Close the shielded app (sends it to background). + completionHandler(.close) + + case .secondaryButtonPressed: + // 4. Defer to the guardian approval flow (child authorization) + // or do nothing (individual authorization). + completionHandler(.defer) + + @unknown default: + completionHandler(.none) + } + } + + // 5. Handle button taps on shields covering web domains (Safari, WebKit views). + override func handle( + action: ShieldAction, + for webDomain: WebDomainToken, + completionHandler: @escaping (ShieldActionResponse) -> Void + ) { + switch action { + case .primaryButtonPressed: + completionHandler(.close) + case .secondaryButtonPressed: + completionHandler(.defer) + @unknown default: + completionHandler(.none) + } + } + + // 6. Handle button taps on shields covering entire activity categories. + override func handle( + action: ShieldAction, + for category: ActivityCategoryToken, + completionHandler: @escaping (ShieldActionResponse) -> Void + ) { + switch action { + case .primaryButtonPressed: + // 7. Example: log the event to shared storage before closing. + let defaults = UserDefaults(suiteName: "group.com.example.myapp") + let count = (defaults?.integer(forKey: "shieldDismissCount") ?? 0) + 1 + defaults?.set(count, forKey: "shieldDismissCount") + + completionHandler(.close) + case .secondaryButtonPressed: + completionHandler(.defer) + @unknown default: + completionHandler(.none) + } + } +} +``` + +## Usage + +- Create: `bun create target shield-action` +- Edit: `targets/shield-action/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 15.0+ | Full support. Individual authorization added in 16.0+. | +| iPadOS | 15.0+ | Same as iOS. | +| macOS | -- | Not supported. Screen Time API is iOS/iPadOS only. | +| watchOS | -- | Not supported. | +| visionOS | -- | Not supported. | +| tvOS | -- | Not supported. | + +## Gotchas + +- **Cannot open the parent app.** There is no `ShieldActionResponse` case that opens the host app. Developers have requested `.openParentApp` on Apple Developer Forums, but as of iOS 18 it does not exist. The only options are `.close`, `.defer`, and `.none`. +- **`.defer` only works with child authorization.** When using `AuthorizationCenter.requestAuthorization(for: .individual)`, returning `.defer` has no meaningful effect because there is no guardian to approve the request. Use `.close` or `.none` instead for individual authorization flows. +- **App Group is required for shared state.** The extension runs in a separate process. To share data (e.g., dismiss counts, user preferences) with the host app or other extensions, add the App Groups capability to both the extension target and the main app target and use `UserDefaults(suiteName:)` or a shared file container. +- **Family Controls entitlement requires Apple approval.** The `com.apple.developer.family-controls` entitlement must be requested through Apple's developer portal. Without approval, development builds work but TestFlight and App Store distribution will be rejected. +- **Completion handler must be called.** Failing to call the `completionHandler` leaves the shield in an undefined state. Always call it in every code path, including error cases. +- **No custom UI in this extension.** The Shield Action extension is headless. It processes the button tap and returns a response. All visual customization of the shield (labels, colors, icons) is handled by the separate Shield Configuration extension. +- **The class was renamed in Xcode 14.** WWDC 2021 materials reference `ShieldActionHandler`. This was renamed to `ShieldActionDelegate` in iOS 16 / Xcode 14. Code samples from early tutorials may use the old name and will not compile. +- **Three separate `handle` overloads.** You must override the correct overload for the token type you are shielding (Application, WebDomain, or ActivityCategory). If you only override the Application variant but shield by category, the category button taps will use the default behavior. diff --git a/plugins/expo/skills/apple-targets/shield-config.md b/plugins/expo/skills/apple-targets/shield-config.md new file mode 100644 index 0000000..fbefaa8 --- /dev/null +++ b/plugins/expo/skills/apple-targets/shield-config.md @@ -0,0 +1,190 @@ +--- +title: Shield Configuration Extension +description: Customizes the appearance of the Screen Time shield overlay by providing titles, subtitles, icons, button labels, and colors for shielded apps and websites. +version: iOS 15.0+ +--- + +# Shield Configuration Extension (`shield-config`) + +A Shield Configuration extension controls the visual appearance of the Screen Time shield overlay. When `ManagedSettingsStore` shields an app, website, or activity category, the system asks this extension for a `ShieldConfiguration` describing the title, subtitle, icon, button labels, button colors, and background color to display. The extension subclasses `ShieldConfigurationDataSource` (not a view controller) and overrides `configuration(shielding:)` methods for applications, web domains, and their category variants. It is one of three extension types in Apple's Screen Time API suite (alongside Device Activity Monitor and Shield Action) and requires the Family Controls entitlement. + +## Apple Documentation + +- [ShieldConfigurationDataSource](https://developer.apple.com/documentation/managedsettingsui/shieldconfigurationdatasource) +- [ShieldConfiguration](https://developer.apple.com/documentation/managedsettingsui/shieldconfiguration) +- [ShieldConfiguration.Label](https://developer.apple.com/documentation/managedsettingsui/shieldconfiguration/label) +- [configuration(shielding:) -- Application](https://developer.apple.com/documentation/managedsettingsui/shieldconfigurationdatasource/configuration(shielding:)-5uqm1) +- [configuration(shielding:in:) -- Application in Category](https://developer.apple.com/documentation/managedsettingsui/shieldconfigurationdatasource/configuration(shielding:in:)-7mqr5) +- [configuration(shielding:) -- WebDomain](https://developer.apple.com/documentation/managedsettingsui/shieldconfigurationdatasource/configuration(shielding:)-98nkb) +- [configuration(shielding:in:) -- WebDomain in Category](https://developer.apple.com/documentation/managedsettingsui/shieldconfigurationdatasource/configuration(shielding:in:)-5moeh) +- [ManagedSettingsUI Framework](https://developer.apple.com/documentation/managedsettingsui) + +## WWDC History + +- **[WWDC 2021, Session 10123 -- Meet the Screen Time API](https://developer.apple.com/videos/play/wwdc2021/10123/)** -- Introduced the Screen Time API and the shield configuration provider (originally `ShieldConfigurationProvider`) for customizing the shield overlay's text, icon, and colors. +- **[WWDC 2022, Session 110336 -- What's New in Screen Time API](https://developer.apple.com/videos/play/wwdc2022/110336/)** -- Renamed `ShieldConfigurationProvider` to `ShieldConfigurationDataSource`. Added individual authorization. Clarified the extension architecture and data flow. + +## What It Does + +1. **Shield is about to appear.** When `ManagedSettingsStore` contains shield entries for applications or categories, the system prepares to display the shield overlay. +2. **System queries the extension.** The system launches the Shield Configuration extension and calls the appropriate `configuration(shielding:)` overload based on whether the shielded item is an individual application, a web domain, or a category member. +3. **Extension returns a `ShieldConfiguration`.** The configuration object includes optional properties: `title`, `subtitle`, `icon`, `primaryButtonLabel`, `primaryButtonBackgroundColor`, `secondaryButtonLabel`, `backgroundColor`, and `backgroundBlurStyle`. +4. **System renders the overlay.** The system uses the returned configuration to render a full-screen shield. Properties set to `nil` use system defaults (for `title` and `primaryButtonLabel`) or are hidden (for `secondaryButtonLabel` and `icon`). +5. **Button taps go to the Shield Action extension.** The labels defined here determine what the user sees, but the behavior when tapped is handled by the separate Shield Action extension. + +## Use Cases + +### Branded parental controls + +A parental controls app customizes the shield to match its brand. The title says "Time's Up!", the subtitle shows the app name, the icon is the app's logo, the primary button reads "Close App", and the secondary button reads "Ask Parent". Colors match the app's theme. + +### Motivational digital wellbeing + +A focus app displays an encouraging message when a social media app is shielded. The title reads "Stay Focused", the subtitle shows how much focus time remains, and only a primary "Got It" button is shown (secondary label is nil to hide the second button). + +### Context-aware messaging + +The extension reads from App Group shared storage to display different messages depending on the time of day, the specific app being shielded, or how many times the user has hit the shield today. Since tokens are opaque, context is typically passed via shared UserDefaults rather than by identifying the app directly. + +## Key Classes + +| Class / Protocol | Role | +|------------------|------| +| `ShieldConfigurationDataSource` | Base class for the extension's principal class. Override `configuration(shielding:)` methods to customize shield appearance. Not a view controller. | +| `ShieldConfiguration` | Data object describing the shield's visual properties: title, subtitle, icon, button labels, colors, and blur style. | +| `ShieldConfiguration.Label` | A struct pairing a localized `text` string with an optional `color` (`UIColor`). Used for title, subtitle, and button labels. | +| `Application` | Represents a shielded application. Provided by the system to the `configuration(shielding:)` method. | +| `WebDomain` | Represents a shielded web domain. Provided by the system to the `configuration(shielding:)` method. | +| `ActivityCategory` | Represents an activity category. Provided to the `configuration(shielding:in:)` variant. | + +## Implementation + +```swift +import ManagedSettings +import ManagedSettingsUI +import UIKit + +// 1. Subclass ShieldConfigurationDataSource (NOT a UIViewController). +// The class name must match the NSExtensionPrincipalClass in Info.plist. +class ShieldConfigurationExtension: ShieldConfigurationDataSource { + + // 2. Customize the shield for an individually shielded application. + override func configuration( + shielding application: Application + ) -> ShieldConfiguration { + return ShieldConfiguration( + // 3. Set the background color and blur style. + backgroundBlurStyle: .systemMaterial, + backgroundColor: UIColor.systemBackground, + + // 4. Provide a custom icon (use a bundled PNG, not a PDF). + icon: UIImage(named: "ShieldIcon"), + + // 5. Set title and subtitle with text and color. + title: ShieldConfiguration.Label( + text: "Time's Up!", + color: .label + ), + subtitle: ShieldConfiguration.Label( + text: "You've reached your screen time limit.", + color: .secondaryLabel + ), + + // 6. Configure button labels. Setting secondaryButtonLabel + // to nil hides the secondary button entirely. + primaryButtonLabel: ShieldConfiguration.Label( + text: "Close App", + color: .white + ), + primaryButtonBackgroundColor: .systemBlue, + secondaryButtonLabel: ShieldConfiguration.Label( + text: "Ask for More Time", + color: .systemBlue + ) + ) + } + + // 7. Customize the shield for an application shielded as part of a category. + override func configuration( + shielding application: Application, + in category: ActivityCategory + ) -> ShieldConfiguration { + // 8. Read shared state to provide context-aware messaging. + let defaults = UserDefaults(suiteName: "group.com.example.myapp") + let dismissCount = defaults?.integer(forKey: "shieldDismissCount") ?? 0 + + return ShieldConfiguration( + backgroundBlurStyle: .systemThickMaterial, + title: ShieldConfiguration.Label( + text: "Stay Focused", + color: .label + ), + subtitle: ShieldConfiguration.Label( + text: dismissCount > 3 + ? "You've tried to open this \(dismissCount) times today." + : "This app is restricted right now.", + color: .secondaryLabel + ), + primaryButtonLabel: ShieldConfiguration.Label( + text: "Got It", + color: .white + ), + primaryButtonBackgroundColor: .systemIndigo + ) + } + + // 9. Customize the shield for a web domain. + override func configuration( + shielding webDomain: WebDomain + ) -> ShieldConfiguration { + return ShieldConfiguration( + title: ShieldConfiguration.Label( + text: "Website Blocked", + color: .label + ), + primaryButtonLabel: ShieldConfiguration.Label( + text: "Go Back", + color: .white + ), + primaryButtonBackgroundColor: .systemRed + ) + } + + // 10. Customize the shield for a web domain within a category. + override func configuration( + shielding webDomain: WebDomain, + in category: ActivityCategory + ) -> ShieldConfiguration { + return ShieldConfiguration() + } +} +``` + +## Usage + +- Create: `bun create target shield-config` +- Edit: `targets/shield-config/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 15.0+ | Full support. Individual authorization added in 16.0+. | +| iPadOS | 15.0+ | Same as iOS. | +| macOS | -- | Not supported. Screen Time API is iOS/iPadOS only. | +| watchOS | -- | Not supported. | +| visionOS | -- | Not supported. | +| tvOS | -- | Not supported. | + +## Gotchas + +- **This is NOT a view controller.** `ShieldConfigurationDataSource` is a data source, not a `UIViewController`. You cannot present custom SwiftUI or UIKit views. You return a `ShieldConfiguration` struct and the system renders the shield. The customization options are intentionally limited to labels, colors, an icon, and a blur style. +- **Use PNG images, not PDFs.** Developers have reported that PDF images for the `icon` property cause the shield to render blank or with a missing icon. Convert assets to PNG format before bundling them in the extension. +- **`nil` properties behave differently.** Setting `primaryButtonLabel` to `nil` shows a system-default label ("OK"). Setting `secondaryButtonLabel` to `nil` hides the secondary button entirely. Setting `icon` to `nil` hides the icon area. Be intentional about which properties you leave nil. +- **The class was renamed in Xcode 14.** WWDC 2021 materials and early tutorials reference `ShieldConfigurationProvider`. This was renamed to `ShieldConfigurationDataSource` in iOS 16 / Xcode 14. Old code samples will not compile. +- **Four overloads to cover all cases.** There are four `configuration(shielding:)` methods: Application, Application-in-Category, WebDomain, and WebDomain-in-Category. If you only override the Application variant but shield by category, the category variant will use the default system appearance. +- **Token matching can be unreliable.** Some developers report that `Application` tokens passed to the Shield Configuration extension do not match tokens stored from `FamilyActivityPicker`, particularly after iOS updates. Avoid relying on token comparison for conditional logic; use App Group shared state instead. +- **Family Controls entitlement requires Apple approval.** The `com.apple.developer.family-controls` entitlement must be requested through Apple's developer portal. Without approval, development builds work but TestFlight and App Store distribution will be rejected. +- **Deployment target must match the main app.** Developers have found that setting a different iOS deployment target on the Shield Configuration extension than the main app causes customizations to silently fail. Ensure both targets have the same minimum deployment target. +- **App Group is required for dynamic content.** The extension runs in a separate sandboxed process. To display dynamic text (like usage counts or user names), read from `UserDefaults(suiteName:)` or a shared App Group file container. +- **Shield may cache configurations.** The system does not always call the extension for every shield presentation. Configurations may be cached. To force a refresh, toggle the shield off and on in `ManagedSettingsStore` or restart the shielded app. diff --git a/plugins/expo/skills/apple-targets/smart-card.md b/plugins/expo/skills/apple-targets/smart-card.md new file mode 100644 index 0000000..cfd9ee1 --- /dev/null +++ b/plugins/expo/skills/apple-targets/smart-card.md @@ -0,0 +1,263 @@ +--- +title: Smart Card Extension +description: CryptoTokenKit-based app extension that acts as a driver for physical smart cards (PIV, CAC) and USB/NFC cryptographic tokens. Makes certificates and keys stored on hardware tokens available system-wide through the keychain and SecKey APIs. +version: iOS 16.0+, macOS 10.12+ +--- + +# Smart Card Extension (`smart-card`) + +## Apple Documentation + +- [CryptoTokenKit Framework](https://developer.apple.com/documentation/cryptotokenkit) +- [Authenticating Users with a Cryptographic Token](https://developer.apple.com/documentation/cryptotokenkit/authenticating-users-with-a-cryptographic-token) +- [Using Cryptographic Assets Stored on a Smart Card](https://developer.apple.com/documentation/cryptotokenkit/using-cryptographic-assets-stored-on-a-smart-card) +- [Configuring Smart Card Authentication](https://developer.apple.com/documentation/cryptotokenkit/configuring-smart-card-authentication) +- [TKSmartCardTokenDriver](https://developer.apple.com/documentation/cryptotokenkit/tksmartcardtokendriver) +- [TKSmartCardToken](https://developer.apple.com/documentation/cryptotokenkit/tksmartcardtoken) +- [TKSmartCardTokenSession](https://developer.apple.com/documentation/cryptotokenkit/tksmartcardtokensession) +- [Apple Deployment Guide: Smart Card Integration](https://support.apple.com/guide/deployment/depd0b888248/web) +- [Supported Smart Card Functions on Mac](https://support.apple.com/guide/deployment/depc47f60521/web) + +## WWDC History + +- **WWDC 2014** -- CryptoTokenKit framework introduced alongside OS X Yosemite (10.10). Low-level smart card communication APIs (`TKSmartCard`, `TKSmartCardSlotManager`) shipped first. +- **WWDC 2016, Session 706 -- What's New in Security** -- The CryptoTokenKit _extension_ mechanism was announced with macOS Sierra (10.12), replacing the deprecated `tokend` system from OS X Lion. This session described it as "system support for cryptographic devices so that smart cards you might be using to prove your identity... we now have out of the box integration." ([transcript](https://asciiwwdc.com/2016/sessions/706)) +- **WWDC 2019, Session 709 -- Cryptography and Your Apps** -- Covered SecKey API improvements relevant to token-backed keys. +- **iOS 16 / WWDC 2022** -- CryptoTokenKit gained iOS support, enabling PIV smart card extensions on iPhone and iPad for the first time. + +## What It Does + +A smart card extension is a CryptoTokenKit "persistent token" driver. When a compatible smart card is inserted (via USB CCID reader or NFC on iOS), the system loads your extension to: + +1. **Enumerate objects** -- Read certificates, public keys, and key references from the card. +2. **Expose to keychain** -- Make discovered certificates and identities available in the system keychain so any app using SecItem/SecKey APIs can find them. +3. **Perform crypto operations** -- Sign, decrypt, and authenticate using the private keys that never leave the hardware token. + +The extension runs sandboxed in its own process and communicates with the card through APDU commands via `TKSmartCardTokenSession`. + +## Use Cases + +### Enterprise / Government Authentication +The most common use case. Organizations using PIV (Personal Identity Verification) or CAC (Common Access Card) smart cards deploy CryptoTokenKit extensions to enable: + +- **macOS login** with smart card + PIN (two-factor authentication) +- **Screensaver unlock** requiring card presence +- **TLS client authentication** in Safari, Chrome, and other browsers +- **VPN authentication** using certificate-based credentials +- **Kerberos SSO (PKINIT)** for single sign-on to enterprise services +- **S/MIME email** signing and encryption in Mail.app +- **Sudo escalation** backed by smart card credentials + +### Hardware Security Key Support (iOS) +Since iOS 16, apps like YubiKey Authenticator use CryptoTokenKit extensions to expose PIV credentials from NFC/Lightning/USB-C security keys to Safari and system services. + +### Custom Token Drivers +Organizations with proprietary or non-PIV smart cards build custom drivers to bridge their card's APDU protocol to CryptoTokenKit. + +## PIV Slot Reference + +Smart cards following the PIV standard use these certificate slots: + +| Slot | Name | Purpose | +|------|------|---------| +| 9a | PIV Authentication | Login, TLS client auth | +| 9c | Digital Signing | Email/document signing | +| 9d | Key Management | Encryption/decryption | +| 9e | Card Authentication | Physical access (no PIN required) | + +## Key Classes + +| Class | Role | +|-------|------| +| `TKSmartCardTokenDriver` | Entry point. Subclass this as your extension's principal class. Receives `createToken(smartCard:aid:reply:)` when a card matching your AID is inserted. | +| `TKSmartCardToken` | Represents the token. Initialize with the card's discovered certificates and keys via `TKTokenKeychainContents`. | +| `TKSmartCardTokenSession` | Handles crypto operations. Override `beginAuthentication(operation:)`, `sign(_:keyObjectID:algorithm:)`, and `decrypt(_:keyObjectID:algorithm:)`. | +| `TKSmartCard` | Low-level card communication. Send/receive APDUs. Available via `session.smartCard`. | + +## Extension Info.plist + +The extension's `Info.plist` must declare: + +```xml +NSExtension + + NSExtensionPointIdentifier + com.apple.ctk-tokens + NSExtensionPrincipalClass + $(PRODUCT_MODULE_NAME).TokenExtension + NSExtensionAttributes + + com.apple.ctk.token-type + smartcard + com.apple.ctk.aid + + + A000000308000010000100 + + + +``` + +The `com.apple.ctk.aid` array tells the system which smart card Application Identifiers (AIDs) this extension handles. The system selects your extension when a card with a matching AID is inserted. + +## Implementation + +### Minimal Driver (Template) + +This is what `create-target` scaffolds: + +```swift +import CryptoTokenKit + +class TokenExtension: TKSmartCardTokenDriver { + +} +``` + +### Full Implementation Pattern + +A real-world smart card extension typically has three files: + +**TokenDriver.swift** -- Entry point that creates a token when a card is detected: + +```swift +import CryptoTokenKit + +class TokenDriver: TKSmartCardTokenDriver, TKSmartCardTokenDriverDelegate { + func tokenDriver( + _ driver: TKSmartCardTokenDriver, + createTokenFor smartCard: TKSmartCard, + aid AID: Data, + reply: @escaping (TKSmartCardToken?, Error?) -> Void + ) { + // Select the PIV applet on the card + smartCard.send( + ins: 0xA4, p1: 0x04, p2: 0x00, data: AID + ) { response, sw1, sw2, error in + guard error == nil, sw1 == 0x90, sw2 == 0x00 else { + reply(nil, error) + return + } + + // Read certificates from standard PIV slots + let token = MySmartCardToken( + smartCard: smartCard, + aid: AID, + instanceID: smartCard.slot.name // unique per reader + ) + reply(token, nil) + } + } +} +``` + +**Token.swift** -- Enumerates keys and certificates on the card: + +```swift +import CryptoTokenKit + +class MySmartCardToken: TKSmartCardToken { + init(smartCard: TKSmartCard, aid: Data, instanceID: String) { + super.init( + smartCard: smartCard, + aid: aid, + instanceID: instanceID, + tokenDriver: TokenDriver() + ) + + // Populate keychain contents with certificates found on card + guard let keychainContents = self.keychainContents else { return } + + // Example: add a certificate from PIV slot 9a + let certItem = TKTokenKeychainCertificate( + certificate: pivCertificate, // SecCertificate from card + objectID: "slot-9a-cert" + ) + let keyItem = TKTokenKeychainKey( + certificate: pivCertificate, + objectID: "slot-9a-key" + ) + keyItem.canSign = true + keyItem.canDecrypt = false + + keychainContents.fill(with: [certItem, keyItem]) + } +} +``` + +**TokenSession.swift** -- Handles signing and decryption requests: + +```swift +import CryptoTokenKit + +class MyTokenSession: TKSmartCardTokenSession { + override func beginAuthentication( + operation: TKTokenOperation, + constraint: Any + ) throws -> TKTokenAuthOperation { + // Return a PIN-based auth operation + let pinAuth = TKTokenSmartCardPINAuthOperation() + pinAuth.pinFormat = .fixedLength(8) + pinAuth.smartCard = self.smartCard + return pinAuth + } + + override func sign( + _ dataToSign: Data, + keyObjectID: Any, + algorithm: TKTokenKeyAlgorithm + ) throws -> Data { + guard let smartCard = self.smartCard else { + throw TKError(.corruptedData) + } + + // Select the key on the card and perform signing via APDU + let signedData = try smartCard.send( + ins: 0x87, p1: 0x07, p2: 0x9A, + data: buildSigningAPDU(dataToSign) + ) + return signedData + } +} +``` + +## Usage + +- Create: `bun create target smart-card` +- Edit: `targets/smart-card/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| macOS | 10.10+ | Full support since Yosemite. Extension mechanism since Sierra 10.12. | +| iOS | 16.0+ | Requires USB-C or NFC reader. Lightning via MFi adapters. | +| iPadOS | 16.0+ | Same as iOS. | +| watchOS | -- | Not supported. | +| tvOS | -- | Not supported. | + +## Entitlements + +On macOS, the extension requires the `com.apple.security.smartcard` entitlement. On iOS, no special entitlement is needed beyond the standard app extension sandbox. + +If you need to share data between the main app and the extension, enable App Groups. + +## Debugging Tips + +- **Enable smart card logging**: `sudo defaults write /Library/Preferences/com.apple.security.smartcard Logging -bool true` +- **Register manually**: `pluginkit -a /path/to/YourApp.app/Contents/PlugIns/YourExtension.appex` +- **List registered tokens**: `pluginkit -m -p com.apple.ctk-tokens` +- **Check system log**: `log stream --predicate 'subsystem == "com.apple.CryptoTokenKit"'` + +## Open-Source References + +- [OpenSCToken](https://github.com/frankmorgner/OpenSCToken) -- OpenSC smart card driver built as a CryptoTokenKit extension (Objective-C). Demonstrates macOS login, TLS auth, and keychain integration. +- [Yubico: How to Implement a CryptoTokenKit Extension on iOS](https://www.yubico.com/blog/how-to-implement-a-cryptotokenkit-extension-on-ios/) -- Detailed walkthrough of building a PIV-based CTK extension for iOS using YubiKey. + +## Gotchas + +- Apple's CryptoTokenKit documentation is widely regarded as sparse. Filing a DTS tech support incident is recommended for complex questions. +- `TKTokenKeychainContents` is `nil` unless your token inherits from `TKSmartCardToken` (not the base `TKToken`). +- The extension runs in a sandbox. Configuration files must be read from the appex bundle resources; writes go to the container's `Documents/` directory. +- On macOS, the built-in PIV token driver (`com.apple.CryptoTokenKit.pivtoken`) handles standard PIV cards automatically. You only need a custom extension for non-standard cards or cards requiring custom APDU sequences. +- Card communication is asynchronous. All `TKSmartCard.send()` calls use completion handlers or async/await (iOS 16+). diff --git a/plugins/expo/skills/apple-targets/spotlight-delegate.md b/plugins/expo/skills/apple-targets/spotlight-delegate.md new file mode 100644 index 0000000..cb5cbc7 --- /dev/null +++ b/plugins/expo/skills/apple-targets/spotlight-delegate.md @@ -0,0 +1,163 @@ +--- +title: CoreSpotlight Delegate Extension +description: Allows the system to request re-indexing of your app's searchable content even when the app is not running, ensuring Spotlight results stay fresh after backups, restores, and index resets. +version: iOS 9.0+, macOS 10.13+ +--- + +# CoreSpotlight Delegate Extension (`spotlight-delegate`) + +A CoreSpotlight delegate extension runs when the system needs your app to re-index its searchable content -- after a device restore from backup, when indexed items are about to expire, or when the on-device Spotlight index is rebuilt. The extension subclasses `CSIndexExtensionRequestHandler` (which conforms to `CSSearchableIndexDelegate`) and overrides two methods: one for full re-indexing and one for re-indexing specific items by identifier. This ensures your app's Spotlight content remains current without requiring the user to open the app. + +## Apple Documentation + +- [CSSearchableIndexDelegate](https://developer.apple.com/documentation/corespotlight/cssearchableindexdelegate) +- [CSIndexExtensionRequestHandler](https://developer.apple.com/documentation/corespotlight/csindexextensionrequesthandler) +- [CSSearchableIndex](https://developer.apple.com/documentation/corespotlight/cssearchableindex) +- [CSSearchableItem](https://developer.apple.com/documentation/corespotlight/cssearchableitem) +- [CSSearchableItemAttributeSet](https://developer.apple.com/documentation/corespotlight/cssearchableitemattributeset) +- [Core Spotlight Framework Overview](https://developer.apple.com/documentation/corespotlight) + +## WWDC History + +- **[WWDC 2015, Session 709 -- Introducing Search APIs](https://developer.apple.com/videos/play/wwdc2015/709/)** -- Introduced Core Spotlight on iOS, covering `CSSearchableItem`, `CSSearchableIndex`, and the fundamentals of indexing app content for Spotlight. +- **[WWDC 2016, Session 223 -- Making the Most of Search APIs](https://developer.apple.com/videos/play/wwdc2016/223/)** -- Introduced `CSSearchableIndexDelegate` and the Core Spotlight extension for maintaining indexed content when the app is not running, including disaster recovery and re-indexing flows. +- **[WWDC 2017, Session 231 -- What's New in Core Spotlight for iOS and macOS](https://developer.apple.com/videos/play/wwdc2017/231/)** -- Extended Core Spotlight to macOS, added Drag-and-Drop support through the extension, and emphasized the importance of the delegate for keeping content fresh. +- **[WWDC 2024, Session 10131 -- Support Semantic Search with Core Spotlight](https://developer.apple.com/videos/play/wwdc2024/10131/)** -- Added semantic search capabilities to Core Spotlight, improving result relevance through contextual understanding of indexed content. + +## What It Does + +1. **System detects stale or missing index data.** This happens after a device restore from backup, when a Spotlight index rebuild occurs, or when previously indexed items are about to expire. +2. **Extension is launched.** The system instantiates your `CSIndexExtensionRequestHandler` subclass even if the app is not running. +3. **Full re-index requested.** The system calls `searchableIndex(_:reindexAllSearchableItemsWithAcknowledgementHandler:)` when the entire index needs rebuilding. Your extension fetches all indexable content and writes it to the provided `CSSearchableIndex`. +4. **Partial re-index requested.** The system calls `searchableIndex(_:reindexSearchableItemsWithIdentifiers:acknowledgementHandler:)` with specific item IDs when only certain items need refreshing (e.g., items about to expire or items that failed to index previously). +5. **Extension acknowledges completion.** After indexing, you call the `acknowledgementHandler()`. This tells the system the re-index is complete. If the extension crashes before acknowledging, the system will retry. +6. **User searches Spotlight.** Indexed items appear in Spotlight results. Tapping a result opens your app via `NSUserActivity` with the `CSSearchableItemActionType` activity type and the item's identifier in `userInfo`. + +## Use Cases + +### Notes and document apps +A note-taking app indexes thousands of notes for Spotlight search. When the user restores from an iCloud backup on a new device, the extension re-indexes all notes from the local database so they immediately appear in Spotlight, without waiting for the user to open the app. + +### Recipe and reference apps +A recipe app indexes its full catalog with titles, ingredients, and cuisine tags. The extension ensures that after an iOS update or index reset, all recipes are re-indexed. Partial re-indexing handles individual recipe updates that were missed while the app was suspended. + +### E-commerce and catalog apps +A shopping app indexes product listings for Spotlight. The extension handles periodic re-indexing to refresh prices and availability, and re-indexes specific product IDs when the system reports them as stale. + +### Communication apps +A messaging app indexes conversation titles and recent messages. The extension re-indexes conversation metadata after a device restore, and the Drag-and-Drop support (WWDC 2017) allows users to drag conversation items from Spotlight into other apps. + +## Key Classes + +| Class / Protocol | Role | +|------------------|------| +| `CSIndexExtensionRequestHandler` | Base class for the extension. Conforms to `CSSearchableIndexDelegate` and `NSExtensionRequestHandling`. Subclass this and override the two re-index methods. | +| `CSSearchableIndexDelegate` | Protocol defining `reindexAllSearchableItems` and `reindexSearchableItemsWithIdentifiers` callbacks. | +| `CSSearchableIndex` | The on-device search index. Call `indexSearchableItems(_:completionHandler:)` on it to add or update items. | +| `CSSearchableItem` | Wraps a `CSSearchableItemAttributeSet` with a unique identifier and optional domain identifier for batch deletion. | +| `CSSearchableItemAttributeSet` | Metadata container with properties like `title`, `contentDescription`, `thumbnailData`, and `keywords`. | + +## Implementation + +```swift +import CoreSpotlight + +// 1. Subclass CSIndexExtensionRequestHandler as the extension's principal class. +class IndexRequestHandler: CSIndexExtensionRequestHandler { + + // 2. Called when the system needs a complete re-index of all content. + override func searchableIndex( + _ searchableIndex: CSSearchableIndex, + reindexAllSearchableItemsWithAcknowledgementHandler acknowledgementHandler: @escaping () -> Void + ) { + // 3. Fetch all indexable content from your data store. + // Use App Groups if your database is shared with the main app. + let allItems = DataStore.shared.fetchAllItems() + + // 4. Convert each item into a CSSearchableItem. + let searchableItems = allItems.map { item -> CSSearchableItem in + let attributes = CSSearchableItemAttributeSet(contentType: .text) + attributes.title = item.title + attributes.contentDescription = item.body + attributes.keywords = item.tags + attributes.thumbnailData = item.thumbnailData + attributes.contentCreationDate = item.createdAt + + return CSSearchableItem( + uniqueIdentifier: item.id, + domainIdentifier: "com.example.myapp.items", + attributeSet: attributes + ) + } + + // 5. Index all items, then acknowledge so the system knows we finished. + searchableIndex.indexSearchableItems(searchableItems) { error in + if let error { + print("Re-index all failed: \(error)") + } + // 6. Always call the handler, even on error, so the system + // does not keep retrying indefinitely. + acknowledgementHandler() + } + } + + // 7. Called when specific items need re-indexing (e.g., about to expire). + override func searchableIndex( + _ searchableIndex: CSSearchableIndex, + reindexSearchableItemsWithIdentifiers identifiers: [String], + acknowledgementHandler: @escaping () -> Void + ) { + // 8. Fetch only the items matching the given identifiers. + let items = DataStore.shared.fetchItems(withIDs: identifiers) + + let searchableItems = items.map { item -> CSSearchableItem in + let attributes = CSSearchableItemAttributeSet(contentType: .text) + attributes.title = item.title + attributes.contentDescription = item.body + attributes.keywords = item.tags + attributes.thumbnailData = item.thumbnailData + attributes.contentCreationDate = item.createdAt + + return CSSearchableItem( + uniqueIdentifier: item.id, + domainIdentifier: "com.example.myapp.items", + attributeSet: attributes + ) + } + + // 9. Index the specific items and acknowledge completion. + searchableIndex.indexSearchableItems(searchableItems) { error in + if let error { + print("Re-index items failed: \(error)") + } + acknowledgementHandler() + } + } +} +``` + +## Usage + +- Create: `bun create target spotlight-delegate` +- Edit: `targets/spotlight-delegate/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 9.0+ | Full support. The extension and in-app delegate share the same protocol. | +| iPadOS | 9.0+ | Full support. | +| macOS | 10.13+ (High Sierra) | Full support. Extension runs even when the app is not running. | +| watchOS | -- | Not supported. | +| tvOS | -- | Not supported. | +| visionOS | -- | Not supported. | + +## Gotchas + +- **Always call the acknowledgement handler.** If your extension crashes or returns without calling `acknowledgementHandler()`, the system will retry the re-index request. Call the handler even if indexing partially fails, or you risk an infinite retry loop consuming resources. +- **Share code between the app and extension.** The `CSSearchableIndexDelegate` protocol is the same whether implemented in your app's `AppDelegate` or in the extension. Factor your indexing logic into a shared framework so both targets use the same code. The extension handles re-indexing when the app is not running; the in-app delegate handles it while the app is active. +- **Use `fetchLastClientState` to resume interrupted work.** Call `searchableIndex.fetchLastClientState(completionHandler:)` to retrieve a `Data` blob you previously stored with `searchableIndex.updateClientState(_:completionHandler:)`. Use this to track your indexing progress and resume where you left off if the extension was terminated mid-operation. +- **Item expiration triggers partial re-indexing.** `CSSearchableItem` has an `expirationDate` property (defaults to 30 days). Before items expire, the system calls `reindexSearchableItemsWithIdentifiers` to give you a chance to refresh them. Set a reasonable expiration or use `.distantFuture` for content that does not change. +- **Extension is required for Drag-and-Drop from Spotlight.** On iOS 11+ and macOS, if a user long-presses a Spotlight result to initiate drag, the system calls your extension to provide the item data. Without the extension, Spotlight results from your app cannot participate in Drag-and-Drop. +- **The extension has limited runtime.** Keep indexing operations fast. If you have thousands of items, index them in batches and persist progress to disk so you can resume if the extension is terminated. Avoid blocking network calls in the extension. +- **Domain identifiers enable batch deletion.** Assign a `domainIdentifier` to related items (e.g., `"com.example.recipes"`). You can then call `CSSearchableIndex.deleteSearchableItems(withDomainIdentifiers:)` to remove an entire category at once, which is much faster than deleting by individual ID. diff --git a/plugins/expo/skills/apple-targets/spotlight.md b/plugins/expo/skills/apple-targets/spotlight.md new file mode 100644 index 0000000..6f226e0 --- /dev/null +++ b/plugins/expo/skills/apple-targets/spotlight.md @@ -0,0 +1,117 @@ +--- +title: Spotlight Import Extension +description: Provides Spotlight search metadata for custom file types by extracting searchable attributes when files are indexed on disk. +version: iOS 15.0+, macOS 12.0+ +--- + +# Spotlight Import Extension (`spotlight`) + +A Spotlight import extension extracts metadata from custom file types so they appear in Spotlight search results. When the system indexes a file matching your declared UTI (Uniform Type Identifier), it loads your extension and calls `update(_:forFileAt:)`, where you populate a `CSSearchableItemAttributeSet` with properties like title, content description, author, and keywords. This is the modern replacement for the legacy `mdimporter` plug-in architecture, using Core Spotlight's `CSImportExtension` base class. + +## Apple Documentation + +- [CSImportExtension](https://developer.apple.com/documentation/corespotlight/csimportextension) +- [CSSearchableItemAttributeSet](https://developer.apple.com/documentation/corespotlight/cssearchableitemattributeset) +- [Core Spotlight Framework Overview](https://developer.apple.com/documentation/corespotlight) +- [Spotlight Importer Programming Guide (Archive)](https://developer.apple.com/library/archive/documentation/Carbon/Conceptual/MDImporters/MDImporters.html) +- [Writing a Spotlight Importer (Archive)](https://developer.apple.com/library/archive/documentation/Carbon/Conceptual/MDImporters/Concepts/WritingAnImp.html) + +## WWDC History + +- **[WWDC 2015, Session 709 -- Introducing Search APIs](https://developer.apple.com/videos/play/wwdc2015/709/)** -- Introduced Core Spotlight on iOS, covering `CSSearchableItem`, `CSSearchableIndex`, and the ability to index app content for Spotlight search. +- **[WWDC 2017, Session 231 -- What's New in Core Spotlight for iOS and macOS](https://developer.apple.com/videos/play/wwdc2017/231/)** -- Extended Core Spotlight to macOS, discussed the Spotlight import extension architecture, and covered index delegates and Drag-and-Drop support. +- **[WWDC 2024, Session 10131 -- Support Semantic Search with Core Spotlight](https://developer.apple.com/videos/play/wwdc2024/10131/)** -- Added semantic search capabilities to Core Spotlight, improving relevance for text, image, and video content through related terms and synonyms. + +## What It Does + +1. **System detects a matching file.** When a file with your declared UTI appears on disk (created, modified, or restored from backup), the system's `mdworker` process identifies it for indexing. +2. **Extension is loaded.** The system instantiates your `CSImportExtension` subclass in a sandboxed process. +3. **`update(_:forFileAt:)` is called.** You receive a pre-allocated `CSSearchableItemAttributeSet` and the file's URL. Read the file, extract metadata, and set properties on the attribute set. +4. **Metadata enters the Spotlight index.** The system stores the attributes and associates them with the file. Spotlight queries matching your metadata will surface the file in results. +5. **User taps a result.** When the user selects the file in Spotlight, the system opens it with the registered default app. + +## Use Cases + +### Custom document formats +A design app saves projects as `.mydesign` files. The Spotlight import extension reads each file's header to extract the project name, canvas dimensions, color palette names, and layer count, making them searchable by project name or content keywords. + +### Scientific data files +A research tool stores experiment results in a custom binary format. The importer parses the file header to extract the experiment date, researcher name, instrument type, and summary statistics, so scientists can find datasets by searching for experiment parameters. + +### Configuration and recipe files +A cooking app exports recipes as `.recipe` files. The extension parses each file to extract the recipe title, ingredients list, cuisine type, and cook time, enabling Spotlight searches like "chicken tikka 30 minutes." + +## Key Classes + +| Class / Protocol | Role | +|------------------|------| +| `CSImportExtension` | Abstract base class you subclass. Override `update(_:forFileAt:)` to extract metadata from files. | +| `CSSearchableItemAttributeSet` | A container for metadata attributes. Set properties like `title`, `contentDescription`, `keywords`, `creator`, and `contentType`. | +| `UTType` | Uniform Type Identifier describing your custom file type. Declare in your app's Info.plist under `UTExportedTypeDeclarations`. | + +## Implementation + +```swift +import CoreSpotlight +import UniformTypeIdentifiers + +// 1. Subclass CSImportExtension as the extension's principal class. +class ImportExtension: CSImportExtension { + + // 2. Override update(_:forFileAt:) to populate metadata for each indexed file. + override func update( + _ attributes: CSSearchableItemAttributeSet, + forFileAt contentURL: URL + ) throws { + // 3. Read the file contents. The extension is sandboxed but has + // read access to the specific file URL provided. + let data = try Data(contentsOf: contentURL) + + // 4. Parse your custom format to extract searchable metadata. + let document = try MyDocumentParser.parse(data) + + // 5. Populate the attribute set with searchable properties. + attributes.title = document.title + attributes.contentDescription = document.summary + attributes.keywords = document.tags + attributes.creator = document.authorName + attributes.contentCreationDate = document.createdAt + attributes.contentModificationDate = document.modifiedAt + + // 6. Set a thumbnail if available. + if let thumbnailData = document.thumbnail { + attributes.thumbnailData = thumbnailData + } + + // 7. Add domain-specific attributes for richer search results. + attributes.numberOfPages = document.pageCount as NSNumber + attributes.subject = document.category + } +} +``` + +## Usage + +- Create: `bun create target spotlight` +- Edit: `targets/spotlight/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 15.0+ | Supported via Core Spotlight. | +| iPadOS | 15.0+ | Supported via Core Spotlight. | +| macOS | 12.0+ (declared) | Historically non-functional. First reports of working on macOS 15.5+. Legacy `mdimporter` plug-ins remain the reliable alternative on macOS. | +| watchOS | -- | Not supported. | +| tvOS | -- | Not supported. | +| visionOS | -- | Not supported. | + +## Gotchas + +- **macOS support has been broken for years.** Despite Apple's documentation listing macOS 12.0+ as supported, `CSImportExtension` did not function on macOS until at least macOS 15.5 (Sequoia). Apple DTS has acknowledged the issue. If you need macOS Spotlight indexing, use the legacy MDImporter/CFPlugIn architecture instead. +- **Sandbox restricts bundle access on macOS.** Even on macOS 15.5 where the extension loads, accessing files inside document bundles (packages) fails with a permission error. `NSFileWrapper(url:)` cannot read into the bundle's subdirectories. Only flat files work reliably. +- **`CSSupportedContentTypes` must match your UTI exactly.** The `CSSupportedContentTypes` array in your extension's Info.plist must contain the exact UTI string for your custom file type. If it does not match, the system will never invoke your extension for those files. +- **Declare your UTI in the main app.** The custom UTI must be declared under `UTExportedTypeDeclarations` (or `UTImportedTypeDeclarations`) in your main app's Info.plist, not just the extension's. The system discovers UTIs from the app bundle. +- **The extension is stateless.** Each invocation of `update(_:forFileAt:)` is independent. The extension may be launched and terminated for each file. Do not store data in instance variables and expect it to persist. +- **Verify with `mdimport`.** On macOS, use `mdimport -L` to list installed importers and `mdimport -d2 /path/to/file` to test your importer. On iOS, there is no equivalent diagnostic tool. +- **No network access.** The extension is sandboxed without network entitlements. All metadata must be extractable from the file itself. diff --git a/plugins/expo/skills/apple-targets/unwanted-communication.md b/plugins/expo/skills/apple-targets/unwanted-communication.md new file mode 100644 index 0000000..0c8373e --- /dev/null +++ b/plugins/expo/skills/apple-targets/unwanted-communication.md @@ -0,0 +1,278 @@ +--- +title: Unwanted Communication Reporting Extension +description: IdentityLookup-based extension that lets users report spam SMS messages and phone calls as junk directly from the Messages and Phone apps. +version: iOS 12.0+ +--- + +# Unwanted Communication Reporting Extension (`unwanted-communication`) + +An app extension that provides a UI for users to report unwanted SMS messages and phone calls as spam. When a user long-presses a message or taps "Report Junk" on a recent call, iOS presents your extension's view controller so the user can classify and submit the report. + +## Apple Documentation + +- [SMS and Call Reporting (IdentityLookup)](https://developer.apple.com/documentation/identitylookup) +- [SMS and Call Spam Reporting](https://developer.apple.com/documentation/IdentityLookup/sms-and-call-spam-reporting) +- [ILClassificationUIExtensionViewController](https://developer.apple.com/documentation/sms_and_call_reporting/ilclassificationuiextensionviewcontroller) +- [ILClassificationRequest](https://developer.apple.com/documentation/identitylookup/ilclassificationrequest) +- [ILClassificationResponse](https://developer.apple.com/documentation/identitylookup/ilclassificationresponse) +- [Creating a Message Filter App Extension](https://developer.apple.com/documentation/identitylookup/creating-a-message-filter-app-extension) +- [Getting Up-to-Date Calling and Blocking Information](https://developer.apple.com/documentation/identitylookup/getting-up-to-date-calling-and-blocking-information-for-your-app) + +## WWDC History + +- **WWDC 2017, Session 249 -- [Filtering Unwanted Messages with Identity Lookup](https://developer.apple.com/videos/play/wwdc2017/249/)** -- The `IdentityLookup` framework was introduced with iOS 11 for SMS message filtering extensions. This was the foundation that the unwanted communication extension builds on. +- **WWDC 2018 (iOS 12)** -- Apple introduced the **Unwanted Communication Reporting** extension type, allowing users to report spam calls and SMS messages to third-party apps. This added the `ILClassificationUIExtensionViewController` and network reporting APIs. +- **WWDC 2022, Session 110341 -- [Explore SMS Message Filters](https://developer.apple.com/videos/play/wwdc2022/110341/)** -- Updated SMS filtering APIs for iOS 16 with sub-classification support and improved filtering capabilities. + +## What It Does + +The Unwanted Communication Reporting extension presents a custom UI when a user reports a message or call as junk. The flow is: + +1. User selects a message in Messages.app or a call in Recents and taps **Report Junk**. +2. iOS presents your extension's `ILClassificationUIExtensionViewController`. +3. Your extension displays a UI for the user to classify the communication (spam, scam, etc.). +4. The user taps **Done** and your extension returns an `ILClassificationResponse`. +5. iOS delivers the report to your server via the configured network endpoint. + +The extension receives: +- **For SMS**: Full message text, sender information, and content. +- **For calls**: Caller's phone number and the date/time of the call. + +## Use Cases + +### Spam/Robocall Reporting Apps +The primary use case. Apps like Truecaller, Hiya, RoboKiller, and similar services use this extension to let users report spam directly from the native Messages and Phone apps. + +### Enterprise Communication Security +Organizations deploy reporting extensions to collect data about phishing SMS campaigns and scam calls targeting employees. + +### Telecom Carrier Apps +Mobile carriers ship unwanted communication extensions so subscribers can report spam, feeding the carrier's spam detection systems. + +## Key Classes + +| Class | Role | +|-------|------| +| `ILClassificationUIExtensionViewController` | Principal view controller. Subclass this to present your reporting UI. | +| `ILClassificationRequest` | Contains the communication details (message text, sender, call info) to classify. | +| `ILClassificationResponse` | Your extension's response with the chosen classification action. | +| `ILClassificationAction` | Enum of possible actions: `.none`, `.reportJunk`, `.reportNotJunk`, `.reportJunkAndBlockSender`. | + +## Classification Actions + +| Action | Description | +|--------|-------------| +| `.none` | User cancelled or chose not to classify. | +| `.reportJunk` | Report the communication as junk/spam. | +| `.reportNotJunk` | Mark as not junk (false positive correction). | +| `.reportJunkAndBlockSender` | Report as junk and block the sender. | + +## Extension Info.plist + +The extension's `Info.plist` must declare the extension point and, optionally, a network reporting destination: + +```xml +NSExtension + + NSExtensionPointIdentifier + com.apple.identitylookup.classification-ui + NSExtensionPrincipalClass + $(PRODUCT_MODULE_NAME).ClassificationViewController + NSExtensionAttributes + + ILClassificationExtensionNetworkReportDestination + https://api.example.com/reports/unwanted + + +``` + +## Implementation + +### Minimal Extension (Template) + +This is what `create-target` scaffolds: + +```swift +import IdentityLookupUI + +class ClassificationViewController: ILClassificationUIExtensionViewController { + + override func viewDidLoad() { + super.viewDidLoad() + } + + override func prepare(for classificationRequest: ILClassificationRequest) { + // Configure the view for the classification request + } + + override func classificationResponse( + for request: ILClassificationRequest + ) -> ILClassificationResponse { + return ILClassificationResponse(action: .none) + } +} +``` + +### Full Implementation Pattern + +A production extension with UI and network reporting: + +```swift +import UIKit +import IdentityLookup +import IdentityLookupUI + +class ClassificationViewController: ILClassificationUIExtensionViewController { + + private var selectedAction: ILClassificationAction = .none + + override func viewDidLoad() { + super.viewDidLoad() + setupUI() + } + + override func viewDidAppear(_ animated: Bool) { + super.viewDidAppear(animated) + // IMPORTANT: Enable the Done button. Without this, the user + // cannot submit the report. + self.extensionContext.isReadyForClassificationResponse = true + } + + override func prepare(for classificationRequest: ILClassificationRequest) { + // Read the communication details and populate your UI. + // + // For SMS: classificationRequest contains message text and sender. + // For calls: classificationRequest contains caller number and date. + // + // Example: display the sender info in a label + if let comms = classificationRequest.communicationIdentifiers.first { + senderLabel.text = "From: \(comms)" + } + } + + override func classificationResponse( + for request: ILClassificationRequest + ) -> ILClassificationResponse { + // Return the user's chosen classification. + // The system sends this to your network endpoint automatically. + return ILClassificationResponse(action: selectedAction) + } + + // MARK: - UI + + private lazy var senderLabel = UILabel() + + private func setupUI() { + let stack = UIStackView() + stack.axis = .vertical + stack.spacing = 16 + stack.translatesAutoresizingMaskIntoConstraints = false + view.addSubview(stack) + + NSLayoutConstraint.activate([ + stack.leadingAnchor.constraint(equalTo: view.leadingAnchor, constant: 20), + stack.trailingAnchor.constraint(equalTo: view.trailingAnchor, constant: -20), + stack.centerYAnchor.constraint(equalTo: view.centerYAnchor), + ]) + + senderLabel.font = .preferredFont(forTextStyle: .headline) + stack.addArrangedSubview(senderLabel) + + let spamButton = makeButton(title: "Report as Spam", action: .reportJunk) + let blockButton = makeButton(title: "Report & Block", action: .reportJunkAndBlockSender) + let notJunkButton = makeButton(title: "Not Junk", action: .reportNotJunk) + + stack.addArrangedSubview(spamButton) + stack.addArrangedSubview(blockButton) + stack.addArrangedSubview(notJunkButton) + } + + private func makeButton( + title: String, + action: ILClassificationAction + ) -> UIButton { + let button = UIButton(type: .system) + button.setTitle(title, for: .normal) + button.tag = action.rawValue + button.addTarget(self, action: #selector(didTapAction(_:)), for: .touchUpInside) + return button + } + + @objc private func didTapAction(_ sender: UIButton) { + selectedAction = ILClassificationAction(rawValue: sender.tag) ?? .none + } +} +``` + +### Network Reporting Server Setup + +When the user submits a report, iOS sends it to your server automatically. You need: + +**1. Associated Domains entitlement on the extension:** + +Add the `classificationreport:` service to your associated domains: + +``` +classificationreport:api.example.com +``` + +**2. Apple App Site Association (AASA) file on your server:** + +Host at `https://api.example.com/.well-known/apple-app-site-association`: + +```json +{ + "classificationreport": { + "apps": [ + "TEAMID.com.example.myapp", + "TEAMID.com.example.myapp.unwanted-communication" + ] + } +} +``` + +**3. HTTPS endpoint:** + +The URL specified in `ILClassificationExtensionNetworkReportDestination` receives a POST with the classification data. Must be HTTPS with a valid certificate (no ATS overrides allowed). + +## User Activation + +Users must manually enable the extension: + +1. Go to **Settings > Phone > SMS/Call Reporting** (or **Settings > Messages > SMS/Call Reporting**). +2. Select your app's extension. + +Only **one** Unwanted Communication Reporting extension can be active at a time. + +## Usage + +- Create: `bun create target unwanted-communication` +- Edit: `targets/unwanted-communication/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 12.0+ | Full support for SMS and call reporting. | +| iPadOS | -- | Not supported (no Phone or SMS on iPad). | +| macOS | -- | Not supported. | +| watchOS | -- | Not supported. | +| tvOS | -- | Not supported. | + +## Privacy Model + +Apple enforces strict privacy constraints: + +- The extension's container is **deleted after every use**. You cannot persist data between invocations. +- The extension **cannot access the network directly**. All report delivery is handled by the system via the configured `ILClassificationExtensionNetworkReportDestination`. +- Shared `UserDefaults` via App Groups do **not** work reliably from this extension type due to sandbox restrictions. +- Any cookies the server attempts to set are ignored. + +## Gotchas + +- **Done button stays disabled** unless you set `self.extensionContext.isReadyForClassificationResponse = true` in `viewDidAppear`. This is not set automatically. +- **Network reports fire only once** after install in some iOS versions. Force-quitting the Messages app is a known workaround. This appears to be an iOS bug that has persisted across multiple versions. +- **No UITextField/UITextView** -- Adding text input fields inside the extension view controller may cause crashes related to the dictation feature. Use buttons or segmented controls instead. +- **Associated domains must match exactly** -- Both the containing app and the extension bundle IDs must be listed in the AASA file under `classificationreport`. Missing either one causes "unauthorized to defer" errors. +- **Only one extension active at a time** -- If another app's extension is already enabled, yours will replace it. Users must manually switch in Settings. +- **SMS filtering regional availability** -- SMS filtering (the separate `message-filter` extension type) may only work in certain countries. The unwanted communication reporting extension itself is available globally, but the underlying SMS/call features depend on the device's carrier and region. diff --git a/plugins/expo/skills/apple-targets/virtual-conference.md b/plugins/expo/skills/apple-targets/virtual-conference.md new file mode 100644 index 0000000..e825902 --- /dev/null +++ b/plugins/expo/skills/apple-targets/virtual-conference.md @@ -0,0 +1,144 @@ +--- +title: Virtual Conference Provider Extension +description: Integrates your video or voice calling app with the system Calendar so users see a "Join" button and room-type picker directly on their events. +version: iOS 15.0+, macOS 12.0+ +--- + +# Virtual Conference Provider Extension (`virtual-conference`) + +A virtual conference extension lets Calendar show a "Join" button on events that use your app's meeting rooms. When a user creates a new event, they can pick one of your conference room types from the location picker (shown alongside your app icon). The system then calls your extension to generate the meeting URL and details, which Calendar attaches to the event. Apps like Zoom, Microsoft Teams, and Webex use this extension to provide one-tap join from Calendar without opening the browser. + +## Apple Documentation + +- [EKVirtualConferenceProvider](https://developer.apple.com/documentation/eventkit/ekvirtualconferenceprovider) +- [EKVirtualConferenceDescriptor](https://developer.apple.com/documentation/eventkit/ekvirtualconferencedescriptor) +- [EKVirtualConferenceURLDescriptor](https://developer.apple.com/documentation/eventkit/ekvirtualconferenceurldescriptor) +- [EKVirtualConferenceRoomTypeDescriptor](https://developer.apple.com/documentation/eventkit/ekvirtualconferenceroomtypedescriptor) +- [EventKit Framework Overview](https://developer.apple.com/documentation/eventkit) + +## WWDC History + +- **[WWDC 2021 -- What's New in Managing Apple Devices](https://developer.apple.com/videos/play/wwdc2021/10130/)** -- `EKVirtualConferenceProvider` was introduced alongside iOS 15 and macOS 12, enabling third-party video conferencing apps to integrate with Calendar. +- **[WWDC 2023, Session 10052 -- Discover Calendar and EventKit](https://developer.apple.com/videos/play/wwdc2023/10052/)** -- Provided the first detailed walkthrough of building a virtual conference extension, including room types, URL descriptors, and the user flow in the Calendar location picker. + +## What It Does + +1. **User creates or edits a Calendar event.** In the location picker, Calendar shows "Video Call" options from installed apps that provide a virtual conference extension. +2. **Room type selection.** The system calls `fetchAvailableRoomTypes(completionHandler:)` on your `EKVirtualConferenceProvider` subclass. You return an array of `EKVirtualConferenceRoomTypeDescriptor` objects, each with a title and unique identifier. +3. **Conference generation.** When the user picks a room type, the system calls `fetchVirtualConference(identifier:completionHandler:)` with the selected room type identifier. Your extension creates the meeting (e.g., calls your backend API) and returns an `EKVirtualConferenceDescriptor` containing a title, one or more URL descriptors, and optional plain-text details. +4. **Calendar displays join affordance.** The event now shows a "Join" button. Tapping it opens the URL, which typically deep-links into your app. +5. **Shared across devices.** Because the conference data is part of the `EKEvent`, it syncs via iCloud/CalDAV to all the user's devices. + +## Use Cases + +### Video Conferencing Apps +Apps like Zoom, Microsoft Teams, Google Meet, and Webex can generate meeting links directly from the Calendar event creation flow. Users never need to open the conferencing app separately to create a room -- they just pick it from the Calendar location field and share the event invite. + +### Internal Enterprise Communication Tools +Companies with proprietary conferencing systems (e.g., Cisco Jabber, custom WebRTC solutions) can integrate with Calendar so employees see a native "Join" button rather than having to copy-paste meeting URLs from emails or chat. + +### Audio-Only Calling Apps +The extension is not limited to video. VoIP or audio bridge apps can provide room types like "Audio Bridge" or "Dial-In Conference" so users can join phone-style meetings from Calendar. + +## Key Classes + +| Class / Protocol | Role | +|------------------|------| +| `EKVirtualConferenceProvider` | Subclass this as your extension's principal class. Override `fetchAvailableRoomTypes` and `fetchVirtualConference`. | +| `EKVirtualConferenceRoomTypeDescriptor` | Describes one room type (e.g., "Team Meeting", "Webinar"). Has a `title` and a unique `identifier`. | +| `EKVirtualConferenceDescriptor` | The meeting details returned by your extension. Contains a title, an array of `EKVirtualConferenceURLDescriptor`, and optional conference details string. | +| `EKVirtualConferenceURLDescriptor` | A single URL to join the conference. Has a `title` (e.g., "Join via App", "Join via Web") and a `URL`. | + +## Implementation + +```swift +import EventKit + +class VirtualConferenceProvider: EKVirtualConferenceProvider { + + // 1. Declare the room types your app supports. + // Calendar shows these in the location picker next to your app icon. + override func fetchAvailableRoomTypes( + completionHandler: @escaping ([EKVirtualConferenceRoomTypeDescriptor]?, Error?) -> Void + ) { + let standardRoom = EKVirtualConferenceRoomTypeDescriptor( + title: "Video Meeting", + identifier: "standard" + ) + let webinarRoom = EKVirtualConferenceRoomTypeDescriptor( + title: "Webinar", + identifier: "webinar" + ) + // 2. Return all available room types. + completionHandler([standardRoom, webinarRoom], nil) + } + + // 3. Called when the user selects one of your room types. + // Generate the meeting and return a descriptor with join URLs. + override func fetchVirtualConference( + identifier: EKVirtualConferenceRoomTypeIdentifier, + completionHandler: @escaping (EKVirtualConferenceDescriptor?, Error?) -> Void + ) { + // 4. In a real app, call your backend to create a meeting room + // and receive a meeting URL. Here we simulate it. + let meetingId = UUID().uuidString.prefix(8).lowercased() + let meetingURL = URL(string: "https://meet.example.com/\(meetingId)")! + let deepLinkURL = URL(string: "myapp://join/\(meetingId)")! + + // 5. Create URL descriptors -- one for the app deep link, + // one as a web fallback. + let appURL = EKVirtualConferenceURLDescriptor( + title: "Open in MyApp", + url: deepLinkURL + ) + let webURL = EKVirtualConferenceURLDescriptor( + title: "Join via Browser", + url: meetingURL + ) + + // 6. Build the conference descriptor. + let title: String + switch identifier { + case "webinar": + title = "MyApp Webinar" + default: + title = "MyApp Meeting" + } + + let descriptor = EKVirtualConferenceDescriptor( + title: title, + urlDescriptors: [appURL, webURL], + conferenceDetails: "Meeting ID: \(meetingId)" + ) + + // 7. Return the descriptor. Calendar attaches it to the event. + completionHandler(descriptor, nil) + } +} +``` + +## Usage + +- Create: `bun create target virtual-conference` +- Edit: `targets/virtual-conference/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 15.0+ | Full support. Calendar shows room types in location picker. | +| iPadOS | 15.0+ | Same as iOS. | +| macOS | 12.0+ | Calendar.app supports virtual conference extensions. | +| watchOS | -- | Not supported. Calendar on watchOS does not load conference extensions. | +| tvOS | -- | Not supported. | +| visionOS | 1.0+ | Calendar is available; virtual conference extensions are expected to work. | + +## Gotchas + +- **Your extension is only invoked at event creation time.** The system calls `fetchVirtualConference` when the user picks your room type while creating or editing an event. It is not called again when the event is opened later -- the stored URL descriptors are used directly. If you need to regenerate meeting links, the user must re-select your room type. +- **Room types appear only if your main app is installed.** The extension is bundled inside your app. If the user deletes the app, your room types disappear from the Calendar location picker. +- **No way to programmatically attach a conference to an existing event.** Developers on Apple Developer Forums have reported that there is no public API to set virtual conference data on an `EKEvent` created via `EKEventEditViewController` or `EKEventStore`. The only path is through the user selecting your room type in the Calendar UI. +- **The completion handler must be called promptly.** If `fetchVirtualConference` takes too long (e.g., a slow network call to your backend), the system may time out and show an error. Keep the operation fast or pre-generate meeting IDs. +- **URL descriptors should include a web fallback.** If your first URL uses a custom scheme (`myapp://...`) and the app is not installed on the device receiving the shared event, the join button will fail silently. Always include an `https://` fallback URL. +- **EventKit permission is not required for the extension.** The extension does not need calendar access (`EKEventStore` authorization) to function. It only provides conference details -- it does not read or write events. +- **Zoom has not yet adopted this API.** Despite community requests, as of 2025, Zoom has not shipped an `EKVirtualConferenceProvider` extension. FaceTime is the only first-party provider. Adoption among third-party apps remains limited. diff --git a/plugins/expo/skills/apple-targets/watch.md b/plugins/expo/skills/apple-targets/watch.md new file mode 100644 index 0000000..d3522e6 --- /dev/null +++ b/plugins/expo/skills/apple-targets/watch.md @@ -0,0 +1,493 @@ +--- +title: watchOS App +description: A standalone watchOS application target that runs natively on Apple Watch with SwiftUI, supporting complications, workout sessions, and phone connectivity. +version: watchOS 7.0+ +--- + +# watchOS App (`watch`) + +A watchOS app target creates a fully independent application that runs on Apple Watch. It uses the product type `com.apple.product-type.application` -- the same as any standard app, not an extension. Since watchOS 7, apps use the SwiftUI `@main` App lifecycle, replacing the legacy WKExtension/WKInterfaceController pattern. Modern watchOS apps are SwiftUI-first: they use `NavigationStack` for hierarchical navigation, `TabView` for paged layouts, and `List` for scrollable content driven by the Digital Crown. The companion iOS app is optional -- watchOS apps can be distributed independently through the App Store and can make their own network requests, store data locally, and access HealthKit, CoreLocation, and other on-device frameworks. + +## Apple Documentation + +- [Building a watchOS App](https://developer.apple.com/documentation/watchos-apps/building_a_watchos_app) -- primary guide for setting up the app lifecycle, creating the user interface, and managing scenes with SwiftUI. +- [Creating a watchOS App (SwiftUI Tutorial)](https://developer.apple.com/tutorials/swiftui/creating-a-watchos-app) -- hands-on tutorial adapting the Landmarks sample app for watchOS. +- [TN3157: Updating Your watchOS Project for SwiftUI and WidgetKit](https://developer.apple.com/documentation/technotes/tn3157-updating-your-watchos-project-for-swiftui-and-widgetkit) -- tech note covering migration from WatchKit storyboards to SwiftUI and from ClockKit to WidgetKit. +- [WCSession (Watch Connectivity)](https://developer.apple.com/documentation/watchconnectivity/wcsession) -- the singleton session object for bidirectional communication between the iOS app and the watchOS app. +- [WCSessionDelegate](https://developer.apple.com/documentation/watchconnectivity/wcsessiondelegate) -- delegate protocol for receiving messages, application context, and file transfers. +- [WKApplicationRefreshBackgroundTask](https://developer.apple.com/documentation/watchkit/wkapplicationrefreshbackgroundtask) -- background task for scheduling periodic data refreshes on watchOS. +- [watchOS App Planning](https://developer.apple.com/watchos/planning/) -- Apple's overview of design and development considerations for watchOS apps. + +## WWDC History + +- **[WWDC 2019, Session 208 -- Creating Independent Watch Apps](https://developer.apple.com/videos/play/wwdc2019/208/)** -- introduced watchOS 6 independent apps that no longer require a companion iOS app, with their own App Store presence and direct network access. +- **[WWDC 2019, Session 219 -- SwiftUI on watchOS](https://asciiwwdc.com/2019/sessions/219)** -- first look at building watchOS interfaces with SwiftUI, replacing WKInterfaceController and storyboards. +- **[WWDC 2020, Session 10048 -- Build Complications in SwiftUI](https://developer.apple.com/videos/play/wwdc2020/10048/)** -- introduced SwiftUI-based ClockKit graphic complications for Meridian, Infograph, and other watch faces in watchOS 7. +- **[WWDC 2020, Session 10171 -- What's New in watchOS Design](https://developer.apple.com/videos/play/wwdc2020/10171/)** -- design principles for watchOS 7 apps, including the SwiftUI App lifecycle and notification handling. +- **[WWDC 2022, Session 10050 -- Complications and Widgets: Reloaded](https://developer.apple.com/videos/play/wwdc2022/10050/)** -- unified WidgetKit across iOS and watchOS 9, replacing ClockKit with four accessory widget families. +- **[WWDC 2022, Session 10051 -- Go Further with Complications in WidgetKit](https://developer.apple.com/videos/play/wwdc2022/10051/)** -- watchOS-specific WidgetKit features, rendering modes (full color, accented, vibrant), and migration from ClockKit. +- **[WWDC 2023, Session 10138 -- Design and Build Apps for watchOS 10](https://developer.apple.com/videos/play/wwdc2023/10138/)** -- major redesign of the watchOS UI paradigm with vertically pageable `TabView`, the Smart Stack, and new NavigationSplitView patterns. + +## What It Does + +1. **System launches the app.** When the user taps the app icon on the watch Home Screen or receives a complication tap, watchOS instantiates the `@main` App struct and renders the initial `WindowGroup` scene. +2. **SwiftUI drives the UI.** The app presents views using `NavigationStack`, `TabView`, and `List`. The Digital Crown controls scrolling. Haptic feedback is provided via `WKInterfaceDevice.current().play()`. +3. **Complications surface data.** The app's WidgetKit extension (watchOS 9+) or ClockKit data source (legacy) provides timeline entries that the system renders on watch faces as complications, giving users at-a-glance information. +4. **Watch Connectivity syncs with iPhone.** `WCSession` enables real-time messaging (`sendMessage`), background transfers (`transferUserInfo`, `transferFile`), and application context (`updateApplicationContext`) between the watch and the paired iPhone. +5. **Background refresh keeps data current.** The app schedules `WKApplicationRefreshBackgroundTask` to wake periodically (budget: approximately one task per hour for dock apps) and fetch updated data. +6. **Workout sessions track activity.** Using `HKWorkoutSession` and `HKLiveWorkoutBuilder`, the app starts, pauses, and ends workout sessions with real-time heart rate, calories, and distance data from HealthKit. +7. **Notifications arrive on the wrist.** The app registers for remote and local notifications. Custom notification views built with SwiftUI replace the default system presentation. + +## Use Cases + +### Fitness and Workout Tracking +A running app starts an `HKWorkoutSession`, displays real-time heart rate and pace via HealthKit queries, and uses extended runtime sessions to keep the screen active. A complication on the watch face shows the weekly distance goal progress. + +### Messaging and Communication +A chat app uses `WCSession.sendMessage` for real-time message relay when the iPhone is reachable, and falls back to `transferUserInfo` for queued delivery. Quick replies use the system text input controller with dictation, scribble, and emoji. + +### Smart Home Control +A home automation app presents a list of devices with toggle controls. Tapping a complication on the watch face immediately opens the "lights" scene. Background refresh updates device status every 15 minutes so complications always show current state. + +### Travel and Navigation +A transit app shows the next departure time as a complication using a relative date text style. The app uses CoreLocation for turn-by-turn haptic directions and `WKInterfaceDevice` haptics for left/right turn signals. + +## Key Classes + +| Class | Role | +|-------|------| +| `App` (SwiftUI) | The `@main` entry point for the watchOS app, declaring scenes via `WindowGroup`. | +| `NavigationStack` | Manages hierarchical navigation with push/pop semantics driven by the Digital Crown back gesture. | +| `TabView` | Provides paged or vertically scrollable tab navigation (watchOS 10+ uses vertical pages). | +| `WCSession` | Singleton for Watch Connectivity. Handles `sendMessage`, `updateApplicationContext`, `transferUserInfo`, and `transferFile`. | +| `WCSessionDelegate` | Delegate receiving messages, context updates, and file transfers from the companion iOS app. | +| `HKWorkoutSession` | Manages a workout session lifecycle (start, pause, resume, end) with automatic sensor activation. | +| `HKLiveWorkoutBuilder` | Collects real-time workout samples (heart rate, distance, calories) during an active session. | +| `WKApplicationRefreshBackgroundTask` | A background task the system delivers to refresh app data. Limited to ~1 per hour for dock apps. | +| `WKExtendedRuntimeSession` | Extends app runtime beyond normal limits for workouts, mindfulness, and physical therapy sessions. | +| `WidgetKit` (TimelineProvider) | Powers watch face complications on watchOS 9+ using the same WidgetKit API as iOS. | +| `CLLocationManager` | Provides location updates and authorization on Apple Watch. | +| `UNUserNotificationCenter` | Registers for and handles local and remote notifications on the watch. | + +## Implementation + +### Full watchOS App with Connectivity and Workout Session + +```swift +import SwiftUI +import HealthKit +import WatchConnectivity + +// 1. Define the main app entry point using the SwiftUI App lifecycle. +// This replaces the legacy WKExtensionDelegate/WKInterfaceController pattern. +@main +struct FitnessWatchApp: App { + @StateObject private var connectivityManager = ConnectivityManager() + @StateObject private var workoutManager = WorkoutManager() + + var body: some Scene { + WindowGroup { + // 2. Use TabView for top-level navigation. On watchOS 10+, + // this renders as vertically pageable sections. + TabView { + DashboardView() + .environmentObject(workoutManager) + .tag("dashboard") + + WorkoutView() + .environmentObject(workoutManager) + .tag("workout") + + SettingsView() + .environmentObject(connectivityManager) + .tag("settings") + } + .tabViewStyle(.verticalPage) + } + } +} + +// 3. ConnectivityManager wraps WCSession for bidirectional phone<->watch +// communication. It must conform to WCSessionDelegate. +class ConnectivityManager: NSObject, ObservableObject, WCSessionDelegate { + @Published var lastReceivedMessage: String = "" + @Published var isReachable = false + + private var session: WCSession? + + override init() { + super.init() + // 4. Check if Watch Connectivity is supported on this device. + // It is always supported on Apple Watch but only supported on + // iPhone when paired with a watch. + if WCSession.isSupported() { + session = WCSession.default + session?.delegate = self + session?.activate() + } + } + + // 5. Required delegate method: called when activation completes. + func session( + _ session: WCSession, + activationDidCompleteWith activationState: WCSessionActivationState, + error: Error? + ) { + DispatchQueue.main.async { + self.isReachable = session.isReachable + } + } + + // 6. Receive real-time messages from the iPhone app. + // sendMessage only works when both apps are reachable. + func session( + _ session: WCSession, + didReceiveMessage message: [String: Any] + ) { + DispatchQueue.main.async { + if let text = message["text"] as? String { + self.lastReceivedMessage = text + } + } + } + + // 7. Receive application context updates. Unlike sendMessage, + // application context is delivered even if the counterpart + // app was not running -- the system caches the latest context. + func session( + _ session: WCSession, + didReceiveApplicationContext applicationContext: [String: Any] + ) { + DispatchQueue.main.async { + if let text = applicationContext["status"] as? String { + self.lastReceivedMessage = text + } + } + } + + // 8. Send a message to the iPhone. The reply handler runs on a + // background thread; dispatch to main for UI updates. + func sendMessageToPhone(_ data: [String: Any]) { + guard let session, session.isReachable else { + // 9. Fallback: use transferUserInfo for queued delivery + // when the phone is not immediately reachable. + session?.transferUserInfo(data) + return + } + + session.sendMessage(data, replyHandler: { reply in + DispatchQueue.main.async { + if let response = reply["response"] as? String { + self.lastReceivedMessage = response + } + } + }, errorHandler: { error in + print("Send failed: \(error.localizedDescription)") + }) + } +} + +// 10. WorkoutManager handles HealthKit workout sessions with real-time +// heart rate, calorie, and distance tracking. +class WorkoutManager: NSObject, ObservableObject, HKWorkoutSessionDelegate, + HKLiveWorkoutBuilderDelegate +{ + @Published var isWorkoutActive = false + @Published var heartRate: Double = 0 + @Published var activeCalories: Double = 0 + @Published var distance: Double = 0 + @Published var elapsedTime: TimeInterval = 0 + + private let healthStore = HKHealthStore() + private var workoutSession: HKWorkoutSession? + private var workoutBuilder: HKLiveWorkoutBuilder? + + // 11. Request HealthKit authorization for the data types we need. + func requestAuthorization() { + let typesToShare: Set = [ + HKQuantityType.workoutType() + ] + let typesToRead: Set = [ + HKQuantityType(.heartRate), + HKQuantityType(.activeEnergyBurned), + HKQuantityType(.distanceWalkingRunning) + ] + + healthStore.requestAuthorization( + toShare: typesToShare, + read: typesToRead + ) { success, error in + if let error { + print("HealthKit auth failed: \(error.localizedDescription)") + } + } + } + + // 12. Start a workout session with a specific activity type. + func startWorkout(activityType: HKWorkoutActivityType = .running) { + let configuration = HKWorkoutConfiguration() + configuration.activityType = activityType + configuration.locationType = .outdoor + + do { + workoutSession = try HKWorkoutSession( + healthStore: healthStore, + configuration: configuration + ) + workoutBuilder = workoutSession?.associatedWorkoutBuilder() + + workoutSession?.delegate = self + workoutBuilder?.delegate = self + + // 13. Set the data source to collect live samples from + // Apple Watch sensors automatically. + workoutBuilder?.dataSource = HKLiveWorkoutDataSource( + healthStore: healthStore, + workoutConfiguration: configuration + ) + + let startDate = Date() + workoutSession?.startActivity(with: startDate) + workoutBuilder?.beginCollection(withStart: startDate) { _, _ in } + + DispatchQueue.main.async { + self.isWorkoutActive = true + } + } catch { + print("Failed to start workout: \(error.localizedDescription)") + } + } + + // 14. End the workout and save it to HealthKit. + func endWorkout() { + workoutSession?.end() + DispatchQueue.main.async { + self.isWorkoutActive = false + } + } + + // MARK: - HKWorkoutSessionDelegate + + func workoutSession( + _ workoutSession: HKWorkoutSession, + didChangeTo toState: HKWorkoutSessionState, + from fromState: HKWorkoutSessionState, + date: Date + ) { + if toState == .ended { + // 15. Finalize and save the workout when the session ends. + workoutBuilder?.endCollection(withEnd: date) { _, _ in + self.workoutBuilder?.finishWorkout { workout, error in + if let error { + print("Failed to save workout: \(error.localizedDescription)") + } + } + } + } + } + + func workoutSession( + _ workoutSession: HKWorkoutSession, + didFailWithError error: Error + ) { + print("Workout session error: \(error.localizedDescription)") + } + + // MARK: - HKLiveWorkoutBuilderDelegate + + func workoutBuilderDidCollectEvent( + _ workoutBuilder: HKLiveWorkoutBuilder + ) {} + + func workoutBuilder( + _ workoutBuilder: HKLiveWorkoutBuilder, + didCollectDataOf collectedTypes: Set + ) { + // 16. Update published properties with the latest workout statistics + // so SwiftUI views refresh automatically. + for type in collectedTypes { + guard let quantityType = type as? HKQuantityType, + let statistics = workoutBuilder.statistics(for: quantityType) + else { continue } + + DispatchQueue.main.async { + switch quantityType { + case HKQuantityType(.heartRate): + let bpm = statistics.mostRecentQuantity()? + .doubleValue(for: .count().unitDivided(by: .minute())) + self.heartRate = bpm ?? 0 + + case HKQuantityType(.activeEnergyBurned): + let cal = statistics.sumQuantity()? + .doubleValue(for: .kilocalorie()) + self.activeCalories = cal ?? 0 + + case HKQuantityType(.distanceWalkingRunning): + let meters = statistics.sumQuantity()? + .doubleValue(for: .meter()) + self.distance = (meters ?? 0) / 1000.0 // Convert to km + + default: + break + } + } + } + } +} + +// 17. Dashboard view showing connectivity status and recent data. +struct DashboardView: View { + @EnvironmentObject var workoutManager: WorkoutManager + + var body: some View { + NavigationStack { + List { + Section("Today") { + HStack { + Image(systemName: "flame.fill") + .foregroundStyle(.orange) + Text("\(Int(workoutManager.activeCalories)) kcal") + } + HStack { + Image(systemName: "figure.run") + .foregroundStyle(.green) + Text(String(format: "%.1f km", workoutManager.distance)) + } + } + } + .navigationTitle("Fitness") + } + } +} + +// 18. Workout view with start/stop controls and live metrics. +struct WorkoutView: View { + @EnvironmentObject var workoutManager: WorkoutManager + + var body: some View { + NavigationStack { + VStack(spacing: 16) { + if workoutManager.isWorkoutActive { + // 19. Live workout metrics update in real time via + // the @Published properties on WorkoutManager. + VStack(spacing: 12) { + MetricView( + icon: "heart.fill", + value: "\(Int(workoutManager.heartRate))", + unit: "BPM", + color: .red + ) + MetricView( + icon: "flame.fill", + value: "\(Int(workoutManager.activeCalories))", + unit: "kcal", + color: .orange + ) + MetricView( + icon: "figure.run", + value: String(format: "%.2f", workoutManager.distance), + unit: "km", + color: .green + ) + } + + Button("End Workout", role: .destructive) { + workoutManager.endWorkout() + } + } else { + Button { + workoutManager.requestAuthorization() + workoutManager.startWorkout() + } label: { + Label("Start Run", systemImage: "figure.run") + } + .buttonStyle(.borderedProminent) + .tint(.green) + } + } + .navigationTitle("Workout") + } + } +} + +struct MetricView: View { + let icon: String + let value: String + let unit: String + let color: Color + + var body: some View { + HStack { + Image(systemName: icon) + .foregroundStyle(color) + .frame(width: 24) + Text(value) + .font(.title2.monospacedDigit().bold()) + Text(unit) + .font(.caption) + .foregroundStyle(.secondary) + Spacer() + } + } +} + +struct SettingsView: View { + @EnvironmentObject var connectivityManager: ConnectivityManager + + var body: some View { + NavigationStack { + List { + Section("Phone Connection") { + HStack { + Circle() + .fill(connectivityManager.isReachable ? .green : .red) + .frame(width: 8, height: 8) + Text( + connectivityManager.isReachable + ? "Connected" : "Not Reachable" + ) + } + } + + Section("Sync") { + Button("Send Status to Phone") { + connectivityManager.sendMessageToPhone([ + "status": "active", + "heartRate": 72, + ]) + } + } + } + .navigationTitle("Settings") + } + } +} +``` + +## Usage + +- Create: `bun create target watch` +- Edit: `targets/watch/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| watchOS | 7.0+ | SwiftUI App lifecycle (`@main`). Storyboards deprecated. Independent app distribution since watchOS 6. | +| watchOS | 9.0+ | WidgetKit complications replace ClockKit. Four accessory families: circular, rectangular, inline, corner. | +| watchOS | 10.0+ | Redesigned UI with vertical `TabView` paging, Smart Stack, and new NavigationSplitView patterns. | +| iOS | -- | The companion iOS app is optional. If present, use WatchConnectivity for data sync. | + +## Gotchas + +- **WatchKit storyboards are deprecated since watchOS 7.** All new watchOS apps must use the SwiftUI `@main` App lifecycle. The legacy `WKExtensionDelegate` and `WKInterfaceController` APIs still work but receive no new features. +- **WCSession requires both a delegate and activation.** Activating a `WCSession` without setting a delegate first is a programming error. Always set the delegate before calling `activate()`, and do so early in the app lifecycle. +- **`sendMessage` requires reachability.** Real-time messaging only works when both the watch and phone apps are active (the phone app can be woken in the background). Always check `session.isReachable` and fall back to `transferUserInfo` or `updateApplicationContext` for non-urgent data. +- **Background refresh budget is limited.** Dock apps get approximately one `WKApplicationRefreshBackgroundTask` per hour. Apps with an active complication on the watch face get a higher budget. Apps not in the dock or on a watch face receive no regular background time. +- **Background tasks have strict time limits.** A `WKApplicationRefreshBackgroundTask` gives your app 4 seconds of CPU time and 15 seconds of wall-clock time. Use `WKURLSessionRefreshBackgroundTask` with background `URLSession` download tasks for network requests -- the system suspends your app during the download and re-wakes it when data arrives. +- **File protection blocks background access.** Files created with the default complete protection level are inaccessible when the watch screen is locked. Background tasks run while locked, so any files you need to read or write during a background task must use `.noFileProtection` or `.completeUntilFirstUserAuthentication`. +- **HealthKit authorization is per-device.** The user must grant HealthKit permissions directly on the Apple Watch. You cannot request HealthKit authorization from the iPhone on behalf of the watch app. +- **ClockKit is fully replaced by WidgetKit on watchOS 9+.** New complications must use WidgetKit's `TimelineProvider` and accessory widget families. ClockKit complications still work on older watch faces but cannot be added to new faces. +- **Only `plist`-encodable types in WCSession messages.** The `[String: Any]` dictionaries sent via `sendMessage` and `transferUserInfo` only support property list types (String, Int, Double, Data, Array, Dictionary, Date, Bool). Encode custom types to `Data` before sending. +- **The watchOS Simulator does not support all sensors.** Heart rate, accelerometer, and gyroscope data are unavailable in the Simulator. Test workout and motion features on a physical Apple Watch. +- **Independent apps still need an iOS companion for some features.** While watchOS apps can be fully independent, features like `WCSession` communication, handoff, and some CloudKit sync patterns require the paired iPhone app to be installed. diff --git a/plugins/expo/skills/apple-targets/widget.md b/plugins/expo/skills/apple-targets/widget.md new file mode 100644 index 0000000..64d8525 --- /dev/null +++ b/plugins/expo/skills/apple-targets/widget.md @@ -0,0 +1,536 @@ +--- +title: WidgetKit Home Screen Widget +description: Displays glanceable, read-only (or interactive on iOS 17+) views on the Home Screen, Lock Screen, StandBy, and Desktop, powered by timeline-based data and SwiftUI. +version: iOS 14.0+, macOS 11.0+ +--- + +# WidgetKit Home Screen Widget (`widget`) + +A widget extension renders small SwiftUI views directly on the Home Screen, Lock Screen, StandBy mode, and macOS Desktop. Widgets refresh on a system-managed timeline, and starting with iOS 17 they can contain interactive controls (buttons and toggles) via App Intents. The same extension can also power Live Activities on the Lock Screen and Dynamic Island, as well as Control Center controls (iOS 18+). + +## Apple Documentation + +- [WidgetKit Framework Overview](https://developer.apple.com/documentation/widgetkit) +- [Creating a Widget Extension](https://developer.apple.com/documentation/widgetkit/creating-a-widget-extension) +- [Widget Protocol (SwiftUI)](https://developer.apple.com/documentation/swiftui/widget) +- [TimelineProvider](https://developer.apple.com/documentation/widgetkit/timelineprovider) +- [AppIntentTimelineProvider](https://developer.apple.com/documentation/widgetkit/appintenttimelineprovider) +- [WidgetFamily](https://developer.apple.com/documentation/widgetkit/widgetfamily) +- [ActivityKit (Live Activities)](https://developer.apple.com/documentation/activitykit) +- [WidgetKit Widgets Overview (Apple landing page)](https://developer.apple.com/widgets/) +- [Keeping a Widget Up to Date](https://developer.apple.com/documentation/widgetkit/keeping-a-widget-up-to-date) + +## WWDC History + +- **[WWDC 2020, Session 10028 -- Meet WidgetKit](https://developer.apple.com/videos/play/wwdc2020/10028/)** -- Introduced WidgetKit with timeline providers, widget families (small/medium/large), and SwiftUI-only rendering. +- **[WWDC 2020, Session 10033 -- Build SwiftUI Views for Widgets](https://developer.apple.com/videos/play/wwdc2020/10033/)** -- Deep dive into SwiftUI view techniques specific to widgets. +- **[WWDC 2020, Sessions 10034-10036 -- Widgets Code-Along (Parts 1-3)](https://developer.apple.com/videos/play/wwdc2020/10034/)** -- Three-part hands-on series covering widget families, timelines, URL sessions, and widget bundles. +- **[WWDC 2021, Session 10048 -- Principles of Great Widgets](https://developer.apple.com/videos/play/wwdc2021/10048/)** -- Design guidance for effective widget experiences. +- **[WWDC 2022, Session 10050 -- Complications and Widgets: Reloaded](https://developer.apple.com/videos/play/wwdc2022/10050/)** -- Added Lock Screen widget families (`accessoryCircular`, `accessoryRectangular`, `accessoryInline`) and unified WidgetKit across iOS and watchOS. +- **[WWDC 2022, Session 10051 -- Go Further with Complications in WidgetKit](https://developer.apple.com/videos/play/wwdc2022/10051/)** -- watchOS-specific WidgetKit features and ClockKit migration. +- **[WWDC 2023, Session 10027 -- Bring Widgets to New Places](https://developer.apple.com/videos/play/wwdc2023/10027/)** -- Widgets on Mac Desktop, iPad Lock Screen, StandBy mode, and Apple Watch Smart Stack. +- **[WWDC 2023, Session 10028 -- Bring Widgets to Life](https://developer.apple.com/videos/play/wwdc2023/10028/)** -- Interactive widgets with App Intents (Button and Toggle), animated transitions. +- **[WWDC 2023, Session 10184 -- Meet ActivityKit](https://developer.apple.com/videos/play/wwdc2023/10184/)** -- Live Activities on Lock Screen and Dynamic Island using ActivityKit. +- **[WWDC 2024, Session 10157 -- Extend Your App's Controls Across the System](https://developer.apple.com/videos/play/wwdc2024/10157/)** -- Control Center controls built with WidgetKit (iOS 18+). +- **[WWDC 2025, Session 278 -- What's New in Widgets](https://developer.apple.com/videos/play/wwdc2025/278/)** -- Widget push updates, glass presentation on iOS 26, visionOS pinned widgets. + +## What It Does + +1. **System requests a timeline.** WidgetKit calls your `TimelineProvider.getTimeline(in:completion:)` (or the `AppIntentTimelineProvider` equivalent) to get an array of dated entries. +2. **Entries are rendered as SwiftUI views.** Each `TimelineEntry` is passed to your widget's `body` view closure. The system snapshots the SwiftUI tree into a static render (pre-iOS 17) or an interactive render (iOS 17+). +3. **The system manages refresh.** Based on your `TimelineReloadPolicy` (`.atEnd`, `.after(date)`, `.never`) the system schedules the next call to your provider. You get a limited daily budget of refreshes. +4. **In-app triggers.** Your main app can call `WidgetCenter.shared.reloadTimelines(ofKind:)` to force a reload after data changes. +5. **Push-driven updates.** A server can send a WidgetKit push notification via APNs to trigger a reload without the app running. +6. **Taps deep-link.** A tap on the widget opens the main app. You control the destination via `.widgetURL()` or `Link` views. +7. **Interactive controls (iOS 17+).** Buttons and Toggles bound to an `AppIntent` execute logic directly in the widget extension process -- no app launch required. + +## Use Cases + +### Weather and forecasts +A medium-sized widget showing current temperature, conditions, and a 5-hour forecast strip. The timeline provider generates entries at hourly intervals, with a network request in `getTimeline` to fetch fresh data. + +### Task and habit trackers +An interactive widget with a checklist. Each row has a Toggle bound to a `ToggleTaskIntent` (AppIntent) so users can mark items complete directly from the Home Screen without opening the app. + +### Sports scores and live events +A Live Activity on the Lock Screen and Dynamic Island showing the current score, game clock, and team logos. The app starts the activity at tip-off and updates it via APNs push or in-app `Activity.update()` calls. + +### Music and media playback +A small widget showing the currently playing album art and track name. An interactive Play/Pause button uses an AppIntent to control playback via the shared audio session. + +## Key Classes + +| Class / Protocol | Role | +|------------------|------| +| `TimelineProvider` | Supplies the widget with a timeline of entries. Implement `placeholder`, `getSnapshot`, and `getTimeline`. | +| `AppIntentTimelineProvider` | iOS 17+ replacement that uses App Intents for user-configurable widgets (replaces `IntentTimelineProvider`). | +| `TimelineEntry` | A single dated data point. Must contain a `date` property; add any custom fields your view needs. | +| `WidgetConfiguration` | Declares the widget. Use `StaticConfiguration` (no config) or `AppIntentConfiguration` (user-configurable). | +| `WidgetFamily` | Enum of supported sizes: `.systemSmall`, `.systemMedium`, `.systemLarge`, `.systemExtraLarge`, `.accessoryCircular`, `.accessoryRectangular`, `.accessoryInline`. | +| `WidgetBundle` | Groups multiple widgets from the same extension into a single target. | +| `WidgetCenter` | Lets the main app reload timelines and query current widget configurations. | +| `ActivityAttributes` | Defines the static and dynamic data schema for a Live Activity. | +| `ActivityConfiguration` | Declares the Live Activity UI (Lock Screen banner + Dynamic Island). | +| `Activity` | Manages the lifecycle of a Live Activity instance (request, update, end). | + +## Implementation + +### Timeline-Based Widget with Interactive Controls (iOS 17+) + +```swift +import WidgetKit +import SwiftUI +import AppIntents + +// 1. Define the data each timeline entry carries. +struct TaskEntry: TimelineEntry { + let date: Date + let tasks: [TaskItem] +} + +struct TaskItem: Identifiable, Codable { + let id: String + var title: String + var isComplete: Bool +} + +// 2. Build a timeline provider that reads from shared storage. +struct TaskTimelineProvider: TimelineProvider { + func placeholder(in context: Context) -> TaskEntry { + TaskEntry(date: .now, tasks: [ + TaskItem(id: "1", title: "Buy groceries", isComplete: false) + ]) + } + + func getSnapshot(in context: Context, completion: @escaping (TaskEntry) -> Void) { + completion(placeholder(in: context)) + } + + func getTimeline(in context: Context, completion: @escaping (Timeline) -> Void) { + // 3. Read tasks from an App Group shared container. + let tasks = loadTasksFromAppGroup() + let entry = TaskEntry(date: .now, tasks: tasks) + + // 4. Reload at the top of the next hour, or rely on in-app triggers. + let nextUpdate = Calendar.current.date(byAdding: .hour, value: 1, to: .now)! + let timeline = Timeline(entries: [entry], policy: .after(nextUpdate)) + completion(timeline) + } + + private func loadTasksFromAppGroup() -> [TaskItem] { + guard let url = FileManager.default + .containerURL(forSecurityApplicationGroupIdentifier: "group.com.example.myapp")? + .appendingPathComponent("tasks.json"), + let data = try? Data(contentsOf: url), + let tasks = try? JSONDecoder().decode([TaskItem].self, from: data) + else { return [] } + return tasks + } +} + +// 5. Define an App Intent for toggling a task (iOS 17+). +struct ToggleTaskIntent: AppIntent { + static var title: LocalizedStringResource = "Toggle Task" + + @Parameter(title: "Task ID") + var taskId: String + + init() {} + init(taskId: String) { self.taskId = taskId } + + func perform() async throws -> some IntentResult { + // 6. Mutate shared data and tell WidgetKit to refresh. + var tasks = SharedTaskStore.load() + if let idx = tasks.firstIndex(where: { $0.id == taskId }) { + tasks[idx].isComplete.toggle() + SharedTaskStore.save(tasks) + } + return .result() + } +} + +// 7. Build the SwiftUI view. +struct TaskWidgetEntryView: View { + var entry: TaskEntry + @Environment(\.widgetFamily) var family + + var body: some View { + switch family { + case .systemSmall: + // Small: show count only + VStack { + Text("\(entry.tasks.filter { !$0.isComplete }.count)") + .font(.system(size: 48, weight: .bold)) + Text("tasks left") + .font(.caption) + .foregroundStyle(.secondary) + } + .containerBackground(.fill.tertiary, for: .widget) + + case .systemMedium: + // Medium: show interactive checklist + VStack(alignment: .leading, spacing: 6) { + ForEach(entry.tasks.prefix(4)) { task in + // 8. Toggle bound to an AppIntent -- executes without launching the app. + Toggle(isOn: task.isComplete, intent: ToggleTaskIntent(taskId: task.id)) { + Text(task.title) + .strikethrough(task.isComplete) + } + .toggleStyle(.checkbox) + } + } + .padding() + .containerBackground(.fill.tertiary, for: .widget) + + default: + Text("Unsupported") + } + } +} + +// 9. Declare the widget configuration. +struct TaskWidget: Widget { + let kind = "TaskWidget" + + var body: some WidgetConfiguration { + StaticConfiguration(kind: kind, provider: TaskTimelineProvider()) { entry in + TaskWidgetEntryView(entry: entry) + } + .configurationDisplayName("Tasks") + .description("Track your to-do list at a glance.") + .supportedFamilies([.systemSmall, .systemMedium]) + } +} +``` + +### Live Activity (Delivery Tracker) + +```swift +import ActivityKit +import WidgetKit +import SwiftUI + +// 1. Define attributes: static data + dynamic ContentState. +struct DeliveryAttributes: ActivityAttributes { + let orderNumber: String + let restaurantName: String + + struct ContentState: Codable, Hashable { + var status: String // e.g. "Preparing", "On the way", "Delivered" + var estimatedArrival: Date + var driverName: String + } +} + +// 2. Build the Live Activity widget. +struct DeliveryLiveActivity: Widget { + var body: some WidgetConfiguration { + ActivityConfiguration(for: DeliveryAttributes.self) { context in + // Lock Screen / banner presentation + HStack { + VStack(alignment: .leading) { + Text(context.state.status) + .font(.headline) + Text("ETA: \(context.state.estimatedArrival, style: .timer)") + .font(.subheadline) + .foregroundStyle(.secondary) + } + Spacer() + Image(systemName: "bicycle") + .font(.title) + } + .padding() + .activityBackgroundTint(.cyan.opacity(0.2)) + } dynamicIsland: { context in + // 3. Dynamic Island presentations. + DynamicIsland { + DynamicIslandExpandedRegion(.leading) { + Image(systemName: "bicycle") + } + DynamicIslandExpandedRegion(.center) { + Text(context.state.status) + .font(.headline) + } + DynamicIslandExpandedRegion(.trailing) { + Text(context.state.estimatedArrival, style: .timer) + .font(.caption) + } + DynamicIslandExpandedRegion(.bottom) { + Text("Driver: \(context.state.driverName)") + } + } compactLeading: { + Image(systemName: "bicycle") + } compactTrailing: { + Text(context.state.estimatedArrival, style: .timer) + } minimal: { + Image(systemName: "bicycle") + } + } + } +} + +// 4. Start, update, and end from the main app. +func startDeliveryActivity(orderNumber: String, restaurant: String) throws { + let attributes = DeliveryAttributes( + orderNumber: orderNumber, + restaurantName: restaurant + ) + let initialState = DeliveryAttributes.ContentState( + status: "Preparing", + estimatedArrival: Date().addingTimeInterval(30 * 60), + driverName: "Alex" + ) + let content = ActivityContent(state: initialState, staleDate: nil) + let _ = try Activity.request( + attributes: attributes, + content: content, + pushType: .token // enables APNs updates + ) +} + +func updateDeliveryActivity(activity: Activity, newStatus: String) async { + let updatedState = DeliveryAttributes.ContentState( + status: newStatus, + estimatedArrival: Date().addingTimeInterval(10 * 60), + driverName: "Alex" + ) + await activity.update(ActivityContent(state: updatedState, staleDate: nil)) +} + +func endDeliveryActivity(activity: Activity) async { + let finalState = DeliveryAttributes.ContentState( + status: "Delivered", + estimatedArrival: .now, + driverName: "Alex" + ) + await activity.end( + ActivityContent(state: finalState, staleDate: nil), + dismissalPolicy: .after(.now.addingTimeInterval(4 * 3600)) + ) +} +``` + +### Lock Screen Accessory Widget + +```swift +import WidgetKit +import SwiftUI + +// 1. Reuse the same provider; declare a separate widget for Lock Screen families. +struct TaskAccessoryWidget: Widget { + let kind = "TaskAccessoryWidget" + + var body: some WidgetConfiguration { + StaticConfiguration(kind: kind, provider: TaskTimelineProvider()) { entry in + // 2. Accessory widgets use a monochrome, compact layout. + let remaining = entry.tasks.filter { !$0.isComplete }.count + ZStack { + AccessoryWidgetBackground() + VStack { + Image(systemName: "checklist") + Text("\(remaining)") + .font(.title2.bold()) + } + } + .widgetLabel("\(remaining) left") + } + .configurationDisplayName("Task Count") + .description("Remaining tasks on your Lock Screen.") + .supportedFamilies([.accessoryCircular, .accessoryRectangular, .accessoryInline]) + } +} +``` + +### Control Center Control Widget (iOS 18+) + +Control Center controls are declared using `ControlWidget` and backed by App Intents. They appear in the customizable Control Center grid, Lock Screen, and Action Button. Each control requires an SF Symbol for its icon. + +```swift +import WidgetKit +import SwiftUI +import AppIntents + +// 1. Define the App Intent that the control executes. +struct ToggleLightsIntent: SetValueIntent { + static var title: LocalizedStringResource = "Toggle Lights" + + @Parameter(title: "Light State") + var value: Bool + + func perform() async throws -> some IntentResult { + LightManager.shared.setLights(on: value) + return .result() + } +} + +// 2. Define a value provider that supplies the current state. +struct LightsValueProvider: ControlWidgetToggleValueProvider { + func currentValue() async throws -> Bool { + LightManager.shared.areLightsOn + } + + // 3. Provide a preview value for the Control Center gallery. + let previewValue: Bool = false +} + +// 4. Declare the control widget with an SF Symbol icon. +struct LightsControl: ControlWidget { + var body: some ControlWidgetConfiguration { + ControlWidgetToggle( + "Living Room", + isOn: LightsValueProvider(), + action: ToggleLightsIntent() + ) { isOn in + Label(isOn ? "On" : "Off", systemImage: isOn ? "lightbulb.fill" : "lightbulb") + } + .displayName("Living Room Lights") + .description("Toggle your living room lights.") + } +} + +// 5. Button-style control (non-toggle). +struct CaffeinateControl: ControlWidget { + var body: some ControlWidgetConfiguration { + ControlWidgetButton(action: LogCaffeineIntent()) { + // SF Symbol used as the control icon + Label("Log Caffeine", systemImage: "cup.and.saucer.fill") + } + .displayName("Log Caffeine") + .description("Quickly log a caffeine intake.") + } +} +``` + +### Control Widget Deep Link (iOS 18+) + +The most common control widget use case is opening a deep link or URL in the main app. Use `ControlConfigurationIntent` with `openAppWhenRun = true` and return an `OpenURLIntent` to navigate to a specific screen. + +```swift +import WidgetKit +import SwiftUI +import AppIntents + +// 1. Declare the control widget with a button that opens the app. +@available(iOS 18.0, *) +struct DeepLinkControl: ControlWidget { + static let kind = "com.example.myapp.deeplink" + + var body: some ControlWidgetConfiguration { + StaticControlConfiguration(kind: Self.kind) { + ControlWidgetButton(action: OpenAppIntent()) { + Label("Settings", systemImage: "gear") + } + } + .displayName("Open Settings") + .description("Jump straight to app settings.") + } +} + +// 2. Define the intent that opens a URL when the control is tapped. +// IMPORTANT: This intent must be compiled into BOTH the widget extension +// target AND the main app target when openAppWhenRun = true. +// See: https://developer.apple.com/forums/thread/763851 +@available(iOS 18.0, *) +struct OpenAppIntent: ControlConfigurationIntent { + static let title: LocalizedStringResource = "Launch App" + static let description = IntentDescription(stringLiteral: "Open the app to a specific screen.") + static let isDiscoverable = false + static let openAppWhenRun: Bool = true + + @MainActor + func perform() async throws -> some IntentResult & OpensIntent { + // 3. Return an OpenURLIntent with your deep link URL. + // This can be a universal link or a custom URL scheme. + return .result(opensIntent: OpenURLIntent(URL(string: "myapp://settings")!)) + } +} +``` + +> **Key detail:** When `openAppWhenRun = true`, the intent struct must be compiled into both the widget extension target and the main app target. If it only exists in the extension, the app will not launch correctly. + +### SF Symbols for Icons + +Widgets, controls, and App Shortcuts use **SF Symbols** for their icons. SF Symbols is Apple's library of thousands of vector icons designed for system integration. + +**Using built-in SF Symbols:** + +```swift +// In widgets and controls +Image(systemName: "heart.fill") +Label("Favorites", systemImage: "star.fill") + +// In App Shortcuts +AppShortcut( + intent: MyIntent(), + phrases: ["Open \(.applicationName)"], + shortTitle: "Open", + systemImageName: "arrow.up.right" // SF Symbol name +) +``` + +**Creating custom SF Symbols from SVG:** + +Custom SF Symbols let you use your own icons that match the weight, scale, and design language of system symbols. Create one from an SVG file or URL: + +```sh +bun create symbol +``` + +Popular brands have SVGs on https://simpleicons.org, e.g. https://simpleicons.org/icons/ferrari.svg + +This generates a `.symbol` file that can be used with `Image("customSymbolName")` instead of `Image(systemName:)`. Custom symbols automatically support all SF Symbol rendering modes (monochrome, hierarchical, palette, multicolor) and adapt to dynamic type sizes. + +**Symbol rendering modes in widgets:** + +```swift +// Monochrome (default) -- single color, respects tint +Image(systemName: "heart.fill") + +// Hierarchical -- primary color with automatic opacity layers +Image(systemName: "heart.circle.fill") + .symbolRenderingMode(.hierarchical) + +// Palette -- explicit multi-color +Image(systemName: "heart.circle.fill") + .symbolRenderingMode(.palette) + .foregroundStyle(.red, .gray) + +// Multicolor -- Apple's predefined colors (not all symbols support this) +Image(systemName: "heart.circle.fill") + .symbolRenderingMode(.multicolor) +``` + +> **Note:** Lock Screen accessory widgets only support monochrome rendering. Control Center controls use monochrome by default and are tinted by the system. + +## Usage + +- Create: `bun create target widget` +- Edit: `targets/widget/` +- This will require a native rebuild. +## Platform Availability + +| Platform | Minimum OS | Notes | +|----------|-----------|-------| +| iOS | 14.0+ | Home Screen widgets. Lock Screen in 16.0+. Interactive in 17.0+. Controls in 18.0+. | +| iPadOS | 14.0+ | Home Screen widgets. Lock Screen in 17.0+. | +| macOS | 11.0+ | Notification Center in 11.0+. Desktop in 14.0+. | +| watchOS | 9.0+ | Complications via WidgetKit (replaces ClockKit). Smart Stack in 10.0+. | +| visionOS | 1.0+ | Supported. Can be pinned to surfaces in visionOS 2.0+. | +| tvOS | -- | Not supported. | + +## Gotchas + +- **Timeline budget is limited.** The system grants approximately 40-70 reloads per day depending on how often the widget is viewed. Calling `reloadTimelines(ofKind:)` from your app is not rate-limited but still consumes from the same budget. Design your reload policy accordingly. +- **Views must be pure SwiftUI.** UIKit views, web views, maps, and cameras are not available in widget extensions. Use SwiftUI equivalents only. +- **Only Button and Toggle are interactive.** iOS 17 interactive widgets support only `Button` and `Toggle` initialized with an `AppIntent`. Text fields, sliders, pickers, and other controls are not supported. +- **App Group is required for shared data.** The widget extension runs in a separate process. You must use an App Group container (`FileManager.containerURL(forSecurityApplicationGroupIdentifier:)` or `UserDefaults(suiteName:)`) to share data between your app and the widget. +- **`containerBackground` is required on iOS 17+.** Starting in iOS 17, every widget view must include a `.containerBackground(for: .widget)` modifier. Without it, the system shows a default background and logs a warning. +- **Network requests in the provider run on a short timer.** If your `getTimeline` makes network requests, they must complete quickly (under ~30 seconds). For longer operations, use background URLSessions and trigger timeline reloads when data arrives. +- **Live Activities require Info.plist key.** You must add `NSSupportsLiveActivities = YES` in your app's (not the extension's) Info.plist, or Live Activities silently fail to start. +- **Live Activity size limit.** Each ActivityKit update payload must be under 4 KB. The Live Activity itself can remain on screen for up to 8 hours on the Dynamic Island and 12 hours on the Lock Screen. +- **Lock Screen widgets use a limited color palette.** Accessory family widgets render in three modes: `vibrant` (iOS), `accented` (watchOS), and `fullColor` (watchOS only). Do not rely on custom colors in accessory widgets on iOS -- they will be tinted by the system. +- **Widget previews require Xcode 15+.** The `#Preview` macro for widgets was introduced in Xcode 15. Older preview syntax using `PreviewProvider` is still supported but deprecated. +- **`systemExtraLarge` is iPad-only.** The `.systemExtraLarge` widget family is only available on iPadOS 15+ and is not offered on iPhone. +- **Control widgets require iOS 18+.** `ControlWidget`, `ControlWidgetToggle`, and `ControlWidgetButton` are only available on iOS 18+. They must be backed by App Intents (`SetValueIntent` for toggles, `AppIntent` for buttons) and require an SF Symbol for their icon -- custom images are not supported. +- **`openAppWhenRun` intents must be in both targets.** When a `ControlConfigurationIntent` has `openAppWhenRun = true`, the intent struct must be compiled into both the widget extension and the main app target. If it only exists in the extension, the app will not launch. Place shared intents in a framework or add the file to both targets. +- **Control widget icons must be SF Symbols.** The `Label` in a `ControlWidgetToggle` or `ControlWidgetButton` must use an SF Symbol -- either a built-in system symbol via `systemImage:` or a custom SF Symbol (`.symbol` file) created from an SVG. Standard asset catalog images, UIImage, and remote images are not supported in control widgets. diff --git a/plugins/expo/skills/release-app/SKILL.md b/plugins/expo/skills/release-app/SKILL.md new file mode 100644 index 0000000..0602cfb --- /dev/null +++ b/plugins/expo/skills/release-app/SKILL.md @@ -0,0 +1,101 @@ +--- +name: release-app +description: Step-by-step guide for first-time developers to release an Expo app on the iOS App Store and Google Play. Walks through monetization, App Store metadata and ASO, privacy policy generation and hosting, a pre-submission rejection scan, EAS Build, EAS Submit, and post-launch marketing. Use when the user wants to launch, release, publish, ship, or submit their app to the app stores, or asks about app store metadata, screenshots, privacy policies, or store rejection reasons. +version: 1.0.0 +license: MIT +--- + +# Expo App Release Checklist + +A guided, end-to-end checklist that walks a first-time developer from a finished Expo app to a live listing on the **Apple App Store** and **Google Play**. Be encouraging and specific — first-time submitters are often overwhelmed, so give exact commands, character counts, and examples, and celebrate each completed phase. + +For the underlying EAS mechanics (build profiles, `eas.json`, credentials, web hosting), the `expo-deployment` skill is the command reference. This skill is the guided launch journey that wraps those mechanics with the non-technical steps — metadata, privacy, rejection scan, and marketing. + +## The Journey + +Introduce the journey when the skill activates, then work through the phases in order (let the user skip ahead if they ask): + +1. **Understand the app** — gather the basics (below) +2. **Monetization** — Small Business Program, Play fees, RevenueCat → `references/monetization.md` +3. **App Store metadata** — name, subtitle, keywords, description, screenshots, categories → `references/app-store-metadata.md` +4. **Privacy policy** — generate a policy and host it (with a home page) on EAS → `references/privacy-policy.md` +5. **Rejection scan** — catch common App Store / Play rejections before submitting → `references/rejection-scan.md` +6. **Build with EAS** — production builds and version-number strategy → `references/eas-build.md` +7. **Submit with EAS** — submit to both stores, first-time checklists, automation → `references/eas-submit.md` +8. **Marketing** — launch-day and week-one promotion, getting reviews → `references/marketing.md` + +Opening message: + +``` +📱 Expo App Release Checklist + +I'll guide you through everything to get your Expo app on the App Store and Google Play: + +1. 💰 Monetization — Small Business Program, RevenueCat, etc. +2. 📝 App Store Metadata — drafts for every required field +3. 🔒 Privacy Policy — generated from your app's data practices, hosted on EAS +4. ⚠️ Rejection Scan — catch common rejection reasons early +5. 🔨 Build with EAS — exact build commands +6. 🚀 Submit with EAS — submit to both stores +7. 📣 Marketing — where to share your launch + +Let's start! What's your app called and what does it do? +``` + +## Phase 1: Understand the App + +Gather the essentials before generating anything, and reuse them throughout the checklist: + +- **App basics** — name, one-sentence pitch, target audience +- **Status** — is it built and working? tested on real devices? +- **Accounts** — Apple Developer account ($99/yr)? Google Play account ($25 one-time)? +- **Monetization** — free or paid? in-app purchases / subscriptions? ads? + +The answers drive metadata (Phase 3), the privacy policy (Phase 4), and the rejection scan (Phase 5). + +## References + +Consult these as each phase comes up: + +- ./references/monetization.md — Apple Small Business Program, Google Play's reduced fee, RevenueCat for subscriptions +- ./references/app-store-metadata.md — Apple + Google Play metadata fields, ASO keyword strategy, screenshots, categories +- ./references/privacy-policy.md — data-practice questions, policy generation, hosting policy + home page on EAS, Play Data Safety +- ./references/rejection-scan.md — critical / high / medium rejection checks and how to fix them +- ./references/eas-build.md — production builds and version-number strategy +- ./references/eas-submit.md — store submission, first-time checklists, the 12-tester rule, EAS Workflows automation +- ./references/marketing.md — launch-day and week-one marketing, requesting reviews + +## Summary Output + +When the journey is complete, recap what was prepared: + +``` +✅ RELEASE CHECKLIST COMPLETE! + +MONETIZATION: [program enrolled / payment method] +METADATA: app name, subtitle, keywords (100 chars), description, screenshots, category +PRIVACY POLICY: generated + hosted at [URL] +REJECTION SCAN: [X] passed, [X] fixed +BUILDS: eas build --platform ios|android --profile production +SUBMISSIONS: eas submit --platform ios|android --latest +MARKETING: launch-day checklist ready + +Good luck with your launch! 🚀 +``` + +## Working Style + +- **Be encouraging** — first-time developers are often overwhelmed. +- **Be specific** — exact commands, character counts, concrete examples. +- **Be proactive** — anticipate the next step. +- **Save their work** — offer to write generated metadata and policies to files they can copy. +- If they're stuck, offer to skip to a phase or link the relevant Expo docs. +- If they lack developer accounts, explain the cost and process, and note that Google Play's 12-tester rule means starting ASAP. + +## Additional Resources + +- Expo docs: https://docs.expo.dev +- EAS Build: https://docs.expo.dev/build/introduction/ +- EAS Submit: https://docs.expo.dev/submit/introduction/ +- App Store Review Guidelines: https://developer.apple.com/app-store/review/guidelines/ +- Google Play policies: https://play.google.com/about/developer-content-policy/ diff --git a/plugins/expo/skills/release-app/references/app-store-metadata.md b/plugins/expo/skills/release-app/references/app-store-metadata.md new file mode 100644 index 0000000..5c35072 --- /dev/null +++ b/plugins/expo/skills/release-app/references/app-store-metadata.md @@ -0,0 +1,94 @@ +# App Store Metadata Generation + +This is the most interactive phase: generate drafts for **every** required field on both stores. Always show the character count next to each generated option and respect the limits exactly. + +## Gather context first + +Ask before drafting: + +1. List 3–5 key features +2. What makes it different from competitors? +3. What problem does it solve? +4. Any awards, press, or testimonials? +5. Best-fit category? (Games, Productivity, Health & Fitness, …) + +## Apple App Store + +### App Name — 30 characters max + +Include a keyword for discoverability. Offer 2–3 options with counts. + +``` +1. [App Name] – [Keyword] (X/30) +2. [Variation] (X/30) +``` + +### Subtitle — 30 characters max + +Secondary keywords; do **not** repeat words from the name. + +### Keywords — exactly 100 characters + +Apple gives you exactly 100 characters — use all of them. + +- Comma-separated, **no spaces after commas** (spaces waste characters) +- Don't repeat words already in the name or subtitle +- Don't include both singular and plural of a word + +``` +keyword1,keyword2,keyword3,keyword4,... (100/100) +``` + +### Description — 4,000 characters max + +- The first ~170 characters show before "Read More" — lead with the main benefit, not a feature list +- Then features-as-benefits, social proof if any, and a closing call to action + +### Promotional Text — 170 characters + +The only field you can update **without** submitting a new build — use it for seasonal content, new features, or promos. + +### What's New + +- First release: `Initial release! [what makes the app great]` +- Updates: bullet `New: …`, `Improved: …`, `Fixed: …` + +## Google Play + +### Short Description — 80 characters + +The elevator pitch. Offer 2 options with counts. + +### Full Description — 4,000 characters + +Unlike Apple, Google **indexes the description for search** — weave keywords in naturally throughout. + +## Screenshots + +``` +Apple App Store: +- 6.7" iPhone (required): 1290 x 2796 +- 6.5" iPhone (required): 1284 x 2778 +- iPad Pro 12.9" (if universal): 2048 x 2732 +- 1–10 screenshots per size + +Google Play: +- Phone: 16:9, min 320px / max 3840px +- 2–8 screenshots per device type +``` + +Propose a 5-screenshot story and a caption for each: + +1. Main value proposition +2. Key feature 1 +3. Key feature 2 +4. Social proof / differentiator +5. Call to action / extra feature + +Screenshots must show the **real** app (no mockups) — mismatches get rejected. + +## Categories + +Recommend a primary (and optional secondary) Apple category, plus a Google Play category and up to 5 tags, with a one-line rationale. + +> Offer to save the finished metadata to a file (e.g. `store-metadata.md`) the user can copy into App Store Connect and Play Console. diff --git a/plugins/expo/skills/release-app/references/eas-build.md b/plugins/expo/skills/release-app/references/eas-build.md new file mode 100644 index 0000000..73e8786 --- /dev/null +++ b/plugins/expo/skills/release-app/references/eas-build.md @@ -0,0 +1,70 @@ +# Build with EAS + +Produce the production builds you'll submit to the stores. For full `eas.json` and build-profile reference, see the `expo-deployment` skill; this file is the first-timer walkthrough. + +## Prerequisites + +``` +□ EAS CLI available: npm install -g eas-cli (or use npx eas-cli@latest) +□ Logged in: eas login +□ Project configured: eas build:configure +``` + +## Step 1 — production profile in `eas.json` + +```json +{ + "build": { + "production": { + "ios": { "distribution": "store" }, + "android": { "buildType": "app-bundle" } + } + } +} +``` + +## Step 2 — confirm `app.json` + +- `ios.bundleIdentifier` — e.g. `com.yourcompany.yourapp` +- `android.package` — e.g. `com.yourcompany.yourapp` +- `version` — e.g. `"1.0.0"` +- `ios.buildNumber` — e.g. `"1"` +- `android.versionCode` — e.g. `1` + +## Step 3 — build + +```bash +# iOS +eas build --platform ios --profile production + +# Android +eas build --platform android --profile production + +# Both +eas build --platform all --profile production +``` + +Builds typically take ~15–30 minutes. + +## Version-number strategy + +``` +version (e.g. "1.0.0") + Shown to users. Semantic MAJOR.MINOR.PATCH. Increase with each store release. + +iOS buildNumber (e.g. "1") + Internal id. Must increase with EVERY build submitted to App Store Connect, + even if the version string is unchanged. + +Android versionCode (e.g. 1) + Internal id, integer only. Must increase with EVERY Play submission. +``` + +Forgetting to bump these is the most common cause of a rejected upload. Let EAS manage it automatically: + +```json +{ + "cli": { "version": ">= 16.0.0", "appVersionSource": "remote" }, + "build": { "production": { "autoIncrement": true } } +} +``` diff --git a/plugins/expo/skills/release-app/references/eas-submit.md b/plugins/expo/skills/release-app/references/eas-submit.md new file mode 100644 index 0000000..a29e985 --- /dev/null +++ b/plugins/expo/skills/release-app/references/eas-submit.md @@ -0,0 +1,162 @@ +# Submit with EAS + +Send the finished builds to App Store Connect and Google Play. For deeper credential and submit-profile reference, see the `expo-deployment` skill. + +## Prerequisites + +``` +□ Build completed successfully +□ App created in App Store Connect (iOS) and Google Play Console (Android) +□ All store metadata filled in +□ Screenshots uploaded +□ Privacy policy URL added +``` + +## Step 1 — credentials + +``` +iOS (App Store Connect): + eas credentials → select iOS → set up an App Store Connect API Key + (or use manual Apple ID login) + +Android (Google Play): + - Create a service account in Google Cloud Console + - Download its JSON key + - Add it under Play Console → API access + - Reference it in eas.json: + { + "submit": { + "production": { + "android": { + "serviceAccountKeyPath": "./google-service-account.json", + "track": "production" + } + } + } + } +``` + +## Step 2 — submit + +```bash +eas submit --platform ios --latest +eas submit --platform android --latest +# or both: +eas submit --platform all --latest +``` + +After submission: iOS goes to App Store review (typically 1–3 days); Android goes to Play review (usually faster). + +## First-time submission checklists + +``` +iOS (App Store Connect) +□ App created; bundle ID matches +□ Metadata complete (name, description, keywords, …) +□ Screenshots for required device sizes +□ Privacy policy URL added +□ Age-rating questionnaire complete +□ Pricing and availability set + +Android (Google Play) +□ App created; package name matches +□ Store listing complete +□ Screenshots uploaded +□ Privacy policy URL added +□ Data Safety questionnaire complete +□ Content rating questionnaire complete +□ Target audience / content settings configured +□ 12 testers for 14 days (new personal-account requirement) +``` + +## Google Play's 12-tester requirement + +New **personal** developer accounts must run a closed test with **12 testers for at least 14 days**, and testers must actually use the app (not just install). + +``` +1. Play Console → Testing → Closed testing → create a track +2. Add tester emails (or a Google Group) +3. Share the opt-in link +4. Wait 14 days with active usage +``` + +Tip: recruit friends/family or swap testers with other developers (e.g. the Expo Discord). + +## Automate future releases with EAS Workflows + +Once the first release is live, automate subsequent builds and submissions. Fingerprinting skips a native build when only JS changed and ships an OTA update instead. + +Create `.eas/workflows/deploy-to-production.yml`: + +```yaml +name: Deploy to production +on: + push: + branches: ['main'] + +jobs: + fingerprint: + name: Fingerprint + type: fingerprint + environment: production + + get_android_build: + name: Check for existing Android build + needs: [fingerprint] + type: get-build + params: + fingerprint_hash: ${{ needs.fingerprint.outputs.android_fingerprint_hash }} + profile: production + + get_ios_build: + name: Check for existing iOS build + needs: [fingerprint] + type: get-build + params: + fingerprint_hash: ${{ needs.fingerprint.outputs.ios_fingerprint_hash }} + profile: production + + build_android: + name: Build Android + needs: [get_android_build] + if: ${{ !needs.get_android_build.outputs.build_id }} + type: build + params: { platform: android, profile: production } + + build_ios: + name: Build iOS + needs: [get_ios_build] + if: ${{ !needs.get_ios_build.outputs.build_id }} + type: build + params: { platform: ios, profile: production } + + submit_android_build: + name: Submit Android Build + needs: [build_android] + type: submit + params: { build_id: ${{ needs.build_android.outputs.build_id }} } + + submit_ios_build: + name: Submit iOS Build + needs: [build_ios] + type: submit + params: { build_id: ${{ needs.build_ios.outputs.build_id }} } + + publish_android_update: + name: Publish Android update + needs: [get_android_build] + if: ${{ needs.get_android_build.outputs.build_id }} + type: update + params: { branch: production, platform: android } + + publish_ios_update: + name: Publish iOS update + needs: [get_ios_build] + if: ${{ needs.get_ios_build.outputs.build_id }} + type: update + params: { branch: production, platform: ios } +``` + +Push to `main` and the workflow triggers: native changes build + submit; JS-only changes ship an OTA update. Docs: https://docs.expo.dev/eas/workflows/examples/deploy-to-production/ + +> For more workflow examples, the `expo-cicd-workflows` skill covers EAS Workflows YAML in depth. diff --git a/plugins/expo/skills/release-app/references/marketing.md b/plugins/expo/skills/release-app/references/marketing.md new file mode 100644 index 0000000..7a4c129 --- /dev/null +++ b/plugins/expo/skills/release-app/references/marketing.md @@ -0,0 +1,68 @@ +# Post-Launch Marketing + +Once the app is approved, help the user get users. Tailor the channels to the app's niche. + +## Launch day + +``` +□ Share on your personal social media +□ Post in relevant subreddits (follow each one's self-promo rules): + r/[your topic], r/SideProject, r/iOSapps or r/androidapps +□ Post with relevant hashtags +□ Ask friends and family to download + leave a review +``` + +## Week 1 + +``` +□ Product Hunt — launch Tue–Thu; prep logo, screenshots, tagline; line up + friends to upvote and comment. https://www.producthunt.com +□ Hacker News "Show HN" — only if technically interesting. + https://news.ycombinator.com +□ Indie Hackers — great for solo devs. https://www.indiehackers.com +□ BetaList — for new apps. https://betalist.com +``` + +## Ongoing + +``` +□ Respond to App Store / Play reviews (boosts ranking) +□ Post updates and content (blog posts, short videos) +□ Reach out to relevant bloggers / reviewers +□ Consider App Store Search Ads +``` + +## Communities + +``` +□ Expo Discord — share in #showcase +□ React Native community +□ Niche communities for your app's topic +``` + +## ASO over time + +``` +□ After 2–4 weeks, review keyword performance +□ Update keywords based on what's ranking +□ A/B test screenshots where supported (Google Play) +``` + +## Ask for reviews in-app + +Reviews strongly affect downloads. Prompt at a positive moment with the native review sheet: + +```ts +import * as StoreReview from "expo-store-review"; + +// After a "win" — completing a task, finishing a session, etc. +if (await StoreReview.hasAction()) { + await StoreReview.requestReview(); +} +``` + +Best practices: + +- Ask **after** a success or "wow" moment, never during onboarding +- Ask after the app has been used a few times +- At most once per version diff --git a/plugins/expo/skills/release-app/references/monetization.md b/plugins/expo/skills/release-app/references/monetization.md new file mode 100644 index 0000000..30e44d0 --- /dev/null +++ b/plugins/expo/skills/release-app/references/monetization.md @@ -0,0 +1,52 @@ +# Monetization Setup + +Help the user lower store fees and, if they have subscriptions, set up the right tooling. Only walk through the parts that apply to their monetization plan from Phase 1. + +## Apple Small Business Program + +If the developer expects less than **$1M/year** in App Store proceeds, they can cut Apple's commission from 30% to **15%**. + +``` +💰 Apple Small Business Program + +If you expect to earn less than $1M per year, you can reduce Apple's +commission from 30% to 15%. + +To enroll: +1. Go to https://developer.apple.com/programs/small-business-program/ +2. Sign in with your Apple Developer account +3. Accept the terms and enroll + +Requirements: +- Less than $1M in App Store proceeds in the previous year +- Re-enroll each year +- All apps under your account qualify +``` + +## Google Play's Reduced Fee + +``` +💰 Google Play Service Fee + +Google automatically applies 15% (instead of 30%) on the first $1M of +earnings each year — no enrollment needed. + +Subscriptions are 15% from day one. +``` + +## RevenueCat (for subscriptions) + +If the app has subscriptions, recommend RevenueCat to handle receipt validation, cross-platform subscription status, analytics, and webhooks. + +``` +For subscriptions, RevenueCat handles: +- Receipt validation +- Cross-platform subscription status +- Analytics and insights +- Webhook integrations + +Install: npx expo install react-native-purchases +Docs: https://www.revenuecat.com/docs/getting-started/installation/reactnative +``` + +> Reminder: all **digital** goods and subscriptions must use Apple/Google in-app purchase — external payment processors (Stripe, PayPal) are only allowed for physical goods and services. The rejection scan covers this (see `rejection-scan.md`). diff --git a/plugins/expo/skills/release-app/references/privacy-policy.md b/plugins/expo/skills/release-app/references/privacy-policy.md new file mode 100644 index 0000000..5e1d22e --- /dev/null +++ b/plugins/expo/skills/release-app/references/privacy-policy.md @@ -0,0 +1,124 @@ +# Privacy Policy Generation & Hosting + +Every app needs a privacy policy reachable at a public URL. Generate one from the app's actual data practices, then host it (alongside a simple home page) with EAS. + +## Gather data practices + +``` +🔒 To create your privacy policy, I need to know what data your app collects: + +1. User accounts / login? Which methods? (Email, Google, Apple, Facebook, Phone) +2. Third-party SDKs (check package.json): + - Analytics (Firebase, Amplitude, Mixpanel, Segment) + - Crash reporting (Crashlytics, Sentry) + - Ads (AdMob, Meta Ads) + - Payments (RevenueCat, Stripe) + - Push (Expo Notifications, FCM) +3. Device access: location, camera, photos/media, contacts, calendar, microphone? +4. Do you share data with third parties? +5. Regions: US only, Europe (GDPR), California (CCPA)? +``` + +## Generate the policy + +Produce a complete policy tailored to their answers: + +```markdown +# Privacy Policy for [App Name] + +**Last Updated: [Date]** + +[Developer Name] ("we") operates the [App Name] mobile application. + +## Information We Collect +[personal info, usage data, location, etc. — from their answers] + +## How We Use Your Information +[purposes tied to their SDKs and features] + +## Third-Party Services +[each SDK, what it collects, link to its privacy policy] + +## Your Privacy Rights +[GDPR section if EU; CCPA section if US/California] + +## Data Retention +[their practices] + +## Contact Us +Email: [their email] +``` + +## Host it on EAS + +Stand up a tiny static site with a home page and the policy, then deploy with EAS Hosting. + +**Step 1 — create the folder and copy the app icon** + +```bash +mkdir dist +cp assets/icon.png dist/icon.png # icon path is usually set in app.json +``` + +**Step 2 — `dist/index.html`** (simple home page) + +```html + + + + + + [App Name] + + + +
+ [App Name] icon +

[App Name]

+

[Your app's tagline]

+
+ Download on the App Store + Get it on Google Play +
+ +
+ + +``` + +**Step 3 — `dist/privacy.html`**: the generated policy as HTML (a centered, readable column linking back to `index.html`). + +**Step 4 — deploy** + +```bash +eas login # if not already +eas deploy +``` + +This uploads `dist/` and returns a URL like `https://[project-slug].expo.app`. The policy is then at `https://[project-slug].expo.app/privacy.html`. + +**Step 5 — use the URL** in: + +- App Store Connect → App Information → Privacy Policy URL +- Google Play Console → Store listing → Privacy Policy +- `app.json` under `expo.ios.privacyPolicyUrl` + +> The `expo-deployment` skill covers EAS Hosting in more depth if the user wants custom domains or CI deploys. + +## Google Play Data Safety + +Help fill out the Data Safety form from the same answers: + +- **Data collected** — each data type, whether collected, and the purpose +- **Data shared** — what (if anything) goes to third parties +- **Security** — encrypted in transit? can users request deletion, and how? diff --git a/plugins/expo/skills/release-app/references/rejection-scan.md b/plugins/expo/skills/release-app/references/rejection-scan.md new file mode 100644 index 0000000..cf989f6 --- /dev/null +++ b/plugins/expo/skills/release-app/references/rejection-scan.md @@ -0,0 +1,83 @@ +# Pre-Submission Rejection Scan + +Catch the common App Store / Play rejection reasons before the user submits. Ask for `app.json`/`app.config.js`, `package.json`, a short feature description, and whether the app has auth or in-app purchases — then walk the checks below and report results. + +## Critical — will be rejected + +``` +🔴 CRITICAL + +□ Sign in with Apple + If you offer Google / Facebook / other social login, you MUST also offer + Sign in with Apple. + +□ In-app purchases + All digital goods/subscriptions MUST use Apple/Google IAP (not Stripe, + PayPal, etc.). Physical goods can use external payment. + +□ Privacy policy + Required for all apps; must be reachable at a public URL. + +□ Account deletion + If users can create accounts, they must be able to delete them in-app. + +□ App completeness + No placeholder content, "coming soon" features, or broken links. + +□ Reviewer access + If login is required, provide demo credentials for App Review (or allow + use without login). +``` + +## High risk + +``` +🟡 HIGH RISK + +□ App Tracking Transparency (iOS) + If you use advertising identifiers (AdMob, Meta Ads, IDFA analytics), you + must show the ATT prompt. + +□ Permission justifications + Every iOS permission needs a specific reason string in Info.plist — + generic reasons get rejected. + +□ Metadata accuracy + Screenshots must show the real app; description must match functionality. + +□ Kids category (if applicable) + No ads, no external links, no data collection, strict parental gate. +``` + +## Medium risk + +``` +🟠 MEDIUM RISK + +□ Cross-platform references + Don't mention Android in an iOS app (or vice versa). + +□ Pricing claims + Don't say "free" if there are IAPs — use "free to download". + +□ Third-party trademarks + Don't use trademarked names/brands without permission. +``` + +## Report results + +``` +📋 REJECTION SCAN RESULTS + +✅ Passed: [X] checks +⚠️ Warnings: [X] items to review +❌ Must fix: [X] blocking issues + +MUST FIX BEFORE SUBMITTING: +1. [issue + how to fix] + +WARNINGS TO REVIEW: +1. [warning + recommendation] +``` + +For a deeper, guideline-by-guideline audit, review the app against Apple's full [App Store Review Guidelines](https://developer.apple.com/app-store/review/guidelines/) in addition to this quick scan.