diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 260001c..ff43467 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -16,7 +16,7 @@ A collector is three pieces: ### Design constraints -Collectors observe a source and emit events. They do not write files, change study state, start activities, schedule prompts, export, or request permissions. This is not a rule imposed on collector authors so much as a consequence of the module graph: a `collector:*` module depends only on `core:collector-api` and `core:study-definition`, so storage, the runtime, and the protocol layer are not on its classpath. +Collectors observe a source and emit events. They do not write files, change study state, start activities, schedule interventions, render surveys, export, or request permissions. This is not a rule imposed on collector authors so much as a consequence of the module graph: a `collector:*` module depends only on `core:collector-api` and `core:study-definition`, so storage, the runtime, and the protocol layer are not on its classpath. That boundary is what keeps a new data source cheap to add and cheap to review. It also means the answer to "how do I persist this myself?" is that you do not — everything goes through the `EventSink` in your `CollectorContext`, which is what makes sequence numbers contiguous and monotone, quota accounting correct, and a bundle able to declare the exact window it carries. diff --git a/README.md b/README.md index 7612451..ce52766 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ Standing up a mobile sensing study normally means writing an Android app, gettin ## How a study works 1. **Generate your keys.** One Ed25519 pair to sign study configurations, one HPKE pair to decrypt exports. `researcher-tools` does both. -2. **Write the study.** A JSON file naming the collectors you want, their parameters, the duration, prompts, the on-device storage quota (8 MiB to 8 GiB), consent text, your export public key, and a `signer` block carrying the key ID and public half of your signing key. +2. **Write the study.** A strict v1 JSON file naming collectors, reusable surveys, scheduled interventions, anonymous or assigned-code identity mode, duration, storage quota, consent text, and signing/export public keys. 3. **Sign it.** `researcher-tools sign` produces a `.adccfg` file. Because the signing public key travels inside the signed bytes, any build of the app can verify it. 4. **Distribute.** Participants install the app and import your `.adccfg`. Setup is five steps, one screen each — the study details, what each enabled collector records and does not record, the consent text with the signer's key fingerprint, the Android access your collectors need, and the start button — and collection begins only when they press it. 5. **Collect.** Events are written to encrypted on-device storage. Participants can pause, resume, finish early, or withdraw. @@ -49,8 +49,10 @@ Studies collect from people's personal phones, so the platform is built to suppo - **Encrypted on the device.** Each study's events and metadata are encrypted with a per-study AES-256-GCM key from the Android Keystore, marked non-exportable, in 4 MiB event segments under the app's no-backup storage, up to the quota the configuration set. An event is appended once and never rewritten; the only thing that removes a segment is confirmed delivery to the study's endpoint, and then only under storage pressure. - **Signed, tamper-evident studies.** A configuration is Ed25519-signed and strictly validated: canonical encoding, exact schema, known collectors, validity window, minimum app version. Verification failures are fail-closed — an unverifiable configuration collects nothing. A signature proves the configuration is unchanged since it was signed; it does not prove who wrote it unless the build pins that signer, and the consent screen states which of the two applies. +- **Durable interventions and native surveys.** Notification actions can use one-time, recurring, or daily-local triggers. Their occurrences survive retries and reboot without duplication. Native surveys support short text, integer scales, single choice, and multiple choice; only a confirmed, complete submission enters the encrypted event stream. +- **Separated participant identities.** Every import gets a fresh random instance UUID. A configuration may additionally carry an opaque researcher-assigned code; both appear inside encrypted exports, while upload headers expose only the instance UUID used for routing and de-duplication. - **Encrypted, participant-directed export.** Getting data to the research team is an export the participant performs and directs, encrypted with a fresh key per export and wrapped to your HPKE public key. The app never holds your private key. -- **Scheduled upload, when the study asks for it.** A configuration may name an HTTPS endpoint, an interval, and whether metered networks are allowed. The app then delivers the same encrypted bundles on that schedule, over Wi-Fi unless the study allows otherwise, so the endpoint stores ciphertext only your private key opens. The endpoint host, the cadence, and the network condition are shown on the consent screen before the participant accepts — and the app honours the interval you signed rather than clamping it to a platform floor. Each run sends everything outstanding up to a 16 MiB size budget and records exactly how far it got, so a backlog drains across runs without ever re-sending or skipping an event. Upload is part of the study rather than a separate switch. A configuration whose `upload` block is empty sends nothing, and collection never depends on an upload succeeding — a study whose endpoint is unreachable keeps recording, manual export still works, and the participant sees a fixed failure code such as `UPLOAD_TIMEOUT` rather than a raw error. Finishing or withdrawing stops collection, its prompts, and its deadline, but delivery continues until the undelivered tail has reached you. Confirmed delivery is also what lets a device reclaim local space once its quota runs high; undelivered events are never released to make room, and the participant is told when earlier ones were. +- **Scheduled upload, when the study asks for it.** A configuration may name an HTTPS endpoint, interval, and metered-network policy. The endpoint host, cadence, and network condition are shown before consent. Each run sends outstanding events up to a 16 MiB plaintext budget and records its exact boundary. Finishing or withdrawing cancels future interventions and the deadline, while delivery continues until the undelivered tail arrives. Undelivered events are never reclaimed to make room. - **Participant control over the lifecycle.** Collection starts only on an explicit action and can be paused, finished, or withdrawn. Pausing takes a monotonic boundary, so delayed callbacks cannot leak post-pause data into the dataset. - **Storage failures stop collection.** Quota exhaustion or a write failure fail-closes the study to `PAUSED` rather than silently dropping events, so a dataset is complete over the window it declares or absent. diff --git a/app/build.gradle.kts b/app/build.gradle.kts index cab7235..306b048 100644 --- a/app/build.gradle.kts +++ b/app/build.gradle.kts @@ -105,6 +105,8 @@ dependencies { debugImplementation(libs.compose.ui.tooling) debugImplementation(libs.compose.ui.test.manifest) + testImplementation(libs.junit4) + androidTestImplementation(platform(libs.compose.bom)) androidTestImplementation(libs.androidx.test.junit) androidTestImplementation(libs.androidx.test.runner) diff --git a/app/src/debug/res/raw/demo_study_envelope.txt b/app/src/debug/res/raw/demo_study_envelope.txt index 5ae0f3f..9e4f9cb 100644 --- a/app/src/debug/res/raw/demo_study_envelope.txt +++ b/app/src/debug/res/raw/demo_study_envelope.txt @@ -1 +1 @@ -QURDQ0ZHMDEAEAAACN0AQGRlbW8tc2lnbmVyLTIwMjZ7InNjaGVtYV92ZXJzaW9uIjoxLCJleHBlcmltZW50X2lkIjoibW9kdWxhci1zZW5zaW5nLWRlbW8iLCJjb25maWd1cmF0aW9uX2lkIjoiZGVtby1jb25maWctMjAyNiIsImlzc3VlZF9hdCI6IjIwMjYtMDEtMDFUMDA6MDA6MDBaIiwiZXhwaXJlc19hdCI6IjIwMzUtMDEtMDFUMDA6MDA6MDBaIiwibWluaW11bV9hcHBfdmVyc2lvbiI6MSwidGl0bGUiOiJNb2R1bGFyIHNlbnNpbmcgZGVtb25zdHJhdGlvbiIsInJlc2VhcmNoZXIiOnsibmFtZSI6IkFuZHJvaWQgRGF0YSBDb2xsZWN0b3IgbWFpbnRhaW5lcnMiLCJjb250YWN0IjoicmVzZWFyY2hAZXhhbXBsZS5pbnZhbGlkIn0sInB1cnBvc2UiOiJWZXJpZnkgdGhlIGNvbXBsZXRlIG9uLWRldmljZSBjb2xsZWN0aW9uLCBwYXVzZSwgZXhwb3J0LCBhbmQgcmVzZWFyY2hlci1kZWNyeXB0aW9uIGxvb3AuIiwiZHVyYXRpb25faG91cnMiOjI0LCJjb25zZW50Ijp7ImRvY3VtZW50X3ZlcnNpb24iOiJkZW1vLTEiLCJzdW1tYXJ5IjoiVGhpcyBkZW1vbnN0cmF0aW9uIGNhbiBjb2xsZWN0IHByZWNpc2UgbG9jYXRpb24sIG1vdGlvbiwgbmV0d29yayBzdGF0ZSwgYWdncmVnYXRlIFdpLUZpL21vYmlsZSB1c2FnZSwgYXBwIGFuZCBzY3JlZW4gdXNhZ2UgZXZlbnRzLCB0aGlzIGFwcCdzIGxpZmVjeWNsZSwgYW5kIHRvdWNoIGR5bmFtaWNzIG1hZGUgaW5zaWRlIHRoZSBvcHRpb25hbCByZXNlYXJjaCBrZXlib2FyZC4gSXQgbmV2ZXIgcmVjb3JkcyBrZXlib2FyZCB0ZXh0LiBEYXRhIHN0YXlzIGVuY3J5cHRlZCBvbiB0aGlzIGRldmljZSB1bnRpbCB5b3UgY2hvb3NlIEV4cG9ydC4gWW91IGNhbiBwYXVzZSwgd2l0aGRyYXcsIGV4cG9ydCByZXBlYXRlZGx5LCBvciBkZWxldGUgbG9jYWwgZGF0YS4ifSwiY29sbGVjdG9ycyI6W3siaWQiOiJhcHBfbGlmZWN5Y2xlLnYxIiwicmVxdWlyZWQiOnRydWUsImNvbmZpZyI6e319LHsiaWQiOiJhY2NlbGVyb21ldGVyLnYxIiwicmVxdWlyZWQiOnRydWUsImNvbmZpZyI6eyJzYW1wbGluZ19wZXJpb2RfdXMiOjEwMDAwMCwibWF4aW11bV9yZXBvcnRfbGF0ZW5jeV91cyI6MTAwMDAwMH19LHsiaWQiOiJuZXR3b3JrX3N0YXRlLnYxIiwicmVxdWlyZWQiOnRydWUsImNvbmZpZyI6eyJpbmNsdWRlX2JhbmR3aWR0aF9lc3RpbWF0ZXMiOnRydWV9fSx7ImlkIjoibmV0d29ya191c2FnZS52MSIsInJlcXVpcmVkIjpmYWxzZSwiY29uZmlnIjp7InRyYW5zcG9ydHMiOlsibW9iaWxlIiwid2lmaSJdLCJwb2xsX2ludGVydmFsX21pbnV0ZXMiOjV9fSx7ImlkIjoidXNhZ2VfZXZlbnRzLnYxIiwicmVxdWlyZWQiOmZhbHNlLCJjb25maWciOnsicG9sbF9pbnRlcnZhbF9taW51dGVzIjoxNX19LHsiaWQiOiJsb2NhdGlvbi52MSIsInJlcXVpcmVkIjpmYWxzZSwiY29uZmlnIjp7ImludGVydmFsX21pbGxpcyI6MTAwMDAsIm1pbmltdW1faW50ZXJ2YWxfbWlsbGlzIjo1MDAwLCJtYXhpbXVtX2JhdGNoX2RlbGF5X21pbGxpcyI6MzAwMDAsIm1pbmltdW1fZGlzcGxhY2VtZW50X21ldGVycyI6NS4wLCJwcmlvcml0eSI6IkJBTEFOQ0VEIn19LHsiaWQiOiJrZXlib2FyZF90b3VjaC52MSIsInJlcXVpcmVkIjpmYWxzZSwiY29uZmlnIjp7InRyYWplY3Rvcnlfc2FtcGxpbmdfaHoiOjYwfX1dLCJwcm9tcHRzIjpbeyJpZCI6ImRlbW8tY2hlY2staW4iLCJkZWxheV9taW51dGVzIjo2MCwibWVzc2FnZSI6IlBsZWFzZSBjaGVjayB0aGF0IHRoZSBzdHVkeSBpcyBzdGlsbCBydW5uaW5nIGFzIGV4cGVjdGVkLiJ9XSwic3RvcmFnZSI6eyJtYXhpbXVtX2xvY2FsX2J5dGVzIjoxNjc3NzIxNn0sInNpZ25lciI6eyJrZXlfaWQiOiJkZW1vLXNpZ25lci0yMDI2IiwicHVibGljX2tleSI6Ik1Db3dCUVlESzJWd0F5RUFzUlNhVHBabVRTQkw3ZU42blMvSEJzTm1MTThuMWhkUm1JdDF2dExac0MwPSJ9LCJleHBvcnQiOnsicmVzZWFyY2hlcl9rZXlfaWQiOiJkZW1vLWhwa2UtMjAyNiIsInRpbmtfaHBrZV9wdWJsaWNfa2V5c2V0Ijp7InByaW1hcnlLZXlJZCI6MjE4OTkyNzI3LCJrZXkiOlt7ImtleURhdGEiOnsidHlwZVVybCI6InR5cGUuZ29vZ2xlYXBpcy5jb20vZ29vZ2xlLmNyeXB0by50aW5rLkhwa2VQdWJsaWNLZXkiLCJ2YWx1ZSI6IkVnWUlBUkFCR0FJYUlCcHlRM3c0ZkZ4OVhnRVV4NWt5elphSVBYTHE3YVlVNlJKK3k5K3JHTkVBIiwia2V5TWF0ZXJpYWxUeXBlIjoiQVNZTU1FVFJJQ19QVUJMSUMifSwic3RhdHVzIjoiRU5BQkxFRCIsImtleUlkIjoyMTg5OTI3MjcsIm91dHB1dFByZWZpeFR5cGUiOiJUSU5LIn1dfX0sInVwbG9hZCI6e319qNwoReutzhOLUp/zBskXBkDTpjAWugkmT6f35dcu0vBfMCmQ8WSLKhv2L41ttYC/dnERe0sxLe6fwOxqEtihAQ== \ No newline at end of file +QURDQ0ZHMDEAEAAACyYAQGRlbW8tc2lnbmVyLTIwMjZ7InNjaGVtYV92ZXJzaW9uIjoxLCJleHBlcmltZW50X2lkIjoibW9kdWxhci1zZW5zaW5nLWRlbW8iLCJjb25maWd1cmF0aW9uX2lkIjoiZGVtby1jb25maWctMjAyNiIsImFzc2lnbmVkX3BhcnRpY2lwYW50X2lkIjpudWxsLCJpc3N1ZWRfYXQiOiIyMDI2LTAxLTAxVDAwOjAwOjAwWiIsImV4cGlyZXNfYXQiOiIyMDM1LTAxLTAxVDAwOjAwOjAwWiIsIm1pbmltdW1fYXBwX3ZlcnNpb24iOjEsInRpdGxlIjoiTW9kdWxhciBzZW5zaW5nIGRlbW9uc3RyYXRpb24iLCJyZXNlYXJjaGVyIjp7Im5hbWUiOiJBbmRyb2lkIERhdGEgQ29sbGVjdG9yIG1haW50YWluZXJzIiwiY29udGFjdCI6InJlc2VhcmNoQGV4YW1wbGUuaW52YWxpZCJ9LCJwdXJwb3NlIjoiVmVyaWZ5IHRoZSBjb21wbGV0ZSBvbi1kZXZpY2UgY29sbGVjdGlvbiwgcGF1c2UsIGV4cG9ydCwgYW5kIHJlc2VhcmNoZXItZGVjcnlwdGlvbiBsb29wLiIsImR1cmF0aW9uX2hvdXJzIjoyNCwiY29uc2VudCI6eyJkb2N1bWVudF92ZXJzaW9uIjoiZGVtby0xIiwic3VtbWFyeSI6IlRoaXMgZGVtb25zdHJhdGlvbiBjYW4gY29sbGVjdCBwcmVjaXNlIGxvY2F0aW9uLCBtb3Rpb24sIG5ldHdvcmsgc3RhdGUsIGFnZ3JlZ2F0ZSBXaS1GaS9tb2JpbGUgdXNhZ2UsIGFwcCBhbmQgc2NyZWVuIHVzYWdlIGV2ZW50cywgdGhpcyBhcHAncyBsaWZlY3ljbGUsIGFuZCB0b3VjaCBkeW5hbWljcyBtYWRlIGluc2lkZSB0aGUgb3B0aW9uYWwgcmVzZWFyY2gga2V5Ym9hcmQuIEl0IG5ldmVyIHJlY29yZHMga2V5Ym9hcmQgdGV4dC4gRGF0YSBzdGF5cyBlbmNyeXB0ZWQgb24gdGhpcyBkZXZpY2UgdW50aWwgeW91IGNob29zZSBFeHBvcnQuIFlvdSBjYW4gcGF1c2UsIHdpdGhkcmF3LCBleHBvcnQgcmVwZWF0ZWRseSwgb3IgZGVsZXRlIGxvY2FsIGRhdGEuIn0sImNvbGxlY3RvcnMiOlt7ImlkIjoiYXBwX2xpZmVjeWNsZS52MSIsInJlcXVpcmVkIjp0cnVlLCJjb25maWciOnt9fSx7ImlkIjoiYWNjZWxlcm9tZXRlci52MSIsInJlcXVpcmVkIjp0cnVlLCJjb25maWciOnsic2FtcGxpbmdfcGVyaW9kX3VzIjoxMDAwMDAsIm1heGltdW1fcmVwb3J0X2xhdGVuY3lfdXMiOjEwMDAwMDB9fSx7ImlkIjoibmV0d29ya19zdGF0ZS52MSIsInJlcXVpcmVkIjp0cnVlLCJjb25maWciOnsiaW5jbHVkZV9iYW5kd2lkdGhfZXN0aW1hdGVzIjp0cnVlfX0seyJpZCI6Im5ldHdvcmtfdXNhZ2UudjEiLCJyZXF1aXJlZCI6ZmFsc2UsImNvbmZpZyI6eyJ0cmFuc3BvcnRzIjpbIm1vYmlsZSIsIndpZmkiXSwicG9sbF9pbnRlcnZhbF9taW51dGVzIjo1fX0seyJpZCI6InVzYWdlX2V2ZW50cy52MSIsInJlcXVpcmVkIjpmYWxzZSwiY29uZmlnIjp7InBvbGxfaW50ZXJ2YWxfbWludXRlcyI6MTV9fSx7ImlkIjoibG9jYXRpb24udjEiLCJyZXF1aXJlZCI6ZmFsc2UsImNvbmZpZyI6eyJpbnRlcnZhbF9taWxsaXMiOjEwMDAwLCJtaW5pbXVtX2ludGVydmFsX21pbGxpcyI6NTAwMCwibWF4aW11bV9iYXRjaF9kZWxheV9taWxsaXMiOjMwMDAwLCJtaW5pbXVtX2Rpc3BsYWNlbWVudF9tZXRlcnMiOjUuMCwicHJpb3JpdHkiOiJCQUxBTkNFRCJ9fSx7ImlkIjoia2V5Ym9hcmRfdG91Y2gudjEiLCJyZXF1aXJlZCI6ZmFsc2UsImNvbmZpZyI6eyJ0cmFqZWN0b3J5X3NhbXBsaW5nX2h6Ijo2MH19XSwic3VydmV5cyI6W3siaWQiOiJkZW1vLXN1cnZleSIsInRpdGxlIjp7ImRlZmF1bHQiOiJTdHVkeSBjaGVjay1pbiIsInRyYW5zbGF0aW9ucyI6eyJ6aC1UVyI6IueglOeptueiuuiqjSJ9fSwiZGVzY3JpcHRpb24iOnsiZGVmYXVsdCI6IlRlbGwgdXMgaG93IHRoZSBzdHVkeSBpcyBnb2luZy4iLCJ0cmFuc2xhdGlvbnMiOnt9fSwicXVlc3Rpb25zIjpbeyJ0eXBlIjoic2hvcnRfdGV4dCIsImlkIjoic3RhdHVzLW5vdGUiLCJwcm9tcHQiOnsiZGVmYXVsdCI6IkhvdyBpcyBpdCBnb2luZz8iLCJ0cmFuc2xhdGlvbnMiOnt9fSwicmVxdWlyZWQiOmZhbHNlLCJtYXhpbXVtX2xlbmd0aCI6NTAwfV19XSwiaW50ZXJ2ZW50aW9ucyI6W3siaWQiOiJkZW1vLWNoZWNrLWluIiwiYWN0aW9uIjp7InR5cGUiOiJzdXJ2ZXkiLCJub3RpZmljYXRpb25fdGl0bGUiOiJTdHVkeSBjaGVjay1pbiIsIm5vdGlmaWNhdGlvbl9tZXNzYWdlIjoiUGxlYXNlIGNvbXBsZXRlIHRoZSBzdHVkeSBjaGVjay1pbi4iLCJzdXJ2ZXlfaWQiOiJkZW1vLXN1cnZleSJ9LCJ0cmlnZ2VycyI6W3siaWQiOiJhZnRlci1vbmUtaG91ciIsInNjaGVkdWxlIjp7InR5cGUiOiJvbmVfdGltZSIsIm9mZnNldF9taW51dGVzIjo2MCwiY2xvY2siOiJBQ1RJVkVfUlVOTklOR19USU1FIn0sImF2YWlsYWJpbGl0eV9taW51dGVzIjoxNDQwfV19XSwic3RvcmFnZSI6eyJtYXhpbXVtX2xvY2FsX2J5dGVzIjoxNjc3NzIxNn0sInNpZ25lciI6eyJrZXlfaWQiOiJkZW1vLXNpZ25lci0yMDI2IiwicHVibGljX2tleSI6Ik1Db3dCUVlESzJWd0F5RUFzUlNhVHBabVRTQkw3ZU42blMvSEJzTm1MTThuMWhkUm1JdDF2dExac0MwPSJ9LCJleHBvcnQiOnsicmVzZWFyY2hlcl9rZXlfaWQiOiJkZW1vLWhwa2UtMjAyNiIsInRpbmtfaHBrZV9wdWJsaWNfa2V5c2V0Ijp7InByaW1hcnlLZXlJZCI6MjE4OTkyNzI3LCJrZXkiOlt7ImtleURhdGEiOnsidHlwZVVybCI6InR5cGUuZ29vZ2xlYXBpcy5jb20vZ29vZ2xlLmNyeXB0by50aW5rLkhwa2VQdWJsaWNLZXkiLCJ2YWx1ZSI6IkVnWUlBUkFCR0FJYUlCcHlRM3c0ZkZ4OVhnRVV4NWt5elphSVBYTHE3YVlVNlJKK3k5K3JHTkVBIiwia2V5TWF0ZXJpYWxUeXBlIjoiQVNZTU1FVFJJQ19QVUJMSUMifSwic3RhdHVzIjoiRU5BQkxFRCIsImtleUlkIjoyMTg5OTI3MjcsIm91dHB1dFByZWZpeFR5cGUiOiJUSU5LIn1dfX0sInVwbG9hZCI6e319eQt/MbwbgKiep/lySOSuib/oEqqRxyR9LuR7lffn30Tz7G7jz1ue7a3mV+1zqhtEbdYAoLnhleIZRa0GmcvcCA== diff --git a/app/src/main/AndroidManifest.xml b/app/src/main/AndroidManifest.xml index e8ca73f..a7f151b 100644 --- a/app/src/main/AndroidManifest.xml +++ b/app/src/main/AndroidManifest.xml @@ -37,6 +37,9 @@ + + + diff --git a/app/src/main/kotlin/cool/linc/androiddatacollector/BootRecoveryReceiver.kt b/app/src/main/kotlin/cool/linc/androiddatacollector/BootRecoveryReceiver.kt index 64502a4..f15f18f 100644 --- a/app/src/main/kotlin/cool/linc/androiddatacollector/BootRecoveryReceiver.kt +++ b/app/src/main/kotlin/cool/linc/androiddatacollector/BootRecoveryReceiver.kt @@ -8,15 +8,20 @@ import kotlinx.coroutines.launch class BootRecoveryReceiver : BroadcastReceiver() { override fun onReceive(context: Context, intent: Intent) { - if (intent.action != Intent.ACTION_BOOT_COMPLETED) return + if (intent.action !in RECOVERY_ACTIONS) return val pending = goAsync() val application = context.applicationContext as CollectorApplication application.applicationScope.launch { try { application.session.snapshot.first { it.initialized } + application.session.rescheduleInterventions() } finally { pending.finish() } } } + + private companion object { + val RECOVERY_ACTIONS = setOf(Intent.ACTION_BOOT_COMPLETED, Intent.ACTION_TIME_CHANGED, Intent.ACTION_TIMEZONE_CHANGED) + } } diff --git a/app/src/main/kotlin/cool/linc/androiddatacollector/CollectorApplication.kt b/app/src/main/kotlin/cool/linc/androiddatacollector/CollectorApplication.kt index 51fa6c2..0a64238 100644 --- a/app/src/main/kotlin/cool/linc/androiddatacollector/CollectorApplication.kt +++ b/app/src/main/kotlin/cool/linc/androiddatacollector/CollectorApplication.kt @@ -102,6 +102,7 @@ class CollectorApplication : Application() { // The delivery chain is one-time work, so it has no platform-side repetition to fall // back on. Re-establishing it here covers a link lost to a crash or a force stop. session.snapshot.value.configuration?.let(workScheduler::reschedulePendingWork) + session.rescheduleInterventions() } } diff --git a/app/src/main/kotlin/cool/linc/androiddatacollector/CollectorDashboard.kt b/app/src/main/kotlin/cool/linc/androiddatacollector/CollectorDashboard.kt index 9d14e57..26a81b0 100644 --- a/app/src/main/kotlin/cool/linc/androiddatacollector/CollectorDashboard.kt +++ b/app/src/main/kotlin/cool/linc/androiddatacollector/CollectorDashboard.kt @@ -423,6 +423,7 @@ private fun ConsentPanel(study: StudyUiState.ActiveStudy, actions: StudyUiAction // prose, so they describe what the app will actually do even if the summary leaves it out. // Their wording is deliberately complete. PublisherDisclosure(study.configuration, study.signerAnchored) + IdentityDisclosure(study.configuration.assignedParticipantId) UploadDisclosure(study.configuration.upload) Row(verticalAlignment = Alignment.CenterVertically) { Checkbox( @@ -441,6 +442,27 @@ private fun ConsentPanel(study: StudyUiState.ActiveStudy, actions: StudyUiAction } } +@Composable +private fun IdentityDisclosure(assignedParticipantId: String?) { + Disclosure( + mark = { GlyphIcon(Glyph.PERSON, MaterialTheme.colorScheme.primary, 16.dp) }, + title = stringResource( + if (assignedParticipantId == null) R.string.consent_identity_anonymous_title + else R.string.consent_identity_personalized_title, + ), + ) { + Column(verticalArrangement = Arrangement.spacedBy(6.dp)) { + if (assignedParticipantId == null) { + Text(stringResource(R.string.consent_identity_anonymous_body)) + } else { + Text(stringResource(R.string.consent_identity_assigned_code, assignedParticipantId), fontWeight = FontWeight.Bold) + Text(stringResource(R.string.consent_identity_personalized_body)) + } + Text(stringResource(R.string.consent_identity_instance_body)) + } + } +} + /** * A block the app asserts itself, set apart from the researcher's prose above it by ground rather * than by being a separate card: it belongs to this step, and the step shows one thing. @@ -727,6 +749,10 @@ private fun StudyDetails(study: StudyUiState.ActiveStudy) { if (expanded) { HorizontalDivider() DetailRow(stringResource(R.string.details_configuration), configuration.configurationId) + DetailRow(stringResource(R.string.details_instance_id), study.metadata.participantInstanceId) + configuration.assignedParticipantId?.let { + DetailRow(stringResource(R.string.details_assigned_id), it) + } DetailRow(stringResource(R.string.details_consent_document), configuration.consentDocumentVersion) DetailRow(stringResource(R.string.details_signature), configuration.signer.fingerprint) study.lastExport?.let { diff --git a/app/src/main/kotlin/cool/linc/androiddatacollector/MainActivity.kt b/app/src/main/kotlin/cool/linc/androiddatacollector/MainActivity.kt index ac9a58d..67d597c 100644 --- a/app/src/main/kotlin/cool/linc/androiddatacollector/MainActivity.kt +++ b/app/src/main/kotlin/cool/linc/androiddatacollector/MainActivity.kt @@ -1,6 +1,7 @@ package cool.linc.androiddatacollector import android.Manifest +import android.content.Intent import android.os.Bundle import androidx.activity.ComponentActivity import androidx.activity.compose.setContent @@ -8,8 +9,12 @@ import androidx.activity.enableEdgeToEdge import androidx.activity.result.contract.ActivityResultContracts import androidx.activity.viewModels import androidx.lifecycle.compose.collectAsStateWithLifecycle +import androidx.lifecycle.lifecycleScope +import cool.linc.androiddatacollector.platform.InterventionWorker import cool.linc.androiddatacollector.core.collector.AccessKind import java.time.Instant +import kotlinx.coroutines.flow.first +import kotlinx.coroutines.launch class MainActivity : ComponentActivity() { private val collectorApplication: CollectorApplication @@ -41,6 +46,7 @@ class MainActivity : ComponentActivity() { override fun onCreate(savedInstanceState: Bundle?) { super.onCreate(savedInstanceState) + openOccurrence(intent) enableEdgeToEdge() setContent { val state = viewModel.state.collectAsStateWithLifecycle().value @@ -73,6 +79,21 @@ class MainActivity : ComponentActivity() { viewModel.refreshAccess() } + override fun onNewIntent(intent: Intent) { + super.onNewIntent(intent) + setIntent(intent) + openOccurrence(intent) + } + + private fun openOccurrence(intent: Intent) { + if (intent.action != InterventionWorker.ACTION_OPEN_OCCURRENCE) return + val occurrenceId = intent.getStringExtra(InterventionWorker.KEY_OCCURRENCE_ID) ?: return + lifecycleScope.launch { + val ready = collectorApplication.session.snapshot.first { it.initialized } + if (ready.configuration != null) collectorApplication.session.openOccurrence(occurrenceId) + } + } + /** * Null in a release build, which ships no demonstration study, so the dashboard leaves the * entry point out entirely rather than showing something that cannot work. diff --git a/app/src/main/kotlin/cool/linc/androiddatacollector/SurveyActivity.kt b/app/src/main/kotlin/cool/linc/androiddatacollector/SurveyActivity.kt new file mode 100644 index 0000000..54ee51e --- /dev/null +++ b/app/src/main/kotlin/cool/linc/androiddatacollector/SurveyActivity.kt @@ -0,0 +1,381 @@ +package cool.linc.androiddatacollector + +import android.os.Bundle +import androidx.activity.ComponentActivity +import androidx.activity.compose.setContent +import androidx.activity.enableEdgeToEdge +import androidx.activity.viewModels +import androidx.compose.foundation.layout.Arrangement +import androidx.compose.foundation.layout.Column +import androidx.compose.foundation.layout.Row +import androidx.compose.foundation.layout.fillMaxSize +import androidx.compose.foundation.layout.fillMaxWidth +import androidx.compose.foundation.layout.padding +import androidx.compose.foundation.rememberScrollState +import androidx.compose.foundation.selection.selectable +import androidx.compose.foundation.selection.selectableGroup +import androidx.compose.foundation.selection.toggleable +import androidx.compose.foundation.verticalScroll +import androidx.compose.material3.AlertDialog +import androidx.compose.material3.ExperimentalMaterial3Api +import androidx.compose.material3.Button +import androidx.compose.material3.Checkbox +import androidx.compose.material3.HorizontalDivider +import androidx.compose.material3.LinearProgressIndicator +import androidx.compose.material3.MaterialTheme +import androidx.compose.material3.OutlinedButton +import androidx.compose.material3.OutlinedTextField +import androidx.compose.material3.RadioButton +import androidx.compose.material3.Scaffold +import androidx.compose.material3.Slider +import androidx.compose.material3.Text +import androidx.compose.material3.TopAppBar +import androidx.compose.runtime.Composable +import androidx.compose.runtime.getValue +import androidx.compose.runtime.mutableStateOf +import androidx.compose.runtime.remember +import androidx.compose.runtime.setValue +import androidx.compose.ui.Modifier +import androidx.compose.ui.Alignment +import androidx.compose.ui.platform.LocalConfiguration +import androidx.compose.ui.res.stringResource +import androidx.compose.ui.semantics.Role +import androidx.compose.ui.semantics.contentDescription +import androidx.compose.ui.semantics.semantics +import androidx.compose.ui.unit.dp +import androidx.lifecycle.ViewModel +import androidx.lifecycle.ViewModelProvider +import androidx.lifecycle.compose.collectAsStateWithLifecycle +import androidx.lifecycle.viewModelScope +import cool.linc.androiddatacollector.core.application.StudySessionManager +import cool.linc.androiddatacollector.core.definition.MultipleChoiceQuestion +import cool.linc.androiddatacollector.core.definition.ScaleQuestion +import cool.linc.androiddatacollector.core.definition.ShortTextQuestion +import cool.linc.androiddatacollector.core.definition.SingleChoiceQuestion +import cool.linc.androiddatacollector.core.definition.SurveyAction +import cool.linc.androiddatacollector.core.definition.SurveyDefinition +import cool.linc.androiddatacollector.core.definition.SurveyQuestion +import cool.linc.androiddatacollector.core.model.OccurrenceState +import cool.linc.androiddatacollector.core.runtime.SurveyAnswer +import cool.linc.androiddatacollector.core.runtime.SurveySubmissionResult +import kotlinx.coroutines.flow.MutableStateFlow +import kotlinx.coroutines.flow.asStateFlow +import kotlinx.coroutines.flow.first +import kotlinx.coroutines.flow.update +import kotlinx.coroutines.launch +import org.json.JSONArray +import org.json.JSONObject + +class SurveyActivity : ComponentActivity() { + private val occurrenceId by lazy { + requireNotNull(intent.getStringExtra(OCCURRENCE_ID)) { "Missing occurrence ID" } + } + private val viewModel by viewModels { + SurveyViewModel.Factory((application as CollectorApplication).session, occurrenceId) + } + + override fun onCreate(savedInstanceState: Bundle?) { + super.onCreate(savedInstanceState) + enableEdgeToEdge() + setContent { + MaterialTheme { + SurveyScreen(viewModel, onClose = ::finish) + } + } + } + + companion object { const val OCCURRENCE_ID = "occurrence_id" } +} + +data class SurveyScreenState( + val loading: Boolean = true, + val survey: SurveyDefinition? = null, + val answers: Map = emptyMap(), + val editable: Boolean = false, + val submitted: Boolean = false, + val message: String? = null, +) + +class SurveyViewModel( + private val session: StudySessionManager, + private val occurrenceId: String, +) : ViewModel() { + private val mutableState = MutableStateFlow(SurveyScreenState()) + val state = mutableState.asStateFlow() + + init { + viewModelScope.launch { + val ready = session.snapshot.first { it.initialized } + if (ready.configuration == null) { + mutableState.value = SurveyScreenState(loading = false, message = "unavailable") + return@launch + } + val dispatch = session.openOccurrence(occurrenceId) + val action = dispatch?.action as? SurveyAction + val configuration = ready.configuration + val survey = configuration?.surveys?.firstOrNull { it.id == action?.surveyId } + if (dispatch == null || survey == null) { + val occurrence = ready.runtime.metadata?.occurrences?.get(occurrenceId) + val message = if (occurrence?.state == OccurrenceState.EXPIRED) "expired" else "unavailable" + mutableState.value = SurveyScreenState(loading = false, message = message) + return@launch + } + val submitted = dispatch.occurrence.state == OccurrenceState.SURVEY_SUBMITTED + val restored = if (submitted) decodeAnswers(survey) else emptyMap() + mutableState.value = SurveyScreenState( + loading = false, + survey = survey, + answers = restored, + editable = !submitted, + submitted = submitted, + message = if (submitted) "submitted" else null, + ) + } + } + + fun answer(questionId: String, answer: SurveyAnswer?) { + if (!mutableState.value.editable) return + mutableState.update { current -> + current.copy(answers = if (answer == null) current.answers - questionId else current.answers + (questionId to answer)) + } + } + + fun submit() { + val current = mutableState.value + if (!current.editable || current.survey == null) return + mutableState.update { it.copy(editable = false, message = null) } + viewModelScope.launch { + val result = session.submitSurvey(occurrenceId, current.answers) + val committed = if (result == SurveySubmissionResult.ALREADY_SUBMITTED) { + decodeAnswers(current.survey) + } else { + emptyMap() + } + mutableState.update { submissionState(it, result, committed) } + } + } + + private suspend fun decodeAnswers(survey: SurveyDefinition): Map { + val encoded = session.surveySubmissionEvent(occurrenceId)?.fields?.get("answers_json") ?: return emptyMap() + val root = JSONObject(encoded) + return survey.questions.mapNotNull { question -> + if (!root.has(question.id)) return@mapNotNull null + question.id to when (question) { + is ShortTextQuestion -> SurveyAnswer.Text(root.getString(question.id)) + is ScaleQuestion -> SurveyAnswer.Integer(root.getInt(question.id)) + is SingleChoiceQuestion, + is MultipleChoiceQuestion -> SurveyAnswer.Choices(root.getJSONArray(question.id).strings()) + } + }.toMap() + } + + private fun JSONArray.strings() = List(length()) { getString(it) } + + class Factory(private val session: StudySessionManager, private val occurrenceId: String) : ViewModelProvider.Factory { + @Suppress("UNCHECKED_CAST") + override fun create(modelClass: Class): T = + SurveyViewModel(session, occurrenceId) as T + } +} + +@Composable +@OptIn(ExperimentalMaterial3Api::class) +private fun SurveyScreen(viewModel: SurveyViewModel, onClose: () -> Unit) { + val state = viewModel.state.collectAsStateWithLifecycle().value + val survey = state.survey + val language = LocalConfiguration.current.locales[0].toLanguageTag() + var confirm by remember { mutableStateOf(false) } + Scaffold( + topBar = { TopAppBar(title = { Text(survey?.title?.resolve(language) ?: stringResource(R.string.survey_title)) }) }, + ) { padding -> + when { + state.loading -> Text(stringResource(R.string.survey_loading), Modifier.padding(padding).padding(24.dp)) + survey == null -> SurveyUnavailable(state.message, onClose, Modifier.padding(padding)) + else -> Column( + Modifier.fillMaxSize().padding(padding).verticalScroll(rememberScrollState()).padding(24.dp), + verticalArrangement = Arrangement.spacedBy(20.dp), + ) { + Text(survey.description.resolve(language), style = MaterialTheme.typography.bodyLarge) + LinearProgressIndicator( + progress = { answeredFraction(survey, state.answers) }, + modifier = Modifier.fillMaxWidth().semantics { + contentDescription = "${state.answers.size} / ${survey.questions.size}" + }, + ) + survey.questions.forEachIndexed { index, question -> + SurveyQuestionField( + index = index, + question = question, + language = language, + answer = state.answers[question.id], + readOnly = !state.editable, + onAnswer = { viewModel.answer(question.id, it) }, + ) + HorizontalDivider() + } + state.message?.let { Text(messageText(it), color = MaterialTheme.colorScheme.primary) } + Row(horizontalArrangement = Arrangement.spacedBy(12.dp)) { + OutlinedButton(onClick = onClose) { Text(stringResource(R.string.survey_close)) } + if (state.editable) { + Button( + onClick = { confirm = true }, + enabled = validSurveyAnswers(survey, state.answers), + ) { Text(stringResource(R.string.survey_submit)) } + } + } + } + } + } + if (confirm) AlertDialog( + onDismissRequest = { confirm = false }, + title = { Text(stringResource(R.string.survey_confirm_title)) }, + text = { Text(stringResource(R.string.survey_confirm_body)) }, + confirmButton = { + Button(onClick = { confirm = false; viewModel.submit() }) { Text(stringResource(R.string.survey_submit)) } + }, + dismissButton = { OutlinedButton(onClick = { confirm = false }) { Text(stringResource(R.string.survey_cancel)) } }, + ) +} + +internal fun submissionState( + current: SurveyScreenState, + result: SurveySubmissionResult, + committedAnswers: Map = emptyMap(), +): SurveyScreenState = when (result) { + SurveySubmissionResult.ACCEPTED -> current.copy(editable = false, submitted = true, message = "submitted") + SurveySubmissionResult.ALREADY_SUBMITTED -> current.copy( + answers = committedAnswers, + editable = false, + submitted = true, + message = "submitted", + ) + SurveySubmissionResult.EXPIRED -> current.copy(editable = false, message = "expired") + SurveySubmissionResult.INVALID -> current.copy(editable = true, message = "invalid") +} + +@Composable +private fun SurveyQuestionField( + index: Int, + question: SurveyQuestion, + language: String, + answer: SurveyAnswer?, + readOnly: Boolean, + onAnswer: (SurveyAnswer?) -> Unit, +) { + Column(verticalArrangement = Arrangement.spacedBy(8.dp)) { + Text("${index + 1}. ${question.prompt.resolve(language)}", style = MaterialTheme.typography.titleMedium) + Text( + stringResource(if (question.required) R.string.survey_required else R.string.survey_optional), + style = MaterialTheme.typography.labelMedium, + ) + when (question) { + is ShortTextQuestion -> OutlinedTextField( + value = (answer as? SurveyAnswer.Text)?.value.orEmpty(), + onValueChange = { if (it.length <= question.maximumLength) onAnswer(SurveyAnswer.Text(it)) }, + enabled = !readOnly, + supportingText = { Text("${(answer as? SurveyAnswer.Text)?.value?.length ?: 0}/${question.maximumLength}") }, + modifier = Modifier.fillMaxWidth().semantics { + contentDescription = question.prompt.resolve(language) + }, + ) + is ScaleQuestion -> { + val value = (answer as? SurveyAnswer.Integer)?.value ?: question.minimum + Slider( + value = value.toFloat(), + onValueChange = { onAnswer(SurveyAnswer.Integer(it.toInt())) }, + valueRange = question.minimum.toFloat()..question.maximum.toFloat(), + steps = (question.maximum - question.minimum - 1).coerceAtLeast(0), + enabled = !readOnly, + modifier = Modifier.semantics { + contentDescription = "${question.prompt.resolve(language)}: $value" + }, + ) + Row(Modifier.fillMaxWidth(), horizontalArrangement = Arrangement.SpaceBetween) { + Text("${question.minimum} · ${question.minimumLabel.resolve(language)}") + Text("$value") + Text("${question.maximum} · ${question.maximumLabel.resolve(language)}") + } + } + is SingleChoiceQuestion -> Column(Modifier.selectableGroup()) { + question.options.forEach { option -> + val selected = (answer as? SurveyAnswer.Choices)?.optionIds?.singleOrNull() == option.id + Row( + Modifier.fillMaxWidth().selectable( + selected = selected, + enabled = !readOnly, + role = Role.RadioButton, + onClick = { onAnswer(SurveyAnswer.Choices(listOf(option.id))) }, + ), + verticalAlignment = Alignment.CenterVertically, + ) { + RadioButton( + selected = selected, + onClick = null, + enabled = !readOnly, + ) + Text(option.label.resolve(language)) + } + } + } + is MultipleChoiceQuestion -> question.options.forEach { option -> + val selected = option.id in ((answer as? SurveyAnswer.Choices)?.optionIds ?: emptyList()) + val update = { + val current = (answer as? SurveyAnswer.Choices)?.optionIds.orEmpty() + val next = if (selected) current - option.id else current + option.id + if (next.size <= question.maximumSelections) onAnswer(SurveyAnswer.Choices(next)) + } + Row( + Modifier.fillMaxWidth().toggleable( + value = selected, + enabled = !readOnly, + role = Role.Checkbox, + onValueChange = { update() }, + ), + verticalAlignment = Alignment.CenterVertically, + ) { + Checkbox( + checked = selected, + onCheckedChange = null, + enabled = !readOnly, + ) + Text(option.label.resolve(language)) + } + } + } + } +} + +@Composable +private fun SurveyUnavailable(code: String?, onClose: () -> Unit, modifier: Modifier = Modifier) { + Column(modifier.padding(24.dp), verticalArrangement = Arrangement.spacedBy(16.dp)) { + Text(if (code == "expired") stringResource(R.string.survey_expired) else stringResource(R.string.survey_unavailable)) + Button(onClick = onClose) { Text(stringResource(R.string.survey_close)) } + } +} + +@Composable +private fun messageText(code: String): String = when (code) { + "submitted" -> stringResource(R.string.survey_submitted) + "expired" -> stringResource(R.string.survey_expired) + "invalid" -> stringResource(R.string.survey_invalid) + else -> stringResource(R.string.survey_unavailable) +} + +private fun answeredFraction(survey: SurveyDefinition, answers: Map): Float = + answers.size.toFloat() / survey.questions.size + +private fun validSurveyAnswers(survey: SurveyDefinition, answers: Map): Boolean = + survey.questions.all { question -> + val answer = answers[question.id] ?: return@all !question.required + when (question) { + is ShortTextQuestion -> answer is SurveyAnswer.Text && + answer.value.length <= question.maximumLength && (!question.required || answer.value.isNotBlank()) + is ScaleQuestion -> answer is SurveyAnswer.Integer && answer.value in question.minimum..question.maximum + is SingleChoiceQuestion -> answer is SurveyAnswer.Choices && answer.optionIds.size == 1 && + answer.optionIds.single() in question.options.map { it.id } + is MultipleChoiceQuestion -> answer is SurveyAnswer.Choices && + answer.optionIds.distinct().size == answer.optionIds.size && + answer.optionIds.size in question.minimumSelections..question.maximumSelections && + answer.optionIds.all { id -> question.options.any { it.id == id } } + } + } diff --git a/app/src/main/kotlin/cool/linc/androiddatacollector/platform/AndroidStudyPlatform.kt b/app/src/main/kotlin/cool/linc/androiddatacollector/platform/AndroidStudyPlatform.kt index 15bdb04..f8a3bf7 100644 --- a/app/src/main/kotlin/cool/linc/androiddatacollector/platform/AndroidStudyPlatform.kt +++ b/app/src/main/kotlin/cool/linc/androiddatacollector/platform/AndroidStudyPlatform.kt @@ -3,8 +3,11 @@ package cool.linc.androiddatacollector.platform import android.Manifest import android.app.NotificationChannel import android.app.NotificationManager +import android.app.PendingIntent import android.content.Context +import android.content.Intent import android.content.pm.PackageManager +import android.net.Uri import androidx.work.BackoffPolicy import androidx.work.Constraints import androidx.work.Data @@ -12,16 +15,22 @@ import androidx.work.ExistingWorkPolicy import androidx.work.NetworkType import androidx.work.OneTimeWorkRequestBuilder import androidx.work.WorkManager -import androidx.work.Worker +import androidx.work.CoroutineWorker import androidx.work.WorkerParameters import cool.linc.androiddatacollector.CollectionService import cool.linc.androiddatacollector.ExperimentDeadlineWorker +import cool.linc.androiddatacollector.MainActivity +import cool.linc.androiddatacollector.R +import cool.linc.androiddatacollector.SurveyActivity import cool.linc.androiddatacollector.UploadWorker import cool.linc.androiddatacollector.core.application.StudyCollectionHost import cool.linc.androiddatacollector.core.application.StudyWorkScheduler import cool.linc.androiddatacollector.core.definition.StudyConfiguration +import cool.linc.androiddatacollector.core.definition.SurveyAction import cool.linc.androiddatacollector.core.definition.UploadConfiguration +import cool.linc.androiddatacollector.core.model.InterventionOccurrence import java.util.concurrent.TimeUnit +import kotlinx.coroutines.flow.first class AndroidStudyCollectionHost( private val context: Context, @@ -41,23 +50,6 @@ class AndroidStudyWorkScheduler( private val workManager = WorkManager.getInstance(context.applicationContext) override fun schedule(configuration: StudyConfiguration) { - configuration.prompts.forEach { prompt -> - val request = OneTimeWorkRequestBuilder() - .setInitialDelay(prompt.delayMinutes.toLong(), TimeUnit.MINUTES) - .setInputData( - Data.Builder() - .putString(PromptWorker.KEY_PROMPT_ID, prompt.id) - .putString(PromptWorker.KEY_MESSAGE, prompt.message) - .build(), - ) - .addTag(promptTag(configuration.experimentId)) - .build() - workManager.enqueueUniqueWork( - promptWorkName(configuration.experimentId, prompt.id), - ExistingWorkPolicy.REPLACE, - request, - ) - } val deadline = OneTimeWorkRequestBuilder() .setInitialDelay(configuration.durationHours.toLong(), TimeUnit.HOURS) .setInputData( @@ -74,6 +66,47 @@ class AndroidStudyWorkScheduler( configuration.upload?.let { scheduleUpload(configuration.experimentId, it, ExistingWorkPolicy.REPLACE) } } + override fun replaceInterventionWork( + configuration: StudyConfiguration, + occurrences: List, + ) { + workManager.cancelAllWorkByTag(interventionTag(configuration.experimentId)) + occurrences.forEach { enqueueOccurrence(configuration, it, ExistingWorkPolicy.REPLACE) } + } + + override fun enqueueOccurrence(configuration: StudyConfiguration, occurrence: InterventionOccurrence) { + enqueueOccurrence(configuration, occurrence, ExistingWorkPolicy.KEEP) + } + + private fun enqueueOccurrence( + configuration: StudyConfiguration, + occurrence: InterventionOccurrence, + policy: ExistingWorkPolicy, + ) { + val now = System.currentTimeMillis() + val delay = (occurrence.scheduledFor.wallTimeUtcMillis - now).coerceAtLeast(0) + val request = OneTimeWorkRequestBuilder() + .setInitialDelay(delay, TimeUnit.MILLISECONDS) + .setInputData(Data.Builder().putString(InterventionWorker.KEY_OCCURRENCE_ID, occurrence.occurrenceId).build()) + .addTag(interventionTag(configuration.experimentId)) + .build() + workManager.enqueueUniqueWork( + occurrenceWorkName(configuration.experimentId, occurrence.occurrenceId), + policy, + request, + ) + val expiry = OneTimeWorkRequestBuilder() + .setInitialDelay((occurrence.expiresAtUtcMillis - now).coerceAtLeast(0), TimeUnit.MILLISECONDS) + .setInputData(Data.Builder().putString(InterventionWorker.KEY_OCCURRENCE_ID, occurrence.occurrenceId).build()) + .addTag(interventionTag(configuration.experimentId)) + .build() + workManager.enqueueUniqueWork( + "${occurrenceWorkName(configuration.experimentId, occurrence.occurrenceId)}-expiry", + policy, + expiry, + ) + } + /** * Enqueues one delivery attempt, which re-enqueues its successor when it finishes. * @@ -121,7 +154,7 @@ class AndroidStudyWorkScheduler( } override fun cancelCollectionWork(experimentId: String) { - workManager.cancelAllWorkByTag(promptTag(experimentId)) + workManager.cancelAllWorkByTag(interventionTag(experimentId)) workManager.cancelUniqueWork(deadlineWorkName(experimentId)) } @@ -130,46 +163,84 @@ class AndroidStudyWorkScheduler( workManager.cancelUniqueWork(uploadWorkName(experimentId)) } - private fun promptTag(experimentId: String) = "adc-prompt-$experimentId" - private fun promptWorkName(experimentId: String, promptId: String) = "adc-prompt-$experimentId-$promptId" + private fun interventionTag(experimentId: String) = "adc-intervention-$experimentId" + private fun occurrenceWorkName(experimentId: String, occurrenceId: String) = "adc-intervention-$experimentId-$occurrenceId" private fun deadlineWorkName(experimentId: String) = "adc-deadline-$experimentId" companion object { fun uploadWorkName(experimentId: String) = "adc-upload-$experimentId" } } -class PromptWorker( +class InterventionWorker( context: Context, parameters: WorkerParameters, -) : Worker(context, parameters) { - override fun doWork(): Result { +) : CoroutineWorker(context, parameters) { + override suspend fun doWork(): Result { + val occurrenceId = inputData.getString(KEY_OCCURRENCE_ID) ?: return Result.failure() + val application = applicationContext as cool.linc.androiddatacollector.CollectorApplication + if (application.session.snapshot.first { it.initialized }.configuration == null) return Result.success() + val dispatch = application.session.claimOccurrence(occurrenceId) ?: return Result.success() if (applicationContext.checkSelfPermission(Manifest.permission.POST_NOTIFICATIONS) != PackageManager.PERMISSION_GRANTED ) { - return Result.failure() + // The durable claim remains POSTING. A retry can post it after permission is restored, + // or atomically expire it once its availability window closes. + return Result.retry() } - val promptId = inputData.getString(KEY_PROMPT_ID) ?: return Result.failure() - val message = inputData.getString(KEY_MESSAGE) ?: return Result.failure() + val target = if (dispatch.action is SurveyAction) SurveyActivity::class.java else MainActivity::class.java + val intent = Intent(applicationContext, target) + .setAction(ACTION_OPEN_OCCURRENCE) + .setData(Uri.Builder().scheme("adc").authority("occurrence").appendPath(occurrenceId).build()) + .putExtra(KEY_OCCURRENCE_ID, occurrenceId) + val pendingIntent = PendingIntent.getActivity( + applicationContext, + 0, + intent, + PendingIntent.FLAG_UPDATE_CURRENT or PendingIntent.FLAG_IMMUTABLE, + ) val manager = applicationContext.getSystemService(NotificationManager::class.java) manager.createNotificationChannel( - NotificationChannel(CHANNEL_ID, "Research prompts", NotificationManager.IMPORTANCE_DEFAULT), + NotificationChannel( + CHANNEL_ID, + applicationContext.getString(R.string.intervention_channel), + NotificationManager.IMPORTANCE_DEFAULT, + ), ) manager.notify( - promptId.hashCode(), + occurrenceId, + 0, android.app.Notification.Builder(applicationContext, CHANNEL_ID) .setSmallIcon(android.R.drawable.ic_dialog_info) - .setContentTitle("Research prompt") - .setContentText(message) - .setStyle(android.app.Notification.BigTextStyle().bigText(message)) + .setContentTitle(dispatch.action.notificationTitle) + .setContentText(dispatch.action.notificationMessage) + .setStyle(android.app.Notification.BigTextStyle().bigText(dispatch.action.notificationMessage)) + .setContentIntent(pendingIntent) .setAutoCancel(true) + .setTimeoutAfter((dispatch.occurrence.expiresAtUtcMillis - System.currentTimeMillis()).coerceAtLeast(1)) .build(), ) + application.session.markNotificationPosted(occurrenceId) return Result.success() } companion object { - const val KEY_PROMPT_ID = "prompt_id" - const val KEY_MESSAGE = "message" - private const val CHANNEL_ID = "research-prompts" + const val KEY_OCCURRENCE_ID = "occurrence_id" + const val ACTION_OPEN_OCCURRENCE = "cool.linc.androiddatacollector.OPEN_OCCURRENCE" + private const val CHANNEL_ID = "research-interventions-v1" + } +} + +/** Records the terminal no-response outcome even when the participant never taps a notification. */ +class InterventionExpiryWorker( + context: Context, + parameters: WorkerParameters, +) : CoroutineWorker(context, parameters) { + override suspend fun doWork(): Result { + val occurrenceId = inputData.getString(InterventionWorker.KEY_OCCURRENCE_ID) ?: return Result.failure() + val application = applicationContext as cool.linc.androiddatacollector.CollectorApplication + if (application.session.snapshot.first { it.initialized }.configuration == null) return Result.success() + application.session.claimOccurrence(occurrenceId) + applicationContext.getSystemService(NotificationManager::class.java).cancel(occurrenceId, 0) + return Result.success() } } diff --git a/app/src/main/kotlin/cool/linc/androiddatacollector/platform/OkHttpStudyUploader.kt b/app/src/main/kotlin/cool/linc/androiddatacollector/platform/OkHttpStudyUploader.kt index 6e17d61..0587e41 100644 --- a/app/src/main/kotlin/cool/linc/androiddatacollector/platform/OkHttpStudyUploader.kt +++ b/app/src/main/kotlin/cool/linc/androiddatacollector/platform/OkHttpStudyUploader.kt @@ -69,20 +69,16 @@ class OkHttpStudyUploader( } } - val request = Request.Builder() - .url(upload.endpoint) - .header("Content-Type", "application/octet-stream") - .header("X-ADC-Bundle-Format", ResearchExport.BUNDLE_FORMAT) - .header("X-ADC-Experiment-Id", configuration.experimentId) - .header("X-ADC-Configuration-Id", configuration.configurationId) - .header("X-ADC-Participant-Instance", metadata.participantInstanceId) - .header("X-ADC-Sequence-From", fromSequence.toString()) + val requestBuilder = Request.Builder().url(upload.endpoint) + uploadHeaders(configuration, metadata, fromSequence, toSequence).forEach { (name, value) -> + requestBuilder.header(name, value) + } + val request = requestBuilder // Named "at most" because headers are sent before the body is generated, and a budget // can stop the bundle at any earlier event boundary. The endpoint cannot learn the true // upper bound — that is inside the ciphertext — so it must not file by this value. // `X-ADC-Sequence-From` is exact and strictly increasing per participant, which is what // makes a usable deduplication key. - .header("X-ADC-Sequence-To-At-Most", toSequence.toString()) .post(body) .build() @@ -127,6 +123,22 @@ class OkHttpStudyUploader( } } +/** The complete unencrypted request surface. Assigned participant IDs are deliberately absent. */ +internal fun uploadHeaders( + configuration: StudyConfiguration, + metadata: StudyMetadata, + fromSequence: Long, + toSequence: Long, +): Map = mapOf( + "Content-Type" to "application/octet-stream", + "X-ADC-Bundle-Format" to ResearchExport.BUNDLE_FORMAT, + "X-ADC-Experiment-Id" to configuration.experimentId, + "X-ADC-Configuration-Id" to configuration.configurationId, + "X-ADC-Participant-Instance" to metadata.participantInstanceId, + "X-ADC-Sequence-From" to fromSequence.toString(), + "X-ADC-Sequence-To-At-Most" to toSequence.toString(), +) + /** Maps a transport failure onto a fixed code, so what reaches a screen or a log is never data. */ private fun Throwable.reasonCode(): String = when (this) { is java.net.SocketTimeoutException -> "UPLOAD_TIMEOUT" diff --git a/app/src/main/res/values-zh-rTW/strings.xml b/app/src/main/res/values-zh-rTW/strings.xml index 3e781c2..8c354f3 100644 --- a/app/src/main/res/values-zh-rTW/strings.xml +++ b/app/src/main/res/values-zh-rTW/strings.xml @@ -4,6 +4,20 @@ side by side. Researcher-supplied text is never translated and never appears here. --> + 研究活動 + 問卷 + 正在開啟問卷… + 必填 + 選填 + 送出一次 + 關閉 + 繼續檢查 + 要送出這份回答嗎？ + 送出後即為最終版本；你仍可查看，但不能修改或再次送出。 + 已送出。這份回答現在只能查看。 + 這次問卷已過期，無法送出。 + 請完成必填題目，並檢查每一項回答。 + 無法開啟這次問卷。 研究資料收集器 準備中 %1$d 分 @@ -84,6 +98,12 @@ 這個 App 內建信任此簽章者。 請與研究團隊公佈的金鑰指紋核對。 簽章證明設定檔在簽署後未被竄改，但不能證明是誰寫的。 + 匿名或假名研究 + 這份設定檔不含研究團隊指定的參與者代碼。 + 個人化研究 + 你的指定代碼：%1$s + 請與研究團隊交給你的不透明代碼核對。App 不會要求姓名、電子郵件或電話號碼。 + 每次匯入也會產生另一組隨機安裝代碼，用來串連這次安裝的加密資料分段，並區分重複匯入。 這個研究不會自動傳送資料 資料只留在這支手機，直到你自己匯出。 @@ -123,6 +143,8 @@ 設定 同意書 + 安裝代碼 + 指定代碼 簽章 上次匯出 %1$s 筆 · %2$s diff --git a/app/src/main/res/values/strings.xml b/app/src/main/res/values/strings.xml index a9bab21..43437ec 100644 --- a/app/src/main/res/values/strings.xml +++ b/app/src/main/res/values/strings.xml @@ -7,6 +7,20 @@ Android Data Collector + Research activities + Survey + Opening survey… + Required + Optional + Submit once + Close + Keep reviewing + Submit this response? + A submitted response is final. You can review it, but you cannot change or submit it again. + Submitted. This response is now read-only. + This survey occurrence has expired and cannot be submitted. + Complete the required questions and check each answer. + This survey occurrence is unavailable. Research Data Collector @@ -101,6 +115,12 @@ This app trusts this signer. Check this against the fingerprint your research team published. A signature shows the file has not been altered since it was signed. It does not show who wrote it. + Anonymous or pseudonymous study + This configuration contains no researcher-assigned participant code. + Personalized study + Your assigned code: %1$s + Check this opaque code against the one the research team gave you. The app does not ask for your name, email, or phone number. + This import also gets a separate random installation code. It links this installation’s encrypted chunks and distinguishes repeated imports. This study does not send data automatically What it collects stays on this phone until you export it yourself. @@ -147,6 +167,8 @@ Configuration Consent document + Installation code + Assigned code Signature Last export %1$s events · %2$s diff --git a/app/src/test/kotlin/cool/linc/androiddatacollector/SurveySubmissionStateTest.kt b/app/src/test/kotlin/cool/linc/androiddatacollector/SurveySubmissionStateTest.kt new file mode 100644 index 0000000..13ba92c --- /dev/null +++ b/app/src/test/kotlin/cool/linc/androiddatacollector/SurveySubmissionStateTest.kt @@ -0,0 +1,40 @@ +package cool.linc.androiddatacollector + +import cool.linc.androiddatacollector.core.runtime.SurveyAnswer +import cool.linc.androiddatacollector.core.runtime.SurveySubmissionResult +import org.junit.Assert.assertEquals +import org.junit.Assert.assertFalse +import org.junit.Assert.assertTrue +import org.junit.Test + +class SurveySubmissionStateTest { + @Test + fun alreadySubmittedReplacesLocalDraftWithDurableAnswer() { + val draft = mapOf("question-one" to SurveyAnswer.Text("losing draft")) + val committed = mapOf("question-one" to SurveyAnswer.Text("durable answer")) + + val state = submissionState( + SurveyScreenState(loading = false, answers = draft), + SurveySubmissionResult.ALREADY_SUBMITTED, + committed, + ) + + assertEquals(committed, state.answers) + assertTrue(state.submitted) + assertFalse(state.editable) + assertEquals("submitted", state.message) + } + + @Test + fun expiryPermanentlyDisablesEditingWhileInvalidInputCanBeCorrected() { + val current = SurveyScreenState(loading = false, editable = false) + + val expired = submissionState(current, SurveySubmissionResult.EXPIRED) + assertFalse(expired.editable) + assertEquals("expired", expired.message) + + val invalid = submissionState(current, SurveySubmissionResult.INVALID) + assertTrue(invalid.editable) + assertEquals("invalid", invalid.message) + } +} diff --git a/app/src/test/kotlin/cool/linc/androiddatacollector/platform/UploadIdentityTest.kt b/app/src/test/kotlin/cool/linc/androiddatacollector/platform/UploadIdentityTest.kt new file mode 100644 index 0000000..2cb681a --- /dev/null +++ b/app/src/test/kotlin/cool/linc/androiddatacollector/platform/UploadIdentityTest.kt @@ -0,0 +1,52 @@ +package cool.linc.androiddatacollector.platform + +import cool.linc.androiddatacollector.core.definition.AppLifecycleConfiguration +import cool.linc.androiddatacollector.core.definition.ExportConfiguration +import cool.linc.androiddatacollector.core.definition.SignerIdentity +import cool.linc.androiddatacollector.core.definition.StudyConfiguration +import cool.linc.androiddatacollector.core.definition.UploadConfiguration +import cool.linc.androiddatacollector.core.model.StudyMetadata +import java.time.Instant +import org.junit.Assert.assertEquals +import org.junit.Assert.assertNotEquals +import org.junit.Assert.assertTrue +import org.junit.Test + +class UploadIdentityTest { + @Test + fun importsAlwaysMintDistinctInstancesAndHeadersNeverExposeAssignedCode() { + val first = StudyMetadata.initial("identity-test", "identity-config", "assigned-secret") + val second = StudyMetadata.initial("identity-test", "identity-config", "assigned-secret") + assertNotEquals(first.participantInstanceId, second.participantInstanceId) + assertEquals("assigned-secret", first.assignedParticipantId) + + val headers = uploadHeaders(configuration(), first, 1, 9) + assertEquals(first.participantInstanceId, headers["X-ADC-Participant-Instance"]) + assertTrue(headers.keys.none { it.contains("Assigned", ignoreCase = true) }) + assertTrue(headers.values.none { it == "assigned-secret" }) + } + + private fun configuration() = StudyConfiguration( + schemaVersion = 1, + experimentId = "identity-test", + configurationId = "identity-config", + issuedAt = Instant.parse("2026-01-01T00:00:00Z"), + expiresAt = Instant.parse("2030-01-01T00:00:00Z"), + minimumAppVersion = 1, + title = "Identity test", + researcherName = "Researcher", + researcherContact = "research@example.invalid", + purpose = "Test upload identity separation.", + durationHours = 1, + consentDocumentVersion = "v1", + consentSummary = "Test consent.", + assignedParticipantId = "assigned-secret", + collectors = listOf(AppLifecycleConfiguration(true)), + surveys = emptyList(), + interventions = emptyList(), + maximumLocalBytes = 16_777_216, + signer = SignerIdentity("test-signer", "x".repeat(32)), + export = ExportConfiguration("export-key", "x".repeat(32)), + upload = UploadConfiguration("https://example.invalid/v1", 60, false), + ) +} diff --git a/core/experiment-runtime/src/main/kotlin/cool/linc/androiddatacollector/core/runtime/ExperimentRuntime.kt b/core/experiment-runtime/src/main/kotlin/cool/linc/androiddatacollector/core/runtime/ExperimentRuntime.kt index 5b16476..9fe04c0 100644 --- a/core/experiment-runtime/src/main/kotlin/cool/linc/androiddatacollector/core/runtime/ExperimentRuntime.kt +++ b/core/experiment-runtime/src/main/kotlin/cool/linc/androiddatacollector/core/runtime/ExperimentRuntime.kt @@ -12,8 +12,17 @@ import cool.linc.androiddatacollector.core.collector.EmitResult import cool.linc.androiddatacollector.core.collector.EventSink import cool.linc.androiddatacollector.core.collector.ResearchClocks import cool.linc.androiddatacollector.core.definition.StudyConfiguration +import cool.linc.androiddatacollector.core.definition.InterventionAction +import cool.linc.androiddatacollector.core.definition.MultipleChoiceQuestion +import cool.linc.androiddatacollector.core.definition.ScaleQuestion +import cool.linc.androiddatacollector.core.definition.ShortTextQuestion +import cool.linc.androiddatacollector.core.definition.SingleChoiceQuestion +import cool.linc.androiddatacollector.core.definition.SurveyAction +import cool.linc.androiddatacollector.core.definition.SurveyDefinition import cool.linc.androiddatacollector.core.model.EventDraft import cool.linc.androiddatacollector.core.model.ExperimentState +import cool.linc.androiddatacollector.core.model.InterventionOccurrence +import cool.linc.androiddatacollector.core.model.OccurrenceState import cool.linc.androiddatacollector.core.model.ExperimentStateMachine import cool.linc.androiddatacollector.core.model.RecordedEvent import cool.linc.androiddatacollector.core.model.StudyMetadata @@ -43,6 +52,19 @@ sealed interface CommandResult { data class Failed(val reasonCode: String) : CommandResult } +data class OccurrenceDispatch( + val occurrence: InterventionOccurrence, + val action: InterventionAction, +) + +sealed interface SurveyAnswer { + data class Text(val value: String) : SurveyAnswer + data class Integer(val value: Int) : SurveyAnswer + data class Choices(val optionIds: List) : SurveyAnswer +} + +enum class SurveySubmissionResult { ACCEPTED, ALREADY_SUBMITTED, EXPIRED, INVALID } + class ExperimentRuntime( val configuration: StudyConfiguration, private val store: StudyStore, @@ -62,16 +84,20 @@ class ExperimentRuntime( private val mutableSnapshot = MutableStateFlow(RuntimeSnapshot()) val snapshot: StateFlow = mutableSnapshot.asStateFlow() + fun now() = clocks.now() + suspend fun initialize(): CommandResult = executeCommand(requireInitialized = false) { check(currentMetadata == null) { "Runtime is already initialized" } val loaded = store.loadMetadata() ?: StudyMetadata.initial( configuration.experimentId, configuration.configurationId, + configuration.assignedParticipantId, ).also { initial -> store.initialize(initial) } check(loaded.experimentId == configuration.experimentId) { "Experiment ID mismatch" } check(loaded.configurationId == configuration.configurationId) { "Configuration ID mismatch" } + check(loaded.assignedParticipantId == configuration.assignedParticipantId) { "Assigned participant ID mismatch" } currentMetadata = loaded createCollectors() mutableSnapshot.update { @@ -178,6 +204,141 @@ class ExperimentRuntime( } } + suspend fun ensureOccurrence(planned: InterventionOccurrence): InterventionOccurrence = metadataMutex.withLock { + val metadata = requireMetadata() + metadata.occurrences[planned.occurrenceId]?.let { existing -> + if (existing.state == OccurrenceState.SCHEDULED && + (existing.scheduledFor.wallTimeUtcMillis != planned.scheduledFor.wallTimeUtcMillis || + existing.expiresAtUtcMillis != planned.expiresAtUtcMillis) + ) { + val revised = existing.copy( + scheduledFor = planned.scheduledFor, + expiresAtUtcMillis = planned.expiresAtUtcMillis, + ) + appendOccurrenceEvent( + metadata.copy(occurrences = metadata.occurrences + (revised.occurrenceId to revised)), + revised, + "INTERVENTION_RESCHEDULED", + clocks.now(), + ) + return@withLock revised + } + return@withLock existing + } + require(configuration.interventions.any { it.id == planned.interventionId }) { "Unknown intervention" } + appendOccurrenceEvent( + metadata.copy(occurrences = metadata.occurrences + (planned.occurrenceId to planned)), + planned, + "INTERVENTION_SCHEDULED", + clocks.now(), + ) + planned + } + + suspend fun claimOccurrence(occurrenceId: String): OccurrenceDispatch? = metadataMutex.withLock { + val metadata = requireMetadata() + val occurrence = metadata.occurrences[occurrenceId] ?: return@withLock null + val now = clocks.now() + if (now.wallTimeUtcMillis >= occurrence.expiresAtUtcMillis) { + expireOccurrence(metadata, occurrence, now) + return@withLock null + } + if (occurrence.state !in setOf(OccurrenceState.SCHEDULED, OccurrenceState.POSTING)) return@withLock null + val claimed = if (occurrence.state == OccurrenceState.SCHEDULED) { + occurrence.copy(state = OccurrenceState.POSTING).also { next -> + val updated = metadata.copy(occurrences = metadata.occurrences + (occurrenceId to next)) + store.saveMetadata(updated) + currentMetadata = updated + publishMetadata(updated) + } + } else { + occurrence + } + OccurrenceDispatch(claimed, intervention(claimed).action) + } + + suspend fun markNotificationPosted(occurrenceId: String) = metadataMutex.withLock { + val metadata = requireMetadata() + val occurrence = metadata.occurrences[occurrenceId] ?: return@withLock + if (occurrence.state != OccurrenceState.POSTING) return@withLock + val posted = occurrence.copy(state = OccurrenceState.NOTIFICATION_POSTED) + appendOccurrenceEvent( + metadata.copy(occurrences = metadata.occurrences + (occurrenceId to posted)), + posted, + "NOTIFICATION_POSTED", + clocks.now(), + ) + } + + suspend fun openOccurrence(occurrenceId: String): OccurrenceDispatch? = metadataMutex.withLock { + val metadata = requireMetadata() + val occurrence = metadata.occurrences[occurrenceId] ?: return@withLock null + val now = clocks.now() + if (now.wallTimeUtcMillis >= occurrence.expiresAtUtcMillis && occurrence.state != OccurrenceState.SURVEY_SUBMITTED) { + expireOccurrence(metadata, occurrence, now) + return@withLock null + } + if (occurrence.state in setOf(OccurrenceState.EXPIRED, OccurrenceState.SCHEDULED, OccurrenceState.POSTING)) { + return@withLock null + } + if (occurrence.state == OccurrenceState.NOTIFICATION_POSTED) { + val opened = occurrence.copy(state = OccurrenceState.OPENED, openedAt = now) + appendOccurrenceEvent( + metadata.copy(occurrences = metadata.occurrences + (occurrenceId to opened)), + opened, + if (intervention(opened).action is SurveyAction) "SURVEY_OPENED" else "INTERVENTION_OPENED", + now, + ) + return@withLock OccurrenceDispatch(opened, intervention(opened).action) + } + OccurrenceDispatch(occurrence, intervention(occurrence).action) + } + + suspend fun submitSurvey( + occurrenceId: String, + answers: Map, + ): SurveySubmissionResult = metadataMutex.withLock { + val metadata = requireMetadata() + val occurrence = metadata.occurrences[occurrenceId] ?: return@withLock SurveySubmissionResult.INVALID + if (occurrence.state == OccurrenceState.SURVEY_SUBMITTED) return@withLock SurveySubmissionResult.ALREADY_SUBMITTED + val now = clocks.now() + if (now.wallTimeUtcMillis >= occurrence.expiresAtUtcMillis) { + expireOccurrence(metadata, occurrence, now) + return@withLock SurveySubmissionResult.EXPIRED + } + if (occurrence.state != OccurrenceState.OPENED) return@withLock SurveySubmissionResult.INVALID + val survey = surveyFor(occurrence) ?: return@withLock SurveySubmissionResult.INVALID + val encoded = validateAndEncodeAnswers(survey, answers) ?: return@withLock SurveySubmissionResult.INVALID + val submitted = occurrence.copy( + state = OccurrenceState.SURVEY_SUBMITTED, + submittedAt = now, + submissionSequence = metadata.nextSequenceNumber, + ) + appendOccurrenceEvent( + metadata.copy(occurrences = metadata.occurrences + (occurrenceId to submitted)), + submitted, + "SURVEY_SUBMITTED", + now, + mapOf( + "survey_id" to survey.id, + "scheduled_time" to researchTimeJson(submitted.scheduledFor), + "opened_time" to researchTimeJson(requireNotNull(submitted.openedAt)), + "submitted_time" to researchTimeJson(now), + "answers_json" to encoded, + ), + ) + SurveySubmissionResult.ACCEPTED + } + + suspend fun surveySubmissionEvent(occurrenceId: String): RecordedEvent? = metadataMutex.withLock { + val metadata = requireMetadata() + val sequence = metadata.occurrences[occurrenceId]?.submissionSequence ?: return@withLock null + if (sequence < metadata.retainedFromSequence) return@withLock null + var found: RecordedEvent? = null + store.readEvents(sequence, sequence) { found = it } + found + } + suspend fun metadataForExport(): StudyMetadata = metadataMutex.withLock { val metadata = requireMetadata() require(metadata.state in EXPORTABLE_STATES) { "Experiment cannot be exported from ${metadata.state}" } @@ -270,7 +431,7 @@ class ExperimentRuntime( lastEvents = metadata.lastEvents + (recorded.collectorId to recorded), ) try { - store.appendEvent(recorded) + store.appendEventAtomically(recorded, updated) currentMetadata = updated publishMetadata(updated) EmitResult.Accepted(recorded.sequenceNumber) @@ -439,6 +600,112 @@ class ExperimentRuntime( private fun requireMetadata(): StudyMetadata = checkNotNull(currentMetadata) { "Runtime is not initialized" } + private fun intervention(occurrence: InterventionOccurrence) = + configuration.interventions.first { it.id == occurrence.interventionId } + + private fun surveyFor(occurrence: InterventionOccurrence): SurveyDefinition? = + (intervention(occurrence).action as? SurveyAction)?.let { action -> + configuration.surveys.firstOrNull { it.id == action.surveyId } + } + + private suspend fun expireOccurrence( + metadata: StudyMetadata, + occurrence: InterventionOccurrence, + now: cool.linc.androiddatacollector.core.model.ResearchTime, + ) { + if (occurrence.state in setOf(OccurrenceState.EXPIRED, OccurrenceState.SURVEY_SUBMITTED)) return + if (occurrence.state == OccurrenceState.OPENED && intervention(occurrence).action !is SurveyAction) return + val expired = occurrence.copy(state = OccurrenceState.EXPIRED) + appendOccurrenceEvent( + metadata.copy(occurrences = metadata.occurrences + (occurrence.occurrenceId to expired)), + expired, + if (intervention(expired).action is SurveyAction) "SURVEY_EXPIRED" else "INTERVENTION_EXPIRED", + now, + ) + } + + private suspend fun appendOccurrenceEvent( + metadataAfterState: StudyMetadata, + occurrence: InterventionOccurrence, + payloadType: String, + observedAt: cool.linc.androiddatacollector.core.model.ResearchTime, + additionalFields: Map = emptyMap(), + ) { + val event = RecordedEvent( + sequenceNumber = metadataAfterState.nextSequenceNumber, + collectorId = "interventions.v1", + payloadSchemaVersion = 1, + observedTime = observedAt, + payloadType = payloadType, + fields = mapOf( + "intervention_id" to occurrence.interventionId, + "trigger_id" to occurrence.triggerId, + "occurrence_id" to occurrence.occurrenceId, + "scheduled_for_utc_millis" to occurrence.scheduledFor.wallTimeUtcMillis.toString(), + ) + additionalFields, + ) + val updated = metadataAfterState.copy( + eventCount = event.sequenceNumber, + nextSequenceNumber = event.sequenceNumber + 1, + lastEvents = metadataAfterState.lastEvents + (event.collectorId to event), + ) + store.appendEventAtomically(event, updated) + currentMetadata = updated + publishMetadata(updated) + } + + private fun validateAndEncodeAnswers(survey: SurveyDefinition, answers: Map): String? { + if (answers.keys.any { key -> survey.questions.none { it.id == key } }) return null + survey.questions.forEach { question -> + val answer = answers[question.id] + if (answer == null) { + if (question.required) return null + return@forEach + } + val valid = when (question) { + is ShortTextQuestion -> answer is SurveyAnswer.Text && + answer.value.length <= question.maximumLength && (!question.required || answer.value.isNotBlank()) + is ScaleQuestion -> answer is SurveyAnswer.Integer && answer.value in question.minimum..question.maximum + is SingleChoiceQuestion -> answer is SurveyAnswer.Choices && answer.optionIds.size == 1 && + answer.optionIds.single() in question.options.map { it.id } + is MultipleChoiceQuestion -> answer is SurveyAnswer.Choices && + answer.optionIds.distinct().size == answer.optionIds.size && + answer.optionIds.size in question.minimumSelections..question.maximumSelections && + answer.optionIds.all { id -> id in question.options.map { it.id } } + } + if (!valid) return null + } + val encoded = answers.toSortedMap().entries.joinToString(separator = ",", prefix = "{", postfix = "}") { (id, answer) -> + "${jsonString(id)}:${when (answer) { + is SurveyAnswer.Text -> jsonString(answer.value) + is SurveyAnswer.Integer -> answer.value.toString() + is SurveyAnswer.Choices -> answer.optionIds.joinToString(separator = ",", prefix = "[", postfix = "]") { jsonString(it) } + }}" + } + return encoded.takeIf { it.toByteArray().size <= MAXIMUM_SURVEY_ANSWERS_BYTES } + } + + private fun researchTimeJson(time: cool.linc.androiddatacollector.core.model.ResearchTime): String = + "{\"wall_time_utc_millis\":${time.wallTimeUtcMillis},\"elapsed_realtime_nanos\":${time.elapsedRealtimeNanos}," + + "\"boot_session_id\":${jsonString(time.bootSessionId)}}" + + private fun jsonString(value: String): String = buildString { + append('"') + value.forEach { character -> + when (character) { + '"' -> append("\\\"") + '\\' -> append("\\\\") + '\b' -> append("\\b") + '\u000C' -> append("\\f") + '\n' -> append("\\n") + '\r' -> append("\\r") + '\t' -> append("\\t") + else -> if (character.code < 0x20) append("\\u%04x".format(character.code)) else append(character) + } + } + append('"') + } + private fun publishMetadata(metadata: StudyMetadata) { mutableSnapshot.update { it.copy(metadata = metadata) } } @@ -477,6 +744,7 @@ class ExperimentRuntime( const val INCIDENT_STORAGE_WRITE_FAILED = "STORAGE_WRITE_FAILED" const val INCIDENT_PAUSE_PERSISTENCE_FAILED = "PAUSE_PERSISTENCE_FAILED" + const val MAXIMUM_SURVEY_ANSWERS_BYTES = 60 * 1024 } } diff --git a/core/experiment-runtime/src/test/kotlin/cool/linc/androiddatacollector/core/runtime/ExperimentRuntimeTest.kt b/core/experiment-runtime/src/test/kotlin/cool/linc/androiddatacollector/core/runtime/ExperimentRuntimeTest.kt index 9e7603f..3735e97 100644 --- a/core/experiment-runtime/src/test/kotlin/cool/linc/androiddatacollector/core/runtime/ExperimentRuntimeTest.kt +++ b/core/experiment-runtime/src/test/kotlin/cool/linc/androiddatacollector/core/runtime/ExperimentRuntimeTest.kt @@ -2,6 +2,8 @@ package cool.linc.androiddatacollector.core.runtime import cool.linc.androiddatacollector.core.model.EventDraft import cool.linc.androiddatacollector.core.model.ExperimentState +import cool.linc.androiddatacollector.core.model.InterventionOccurrence +import cool.linc.androiddatacollector.core.model.OccurrenceState import cool.linc.androiddatacollector.core.model.RecordedEvent import cool.linc.androiddatacollector.core.model.ResearchTime import cool.linc.androiddatacollector.core.model.StorageUsage @@ -19,11 +21,25 @@ import cool.linc.androiddatacollector.core.collector.EmitResult import cool.linc.androiddatacollector.core.definition.AppLifecycleConfiguration import cool.linc.androiddatacollector.core.definition.CollectorConfiguration import cool.linc.androiddatacollector.core.definition.ExportConfiguration +import cool.linc.androiddatacollector.core.definition.ChoiceOption +import cool.linc.androiddatacollector.core.definition.InterventionConfiguration +import cool.linc.androiddatacollector.core.definition.InterventionTrigger +import cool.linc.androiddatacollector.core.definition.LocalizedText +import cool.linc.androiddatacollector.core.definition.MultipleChoiceQuestion +import cool.linc.androiddatacollector.core.definition.OneTimeSchedule +import cool.linc.androiddatacollector.core.definition.RelativeClock +import cool.linc.androiddatacollector.core.definition.ScaleQuestion +import cool.linc.androiddatacollector.core.definition.ShortTextQuestion +import cool.linc.androiddatacollector.core.definition.SingleChoiceQuestion import cool.linc.androiddatacollector.core.collector.PrivacyClass import cool.linc.androiddatacollector.core.collector.ResearchClocks import cool.linc.androiddatacollector.core.definition.SignerIdentity import cool.linc.androiddatacollector.core.definition.StudyConfiguration +import cool.linc.androiddatacollector.core.definition.SurveyAction +import cool.linc.androiddatacollector.core.definition.SurveyDefinition import java.time.Instant +import kotlinx.coroutines.async +import kotlinx.coroutines.awaitAll import kotlinx.coroutines.ExperimentalCoroutinesApi import kotlinx.coroutines.flow.MutableStateFlow import kotlinx.coroutines.flow.StateFlow @@ -106,6 +122,88 @@ class ExperimentRuntimeTest { assertTrue(plugin.emit("ACTIVITY_RESUMED") is EmitResult.Accepted) } + @Test + fun surveySubmissionValidatesEveryQuestionTypeAndCommitsExactlyOnce() = runTest { + val store = InMemoryStudyStore() + val clocks = FakeClocks() + val runtime = ExperimentRuntime( + configuration = configuration(surveys = listOf(survey()), interventions = listOf(surveyIntervention())), + store = store, + collectorRegistry = CollectorRegistry(listOf(FakeCollectorPlugin(clocks))), + clocks = clocks, + scope = backgroundScope, + availableAccess = { emptySet() }, + ) + start(runtime) + val occurrence = InterventionOccurrence( + occurrenceId = "a".repeat(64), + interventionId = "survey-notice", + triggerId = "after-minute", + scheduleKey = "relative:1", + scheduledFor = ResearchTime(1_000, 1_000, "boot-test"), + expiresAtUtcMillis = 60_000, + state = OccurrenceState.SCHEDULED, + ) + runtime.ensureOccurrence(occurrence) + assertTrue(runtime.claimOccurrence(occurrence.occurrenceId)?.action is SurveyAction) + runtime.markNotificationPosted(occurrence.occurrenceId) + assertEquals(OccurrenceState.OPENED, runtime.openOccurrence(occurrence.occurrenceId)?.occurrence?.state) + + val incomplete = mapOf("mood-scale" to SurveyAnswer.Integer(3)) + assertEquals(SurveySubmissionResult.INVALID, runtime.submitSurvey(occurrence.occurrenceId, incomplete)) + val answers = mapOf( + "daily-note" to SurveyAnswer.Text("felt focused"), + "mood-scale" to SurveyAnswer.Integer(4), + "primary-place" to SurveyAnswer.Choices(listOf("place-home")), + "symptoms" to SurveyAnswer.Choices(listOf("symptom-tired", "symptom-headache")), + ) + val concurrent = listOf( + async { runtime.submitSurvey(occurrence.occurrenceId, answers) }, + async { runtime.submitSurvey(occurrence.occurrenceId, answers) }, + ).awaitAll() + assertEquals(1, concurrent.count { it == SurveySubmissionResult.ACCEPTED }) + assertEquals(1, concurrent.count { it == SurveySubmissionResult.ALREADY_SUBMITTED }) + + val submitted = requireNotNull(runtime.surveySubmissionEvent(occurrence.occurrenceId)) + assertEquals("SURVEY_SUBMITTED", submitted.payloadType) + assertEquals("daily-survey", submitted.fields["survey_id"]) + val encoded = requireNotNull(submitted.fields["answers_json"]) + assertTrue(encoded.contains("\"daily-note\":\"felt focused\"")) + assertTrue(encoded.contains("\"primary-place\":[\"place-home\"]")) + assertTrue(encoded.contains("\"symptoms\":[\"symptom-tired\",\"symptom-headache\"]")) + assertTrue(!encoded.contains("Home")) + assertEquals(4L, store.events.count().toLong()) + } + + @Test + fun lateSurveyOccurrenceExpiresWithoutOpeningOrSubmission() = runTest { + val store = InMemoryStudyStore() + val clocks = FakeClocks() + val runtime = ExperimentRuntime( + configuration = configuration(surveys = listOf(survey()), interventions = listOf(surveyIntervention())), + store = store, + collectorRegistry = CollectorRegistry(listOf(FakeCollectorPlugin(clocks))), + clocks = clocks, + scope = backgroundScope, + availableAccess = { emptySet() }, + ) + start(runtime) + val occurrence = InterventionOccurrence( + occurrenceId = "b".repeat(64), + interventionId = "survey-notice", + triggerId = "after-minute", + scheduleKey = "relative:1", + scheduledFor = ResearchTime(1, 1, "boot-test"), + expiresAtUtcMillis = 2_500, + state = OccurrenceState.SCHEDULED, + ) + runtime.ensureOccurrence(occurrence) + assertNull(runtime.claimOccurrence(occurrence.occurrenceId)) + assertNull(runtime.openOccurrence(occurrence.occurrenceId)) + assertEquals(OccurrenceState.EXPIRED, runtime.snapshot.value.metadata?.occurrences?.get(occurrence.occurrenceId)?.state) + assertEquals(listOf("INTERVENTION_SCHEDULED", "SURVEY_EXPIRED"), store.events.map { it.payloadType }) + } + @Test fun illegalCommandFailsWithoutMutatingDurableState() = runTest { val store = InMemoryStudyStore() @@ -206,6 +304,13 @@ class ExperimentRuntimeTest { saveCount += 1 } + override suspend fun appendEventAtomically(event: RecordedEvent, metadata: StudyMetadata) { + require(event.sequenceNumber == requireNotNull(this.metadata).nextSequenceNumber) + events += event + this.metadata = metadata + saveCount += 1 + } + override suspend fun readEvents( fromSequenceInclusive: Long, upToSequenceInclusive: Long, @@ -311,10 +416,13 @@ class ExperimentRuntimeTest { fun configuration( collectors: List = listOf(AppLifecycleConfiguration(required = true)), + surveys: List = emptyList(), + interventions: List = emptyList(), ) = StudyConfiguration( schemaVersion = StudyConfiguration.CURRENT_SCHEMA_VERSION, experimentId = EXPERIMENT_ID, configurationId = CONFIGURATION_ID, + assignedParticipantId = null, issuedAt = Instant.parse("2026-01-01T00:00:00Z"), expiresAt = Instant.parse("2030-01-01T00:00:00Z"), minimumAppVersion = 1, @@ -326,7 +434,8 @@ class ExperimentRuntimeTest { consentDocumentVersion = "test-1", consentSummary = "Test consent", collectors = collectors, - prompts = emptyList(), + surveys = surveys, + interventions = interventions, maximumLocalBytes = 16_777_216, signer = SignerIdentity("test-signer", TEST_SIGNER_PUBLIC_KEY), export = ExportConfiguration( @@ -335,6 +444,48 @@ class ExperimentRuntimeTest { ), upload = null, ) + + suspend fun start(runtime: ExperimentRuntime) { + assertEquals(CommandResult.Success, runtime.initialize()) + assertEquals(CommandResult.Success, runtime.reviewStudy()) + assertEquals(CommandResult.Success, runtime.acceptConsent()) + assertEquals(CommandResult.Success, runtime.completeAccessSetup(emptySet())) + assertEquals(CommandResult.Success, runtime.start()) + } + + fun survey() = SurveyDefinition( + "daily-survey", + LocalizedText("Daily survey", mapOf("zh-TW" to "每日問卷")), + LocalizedText("Answer four questions."), + listOf( + ShortTextQuestion("daily-note", LocalizedText("How was today?"), false, 40), + ScaleQuestion("mood-scale", LocalizedText("Mood"), true, 1, 5, LocalizedText("Low"), LocalizedText("High")), + SingleChoiceQuestion( + "primary-place", + LocalizedText("Where were you?"), + true, + listOf(ChoiceOption("place-home", LocalizedText("Home")), ChoiceOption("place-work", LocalizedText("Work"))), + ), + MultipleChoiceQuestion( + "symptoms", + LocalizedText("Symptoms"), + true, + listOf( + ChoiceOption("symptom-tired", LocalizedText("Tired")), + ChoiceOption("symptom-headache", LocalizedText("Headache")), + ChoiceOption("symptom-none", LocalizedText("None")), + ), + 1, + 2, + ), + ), + ) + + fun surveyIntervention() = InterventionConfiguration( + "survey-notice", + SurveyAction("Daily survey", "Your survey is ready.", "daily-survey"), + listOf(InterventionTrigger("after-minute", OneTimeSchedule(1, RelativeClock.CALENDAR_TIME), 60)), + ) } } diff --git a/core/export/src/main/kotlin/cool/linc/androiddatacollector/core/export/ResearchExport.kt b/core/export/src/main/kotlin/cool/linc/androiddatacollector/core/export/ResearchExport.kt index cc59e86..865c6ad 100644 --- a/core/export/src/main/kotlin/cool/linc/androiddatacollector/core/export/ResearchExport.kt +++ b/core/export/src/main/kotlin/cool/linc/androiddatacollector/core/export/ResearchExport.kt @@ -197,6 +197,7 @@ object ResearchExport { writer.name("experiment_id").value(metadata.experimentId) writer.name("configuration_id").value(metadata.configurationId) writer.name("participant_instance_id").value(metadata.participantInstanceId) + metadata.assignedParticipantId?.let { writer.name("assigned_participant_id").value(it) } writer.name("state").value(metadata.state.name) writer.name("next_sequence_number").value(metadata.nextSequenceNumber) writer.name("transitions").beginArray() diff --git a/core/export/src/test/kotlin/cool/linc/androiddatacollector/core/export/ResearchExportTest.kt b/core/export/src/test/kotlin/cool/linc/androiddatacollector/core/export/ResearchExportTest.kt index d5ad123..4bf194a 100644 --- a/core/export/src/test/kotlin/cool/linc/androiddatacollector/core/export/ResearchExportTest.kt +++ b/core/export/src/test/kotlin/cool/linc/androiddatacollector/core/export/ResearchExportTest.kt @@ -22,6 +22,33 @@ import org.junit.Assert.assertTrue import org.junit.Test class ResearchExportTest { + @Test + fun personalizedExportEncryptsAssignedAndInstanceIdentifiersTogether() = runBlocking { + val keys = HpkeCrypto.generateKeyset() + val configuration = configuration(keys.publicKeysetJson).copy(assignedParticipantId = "arm-a-017") + val time = ResearchTime(1_000, 2_000, "boot-test") + val event = RecordedEvent(1, "app_lifecycle.v1", 1, time, "ACTIVITY_RESUMED", emptyMap()) + val metadata = StudyMetadata.initial( + "export-test", + "export-config", + assignedParticipantId = "arm-a-017", + participantInstanceId = "00000000-0000-4000-8000-000000000017", + ).copy( + state = ExperimentState.RUNNING, + eventCount = 1, + nextSequenceNumber = 2, + lastEvents = mapOf(event.collectorId to event), + ) + val encrypted = ByteArrayOutputStream() + ResearchExport.encrypt(ExportSnapshot(configuration, metadata, 10_000), SnapshotStore(metadata, listOf(event)), encrypted) + + val plaintext = ResearchExport.decrypt(encrypted.toByteArray(), keys.privateKeysetJson, configuration) + .toString(Charsets.UTF_8) + assertTrue(plaintext.contains("\"assigned_participant_id\":\"arm-a-017\"")) + assertTrue(plaintext.contains("\"participant_instance_id\":\"${metadata.participantInstanceId}\"")) + assertFalse(encrypted.toByteArray().toString(Charsets.ISO_8859_1).contains("arm-a-017")) + } + @Test fun repeatedExportsAreIndependentDecryptableSnapshotsWithoutAStateTransition() = runBlocking { val keys = HpkeCrypto.generateKeyset() @@ -248,6 +275,7 @@ class ResearchExportTest { override suspend fun initialize(metadata: StudyMetadata) { this.metadata = metadata } override suspend fun saveMetadata(metadata: StudyMetadata) { this.metadata = metadata } override suspend fun appendEvent(event: RecordedEvent) { storedEvents += event } + override suspend fun appendEventAtomically(event: RecordedEvent, metadata: StudyMetadata) { storedEvents += event } override suspend fun readEvents( fromSequenceInclusive: Long, upToSequenceInclusive: Long, @@ -269,6 +297,7 @@ class ResearchExportTest { schemaVersion = StudyConfiguration.CURRENT_SCHEMA_VERSION, experimentId = "export-test", configurationId = "export-config", + assignedParticipantId = null, issuedAt = Instant.parse("2026-01-01T00:00:00Z"), expiresAt = Instant.parse("2030-01-01T00:00:00Z"), minimumAppVersion = 1, @@ -280,7 +309,8 @@ class ResearchExportTest { consentDocumentVersion = "v1", consentSummary = "Export test consent.", collectors = listOf(AppLifecycleConfiguration(true)), - prompts = emptyList(), + surveys = emptyList(), + interventions = emptyList(), maximumLocalBytes = 16_777_216, signer = SignerIdentity("test-signer", TEST_SIGNER_PUBLIC_KEY), export = ExportConfiguration("export-key", publicKeyset), diff --git a/core/model/src/main/kotlin/cool/linc/androiddatacollector/core/model/ExperimentModels.kt b/core/model/src/main/kotlin/cool/linc/androiddatacollector/core/model/ExperimentModels.kt index a5c192e..76e95a8 100644 --- a/core/model/src/main/kotlin/cool/linc/androiddatacollector/core/model/ExperimentModels.kt +++ b/core/model/src/main/kotlin/cool/linc/androiddatacollector/core/model/ExperimentModels.kt @@ -70,7 +70,9 @@ data class EventDraft( private val PAYLOAD_TYPE = Regex("[A-Z][A-Z0-9_]{1,63}") private val FIELD_KEY = Regex("[a-z][a-z0-9_]{0,63}") private const val MAX_FIELDS = 32 - private const val MAX_FIELD_VALUE_LENGTH = 1_024 + // Survey submissions are one bounded, immutable JSON value so they cannot be partially + // committed. The encrypted event frame remains capped at 64 KiB by storage. + private const val MAX_FIELD_VALUE_LENGTH = 60 * 1_024 } } diff --git a/core/model/src/main/kotlin/cool/linc/androiddatacollector/core/model/StudyData.kt b/core/model/src/main/kotlin/cool/linc/androiddatacollector/core/model/StudyData.kt index fbeab3f..b8700c9 100644 --- a/core/model/src/main/kotlin/cool/linc/androiddatacollector/core/model/StudyData.kt +++ b/core/model/src/main/kotlin/cool/linc/androiddatacollector/core/model/StudyData.kt @@ -13,9 +13,13 @@ data class StudyMetadata( /** * Pseudonymous per-install identifier. A study that uploads has no other way to tell one * participant's events from another's, because a manual export carries that information - * out of band. It never leaves the device except inside an encrypted bundle. + * out of band. Unlike the assigned code, this UUID is also the cleartext upload routing key. */ val participantInstanceId: String, + /** Optional researcher-assigned code; protected inside encrypted metadata and exports. */ + val assignedParticipantId: String?, + /** Durable intervention state keyed by globally unique occurrence ID. */ + val occurrences: Map, /** Highest sequence an endpoint has confirmed receiving; 0 when nothing has been delivered. */ val uploadedThroughSequence: Long, /** @@ -31,6 +35,12 @@ data class StudyMetadata( require(ID.matches(experimentId)) { "Invalid experiment ID" } require(ID.matches(configurationId)) { "Invalid configuration ID" } require(INSTANCE_ID.matches(participantInstanceId)) { "Invalid participant instance ID" } + assignedParticipantId?.let { + require(ASSIGNED_ID.matches(it) && it.toByteArray().size <= 64) { "Invalid assigned participant ID" } + } + require(occurrences.all { (id, occurrence) -> id == occurrence.occurrenceId }) { + "Occurrence map key mismatch" + } require(eventCount >= 0) { "Event count must be non-negative" } require(nextSequenceNumber == eventCount + 1) { "Next sequence must follow the lifetime event count" } require(retainedFromSequence in 1..nextSequenceNumber) { "Invalid retained range start" } @@ -55,6 +65,7 @@ data class StudyMetadata( fun initial( experimentId: String, configurationId: String, + assignedParticipantId: String? = null, participantInstanceId: String = UUID.randomUUID().toString(), ) = StudyMetadata( experimentId, @@ -65,9 +76,52 @@ data class StudyMetadata( 1, emptyMap(), participantInstanceId, + assignedParticipantId, + emptyMap(), 0, 1, ) + + private val ASSIGNED_ID = Regex("[A-Za-z0-9][A-Za-z0-9._-]{0,63}") + } +} + +enum class OccurrenceState { + SCHEDULED, + POSTING, + NOTIFICATION_POSTED, + OPENED, + SURVEY_SUBMITTED, + EXPIRED, +} + +data class InterventionOccurrence( + val occurrenceId: String, + val interventionId: String, + val triggerId: String, + val scheduleKey: String, + val scheduledFor: ResearchTime, + val expiresAtUtcMillis: Long, + val state: OccurrenceState, + val openedAt: ResearchTime? = null, + val submittedAt: ResearchTime? = null, + val submissionSequence: Long? = null, +) { + init { + require(OCCURRENCE_ID.matches(occurrenceId)) { "Invalid occurrence ID" } + require(ID.matches(interventionId) && ID.matches(triggerId)) { "Invalid occurrence reference" } + require(scheduleKey.length in 1..160) { "Invalid occurrence schedule key" } + require(expiresAtUtcMillis > scheduledFor.wallTimeUtcMillis) { "Invalid occurrence expiry" } + if (state == OccurrenceState.SURVEY_SUBMITTED) { + require(submittedAt != null && submissionSequence != null) { "Missing submission record" } + } + submissionSequence?.let { require(it > 0) { "Invalid submission sequence" } } + submittedAt?.let { require(openedAt != null) { "A submission must have been opened" } } + } + + companion object { + private val ID = Regex("[a-z0-9][a-z0-9-]{2,63}") + private val OCCURRENCE_ID = Regex("[0-9a-f]{64}") } } @@ -98,6 +152,9 @@ interface StudyStore { suspend fun appendEvent(event: RecordedEvent) + /** Commits one event and its resulting metadata as a recoverable transaction. */ + suspend fun appendEventAtomically(event: RecordedEvent, metadata: StudyMetadata) + /** * Streams `[fromSequenceInclusive, upToSequenceInclusive]`. Implementations must deliver the * whole requested range or throw; a short read is never returned, because a caller cannot diff --git a/core/protocol/src/test/kotlin/cool/linc/androiddatacollector/core/protocol/ConfigurationProtocolTest.kt b/core/protocol/src/test/kotlin/cool/linc/androiddatacollector/core/protocol/ConfigurationProtocolTest.kt index d8708bd..95b4ecc 100644 --- a/core/protocol/src/test/kotlin/cool/linc/androiddatacollector/core/protocol/ConfigurationProtocolTest.kt +++ b/core/protocol/src/test/kotlin/cool/linc/androiddatacollector/core/protocol/ConfigurationProtocolTest.kt @@ -73,6 +73,25 @@ class ConfigurationProtocolTest { } } + @Test + fun v1IdentityLocalizationAndOccurrenceBoundsAreStrict() { + assertEquals("每日確認", LocalizedText("Default", mapOf("zh-TW" to "每日確認")).resolve("zh-Hant-TW")) + assertThrows(IllegalArgumentException::class.java) { + configuration().copy(assignedParticipantId = "contains space") + } + assertThrows(IllegalArgumentException::class.java) { + configuration().copy( + interventions = listOf( + InterventionConfiguration( + "too-frequent", + NotificationAction("Check-in", "Check in now."), + listOf(InterventionTrigger("every-minute", IntervalSchedule(0, 1, RelativeClock.CALENDAR_TIME), 5)), + ), + ), + ) + } + } + @Test fun verifierAuthenticatesSignerAndValidityWindow() { val keyPair = KeyPairGenerator.getInstance("Ed25519").generateKeyPair() @@ -182,6 +201,7 @@ class ConfigurationProtocolTest { schemaVersion = StudyConfiguration.CURRENT_SCHEMA_VERSION, experimentId = "protocol-test", configurationId = "protocol-config", + assignedParticipantId = "arm-a-017", issuedAt = Instant.parse("2026-01-01T00:00:00Z"), expiresAt = Instant.parse("2030-01-01T00:00:00Z"), minimumAppVersion = 1, @@ -201,7 +221,29 @@ class ConfigurationProtocolTest { LocationConfiguration(false, 10_000, 5_000, 30_000, 5f, LocationPriority.BALANCED), KeyboardTouchConfiguration(false, 60), ), - prompts = listOf(PromptConfiguration("daily-check", 60, "Check in.")), + surveys = listOf( + SurveyDefinition( + id = "daily-survey", + title = LocalizedText("Daily check-in", mapOf("zh-TW" to "每日確認")), + description = LocalizedText("Tell us how today went."), + questions = listOf( + ShortTextQuestion("daily-note", LocalizedText("Anything to share?"), false, 500), + ), + ), + ), + interventions = listOf( + InterventionConfiguration( + id = "daily-check", + action = SurveyAction("Daily check-in", "A short survey is ready.", "daily-survey"), + triggers = listOf( + InterventionTrigger( + "after-hour", + OneTimeSchedule(60, RelativeClock.ACTIVE_RUNNING_TIME), + 1_440, + ), + ), + ), + ), maximumLocalBytes = 16_777_216, signer = SignerIdentity("test-signer", signerPublicKey), export = ExportConfiguration( diff --git a/core/storage/src/main/kotlin/cool/linc/androiddatacollector/core/storage/EncryptedExperimentStore.kt b/core/storage/src/main/kotlin/cool/linc/androiddatacollector/core/storage/EncryptedExperimentStore.kt index 618d215..44d2507 100644 --- a/core/storage/src/main/kotlin/cool/linc/androiddatacollector/core/storage/EncryptedExperimentStore.kt +++ b/core/storage/src/main/kotlin/cool/linc/androiddatacollector/core/storage/EncryptedExperimentStore.kt @@ -33,6 +33,7 @@ class EncryptedExperimentStore( private val keyAlias = "adc-core-$opaqueId" private val rootDirectory = context.noBackupFilesDir.resolve(STORAGE_DIRECTORY) private val metadataFile = AtomicFile(rootDirectory.resolve("$opaqueId.metadata.adc")) + private val transactionFile = AtomicFile(rootDirectory.resolve("$opaqueId.transaction.adc")) private val eventDirectory = rootDirectory.resolve("$opaqueId.events") private var persistedSequenceBoundary = 0L private var persistedRetainedFrom = 1L @@ -62,11 +63,27 @@ class EncryptedExperimentStore( decryptPayloads = false, ) val encodedMetadata = decryptMetadata(metadataFile.readFully(), key) - val metadata = StudyDataJsonCodec.decodeMetadata( + var metadata = StudyDataJsonCodec.decodeMetadata( encodedMetadata, scan.firstSequence, scan.lastSequence, ) + if (transactionFile.baseFile.exists()) { + val recovered = runCatching { + StudyDataJsonCodec.decodeMetadata( + decryptDocument(transactionFile.readFully(), key, TRANSACTION_HEADER), + scan.firstSequence, + scan.lastSequence, + ) + }.getOrNull() + if (recovered != null && recovered.eventCount == scan.lastSequence && + recovered.eventCount == metadata.eventCount + 1 + ) { + metadata = recovered + writeMetadata(encryptDocument(StudyDataJsonCodec.encodeMetadata(metadata), key, METADATA_HEADER)) + } + transactionFile.delete() + } require(metadata.experimentId == experimentId) { "Encrypted experiment ID mismatch" } // The lifetime counter comes from metadata, not from the scan: reclaimed events are // gone from disk but their sequence numbers must never be handed out again. @@ -100,22 +117,38 @@ class EncryptedExperimentStore( override suspend fun appendEvent(event: RecordedEvent) = withContext(Dispatchers.IO) { mutex.withLock { val metadata = requireNotNull(persistedMetadata) { "Study storage is not initialized" } - require(event.sequenceNumber == persistedSequenceBoundary + 1) { "Non-contiguous event append" } - val key = existingKey() ?: error("Encrypted experiment key is unavailable") - appendEncryptedEvent(event, key) - persistedSequenceBoundary = event.sequenceNumber - persistMetadata( + appendTransaction( + event, metadata.copy( - eventCount = persistedSequenceBoundary, - nextSequenceNumber = persistedSequenceBoundary + 1, + eventCount = event.sequenceNumber, + nextSequenceNumber = event.sequenceNumber + 1, lastEvents = metadata.lastEvents + (event.collectorId to event), retainedFromSequence = persistedRetainedFrom, ), - key, ) } } + override suspend fun appendEventAtomically(event: RecordedEvent, metadata: StudyMetadata) = + withContext(Dispatchers.IO) { mutex.withLock { appendTransaction(event, metadata) } } + + private fun appendTransaction(event: RecordedEvent, metadata: StudyMetadata) { + requireNotNull(persistedMetadata) { "Study storage is not initialized" } + require(event.sequenceNumber == persistedSequenceBoundary + 1) { "Non-contiguous event append" } + require(metadata.eventCount == event.sequenceNumber && metadata.nextSequenceNumber == event.sequenceNumber + 1) { + "Atomic metadata boundary mismatch" + } + require(metadata.experimentId == experimentId) { "Experiment ID mismatch" } + val key = existingKey() ?: error("Encrypted experiment key is unavailable") + val encoded = StudyDataJsonCodec.encodeMetadata(metadata) + require(encoded.size <= MAXIMUM_METADATA_BYTES) { "Experiment metadata quota exceeded" } + writeAtomic(transactionFile, encryptDocument(encoded, key, TRANSACTION_HEADER)) + appendEncryptedEvent(event, key) + persistedSequenceBoundary = event.sequenceNumber + persistMetadata(metadata, key) + transactionFile.delete() + } + override suspend fun readEvents( fromSequenceInclusive: Long, upToSequenceInclusive: Long, @@ -185,6 +218,7 @@ class EncryptedExperimentStore( override suspend fun clear() = withContext(Dispatchers.IO) { mutex.withLock { metadataFile.delete() + transactionFile.delete() eventDirectory.listFiles()?.forEach { file -> check(file.delete()) { "Cannot delete event segment" } } if (eventDirectory.exists()) check(eventDirectory.delete()) { "Cannot delete event directory" } if (keyStore.containsAlias(keyAlias)) keyStore.deleteEntry(keyAlias) @@ -200,7 +234,7 @@ class EncryptedExperimentStore( ) { val encoded = StudyDataJsonCodec.encodeMetadata(metadata) require(encoded.size <= MAXIMUM_METADATA_BYTES) { "Experiment metadata quota exceeded" } - writeMetadata(encryptMetadata(encoded, key)) + writeMetadata(encryptDocument(encoded, key, METADATA_HEADER)) persistedMetadata = metadata } @@ -368,18 +402,19 @@ class EncryptedExperimentStore( private fun storageBytes(): Long = metadataFile.baseFile.length() + eventDirectory.listFiles().orEmpty().sumOf(File::length) - private fun encryptMetadata( + private fun encryptDocument( plaintext: ByteArray, key: SecretKey, + header: ByteArray, ): ByteArray { val cipher = Cipher.getInstance(CIPHER_TRANSFORMATION).apply { init(Cipher.ENCRYPT_MODE, key) - updateAAD(METADATA_HEADER) + updateAAD(header) } val ciphertext = cipher.doFinal(plaintext) check(cipher.iv.size == IV_BYTES) { "Android Keystore returned an invalid GCM IV" } - return ByteBuffer.allocate(METADATA_HEADER.size + IV_BYTES + ciphertext.size) - .put(METADATA_HEADER) + return ByteBuffer.allocate(header.size + IV_BYTES + ciphertext.size) + .put(header) .put(cipher.iv) .put(ciphertext) .array() @@ -388,18 +423,20 @@ class EncryptedExperimentStore( private fun decryptMetadata( encoded: ByteArray, key: SecretKey, - ): ByteArray { + ): ByteArray = decryptDocument(encoded, key, METADATA_HEADER) + + private fun decryptDocument(encoded: ByteArray, key: SecretKey, expectedHeader: ByteArray): ByteArray { require(encoded.size in MINIMUM_METADATA_FILE_BYTES..MAXIMUM_METADATA_FILE_BYTES) { "Encrypted experiment metadata has an invalid size" } val buffer = ByteBuffer.wrap(encoded) - val header = ByteArray(METADATA_HEADER.size).also(buffer::get) - require(header.contentEquals(METADATA_HEADER)) { "Unsupported experiment metadata format" } + val header = ByteArray(expectedHeader.size).also(buffer::get) + require(header.contentEquals(expectedHeader)) { "Unsupported encrypted document format" } val iv = ByteArray(IV_BYTES).also(buffer::get) val ciphertext = ByteArray(buffer.remaining()).also(buffer::get) return Cipher.getInstance(CIPHER_TRANSFORMATION).run { init(Cipher.DECRYPT_MODE, key, GCMParameterSpec(GCM_TAG_BITS, iv)) - updateAAD(METADATA_HEADER) + updateAAD(expectedHeader) doFinal(ciphertext) } } @@ -436,14 +473,16 @@ class EncryptedExperimentStore( .putLong(sequenceNumber) .array() - private fun writeMetadata(encrypted: ByteArray) { + private fun writeMetadata(encrypted: ByteArray) = writeAtomic(metadataFile, encrypted) + + private fun writeAtomic(file: AtomicFile, encrypted: ByteArray) { require(rootDirectory.exists() || rootDirectory.mkdirs()) { "Cannot create experiment storage directory" } - val output = metadataFile.startWrite() + val output = file.startWrite() try { output.write(encrypted) - metadataFile.finishWrite(output) + file.finishWrite(output) } catch (failure: Throwable) { - metadataFile.failWrite(output) + file.failWrite(output) throw failure } } @@ -514,6 +553,7 @@ class EncryptedExperimentStore( // The metadata codec has no fallback reader by design: a file whose header is not this // exact string is refused rather than migrated. val METADATA_HEADER = "ADCMET01".toByteArray(Charsets.US_ASCII) + val TRANSACTION_HEADER = "ADCTXN01".toByteArray(Charsets.US_ASCII) val SEGMENT_HEADER = "ADCEVT01".toByteArray(Charsets.US_ASCII) val SEGMENT_PATTERN = Regex("events-([0-9]{8})\\.adcs") val SEGMENT_HEADER_BYTES = SEGMENT_HEADER.size + Int.SIZE_BYTES diff --git a/core/storage/src/main/kotlin/cool/linc/androiddatacollector/core/storage/StudyDataJsonCodec.kt b/core/storage/src/main/kotlin/cool/linc/androiddatacollector/core/storage/StudyDataJsonCodec.kt index ba1c9b3..8d82d3f 100644 --- a/core/storage/src/main/kotlin/cool/linc/androiddatacollector/core/storage/StudyDataJsonCodec.kt +++ b/core/storage/src/main/kotlin/cool/linc/androiddatacollector/core/storage/StudyDataJsonCodec.kt @@ -2,6 +2,8 @@ package cool.linc.androiddatacollector.core.storage import cool.linc.androiddatacollector.core.model.ExperimentState import cool.linc.androiddatacollector.core.model.ExperimentTransition +import cool.linc.androiddatacollector.core.model.InterventionOccurrence +import cool.linc.androiddatacollector.core.model.OccurrenceState import cool.linc.androiddatacollector.core.model.RecordedEvent import cool.linc.androiddatacollector.core.model.ResearchTime import cool.linc.androiddatacollector.core.model.StudyMetadata @@ -17,6 +19,8 @@ internal object StudyDataJsonCodec { "next_sequence_number", "transitions", "participant_instance_id", + "assigned_participant_id", + "occurrences", "uploaded_through_sequence", "retained_from_sequence", "last_events", @@ -31,6 +35,18 @@ internal object StudyDataJsonCodec { "payload_type", "fields", ) + private val OCCURRENCE_KEYS = setOf( + "occurrence_id", + "intervention_id", + "trigger_id", + "schedule_key", + "scheduled_for", + "expires_at_utc_millis", + "state", + "opened_at", + "submitted_at", + "submission_sequence", + ) fun encodeMetadata(metadata: StudyMetadata): ByteArray = JSONObject() .put("experiment_id", metadata.experimentId) @@ -41,6 +57,10 @@ internal object StudyDataJsonCodec { metadata.transitions.forEach { put(encodeTransition(it)) } }) .put("participant_instance_id", metadata.participantInstanceId) + .put("assigned_participant_id", metadata.assignedParticipantId ?: JSONObject.NULL) + .put("occurrences", JSONObject().apply { + metadata.occurrences.toSortedMap().forEach { (id, occurrence) -> put(id, encodeOccurrence(occurrence)) } + }) .put("uploaded_through_sequence", metadata.uploadedThroughSequence) .put("retained_from_sequence", metadata.retainedFromSequence) // Durable rather than rebuilt by scanning, so opening a study never has to decrypt its @@ -119,6 +139,10 @@ internal object StudyDataJsonCodec { } }, participantInstanceId = root.getString("participant_instance_id"), + assignedParticipantId = if (root.isNull("assigned_participant_id")) null else root.getString("assigned_participant_id"), + occurrences = root.getJSONObject("occurrences").let { occurrences -> + occurrences.keys().asSequence().associateWith { id -> decodeOccurrence(occurrences.getJSONObject(id)) } + }, uploadedThroughSequence = uploadedThrough, retainedFromSequence = retainedFrom, ) @@ -167,6 +191,34 @@ internal object StudyDataJsonCodec { ) } + private fun encodeOccurrence(occurrence: InterventionOccurrence): JSONObject = JSONObject() + .put("occurrence_id", occurrence.occurrenceId) + .put("intervention_id", occurrence.interventionId) + .put("trigger_id", occurrence.triggerId) + .put("schedule_key", occurrence.scheduleKey) + .put("scheduled_for", encodeTime(occurrence.scheduledFor)) + .put("expires_at_utc_millis", occurrence.expiresAtUtcMillis) + .put("state", occurrence.state.name) + .put("opened_at", occurrence.openedAt?.let(::encodeTime) ?: JSONObject.NULL) + .put("submitted_at", occurrence.submittedAt?.let(::encodeTime) ?: JSONObject.NULL) + .put("submission_sequence", occurrence.submissionSequence ?: JSONObject.NULL) + + private fun decodeOccurrence(json: JSONObject): InterventionOccurrence { + json.requireExactKeys(OCCURRENCE_KEYS) + return InterventionOccurrence( + occurrenceId = json.getString("occurrence_id"), + interventionId = json.getString("intervention_id"), + triggerId = json.getString("trigger_id"), + scheduleKey = json.getString("schedule_key"), + scheduledFor = decodeTime(json.getJSONObject("scheduled_for")), + expiresAtUtcMillis = json.getLong("expires_at_utc_millis"), + state = enumValueOf(json.getString("state")), + openedAt = if (json.isNull("opened_at")) null else decodeTime(json.getJSONObject("opened_at")), + submittedAt = if (json.isNull("submitted_at")) null else decodeTime(json.getJSONObject("submitted_at")), + submissionSequence = if (json.isNull("submission_sequence")) null else json.getLong("submission_sequence"), + ) + } + private fun encodeTime(time: ResearchTime): JSONObject = JSONObject() .put("wall_time_utc_millis", time.wallTimeUtcMillis) .put("elapsed_realtime_nanos", time.elapsedRealtimeNanos) diff --git a/core/study-application/src/main/kotlin/cool/linc/androiddatacollector/core/application/InterventionSchedulePlanner.kt b/core/study-application/src/main/kotlin/cool/linc/androiddatacollector/core/application/InterventionSchedulePlanner.kt new file mode 100644 index 0000000..48f2ae8 --- /dev/null +++ b/core/study-application/src/main/kotlin/cool/linc/androiddatacollector/core/application/InterventionSchedulePlanner.kt @@ -0,0 +1,165 @@ +package cool.linc.androiddatacollector.core.application + +import cool.linc.androiddatacollector.core.definition.DailyLocalSchedule +import cool.linc.androiddatacollector.core.definition.IntervalSchedule +import cool.linc.androiddatacollector.core.definition.OneTimeSchedule +import cool.linc.androiddatacollector.core.definition.RelativeClock +import cool.linc.androiddatacollector.core.definition.StudyConfiguration +import cool.linc.androiddatacollector.core.model.ExperimentState +import cool.linc.androiddatacollector.core.model.InterventionOccurrence +import cool.linc.androiddatacollector.core.model.OccurrenceState +import cool.linc.androiddatacollector.core.model.ResearchTime +import cool.linc.androiddatacollector.core.model.StudyMetadata +import cool.linc.androiddatacollector.core.model.TransitionReason +import java.nio.charset.StandardCharsets +import java.security.MessageDigest +import java.time.Instant +import java.time.LocalTime +import java.time.ZoneId + +/** Pure, deterministic planner. Android owns only the final best-effort WorkManager delay. */ +class InterventionSchedulePlanner { + fun next( + configuration: StudyConfiguration, + metadata: StudyMetadata, + now: ResearchTime, + zoneId: ZoneId, + triggerId: String? = null, + ): List { + if (metadata.state in TERMINAL_STATES) return emptyList() + val firstStart = metadata.transitions.firstOrNull { it.reason == TransitionReason.PARTICIPANT_STARTED }?.time + ?: return emptyList() + val lifetimeEnd = now.wallTimeUtcMillis - elapsedMillis(firstStart, now) + + configuration.durationHours * HOUR_MILLIS + return configuration.interventions.flatMap { intervention -> + intervention.triggers.filter { triggerId == null || it.id == triggerId }.mapNotNull { trigger -> + val candidates = when (val schedule = trigger.schedule) { + is OneTimeSchedule -> sequenceOf( + relativeCandidate(schedule.offsetMinutes.toLong(), schedule.clock, metadata, firstStart, now) to + "relative:${schedule.offsetMinutes}", + ) + is IntervalSchedule -> generateSequence(0) { it + 1 } + .map { index -> schedule.startOffsetMinutes + index.toLong() * schedule.intervalMinutes } + .takeWhile { it * MINUTE_MILLIS < configuration.durationHours * HOUR_MILLIS } + .map { target -> + relativeCandidate(target, schedule.clock, metadata, firstStart, now) to "relative:$target" + } + is DailyLocalSchedule -> dailyCandidates(firstStart, lifetimeEnd, schedule.localTime, zoneId) + .mapIndexed { index, scheduled -> scheduled to "daily:$index" } + } + candidates.takeWhile { it.first.wallTimeUtcMillis < lifetimeEnd } + .map { (scheduled, key) -> + val id = occurrenceId(configuration, intervention.id, trigger.id, key) + InterventionOccurrence( + occurrenceId = id, + interventionId = intervention.id, + triggerId = trigger.id, + scheduleKey = key, + scheduledFor = scheduled, + expiresAtUtcMillis = minOf( + scheduled.wallTimeUtcMillis + trigger.availabilityMinutes * MINUTE_MILLIS, + lifetimeEnd, + ), + state = OccurrenceState.SCHEDULED, + ) + } + .firstOrNull { candidate -> + val existing = metadata.occurrences[candidate.occurrenceId] + existing == null || existing.state in PENDING_STATES + } + } + } + } + + private fun relativeCandidate( + offsetMinutes: Long, + clock: RelativeClock, + metadata: StudyMetadata, + firstStart: ResearchTime, + now: ResearchTime, + ): ResearchTime { + val offsetMillis = offsetMinutes * MINUTE_MILLIS + if (clock == RelativeClock.CALENDAR_TIME) { + val wall = now.wallTimeUtcMillis - elapsedMillis(firstStart, now) + offsetMillis + return estimateResearchTime(wall, now) + } + val runningTransitions = metadata.transitions.filter { + it.to == ExperimentState.RUNNING || it.from == ExperimentState.RUNNING + } + var accumulated = 0L + var opened: ResearchTime? = null + runningTransitions.forEach { transition -> + if (transition.to == ExperimentState.RUNNING) opened = transition.time + if (transition.from == ExperimentState.RUNNING) { + val start = checkNotNull(opened) + val span = elapsedMillis(start, transition.time) + if (offsetMillis <= accumulated + span) { + val delta = offsetMillis - accumulated + return start.copy( + wallTimeUtcMillis = start.wallTimeUtcMillis + delta, + elapsedRealtimeNanos = start.elapsedRealtimeNanos + delta * 1_000_000, + ) + } + accumulated += span + opened = null + } + } + val activeStart = opened + val wall = if (activeStart != null) { + val accrued = accumulated + elapsedMillis(activeStart, now) + now.wallTimeUtcMillis + (offsetMillis - accrued).coerceAtLeast(0) + } else { + Long.MAX_VALUE + } + return if (wall == Long.MAX_VALUE) now.copy(wallTimeUtcMillis = wall) else estimateResearchTime(wall, now) + } + + private fun dailyCandidates( + firstStart: ResearchTime, + lifetimeEnd: Long, + localTime: String, + zoneId: ZoneId, + ): Sequence { + val firstDate = Instant.ofEpochMilli(firstStart.wallTimeUtcMillis).atZone(zoneId).toLocalDate() + val time = LocalTime.parse(localTime) + return generateSequence(firstDate) { it.plusDays(1) } + .map { date -> date.atTime(time).atZone(zoneId).toInstant().toEpochMilli() } + .filter { it >= firstStart.wallTimeUtcMillis } + .takeWhile { it < lifetimeEnd } + .map { estimateResearchTime(it, firstStart) } + } + + private fun estimateResearchTime(wallMillis: Long, reference: ResearchTime): ResearchTime { + val deltaNanos = (wallMillis - reference.wallTimeUtcMillis).coerceAtLeast(0) * 1_000_000 + return ResearchTime(wallMillis, reference.elapsedRealtimeNanos + deltaNanos, reference.bootSessionId) + } + + /** Uses the monotonic clock whenever both endpoints belong to one boot, so wall-clock edits do + * not turn paused time into active study time. Across boots wall time is the only shared base. */ + private fun elapsedMillis(start: ResearchTime, end: ResearchTime): Long = + if (start.bootSessionId == end.bootSessionId && end.elapsedRealtimeNanos >= start.elapsedRealtimeNanos) { + (end.elapsedRealtimeNanos - start.elapsedRealtimeNanos) / 1_000_000 + } else { + (end.wallTimeUtcMillis - start.wallTimeUtcMillis).coerceAtLeast(0) + } + + private fun occurrenceId( + configuration: StudyConfiguration, + interventionId: String, + triggerId: String, + scheduleKey: String, + ): String = MessageDigest.getInstance("SHA-256") + .digest( + listOf(configuration.experimentId, configuration.configurationId, interventionId, triggerId, scheduleKey) + .joinToString("\u0000") + .toByteArray(StandardCharsets.UTF_8), + ) + .joinToString("") { "%02x".format(it) } + + private companion object { + const val MINUTE_MILLIS = 60_000L + const val HOUR_MILLIS = 60 * MINUTE_MILLIS + val TERMINAL_STATES = setOf(ExperimentState.COMPLETED, ExperimentState.WITHDRAWN) + val PENDING_STATES = setOf(OccurrenceState.SCHEDULED, OccurrenceState.POSTING) + } +} diff --git a/core/study-application/src/main/kotlin/cool/linc/androiddatacollector/core/application/StudyApplication.kt b/core/study-application/src/main/kotlin/cool/linc/androiddatacollector/core/application/StudyApplication.kt index 715db60..ccf14f0 100644 --- a/core/study-application/src/main/kotlin/cool/linc/androiddatacollector/core/application/StudyApplication.kt +++ b/core/study-application/src/main/kotlin/cool/linc/androiddatacollector/core/application/StudyApplication.kt @@ -8,13 +8,18 @@ import cool.linc.androiddatacollector.core.collector.StudyAccessGateway import cool.linc.androiddatacollector.core.definition.StudyConfiguration import cool.linc.androiddatacollector.core.export.ExportReceipt import cool.linc.androiddatacollector.core.model.ExperimentState +import cool.linc.androiddatacollector.core.model.InterventionOccurrence +import cool.linc.androiddatacollector.core.model.ResearchTime import cool.linc.androiddatacollector.core.model.StudyMetadata import cool.linc.androiddatacollector.core.model.StudyStore import cool.linc.androiddatacollector.core.protocol.ActiveStudyStore import cool.linc.androiddatacollector.core.protocol.VerifiedConfiguration import cool.linc.androiddatacollector.core.runtime.CommandResult import cool.linc.androiddatacollector.core.runtime.ExperimentRuntime +import cool.linc.androiddatacollector.core.runtime.OccurrenceDispatch import cool.linc.androiddatacollector.core.runtime.RuntimeSnapshot +import cool.linc.androiddatacollector.core.runtime.SurveyAnswer +import cool.linc.androiddatacollector.core.runtime.SurveySubmissionResult import java.io.OutputStream import kotlinx.coroutines.CancellationException import kotlinx.coroutines.CoroutineScope @@ -44,6 +49,12 @@ interface StudyCollectionHost { interface StudyWorkScheduler { fun schedule(configuration: StudyConfiguration) + /** Replaces all pending occurrence work after state, clock, reboot, or time-zone recovery. */ + fun replaceInterventionWork(configuration: StudyConfiguration, occurrences: List) + + /** Adds the successor of a completed trigger without disturbing unrelated work. */ + fun enqueueOccurrence(configuration: StudyConfiguration, occurrence: InterventionOccurrence) + /** * Cancels reminders and the study deadline, leaving scheduled delivery in place. * @@ -124,12 +135,12 @@ data class StudySessionSnapshot( class StudyAccessPolicy { fun requirements(configuration: StudyConfiguration, collectorRequirements: Set): Set { - val promptRequirements = if (configuration.prompts.isEmpty()) { + val interventionRequirements = if (configuration.interventions.isEmpty()) { emptySet() } else { setOf(AccessRequirement(AccessKind.NOTIFICATIONS, required = true)) } - return (collectorRequirements + promptRequirements) + return (collectorRequirements + interventionRequirements) .groupBy(AccessRequirement::kind) .mapTo(mutableSetOf()) { (kind, entries) -> AccessRequirement(kind, entries.any(AccessRequirement::required)) @@ -150,6 +161,7 @@ class StudySessionManager( private val uploader: StudyUploader, private val accessPolicy: StudyAccessPolicy, private val scope: CoroutineScope, + private val schedulePlanner: InterventionSchedulePlanner = InterventionSchedulePlanner(), ) { private val sessionMutex = Mutex() @@ -223,6 +235,7 @@ class StudySessionManager( } try { workScheduler.schedule(current.configuration) + syncInterventionsLocked(current, replace = true) publish(result) } catch (failure: Throwable) { failure.rethrowCancellation() @@ -233,10 +246,15 @@ class StudySessionManager( } } - suspend fun pause(): CommandResult = command( - execute = { it.pause() }, - onSuccess = collectionHost::stop, - ) + suspend fun pause(): CommandResult = sessionMutex.withLock { + val current = requireRuntime() + val result = current.pause() + if (result == CommandResult.Success) { + collectionHost.stop() + syncInterventionsLocked(current, replace = true) + } + publish(result) + } suspend fun resume(): CommandResult = sessionMutex.withLock { refreshAccess() val current = requireRuntime() @@ -247,10 +265,43 @@ class StudySessionManager( return@withLock publish(CommandResult.Failed(INCIDENT_COLLECTION_HOST_FAILED)) } val result = current.resume() - if (result != CommandResult.Success) collectionHost.stop() + if (result != CommandResult.Success) { + collectionHost.stop() + } else { + syncInterventionsLocked(current, replace = true) + } publish(result) } + suspend fun rescheduleInterventions() = sessionMutex.withLock { + runtime?.let { syncInterventionsLocked(it, replace = true) } + } + + suspend fun claimOccurrence(occurrenceId: String): OccurrenceDispatch? = sessionMutex.withLock { + val current = requireRuntime() + current.claimOccurrence(occurrenceId).also { dispatch -> + if (dispatch == null) scheduleNextLocked(current, occurrenceId) + } + } + + suspend fun markNotificationPosted(occurrenceId: String) = sessionMutex.withLock { + requireRuntime().markNotificationPosted(occurrenceId) + scheduleNextLocked(requireRuntime(), occurrenceId) + } + + suspend fun openOccurrence(occurrenceId: String): OccurrenceDispatch? = + sessionMutex.withLock { requireRuntime().openOccurrence(occurrenceId) } + + suspend fun submitSurvey( + occurrenceId: String, + answers: Map, + ): SurveySubmissionResult = sessionMutex.withLock { + requireRuntime().submitSurvey(occurrenceId, answers) + } + + suspend fun surveySubmissionEvent(occurrenceId: String) = + sessionMutex.withLock { requireRuntime().surveySubmissionEvent(occurrenceId) } + suspend fun finish(): CommandResult = terminalCommand { it.finishEarly() } suspend fun completeAfterDuration(): CommandResult = terminalCommand { it.completeAfterDuration() } suspend fun withdraw(): CommandResult = terminalCommand { it.withdraw() } @@ -446,6 +497,31 @@ class StudySessionManager( publish(result) } + private suspend fun syncInterventionsLocked(current: ExperimentRuntime, replace: Boolean) { + val metadata = current.snapshot.value.metadata ?: return + val plans = schedulePlanner.next( + current.configuration, + metadata, + current.now(), + java.time.ZoneId.systemDefault(), + ).map { current.ensureOccurrence(it) } + if (replace) workScheduler.replaceInterventionWork(current.configuration, plans) + else plans.forEach { workScheduler.enqueueOccurrence(current.configuration, it) } + } + + private suspend fun scheduleNextLocked(current: ExperimentRuntime, completedOccurrenceId: String) { + val triggerId = current.snapshot.value.metadata?.occurrences?.get(completedOccurrenceId)?.triggerId ?: return + val metadata = current.snapshot.value.metadata ?: return + schedulePlanner.next( + current.configuration, + metadata, + current.now(), + java.time.ZoneId.systemDefault(), + triggerId, + ).map { current.ensureOccurrence(it) } + .forEach { workScheduler.enqueueOccurrence(current.configuration, it) } + } + private suspend fun terminalCommand( execute: suspend (ExperimentRuntime) -> CommandResult, ): CommandResult = sessionMutex.withLock { diff --git a/core/study-application/src/test/kotlin/cool/linc/androiddatacollector/core/application/InterventionSchedulePlannerTest.kt b/core/study-application/src/test/kotlin/cool/linc/androiddatacollector/core/application/InterventionSchedulePlannerTest.kt new file mode 100644 index 0000000..cc26208 --- /dev/null +++ b/core/study-application/src/test/kotlin/cool/linc/androiddatacollector/core/application/InterventionSchedulePlannerTest.kt @@ -0,0 +1,193 @@ +package cool.linc.androiddatacollector.core.application + +import cool.linc.androiddatacollector.core.definition.AppLifecycleConfiguration +import cool.linc.androiddatacollector.core.definition.DailyLocalSchedule +import cool.linc.androiddatacollector.core.definition.ExportConfiguration +import cool.linc.androiddatacollector.core.definition.InterventionConfiguration +import cool.linc.androiddatacollector.core.definition.InterventionSchedule +import cool.linc.androiddatacollector.core.definition.InterventionTrigger +import cool.linc.androiddatacollector.core.definition.IntervalSchedule +import cool.linc.androiddatacollector.core.definition.NotificationAction +import cool.linc.androiddatacollector.core.definition.OneTimeSchedule +import cool.linc.androiddatacollector.core.definition.RelativeClock +import cool.linc.androiddatacollector.core.definition.SignerIdentity +import cool.linc.androiddatacollector.core.definition.StudyConfiguration +import cool.linc.androiddatacollector.core.model.ExperimentState +import cool.linc.androiddatacollector.core.model.ExperimentTransition +import cool.linc.androiddatacollector.core.model.OccurrenceState +import cool.linc.androiddatacollector.core.model.ResearchTime +import cool.linc.androiddatacollector.core.model.StudyMetadata +import cool.linc.androiddatacollector.core.model.TransitionReason +import java.time.Instant +import java.time.ZoneId +import org.junit.Assert.assertEquals +import org.junit.Assert.assertNotEquals +import org.junit.Assert.assertTrue +import org.junit.Test + +class InterventionSchedulePlannerTest { + private val planner = InterventionSchedulePlanner() + + @Test + fun oneTimeCalendarAndActiveSchedulesDivergeOnlyAcrossPause() { + val calendar = plan(OneTimeSchedule(60, RelativeClock.CALENDAR_TIME), runningMetadata(), 30) + val active = plan(OneTimeSchedule(60, RelativeClock.ACTIVE_RUNNING_TIME), runningMetadata(), 30) + assertEquals(at(60).wallTimeUtcMillis, calendar.scheduledFor.wallTimeUtcMillis) + assertEquals(at(60).wallTimeUtcMillis, active.scheduledFor.wallTimeUtcMillis) + + val paused = runningMetadata().copy( + transitions = listOf( + transition(ExperimentState.READY, ExperimentState.RUNNING, TransitionReason.PARTICIPANT_STARTED, 0), + transition(ExperimentState.RUNNING, ExperimentState.PAUSED, TransitionReason.PARTICIPANT_PAUSED, 30), + transition(ExperimentState.PAUSED, ExperimentState.RUNNING, TransitionReason.PARTICIPANT_RESUMED, 90), + ), + ) + assertEquals(at(60).wallTimeUtcMillis, plan(OneTimeSchedule(60, RelativeClock.CALENDAR_TIME), paused, 100).scheduledFor.wallTimeUtcMillis) + assertEquals(at(120).wallTimeUtcMillis, plan(OneTimeSchedule(60, RelativeClock.ACTIVE_RUNNING_TIME), paused, 100).scheduledFor.wallTimeUtcMillis) + } + + @Test + fun intervalAdvancesOnlyAfterPriorOccurrenceIsTerminal() { + val configuration = configuration(IntervalSchedule(10, 30, RelativeClock.CALENDAR_TIME)) + val metadata = runningMetadata() + val first = planner.next(configuration, metadata, at(5), ZoneId.of("UTC")).single() + assertEquals(at(10).wallTimeUtcMillis, first.scheduledFor.wallTimeUtcMillis) + val retry = planner.next(configuration, metadata, at(20), ZoneId.of("UTC")).single() + assertEquals(first.occurrenceId, retry.occurrenceId) + assertEquals(first.scheduledFor.wallTimeUtcMillis, retry.scheduledFor.wallTimeUtcMillis) + + val completed = metadata.copy( + occurrences = mapOf(first.occurrenceId to first.copy(state = OccurrenceState.NOTIFICATION_POSTED)), + ) + val second = planner.next(configuration, completed, at(20), ZoneId.of("UTC")).single() + assertEquals(at(40).wallTimeUtcMillis, second.scheduledFor.wallTimeUtcMillis) + assertNotEquals(first.occurrenceId, second.occurrenceId) + } + + @Test + fun dailyOccurrenceFollowsCurrentZoneWithoutChangingItsIdentity() { + val configuration = configuration(DailyLocalSchedule("08:00")) + val metadata = runningMetadata() + val utc = planner.next(configuration, metadata, at(0), ZoneId.of("UTC")).single() + val dateLine = planner.next(configuration, metadata, at(0), ZoneId.of("Pacific/Kiritimati")).single() + + assertEquals(at(8 * 60).wallTimeUtcMillis, utc.scheduledFor.wallTimeUtcMillis) + assertEquals(at(18 * 60).wallTimeUtcMillis, dateLine.scheduledFor.wallTimeUtcMillis) + assertEquals("daily:0", utc.scheduleKey) + assertEquals(utc.occurrenceId, dateLine.occurrenceId) + } + + @Test + fun activeScheduleUsesElapsedTimeAcrossWallClockChange() { + val metadata = runningMetadata() + val wallJumpedForward = ResearchTime( + wallTimeUtcMillis = at(180).wallTimeUtcMillis, + elapsedRealtimeNanos = at(30).elapsedRealtimeNanos, + bootSessionId = "boot-one", + ) + val occurrence = planner.next( + configuration(OneTimeSchedule(60, RelativeClock.ACTIVE_RUNNING_TIME)), + metadata, + wallJumpedForward, + ZoneId.of("UTC"), + ).single() + + assertEquals(at(210).wallTimeUtcMillis, occurrence.scheduledFor.wallTimeUtcMillis) + val calendar = planner.next( + configuration(OneTimeSchedule(60, RelativeClock.CALENDAR_TIME)), + metadata, + wallJumpedForward, + ZoneId.of("UTC"), + ).single() + assertEquals(at(210).wallTimeUtcMillis, calendar.scheduledFor.wallTimeUtcMillis) + } + + @Test + fun reconstructionIsIdempotentAndTerminalStudiesHaveNoWork() { + val configuration = configuration(OneTimeSchedule(15, RelativeClock.CALENDAR_TIME)) + val metadata = runningMetadata() + val first = planner.next(configuration, metadata, at(1), ZoneId.of("UTC")) + val afterProcessDeath = planner.next(configuration, metadata, at(2, "new-boot"), ZoneId.of("UTC")) + assertEquals(first.single().occurrenceId, afterProcessDeath.single().occurrenceId) + assertEquals(first.single().scheduledFor.wallTimeUtcMillis, afterProcessDeath.single().scheduledFor.wallTimeUtcMillis) + + val completed = metadata.copy(state = ExperimentState.COMPLETED) + assertTrue(planner.next(configuration, completed, at(2), ZoneId.of("UTC")).isEmpty()) + } + + @Test + fun activeScheduleWaitsWhilePaused() { + val metadata = runningMetadata().copy( + state = ExperimentState.PAUSED, + transitions = listOf( + transition(ExperimentState.READY, ExperimentState.RUNNING, TransitionReason.PARTICIPANT_STARTED, 0), + transition(ExperimentState.RUNNING, ExperimentState.PAUSED, TransitionReason.PARTICIPANT_PAUSED, 30), + ), + ) + assertTrue( + planner.next( + configuration(OneTimeSchedule(60, RelativeClock.ACTIVE_RUNNING_TIME)), + metadata, + at(180), + ZoneId.of("UTC"), + ).isEmpty(), + ) + } + + private fun plan(schedule: InterventionSchedule, metadata: StudyMetadata, nowMinutes: Long) = + planner.next(configuration(schedule), metadata, at(nowMinutes), ZoneId.of("UTC")).single() + + private fun runningMetadata() = StudyMetadata.initial("schedule-test", "schedule-config").copy( + state = ExperimentState.RUNNING, + transitions = listOf( + transition(ExperimentState.READY, ExperimentState.RUNNING, TransitionReason.PARTICIPANT_STARTED, 0), + ), + ) + + private fun transition( + from: ExperimentState, + to: ExperimentState, + reason: TransitionReason, + minutes: Long, + ) = ExperimentTransition(from, to, reason, at(minutes)) + + private fun configuration(schedule: InterventionSchedule) = StudyConfiguration( + schemaVersion = 1, + experimentId = "schedule-test", + configurationId = "schedule-config", + issuedAt = Instant.parse("2025-01-01T00:00:00Z"), + expiresAt = Instant.parse("2030-01-01T00:00:00Z"), + minimumAppVersion = 1, + title = "Schedule test", + researcherName = "Researcher", + researcherContact = "research@example.invalid", + purpose = "Test deterministic intervention scheduling.", + durationHours = 48, + consentDocumentVersion = "v1", + consentSummary = "Test consent.", + assignedParticipantId = null, + collectors = listOf(AppLifecycleConfiguration(true)), + surveys = emptyList(), + interventions = listOf( + InterventionConfiguration( + "test-notice", + NotificationAction("Study notice", "A study notice is ready."), + listOf(InterventionTrigger("test-trigger", schedule, 180)), + ), + ), + maximumLocalBytes = 16_777_216, + signer = SignerIdentity("test-signer", "x".repeat(32)), + export = ExportConfiguration("export-key", "x".repeat(32)), + upload = null, + ) + + private fun at(minutes: Long, boot: String = "boot-one") = ResearchTime( + wallTimeUtcMillis = BASE_UTC_MILLIS + minutes * 60_000, + elapsedRealtimeNanos = 1_000_000_000 + minutes * 60_000_000_000, + bootSessionId = boot, + ) + + private companion object { + val BASE_UTC_MILLIS: Long = Instant.parse("2026-01-01T00:00:00Z").toEpochMilli() + } +} diff --git a/core/study-application/src/test/kotlin/cool/linc/androiddatacollector/core/application/StudySessionManagerTest.kt b/core/study-application/src/test/kotlin/cool/linc/androiddatacollector/core/application/StudySessionManagerTest.kt index 6c87caa..213b90a 100644 --- a/core/study-application/src/test/kotlin/cool/linc/androiddatacollector/core/application/StudySessionManagerTest.kt +++ b/core/study-application/src/test/kotlin/cool/linc/androiddatacollector/core/application/StudySessionManagerTest.kt @@ -16,13 +16,18 @@ import cool.linc.androiddatacollector.core.collector.StudyAccessGateway import cool.linc.androiddatacollector.core.definition.AppLifecycleConfiguration import cool.linc.androiddatacollector.core.definition.CollectorConfiguration import cool.linc.androiddatacollector.core.definition.ExportConfiguration -import cool.linc.androiddatacollector.core.definition.PromptConfiguration +import cool.linc.androiddatacollector.core.definition.InterventionConfiguration +import cool.linc.androiddatacollector.core.definition.InterventionTrigger +import cool.linc.androiddatacollector.core.definition.NotificationAction +import cool.linc.androiddatacollector.core.definition.OneTimeSchedule +import cool.linc.androiddatacollector.core.definition.RelativeClock import cool.linc.androiddatacollector.core.definition.SignerIdentity import cool.linc.androiddatacollector.core.definition.StudyConfiguration import cool.linc.androiddatacollector.core.definition.UploadConfiguration import cool.linc.androiddatacollector.core.export.ExportReceipt import cool.linc.androiddatacollector.core.model.EventDraft import cool.linc.androiddatacollector.core.model.ExperimentState +import cool.linc.androiddatacollector.core.model.InterventionOccurrence import cool.linc.androiddatacollector.core.model.RecordedEvent import cool.linc.androiddatacollector.core.model.ResearchTime import cool.linc.androiddatacollector.core.model.StorageUsage @@ -94,9 +99,21 @@ class StudySessionManagerTest { } @Test - fun promptNotificationIsPartOfCanonicalRequiredAccess() = runTest { + fun interventionNotificationIsPartOfCanonicalRequiredAccess() = runTest { val configuration = configuration( - prompts = listOf(PromptConfiguration("prompt-one", 1, "Check in")), + interventions = listOf( + InterventionConfiguration( + "notice-one", + NotificationAction("Study check-in", "Check in"), + listOf( + InterventionTrigger( + "after-minute", + OneTimeSchedule(1, RelativeClock.CALENDAR_TIME), + 60, + ), + ), + ), + ), ) val fixture = fixture(configuration) fixture.manager.initialize() @@ -444,6 +461,10 @@ class StudySessionManagerTest { override suspend fun initialize(metadata: StudyMetadata) { this.metadata = metadata } override suspend fun saveMetadata(metadata: StudyMetadata) { this.metadata = metadata } override suspend fun appendEvent(event: RecordedEvent) { events += event } + override suspend fun appendEventAtomically(event: RecordedEvent, metadata: StudyMetadata) { + events += event + this.metadata = metadata + } override suspend fun readEvents( fromSequenceInclusive: Long, upToSequenceInclusive: Long, @@ -539,6 +560,14 @@ class StudySessionManagerTest { scheduleCount += 1 if (failSchedule) error("Scheduling failed") } + override fun replaceInterventionWork( + configuration: StudyConfiguration, + occurrences: List, + ) = Unit + override fun enqueueOccurrence( + configuration: StudyConfiguration, + occurrence: InterventionOccurrence, + ) = Unit override fun cancelCollectionWork(experimentId: String) { cancelCollectionCount += 1 } override fun cancel(experimentId: String) { cancelCount += 1 } } @@ -587,12 +616,13 @@ class StudySessionManagerTest { } private fun configuration( - prompts: List = emptyList(), + interventions: List = emptyList(), upload: UploadConfiguration? = null, ) = StudyConfiguration( schemaVersion = StudyConfiguration.CURRENT_SCHEMA_VERSION, experimentId = "session-test", configurationId = "session-config", + assignedParticipantId = null, issuedAt = Instant.parse("2026-01-01T00:00:00Z"), expiresAt = Instant.parse("2030-01-01T00:00:00Z"), minimumAppVersion = 1, @@ -604,7 +634,8 @@ class StudySessionManagerTest { consentDocumentVersion = "v1", consentSummary = "Test consent", collectors = listOf(AppLifecycleConfiguration(required = true)), - prompts = prompts, + surveys = emptyList(), + interventions = interventions, maximumLocalBytes = 16_777_216, signer = SignerIdentity("test-signer", TEST_SIGNER_PUBLIC_KEY), export = ExportConfiguration("export-key", "x".repeat(32)), diff --git a/core/study-definition/src/main/kotlin/cool/linc/androiddatacollector/core/definition/StudyConfiguration.kt b/core/study-definition/src/main/kotlin/cool/linc/androiddatacollector/core/definition/StudyConfiguration.kt index 34c41b4..be99bf7 100644 --- a/core/study-definition/src/main/kotlin/cool/linc/androiddatacollector/core/definition/StudyConfiguration.kt +++ b/core/study-definition/src/main/kotlin/cool/linc/androiddatacollector/core/definition/StudyConfiguration.kt @@ -16,8 +16,11 @@ data class StudyConfiguration( val durationHours: Int, val consentDocumentVersion: String, val consentSummary: String, + /** Researcher-assigned opaque code. Null means anonymous/pseudonymous distribution. */ + val assignedParticipantId: String?, val collectors: List, - val prompts: List, + val surveys: List, + val interventions: List, val maximumLocalBytes: Long, val signer: SignerIdentity, val export: ExportConfiguration, @@ -36,9 +39,29 @@ data class StudyConfiguration( require(durationHours in 1..8_760) { "Invalid study duration" } require(consentDocumentVersion.length in 1..64) { "Invalid consent document version" } require(consentSummary.length in 1..8_000) { "Invalid consent summary" } + assignedParticipantId?.let { + require(ASSIGNED_PARTICIPANT_ID.matches(it) && it.toByteArray().size <= 64) { + "Invalid assigned participant ID" + } + } require(collectors.isNotEmpty()) { "At least one collector is required" } require(collectors.map { it.id }.distinct().size == collectors.size) { "Duplicate collector ID" } - require(prompts.map { it.id }.distinct().size == prompts.size) { "Duplicate prompt ID" } + require(surveys.map { it.id }.distinct().size == surveys.size) { "Duplicate survey ID" } + require(interventions.map { it.id }.distinct().size == interventions.size) { "Duplicate intervention ID" } + require(interventions.flatMap { intervention -> intervention.triggers.map { it.id } }.let { + it.distinct().size == it.size + }) { "Duplicate intervention trigger ID" } + var maximumOccurrenceCount = 0L + interventions.forEach { intervention -> + intervention.triggers.forEach { it.schedule.requireWithin(durationHours * 60) } + maximumOccurrenceCount += intervention.triggers.sumOf { + it.schedule.maximumOccurrences(durationHours * 60) + } + (intervention.action as? SurveyAction)?.let { action -> + require(surveys.any { it.id == action.surveyId }) { "Unknown survey ID" } + } + } + require(maximumOccurrenceCount <= MAXIMUM_OCCURRENCES) { "Too many intervention occurrences" } require(maximumLocalBytes in MINIMUM_LOCAL_BYTES..MAXIMUM_LOCAL_BYTES) { "Invalid local quota" } } @@ -59,6 +82,8 @@ data class StudyConfiguration( const val MINIMUM_LOCAL_BYTES = 8L shl 20 const val MAXIMUM_LOCAL_BYTES = 8L shl 30 val ID = Regex("[a-z0-9][a-z0-9-]{2,63}") + val ASSIGNED_PARTICIPANT_ID = Regex("[A-Za-z0-9][A-Za-z0-9._-]{0,63}") + const val MAXIMUM_OCCURRENCES = 512L } } @@ -164,18 +189,219 @@ data class KeyboardTouchConfiguration( companion object { const val ID = "keyboard_touch.v1" } } -data class PromptConfiguration( +data class InterventionConfiguration( + val id: String, + val action: InterventionAction, + val triggers: List, +) { + init { + require(StudyConfiguration.ID.matches(id)) { "Invalid intervention ID" } + require(triggers.isNotEmpty()) { "An intervention needs at least one trigger" } + require(triggers.map { it.id }.distinct().size == triggers.size) { "Duplicate trigger ID" } + } +} + +sealed interface InterventionAction { + val notificationTitle: String + val notificationMessage: String +} + +data class NotificationAction( + override val notificationTitle: String, + override val notificationMessage: String, +) : InterventionAction { + init { validateNotificationText(notificationTitle, notificationMessage) } +} + +data class SurveyAction( + override val notificationTitle: String, + override val notificationMessage: String, + val surveyId: String, +) : InterventionAction { + init { + validateNotificationText(notificationTitle, notificationMessage) + require(StudyConfiguration.ID.matches(surveyId)) { "Invalid survey ID" } + } +} + +private fun validateNotificationText(title: String, message: String) { + require(title.length in 1..120) { "Invalid notification title" } + require(message.length in 1..500) { "Invalid notification message" } +} + +data class InterventionTrigger( + val id: String, + val schedule: InterventionSchedule, + val availabilityMinutes: Int, +) { + init { + require(StudyConfiguration.ID.matches(id)) { "Invalid trigger ID" } + require(availabilityMinutes in 1..525_600) { "Invalid availability window" } + } +} + +sealed interface InterventionSchedule { + fun requireWithin(studyMinutes: Int) + fun maximumOccurrences(studyMinutes: Int): Long +} + +enum class RelativeClock { CALENDAR_TIME, ACTIVE_RUNNING_TIME } + +data class OneTimeSchedule( + val offsetMinutes: Int, + val clock: RelativeClock, +) : InterventionSchedule { + init { require(offsetMinutes >= 0) { "Invalid one-time offset" } } + override fun requireWithin(studyMinutes: Int) { + require(offsetMinutes < studyMinutes) { "One-time trigger is outside the study" } + } + override fun maximumOccurrences(studyMinutes: Int) = 1L +} + +data class IntervalSchedule( + val startOffsetMinutes: Int, + val intervalMinutes: Int, + val clock: RelativeClock, +) : InterventionSchedule { + init { + require(startOffsetMinutes >= 0) { "Invalid interval start" } + require(intervalMinutes in 1..525_600) { "Invalid trigger interval" } + } + override fun requireWithin(studyMinutes: Int) { + require(startOffsetMinutes < studyMinutes) { "Interval trigger is outside the study" } + } + override fun maximumOccurrences(studyMinutes: Int): Long = + (studyMinutes - startOffsetMinutes + intervalMinutes - 1L) / intervalMinutes +} + +data class DailyLocalSchedule( + /** Strict 24-hour local time, `HH:mm`. */ + val localTime: String, +) : InterventionSchedule { + init { require(LOCAL_TIME.matches(localTime)) { "Invalid daily local time" } } + override fun requireWithin(studyMinutes: Int) = Unit + override fun maximumOccurrences(studyMinutes: Int): Long = (studyMinutes + 1_439L) / 1_440L + 1 + + companion object { private val LOCAL_TIME = Regex("(?:[01][0-9]|2[0-3]):[0-5][0-9]") } +} + +data class SurveyDefinition( val id: String, - val delayMinutes: Int, - val message: String, + val title: LocalizedText, + val description: LocalizedText, + val questions: List, ) { init { - require(StudyConfiguration.ID.matches(id)) { "Invalid prompt ID" } - require(delayMinutes in 1..525_600) { "Invalid prompt delay" } - require(message.length in 1..500) { "Invalid prompt message" } + require(StudyConfiguration.ID.matches(id)) { "Invalid survey ID" } + require(questions.size in 1..100) { "Invalid survey question count" } + require(questions.map { it.id }.distinct().size == questions.size) { "Duplicate survey question ID" } } } +data class LocalizedText( + val default: String, + val translations: Map = emptyMap(), +) { + init { + require(default.length in 1..2_000) { "Invalid default localized text" } + require(translations.size <= 32) { "Too many localized values" } + require(translations.keys.map(String::lowercase).distinct().size == translations.size) { + "Duplicate localized language tag" + } + translations.forEach { (language, value) -> + require(BCP47.matches(language)) { "Invalid language tag" } + require(value.length in 1..2_000) { "Invalid localized text" } + } + } + fun resolve(languageTag: String): String { + translations.entries.firstOrNull { it.key.equals(languageTag, ignoreCase = true) }?.let { return it.value } + val requested = languageTag.lowercase().split('-') + return translations.entries + .filter { it.key.substringBefore('-').equals(requested.first(), ignoreCase = true) } + .sortedByDescending { it.key.count { character -> character == '-' } } + .firstOrNull { (tag) -> tag.lowercase().split('-').drop(1).all(requested::contains) } + ?.value + ?: default + } + companion object { private val BCP47 = Regex("[A-Za-z]{2,3}(?:-[A-Za-z0-9]{2,8})*") } +} + +sealed interface SurveyQuestion { + val id: String + val prompt: LocalizedText + val required: Boolean +} + +data class ShortTextQuestion( + override val id: String, + override val prompt: LocalizedText, + override val required: Boolean, + val maximumLength: Int, +) : SurveyQuestion { + init { + validateQuestionId(id) + require(maximumLength in 1..4_000) { "Invalid short-text limit" } + } +} + +data class ScaleQuestion( + override val id: String, + override val prompt: LocalizedText, + override val required: Boolean, + val minimum: Int, + val maximum: Int, + val minimumLabel: LocalizedText, + val maximumLabel: LocalizedText, +) : SurveyQuestion { + init { + validateQuestionId(id) + require(minimum in -1_000..1_000 && maximum in -1_000..1_000 && minimum < maximum) { + "Invalid scale bounds" + } + } +} + +data class SingleChoiceQuestion( + override val id: String, + override val prompt: LocalizedText, + override val required: Boolean, + val options: List, +) : SurveyQuestion { + init { + validateQuestionId(id) + validateOptions(options) + } +} + +data class MultipleChoiceQuestion( + override val id: String, + override val prompt: LocalizedText, + override val required: Boolean, + val options: List, + val minimumSelections: Int, + val maximumSelections: Int, +) : SurveyQuestion { + init { + validateQuestionId(id) + validateOptions(options) + require(minimumSelections in 0..options.size) { "Invalid minimum selections" } + require(maximumSelections in maxOf(1, minimumSelections)..options.size) { "Invalid maximum selections" } + if (required) require(minimumSelections > 0) { "Required multiple choice needs a selection" } + } +} + +data class ChoiceOption(val id: String, val label: LocalizedText) { + init { require(StudyConfiguration.ID.matches(id)) { "Invalid choice option ID" } } +} + +private fun validateQuestionId(id: String) = + require(StudyConfiguration.ID.matches(id)) { "Invalid survey question ID" } + +private fun validateOptions(options: List) { + require(options.size in 2..50) { "Invalid choice option count" } + require(options.map { it.id }.distinct().size == options.size) { "Duplicate choice option ID" } +} + /** * Who signed this configuration. * diff --git a/core/study-definition/src/main/kotlin/cool/linc/androiddatacollector/core/definition/StudyConfigurationCodec.kt b/core/study-definition/src/main/kotlin/cool/linc/androiddatacollector/core/definition/StudyConfigurationCodec.kt index 1d66322..e6b9652 100644 --- a/core/study-definition/src/main/kotlin/cool/linc/androiddatacollector/core/definition/StudyConfigurationCodec.kt +++ b/core/study-definition/src/main/kotlin/cool/linc/androiddatacollector/core/definition/StudyConfigurationCodec.kt @@ -13,6 +13,7 @@ object StudyConfigurationCodec { "schema_version", "experiment_id", "configuration_id", + "assigned_participant_id", "issued_at", "expires_at", "minimum_app_version", @@ -22,7 +23,8 @@ object StudyConfigurationCodec { "duration_hours", "consent", "collectors", - "prompts", + "surveys", + "interventions", "storage", "signer", "export", @@ -48,6 +50,7 @@ object StudyConfigurationCodec { schemaVersion = root.requireInt("schema_version"), experimentId = root.requireString("experiment_id"), configurationId = root.requireString("configuration_id"), + assignedParticipantId = root.requireNullableString("assigned_participant_id"), issuedAt = Instant.parse(root.requireString("issued_at")), expiresAt = Instant.parse(root.requireString("expires_at")), minimumAppVersion = root.requireInt("minimum_app_version"), @@ -63,7 +66,8 @@ object StudyConfigurationCodec { }.requireString("document_version"), consentSummary = root.requireObject("consent").requireString("summary"), collectors = root.requireArray("collectors").mapElements(::decodeCollector), - prompts = root.requireArray("prompts").mapElements(::decodePrompt), + surveys = root.requireArray("surveys").mapElements(::decodeSurvey), + interventions = root.requireArray("interventions").mapElements(::decodeIntervention), maximumLocalBytes = root.requireObject("storage").also { it.requireExactKeys(setOf("maximum_local_bytes")) }.requireLong("maximum_local_bytes"), @@ -83,6 +87,7 @@ object StudyConfigurationCodec { writer.name("schema_version").value(configuration.schemaVersion) writer.name("experiment_id").value(configuration.experimentId) writer.name("configuration_id").value(configuration.configurationId) + writer.name("assigned_participant_id").value(configuration.assignedParticipantId) writer.name("issued_at").value(configuration.issuedAt.toString()) writer.name("expires_at").value(configuration.expiresAt.toString()) writer.name("minimum_app_version").value(configuration.minimumAppVersion) @@ -100,14 +105,11 @@ object StudyConfigurationCodec { writer.name("collectors").beginArray() configuration.collectors.forEach { encodeCollector(writer, it) } writer.endArray() - writer.name("prompts").beginArray() - configuration.prompts.forEach { prompt -> - writer.beginObject() - writer.name("id").value(prompt.id) - writer.name("delay_minutes").value(prompt.delayMinutes) - writer.name("message").value(prompt.message) - writer.endObject() - } + writer.name("surveys").beginArray() + configuration.surveys.forEach { encodeSurvey(writer, it) } + writer.endArray() + writer.name("interventions").beginArray() + configuration.interventions.forEach { encodeIntervention(writer, it) } writer.endArray() writer.name("storage").beginObject() writer.name("maximum_local_bytes").value(configuration.maximumLocalBytes) @@ -226,16 +228,236 @@ object StudyConfigurationCodec { writer.endObject() } - private fun decodePrompt(element: JsonElement): PromptConfiguration { - val root = element.requireObject("prompt") - root.requireExactKeys(setOf("id", "delay_minutes", "message")) - return PromptConfiguration( + private fun decodeIntervention(element: JsonElement): InterventionConfiguration { + val root = element.requireObject("intervention") + root.requireExactKeys(setOf("id", "action", "triggers")) + return InterventionConfiguration( + id = root.requireString("id"), + action = decodeAction(root.requireObject("action")), + triggers = root.requireArray("triggers").mapElements(::decodeTrigger), + ) + } + + private fun decodeAction(root: JsonObject): InterventionAction = when (root.requireString("type")) { + "notification" -> { + root.requireExactKeys(setOf("type", "notification_title", "notification_message")) + NotificationAction(root.requireString("notification_title"), root.requireString("notification_message")) + } + "survey" -> { + root.requireExactKeys(setOf("type", "notification_title", "notification_message", "survey_id")) + SurveyAction( + root.requireString("notification_title"), + root.requireString("notification_message"), + root.requireString("survey_id"), + ) + } + else -> throw IllegalArgumentException("Unknown intervention action") + } + + private fun decodeTrigger(element: JsonElement): InterventionTrigger { + val root = element.requireObject("trigger") + root.requireExactKeys(setOf("id", "schedule", "availability_minutes")) + val schedule = root.requireObject("schedule") + return InterventionTrigger( root.requireString("id"), - root.requireInt("delay_minutes"), - root.requireString("message"), + when (schedule.requireString("type")) { + "one_time" -> { + schedule.requireExactKeys(setOf("type", "offset_minutes", "clock")) + OneTimeSchedule(schedule.requireInt("offset_minutes"), enumValueOf(schedule.requireString("clock"))) + } + "interval" -> { + schedule.requireExactKeys(setOf("type", "start_offset_minutes", "interval_minutes", "clock")) + IntervalSchedule( + schedule.requireInt("start_offset_minutes"), + schedule.requireInt("interval_minutes"), + enumValueOf(schedule.requireString("clock")), + ) + } + "daily_local" -> { + schedule.requireExactKeys(setOf("type", "local_time")) + DailyLocalSchedule(schedule.requireString("local_time")) + } + else -> throw IllegalArgumentException("Unknown intervention schedule") + }, + root.requireInt("availability_minutes"), + ) + } + + private fun decodeSurvey(element: JsonElement): SurveyDefinition { + val root = element.requireObject("survey") + root.requireExactKeys(setOf("id", "title", "description", "questions")) + return SurveyDefinition( + root.requireString("id"), + decodeLocalizedText(root.requireObject("title")), + decodeLocalizedText(root.requireObject("description")), + root.requireArray("questions").mapElements(::decodeQuestion), ) } + private fun decodeLocalizedText(root: JsonObject): LocalizedText { + root.requireExactKeys(setOf("default", "translations")) + val translations = root.requireObject("translations") + return LocalizedText( + root.requireString("default"), + translations.keySet().associateWith { language -> translations.requireString(language) }, + ) + } + + private fun decodeQuestion(element: JsonElement): SurveyQuestion { + val root = element.requireObject("question") + val id = root.requireString("id") + val prompt = decodeLocalizedText(root.requireObject("prompt")) + val required = root.requireBoolean("required") + return when (root.requireString("type")) { + "short_text" -> { + root.requireExactKeys(setOf("type", "id", "prompt", "required", "maximum_length")) + ShortTextQuestion(id, prompt, required, root.requireInt("maximum_length")) + } + "scale" -> { + root.requireExactKeys( + setOf("type", "id", "prompt", "required", "minimum", "maximum", "minimum_label", "maximum_label"), + ) + ScaleQuestion( + id, + prompt, + required, + root.requireInt("minimum"), + root.requireInt("maximum"), + decodeLocalizedText(root.requireObject("minimum_label")), + decodeLocalizedText(root.requireObject("maximum_label")), + ) + } + "single_choice" -> { + root.requireExactKeys(setOf("type", "id", "prompt", "required", "options")) + SingleChoiceQuestion(id, prompt, required, root.requireArray("options").mapElements(::decodeChoice)) + } + "multiple_choice" -> { + root.requireExactKeys( + setOf("type", "id", "prompt", "required", "options", "minimum_selections", "maximum_selections"), + ) + MultipleChoiceQuestion( + id, + prompt, + required, + root.requireArray("options").mapElements(::decodeChoice), + root.requireInt("minimum_selections"), + root.requireInt("maximum_selections"), + ) + } + else -> throw IllegalArgumentException("Unknown survey question type") + } + } + + private fun decodeChoice(element: JsonElement): ChoiceOption { + val root = element.requireObject("choice") + root.requireExactKeys(setOf("id", "label")) + return ChoiceOption(root.requireString("id"), decodeLocalizedText(root.requireObject("label"))) + } + + private fun encodeIntervention(writer: JsonWriter, intervention: InterventionConfiguration) { + writer.beginObject() + writer.name("id").value(intervention.id) + writer.name("action").beginObject() + when (val action = intervention.action) { + is NotificationAction -> writer.name("type").value("notification") + is SurveyAction -> writer.name("type").value("survey") + } + writer.name("notification_title").value(intervention.action.notificationTitle) + writer.name("notification_message").value(intervention.action.notificationMessage) + (intervention.action as? SurveyAction)?.let { writer.name("survey_id").value(it.surveyId) } + writer.endObject() + writer.name("triggers").beginArray() + intervention.triggers.forEach { trigger -> + writer.beginObject() + writer.name("id").value(trigger.id) + writer.name("schedule").beginObject() + when (val schedule = trigger.schedule) { + is OneTimeSchedule -> { + writer.name("type").value("one_time") + writer.name("offset_minutes").value(schedule.offsetMinutes) + writer.name("clock").value(schedule.clock.name) + } + is IntervalSchedule -> { + writer.name("type").value("interval") + writer.name("start_offset_minutes").value(schedule.startOffsetMinutes) + writer.name("interval_minutes").value(schedule.intervalMinutes) + writer.name("clock").value(schedule.clock.name) + } + is DailyLocalSchedule -> { + writer.name("type").value("daily_local") + writer.name("local_time").value(schedule.localTime) + } + } + writer.endObject() + writer.name("availability_minutes").value(trigger.availabilityMinutes) + writer.endObject() + } + writer.endArray() + writer.endObject() + } + + private fun encodeSurvey(writer: JsonWriter, survey: SurveyDefinition) { + writer.beginObject() + writer.name("id").value(survey.id) + writer.name("title").also { encodeLocalizedText(writer, survey.title) } + writer.name("description").also { encodeLocalizedText(writer, survey.description) } + writer.name("questions").beginArray() + survey.questions.forEach { encodeQuestion(writer, it) } + writer.endArray() + writer.endObject() + } + + private fun encodeLocalizedText(writer: JsonWriter, text: LocalizedText) { + writer.beginObject() + writer.name("default").value(text.default) + writer.name("translations").beginObject() + text.translations.toSortedMap().forEach { (language, value) -> writer.name(language).value(value) } + writer.endObject() + writer.endObject() + } + + private fun encodeQuestion(writer: JsonWriter, question: SurveyQuestion) { + writer.beginObject() + writer.name("type").value( + when (question) { + is ShortTextQuestion -> "short_text" + is ScaleQuestion -> "scale" + is SingleChoiceQuestion -> "single_choice" + is MultipleChoiceQuestion -> "multiple_choice" + }, + ) + writer.name("id").value(question.id) + writer.name("prompt").also { encodeLocalizedText(writer, question.prompt) } + writer.name("required").value(question.required) + when (question) { + is ShortTextQuestion -> writer.name("maximum_length").value(question.maximumLength) + is ScaleQuestion -> { + writer.name("minimum").value(question.minimum) + writer.name("maximum").value(question.maximum) + writer.name("minimum_label").also { encodeLocalizedText(writer, question.minimumLabel) } + writer.name("maximum_label").also { encodeLocalizedText(writer, question.maximumLabel) } + } + is SingleChoiceQuestion -> writer.name("options").also { encodeChoices(writer, question.options) } + is MultipleChoiceQuestion -> { + writer.name("options").also { encodeChoices(writer, question.options) } + writer.name("minimum_selections").value(question.minimumSelections) + writer.name("maximum_selections").value(question.maximumSelections) + } + } + writer.endObject() + } + + private fun encodeChoices(writer: JsonWriter, choices: List) { + writer.beginArray() + choices.forEach { choice -> + writer.beginObject() + writer.name("id").value(choice.id) + writer.name("label").also { encodeLocalizedText(writer, choice.label) } + writer.endObject() + } + writer.endArray() + } + private fun decodeExport(root: JsonObject): ExportConfiguration { root.requireExactKeys(setOf("researcher_key_id", "tink_hpke_public_keyset")) val keyset = root.get("tink_hpke_public_keyset") @@ -264,6 +486,10 @@ object StudyConfigurationCodec { private fun JsonObject.requireString(name: String): String = requireNotNull(get(name)).requireStringValue(name) + private fun JsonObject.requireNullableString(name: String): String? = requireNotNull(get(name)).let { + if (it.isJsonNull) null else it.requireStringValue(name) + } + private fun JsonElement.requireStringValue(name: String): String { require(isJsonPrimitive && asJsonPrimitive.isString) { "$name must be a string" } return asString diff --git a/docs/data-collector-implementation-guide.md b/docs/data-collector-implementation-guide.md index 60b5287..3b4db2c 100644 --- a/docs/data-collector-implementation-guide.md +++ b/docs/data-collector-implementation-guide.md @@ -206,7 +206,7 @@ change one of these, raise it as a design discussion first. | Write files, databases, or preferences | Every research byte must go through the encrypted store so that sequence numbers stay contiguous and export, upload, and reclaiming can all reason about one window. A side file is invisible to export, to the storage quota, and to deletion. | The only project dependencies in a `:collector:*` build file are `:core:collector-api` and `:core:study-definition`. `:core:storage` is not on the classpath, and `CollectorContext` carries no `StudyStore`. | | Change study state | `IMPORTED` → … → `WITHDRAWN` is the participant's control surface. A collector that could move it could un-pause a study the participant paused. | `ExperimentStateMachine.transition` is called only from `ExperimentRuntime` in `:core:experiment-runtime`, which no collector module depends on. | | Start an `Activity` or drive UI | The app must never interrupt the participant on a collector's schedule. | Collector modules do not depend on `:app`. Plugins are constructed with `context.applicationContext`. | -| Schedule prompts or notifications | Prompt timing comes from the signed configuration and is scheduled once, by the session manager. | The `StudyWorkScheduler` port is declared in `:core:study-application`; the WorkManager adapter is in `:app`. Neither is on a collector's classpath. | +| Schedule interventions or notifications | Intervention timing and occurrence identity come from the signed configuration and are reconciled by the session manager. | The `StudyWorkScheduler` port is declared in `:core:study-application`; the WorkManager adapter is in `:app`. Neither is on a collector's classpath. | | Export, encrypt, or package data | Export is a participant-initiated act over a bounded sequence window, encrypted to a researcher HPKE key. | `:core:export` and `:core:crypto` are not on any collector's classpath. | | Open a socket or upload | Network transport lives in the study application layer, where the `StudyUploader` port sends only the encrypted bundle, only to the endpoint the signed configuration names, and only after the consent screen has disclosed it. A collector reaching the network would bypass all three, and the participant guide and deployed consent texts describe a study's transmission as coming from that one place. | The app declares `android.permission.INTERNET` for the upload worker, so the permission is present in the process and no manifest check will catch a collector using it. `:core:export`, `:core:crypto`, and the uploader are off a collector's classpath, and `CollectorContext` exposes no network client. Review is what enforces the rest. | | Record text, characters, or content typed on the research keyboard | The consent text tells participants the keyboard never sees what they write. Touch dynamics research does not need the characters, so the characters are never carried across the boundary. | `ResearchKeyboardView.onTouchEvent` passes `key.category.name` to `ImeObservationBridge.publish`, never `key.text`. `ImeTouchObservation` has no field that could hold a character. The committed text goes to `InputConnection` and stops there. | diff --git a/docs/data-dictionary.md b/docs/data-dictionary.md index d6b530d..ac74a42 100644 --- a/docs/data-dictionary.md +++ b/docs/data-dictionary.md @@ -17,6 +17,7 @@ A decrypted bundle is a `research-bundle-v1` JSON document. "experiment_id": "...", "configuration_id": "...", "participant_instance_id": "0a1b2c3d-4e5f-6071-8293-a4b5c6d7e8f9", + "assigned_participant_id": "cohortA-0042", "state": "RUNNING", "next_sequence_number": 4211, "transitions": [ { "from": "READY", "to": "RUNNING", "reason": "...", "time": { } } ], @@ -32,9 +33,7 @@ a stream, and an uploaded one stops at the first event boundary past a plaintext last sequence it contains is not known until the events have been written. Declaring the window before them would let a bundle claim a range it does not contain. JSON object member order carries no meaning, so a parser that reads by key is unaffected, and decryption is unaffected -too — the format string is unchanged and bundles written by earlier builds decrypt exactly as -they did. Code that consumes a bundle as a token stream and expects the window before the events -it describes does need to change. +too. Code that consumes a bundle as a token stream must still follow the v1 order shown here. `configuration` is the canonical study configuration the participant consented to, reproduced verbatim. Every dataset therefore carries its own definition of what was supposed to be collected — including its `upload` block, so a dataset states whether the study it came from delivered data to an endpoint. @@ -42,7 +41,8 @@ It also includes the `signer` block, so provenance travels with the data: `confi | Field | Meaning | | --- | --- | -| `participant_instance_id` | A random UUID generated on the device when this participant imported the study, and stable for that install. It is pseudonymous: no name, account, device identifier, or advertising ID. It exists so that bundles from different participants in an uploading study can be told apart, and it is disclosed on the consent screen. Treat it as personal data — it links every bundle one person produced. | +| `participant_instance_id` | A random UUID generated on the device for each import. Importing the same signed configuration again creates a different ID and independent sequence space. It is pseudonymous: no name, account, device identifier, or advertising ID. It is disclosed on the consent screen and exposed in upload routing, so treat it as personal data. | +| `assigned_participant_id` | Optional researcher-assigned opaque code copied from the signed configuration. It exists only for a personalized study. It is stored in encrypted metadata and appears in the encrypted export, but is deliberately absent from clear upload headers. It can link the dataset to a research roster and must be governed as personal data. | | `next_sequence_number` | The device's counter at the moment the bundle was written: one past the last event durably stored, across the whole study rather than this bundle. | | `first_sequence_number` | The first event sequence this bundle contains, inclusive. | | `last_sequence_number` | The last event sequence this bundle contains, inclusive. | @@ -96,7 +96,7 @@ This is the most important thing to know before writing a parser. `fields` is a string-to-string map. Numbers and booleans are stringified: acceleration appears as `"9.81"`, not `9.81`, and flags appear as `"true"` / `"false"`, not `true` / `false`. Only the envelope's `sequence_number`, `payload_schema_version`, and the three `observed_time` values are real JSON numbers. -Field keys match `[a-z][a-z0-9_]{0,63}`, at most 32 fields per event, each value at most 1024 characters. +Field keys match `[a-z][a-z0-9_]{0,63}` and an event has at most 32 fields. A field value is capped at 60 KiB of characters; storage independently caps the complete encoded event at 64 KiB. Ordinary collectors emit much smaller scalar values. The larger bound exists so one survey submission can be committed as a single immutable value. ### Time @@ -119,7 +119,26 @@ Several collectors also carry a source-supplied time in their payload. **Do not ### Deduplication -Exports overlap by design — a participant can export repeatedly, and each export contains everything from its retained floor up to its boundary. Uploaded chunks do not overlap each other, and a manual export overlaps every chunk after that floor. Deduplicate on `experiment_id` + `configuration_id` + `collector_id` + `sequence_number`, and separate participants on `participant_instance_id` when bundles arrive at an endpoint rather than by hand. Sequence numbers are never reissued, so a number identifies the same event across every bundle regardless of what has been reclaimed. +Exports overlap by design — a participant can export repeatedly, and each export contains everything from its retained floor up to its boundary. Uploaded chunks do not overlap each other, and a manual export overlaps every chunk after that floor. Deduplicate on `participant_instance_id` + `sequence_number`. Do not merge solely on an assigned ID: two imports for one assigned participant intentionally have different instance IDs and independent sequence spaces. Sequence numbers are never reissued within one instance. + +### Intervention and survey events (`interventions.v1`) + +The runtime, not a data collector, emits these v1 events. All share `intervention_id`, `trigger_id`, `occurrence_id`, and `scheduled_for_utc_millis`. The occurrence ID is a 64-character lowercase SHA-256 identity derived from the logical schedule position; it is the primary join key across lifecycle events. + +| `payload_type` | Meaning | Additional fields | +| --- | --- | --- | +| `INTERVENTION_SCHEDULED` | The logical occurrence became durable | none | +| `INTERVENTION_RESCHEDULED` | A pending active-time or daily-local occurrence received a new target after pause, clock, or timezone reconciliation; its occurrence ID is unchanged | none | +| `NOTIFICATION_POSTED` | Android was asked to display its notification; this does **not** prove the participant saw it | none | +| `INTERVENTION_OPENED` | A notification-only occurrence was opened | none | +| `SURVEY_OPENED` | The exact survey occurrence was opened | none | +| `SURVEY_EXPIRED` | Its signed availability window ended without a submission | none | +| `INTERVENTION_EXPIRED` | A notification-only occurrence expired | none | +| `SURVEY_SUBMITTED` | One validated, final survey response was atomically committed | `survey_id`, `scheduled_time`, `opened_time`, `submitted_time`, `answers_json` | + +The three `*_time` fields are compact JSON encodings of the same three-clock `ResearchTime` shape documented above. `answers_json` is one compact JSON object keyed only by stable question IDs. Short text is a JSON string, scale is an integer, and choice answers are arrays of stable option IDs. Question wording, translated labels, and option display text are never copied into the answer. Optional unanswered questions are absent. There are no draft, answer-change, or partial-submission events. + +For compliance metrics, start with the lifecycle event that actually supports the claim: scheduled is not posted, posted is not seen, opened is not submitted, and expired is not declined. Join on `participant_instance_id` and `occurrence_id`; use `assigned_participant_id` only when an approved personalized-study roster requires it. ### Gaps are real and are not errors diff --git a/docs/participant-guide.md b/docs/participant-guide.md index 7cea098..93d8f90 100644 --- a/docs/participant-guide.md +++ b/docs/participant-guide.md @@ -16,7 +16,7 @@ That picker is not a private setting inside the app. It writes Android's own per Because English is the default, this guide quotes the screen in English and gives the Traditional Chinese in parentheses where you might be running it: Start study (開始研究). -**Text the research team wrote is never translated.** The study title, the purpose, the contact details, and the consent text are shown exactly as they were signed, in whatever language the research team wrote them. Only the app's own words change with this setting. +**Ordinary study prose is never translated.** The study title, purpose, contact details, and consent text are shown exactly as signed. Survey titles, descriptions, questions, and choices are different: the signed configuration can carry an English and Traditional Chinese version, and the survey uses the best exact language match with the signed default as its fallback. Two pieces of text stay in English whichever language you pick, because they are written into the code rather than into the translated set: @@ -108,6 +108,12 @@ In that case the check has already been done for you, at the time the app was bu Nothing in the researcher name, contact details, or study description proves who wrote them — they are part of the file, so whoever signed the file chose them. The fingerprint is the part you can check independently. +### Your study codes + +The consent step also says whether the study is **Anonymous or pseudonymous** (匿名或假名研究) or **Personalized** (個人化研究). An anonymous configuration contains no code assigned by the research team. A personalized configuration shows the exact opaque code embedded in your signed file; compare it with the code the team gave you. The app never asks you to enter a name, email address, or phone number. + +Every import also creates a fresh random installation code. Importing the same configuration twice therefore produces two different installation codes. A personalized export contains both codes inside its encrypted body; an upload exposes only the random installation code in its headers, never the researcher-assigned code. Treat either code as linkable study data even though neither is required to be a name. + ### Whether the study sends data automatically The consent step always answers this, in a block above the agreement checkbox. A study that sends nothing says so, under a check mark: @@ -191,7 +197,7 @@ Tap anywhere on a row that is not granted yet and the app sends you straight to ### Notifications (通知) -Used for study prompts and for the notification that stays visible while collection is running. Prompts are scheduled through Android's background work system, which is not an exact alarm: battery saving, Doze, or system scheduling can delay them. +Used for scheduled study activities and for the notification that stays visible while collection is running. Activities use Android's background work system, which is not an exact alarm: battery saving, Doze, or system scheduling can delay them. ### Motion sensor (加速度感測器) and basic network state @@ -237,11 +243,17 @@ The moment you do, the five dots are replaced, permanently, by a status line, an While a study is collecting: - Continuous sources run under a visible research foreground service, so the ongoing "Research collection active" notification is present the whole time. -- Prompts are scheduled according to the study configuration, counted from the first start. +- Signed interventions are scheduled as one-time, repeating-interval, or daily local-time activities. A schedule may count calendar time or only time spent actively collecting. - When the study's duration is reached, the system completes a collecting or paused study for you. Battery-saving scheduling can delay this. After you restart your phone, only a study that was Collecting (收集中) tries to resume. If you force stop the app, Android may block its work until you open it again. Open the app and check the status line if you are unsure. +### Scheduled activities and surveys + +Each scheduled activity has a durable occurrence identity. Restarting the phone, reopening the app, changing time zone, pausing, or recovering WorkManager reschedules that same occurrence instead of creating another one. Android can deliver a notification late, but the app records separately when it was scheduled, posted, opened, submitted, or expired; a notification being posted is not evidence that you saw it. + +Tapping a survey notification opens exactly that occurrence. Surveys are native app screens with screen-reader labels, progress, required/optional indicators, and four answer types: short text, numeric scale, one choice, or multiple choices. Closing before submission stores no research answer or draft. Reopening returns to the same unanswered survey. Submission requires confirmation and is atomic: after one successful submission, the answer is read-only and cannot be edited or submitted again, even after restart or competing taps. An expired occurrence cannot be submitted. + ### The status line Under the study title, where the dots used to be, there are three things: @@ -392,7 +404,8 @@ Uninstalling the app or clearing its app data also destroys the local keys and d | A source shows a red dot and any other code | Pause first, check that permission or special access, then try to resume; if it still fails, contact the research team and quote the code | | Data volume does not change in real time | Android's accounting is coarse and delayed, and the study sets how often the app polls; this is a normal limit | | There are gaps in location | Check location access, your phone's location services, and the research foreground service; indoors it can still be inaccurate | -| Prompts do not arrive on time | Check the notification permission and battery-saving settings; prompts use inexact background scheduling | +| A scheduled activity does not arrive on time | Check the notification permission and battery-saving settings; interventions use inexact background scheduling | +| A survey says it expired or is unavailable | It cannot be submitted after its signed response window; contact the research team if the timing was unexpected | | Export fails | Check that the destination is writable and has enough space, and retry in another location; the study status does not change because of a failed export | | An `UPLOAD_…` code appears where the sent figure usually is | Nothing to do. Collection carries on and the app retries by itself; connect to Wi-Fi and charge the phone if it persists for days, then contact the research team and quote the code | | Storage failure / paused | The app fails closed and stops accepting events; do not clear the app's data, contact the research team first, or export if you need to | diff --git a/docs/researcher-guide.md b/docs/researcher-guide.md index ea5af7a..e2babf8 100644 --- a/docs/researcher-guide.md +++ b/docs/researcher-guide.md @@ -2,7 +2,7 @@ Android Data Collector runs a study from a signed configuration file, so a new study does not need a new app. You describe the study in JSON — which collectors run and with what -sampling parameters, how long it lasts, what the consent summary says, which prompts are +sampling parameters, how long it lasts, what the consent summary says, which interventions and surveys are scheduled, how much local storage it may use, which public key its bundles are encrypted to, and whether it delivers them to an endpoint on a schedule — sign that file with your study key, and hand it to participants. The participant app verifies the signature, presents the @@ -169,8 +169,9 @@ The full command surface is: ```text signing-keygen --private FILE --public FILE hpke-keygen --private FILE --public FILE -canonicalize --input FILE --output FILE -sign --config FILE --private FILE --key-id ID --output FILE +canonicalize --input FILE --output FILE [--assigned-participant-id ID] +sign --config FILE --private FILE --key-id ID --output FILE [--assigned-participant-id ID] +personalize --config FILE --mapping TSV --private FILE --key-id ID --output-dir DIRECTORY check-config --envelope FILE [--public FILE --key-id ID] [--app-version N] [--now ISO_INSTANT] decrypt --bundle FILE --private FILE --config FILE --output FILE ``` @@ -182,10 +183,10 @@ as a runnable starting point. The root object must contain exactly these keys, n no fewer: ```text -schema_version, experiment_id, configuration_id, +schema_version, experiment_id, configuration_id, assigned_participant_id, issued_at, expires_at, minimum_app_version, title, researcher, purpose, duration_hours, -consent, collectors, prompts, storage, signer, export, upload +consent, collectors, surveys, interventions, storage, signer, export, upload ``` The decoder rejects unknown keys, missing keys, and wrong JSON types outright. There is no @@ -196,7 +197,7 @@ Constraints enforced by - `schema_version` is always `1`. There is no fallback reader and no migration path: a configuration either matches the current schema exactly or is refused. -- IDs (`experiment_id`, `configuration_id`, prompt `id`, `signer.key_id`, +- Stable IDs (`experiment_id`, `configuration_id`, survey/question/option/intervention/trigger IDs, `signer.key_id`, `export.researcher_key_id`) are 3–64 characters matching `[a-z0-9][a-z0-9-]{2,63}`: lowercase alphanumerics and `-`, with an alphanumeric first character. @@ -213,10 +214,18 @@ Constraints enforced by - At least one collector; collector IDs must be unique. An unknown collector ID is rejected for the whole configuration — it is never skipped because it was marked optional. -- Prompt IDs must be unique, `delay_minutes` is 1–525,600 from first start, and `message` - is 1–500 characters. Prompt delivery uses WorkManager and is inexact; do not build a - protocol that assumes a prompt lands at a precise minute. A configuration containing any - prompt makes notification access a required access. +- `assigned_participant_id` is either `null` (anonymous/pseudonymous distribution) or an opaque + 1–64 byte code matching `[A-Za-z0-9][A-Za-z0-9._-]{0,63}`. Do not put names, email addresses, + phone numbers, or other direct identifiers in it. +- Survey, question, option, intervention, and trigger IDs are unique in their respective scope. + A survey has 1–100 questions and localized text has a required default plus at most 32 BCP 47 + overrides. Question types are `short_text`, `scale`, `single_choice`, and `multiple_choice`. +- An intervention owns one notification or survey action and one or more reusable triggers. + One-time and interval triggers explicitly choose `CALENDAR_TIME` or `ACTIVE_RUNNING_TIME`; + daily triggers use `HH:mm` in the device's current time zone. Offsets must fall inside the study, + availability is 1–525,600 minutes, and the signed study is capped at 512 total occurrences so + durable idempotency metadata remains inside its encrypted 1 MiB bound. + WorkManager timing is best effort, not an exact alarm. Any intervention requires notification access. - `storage.maximum_local_bytes` is 8 MiB–8 GiB (8,388,608–8,589,934,592). - `signer` carries exactly `key_id` and `public_key`. `public_key` is the base64 X.509 Ed25519 public half of the key you sign with, 32–1,024 characters. `key_id` must equal the @@ -232,6 +241,41 @@ Constraints enforced by within a minute whether delivery works at all, and the ceiling is a week. `allow_metered` is a boolean; `false` restricts delivery to unmetered networks. +### Identity modes and personalized batches + +With `"assigned_participant_id": null`, one signed artifact may be distributed to everyone. Each +import independently mints a random `participant_instance_id`, including repeated imports of the +same file. With a non-null assigned code, make a distinct artifact and `configuration_id` for each +participant. The assigned code remains inside encrypted metadata and exports; only the random +instance UUID appears in upload routing headers. + +For a batch, supply a UTF-8 tab-separated mapping with exactly +`configuration_idassigned_participant_id` per line, then run: + +```bash +researcher-tools personalize --config template.json --mapping participants.tsv \ + --private /secure/signing.key --key-id lab-signer-2026 --output-dir issued +``` + +The command validates every row and rejects duplicate configuration IDs before creating the output +directory. It writes `.json` and `.adccfg`; assigned codes are +never placed in filenames or printed. `canonicalize` and `sign` also accept +`--assigned-participant-id` for issuing one artifact. + +### Interventions and surveys + +An action is defined once and reused by all of an intervention's triggers. Calendar-relative +schedules include pauses; active-running schedules exclude them. Daily local schedules follow the +phone's current time zone and are recomputed after time or zone changes. Each planned firing has a +SHA-256 `occurrence_id` derived from its configuration, intervention, trigger, and schedule key, so +reboot, process recovery, WorkManager retry, or duplicate execution cannot create a second firing. +Occurrences stop at the study lifetime and expire after their availability window. + +A survey action references a reusable survey by ID. Display text uses `{ "default": "...", +"translations": { "zh-TW": "..." } }`; stable IDs, never labels, appear in answers. Submissions +are validated and appended atomically once to the encrypted event stream. Draft typing, abandoned +answers, and validation failures are not research events. A submitted response is review-only. + The `signer` block looks like this: ```json @@ -309,16 +353,14 @@ language screen rather than a second one that can disagree with it. Everything t authors is translated: the step names, the collector descriptions, the signature and upload disclosures, the dashboard, and the confirmation dialogs. -**Nothing you supply is translated.** `title`, `purpose`, `researcher.name`, -`researcher.contact`, and `consent.summary` are part of the signed bytes and render exactly -as they were signed, in whatever language you wrote them, whatever language the app is in. A -configuration written in English stays English on a phone set to Chinese, and the reverse. -That is a property of signing rather than an omission: text translated on the device would be -text nobody signed, and the consent summary has to be the wording your ethics committee -approved. +**Study-level prose is not translated.** `title`, `purpose`, `researcher.name`, +`researcher.contact`, and `consent.summary` render exactly as signed. Survey titles, descriptions, +questions, endpoint labels, and choice labels are the exception: author their required default and +explicit BCP 47 overrides in the signed localized-text objects. The app selects an exact or +compatible signed language tag, then the signed default; it never invents a translation. The deployment consequence is real and worth planning for. **A study recruiting across -languages needs one signed configuration per language** — same collectors, same parameters, +languages may still need one signed configuration per language for study and consent prose — same collectors, same parameters, its own consent document version, its own `configuration_id`, and its own signature — with each participant given the one written in theirs. Keep `experiment_id` shared across them so the arms are recognisable as one study, and remember that bundles are de-duplicated on @@ -344,7 +386,7 @@ A populated block looks like this: ```json "upload": { - "endpoint": "https://collect.example.edu/adc/v2/bundle", + "endpoint": "https://collect.example.edu/adc/v1/bundle", "interval_minutes": 360, "allow_metered": false } @@ -364,7 +406,7 @@ attempt retries with exponential backoff from one minute. Delivery continues while the study is `PAUSED`, for data collected before the pause, and it continues after the study ends: finishing, completing on the duration deadline, and withdrawing -cancel prompts and the study deadline but leave delivery running, so an undelivered tail still +cancel future interventions and the study deadline but leave delivery running, so an undelivered tail still reaches you. The chain stops renewing once the study is `COMPLETED` or `WITHDRAWN` and everything it collected has been delivered. Deleting local data cancels delivery outright, so plan for a tail you may never receive and keep manual export in your protocol as the fallback. @@ -431,18 +473,24 @@ cannot read — that is ciphertext. **File and de-duplicate on `X-ADC-Sequence-F pair is unique per chunk. An endpoint that records `X-ADC-Sequence-To-At-Most` as a held range will claim sequences it does not have, and nothing later will correct it. +`assigned_participant_id` is intentionally absent from the URL and every header. It is sensitive +join data and exists only inside the HPKE-encrypted configuration/experiment content. Do not add it +to reverse-proxy logs or invent a routing header for it. + Your endpoint must answer 2xx only once it has durably stored the body. The device advances its watermark to wherever the bundle actually stopped, never sends those sequences again, and may release them locally if the study's storage runs high — so a 2xx you have not earned can cost data that exists nowhere else. Answer 408, 429, or 5xx to ask for a retry; any other 4xx is treated as a request that will keep failing and is not worth the participant's battery. -**The participant instance ID.** A random UUID generated on the device when the study is +**The participant instance ID.** A fresh random UUID generated on the device for every import when the study is imported, stored in that study's metadata, and included in every bundle and every upload request. Without it, bundles from different participants arrive indistinguishable — a manual export carries that information out of band, an upload does not. It is pseudonymous: it contains no name, account, device identifier, or advertising ID, and it is not shared across -studies. Treat it as personal data anyway, because it links every chunk one person produced. +studies. Re-importing the same anonymous or personalized artifact generates a different UUID, so +its upload chunk identity cannot collide. Treat it as personal data anyway, because it links every +chunk one import produced. **You must disclose upload in your consent text.** The app renders the endpoint host, the cadence, the network condition, the fact that only your key can open the payload, and the @@ -531,7 +579,7 @@ configuration is refused before `issued_at` and after `expires_at` without chang system clock. Any change to the configuration bytes invalidates the signature. When consent text, -collector optionality or frequency, prompts, quota, or the export key changes, mint a new +collector optionality or frequency, interventions, surveys, identity mode, quota, or the export key changes, mint a new `configuration_id`, re-sign, and obtain consent again. Never edit a `.adccfg` that has already been distributed. @@ -732,6 +780,7 @@ exported_at_utc_millis configuration the canonical study configuration experiment: experiment_id, configuration_id, participant_instance_id, + assigned_participant_id (personalized studies only), state, next_sequence_number, transitions[]: from, to, reason, @@ -747,7 +796,8 @@ experiment: so a chunk is never mistaken for a whole study. An uploaded chunk starts after the last sequence the endpoint confirmed. A manual export starts at 1, or at the lowest sequence still on the phone if the device has reclaimed a delivered prefix. `participant_instance_id` is the -pseudonymous per-install identifier described in section 4. +pseudonymous per-import identifier described in section 4. A personalized export additionally +carries `assigned_participant_id`; use it only as the researcher's opaque join key. The two window fields are written after `events`, not before it, because a budget decides where an uploaded bundle stops while it is still streaming. Declaring the window up front would let a @@ -793,11 +843,11 @@ An export is a snapshot, not a state change: then the reassembled chunks plus the final export, and `first_sequence_number` on each bundle tells you where it starts. Keep the chunks; do not treat a late manual export as a replacement for them. -- De-duplicate on `experiment_id` + `configuration_id` + `collector_id` + - `sequence_number`. Sequence numbers come from a single monotonic counter per study, so +- De-duplicate events on `participant_instance_id` + `sequence_number`; the sequence is global to + collectors, intervention lifecycle, and survey responses within one import. `experiment_id` and + `configuration_id` identify the signed artifact rather than a unique device run. Sequence numbers come from a single monotonic counter per study, so they are stable across exports and uploads alike, and reclaiming never reissues one. In an - uploading study, `participant_instance_id` is what separates one participant's counter from - another's. + uploading study, `participant_instance_id` is what separates repeated imports and devices. - A gap in the delivered sequence range is not proof of data loss. A chunk may not have been delivered yet, or may have been cut short when the study ended, and events below a participant's retained floor were released only because your endpoint confirmed them — look @@ -809,6 +859,14 @@ An export is a snapshot, not a state change: ## 11. Analysis notes +### Intervention and survey events + +Join `INTERVENTION_SCHEDULED`, `INTERVENTION_RESCHEDULED`, `NOTIFICATION_POSTED`, `SURVEY_OPENED`, `SURVEY_SUBMITTED`, and +`SURVEY_EXPIRED` on `occurrence_id`. These are app-observable states: `NOTIFICATION_POSTED` means +Android accepted `notify()`, never that the participant saw it. Parse `answers_json` by stable +question IDs and choice option IDs; labels are presentation text and may differ by language. Use +the scheduled/opened/submitted research-time objects to preserve wall, elapsed, and boot context. + ### Acceleration, posture, and movement Raw x/y/z includes gravity and is expressed in device coordinates, whose orientation diff --git a/docs/system-design.md b/docs/system-design.md index b1c931f..3a0be1a 100644 --- a/docs/system-design.md +++ b/docs/system-design.md @@ -9,8 +9,8 @@ a researcher endpoint is an option a study configuration turns on. ## 1. Goals and boundaries - Android 14-17 (`minSdk 34`, `compileSdk`/`targetSdk 37`). -- A signed study configuration determines the study content, the collectors, their parameters, the - prompts, the local storage quota, the export public key, and whether the study uploads. +- A signed v1 study configuration determines study content, participant identity mode, collectors, + localized surveys, intervention actions and triggers, local storage quota, export key, and upload. - Every study event is encrypted on the device before it is stored, and nothing leaves the device in plaintext. - Data reaches the researcher two ways: an export the participant directs, and — when the @@ -73,8 +73,8 @@ flowchart LR | `:core:collector-api` | Collector lifecycle, health, registry, access contract, and the shared callback dispatcher | | `:core:crypto` | Tink HPKE key handling, wrapping, and unwrapping | | `:core:access` | Runtime permission, Usage Access, input-method, and hardware preflight | -| `:core:experiment-runtime` | Command serialization, state machine, collector supervision, event admission gate | -| `:core:study-application` | The single active-study session; recovery and coordination of the storage/access/host/work/export/upload ports, and the upload watermark | +| `:core:experiment-runtime` | Command serialization, state machine, collector supervision, event admission gate, durable occurrence lifecycle, and atomic survey submission | +| `:core:study-application` | The single active-study session; recovery and coordination of storage/access/host/work/export/upload ports, schedule reconciliation, and the upload watermark | | `:core:storage` | Android Keystore, encrypted metadata, appended event segments, reclaiming delivered ones, recovery | | `:core:export` | Streaming JSON/AES-GCM over a requested sequence window under an optional plaintext budget, HPKE key wrapping, and receipts | | `:collector:*` | One independent module per data source | @@ -254,7 +254,8 @@ SHA-256 into an opaque file and key locator. All data lives under `noBackupFiles The manifest disables backup, and the cloud-backup and device-transfer rules exclude all app data. - Metadata: an `AtomicFile` in the format `ADCMET01 | random 96-bit IV | ciphertext+tag`. `ADCMET01` - carries the participant instance ID, the upload watermark, and the retained floor. It also holds + carries the fresh-per-import instance ID, optional researcher-assigned ID, upload watermark, + retained floor, and durable intervention occurrence states. It also holds `last_events`, the most recent event per collector, which is why opening a study needs no scan of the log to rebuild it. There is no fallback reader, so an `ADCMET01` file is refused rather than migrated. @@ -268,6 +269,11 @@ The manifest disables backup, and the cloud-backup and device-transfer rules exc - Each frame: `sequence(u64) | ciphertextLength(u32) | random IV(12) | ciphertext+tag`. - The AAD binds the event format, the opaque study locator, and the sequence number. - Every event append is followed by an `fsync`; metadata is committed through `AtomicFile`. +- An event plus its resulting metadata is one recoverable commit. Before appending, the store writes + an encrypted `ADCTXN01` journal containing the resulting metadata before the event append. + Recovery either completes that exact commit or + recognizes it as already complete, then removes the journal. This is the one write path used for + occurrence lifecycle events and survey submissions; there is no independent draft store. - The active signed configuration is held separately, under its own Keystore key, as `ADCACT01 | random 96-bit IV | ciphertext+tag`. - The local quota comes from the configuration and is bounded to 8 MiB-8 GiB. Encoded metadata is @@ -391,7 +397,8 @@ researcherKeyId | TinkHPKEWrappedAESKey | AES-GCMCiphertext ``` - Content: `research-bundle-v1` JSON containing the canonical configuration, the snapshot time, the - current state, the participant instance ID, all transitions, the events in the window, and then + current state, the participant instance ID, optional assigned participant ID, all transitions, + the events in the window, and then the bundle's own `first_sequence_number` and `last_sequence_number`. Those two sit *after* the `events` array, because a budget decides where a bundle stops while it streams; declaring the window up front would let a bundle claim a range it does not contain. A reader that needs the @@ -406,7 +413,7 @@ researcherKeyId | TinkHPKEWrappedAESKey | AES-GCMCiphertext A state can be exported any number of times, and each file uses a new random key. Repeated exports normally overlap, so the research side should deduplicate on -`configuration_id + collector_id + sequence_number`. In a study that has reclaimed space, an export +`participant_instance_id + sequence_number`. In a study that has reclaimed space, an export starts at the retained floor instead of at 1 and its `first_sequence_number` says so, which makes it a window over the events still on the device rather than the whole history. The wrong private key, the wrong configuration, or any tampering with the header or the ciphertext leaves the bundle @@ -437,7 +444,7 @@ a collector's health reason, so nothing that reaches a screen or a log can hold response becomes `UPLOAD_HTTP_`. The dashboard renders that code in place of the delivered count, and a collector in `FAILED` or `BLOCKED_ACCESS` shows its own reason code the same way. -## 9. Background execution, prompts, and recovery +## 9. Background execution, interventions, and recovery - `CollectionService` runs as a `specialUse` foreground service on start and resume. The `location` service type is added when a location collector is present and fine location has been granted. @@ -446,15 +453,28 @@ count, and a collector in `FAILED` or `BLOCKED_ACCESS` shows its own reason code re-verifies the signed envelope and loads the encrypted metadata. Collectors are constructed on every initialization, but the admission gate, collector activation, and the foreground service are restored only when the persisted state was `RUNNING`. -- Prompts are WorkManager one-time work, scheduled as a delay in minutes measured from the first - start. The timing is not precise and is not guaranteed. +- Each intervention combines a reusable action with one or more triggers. Actions are localized + notifications or localized native surveys. Triggers are one-time offsets, repeating intervals, + or daily local times; each declares whether elapsed study time means calendar time or active + collecting time. WorkManager timing is inexact and delivery can be late. +- `InterventionSchedulePlanner` derives every occurrence ID from configuration, intervention, + trigger, and logical schedule position. The ID is independent of current timezone and process + history. The durable occurrence record owns its scheduled instant, expiry, and lifecycle + (`SCHEDULED`, `POSTING`, `NOTIFICATION_POSTED`, `OPENED`, `SURVEY_SUBMITTED`, `EXPIRED`). Recovery, + boot, time changes, timezone changes, pause, and resume reconcile by that identity, so they do not + enqueue a second logical occurrence. A configuration is bounded to 512 lifetime occurrences so + this exact durable set remains inside the encrypted metadata ceiling. +- A notification content intent carries only the exact occurrence ID. Opening resolves its signed + action from durable state. Survey answers validate against stable survey/question/option IDs and + commit as one immutable `SURVEY_SUBMITTED` event plus terminal occurrence state. Closing the UI + before that commit persists no answer or draft. - The study deadline is a unique WorkManager job. On expiry it moves `RUNNING` or `PAUSED` to `COMPLETED`. - `UploadWorker` is a self-renewing chain of unique one-time work rather than a `PeriodicWorkRequest`. Each link is enqueued with an initial delay of the configuration's `interval_minutes` and enqueues its successor when it finishes. The reason is that WorkManager's periodic floor is 15 minutes: silently clamping a shorter configured cadence would make the - frequency stated on the consent screen untrue. The first link goes out alongside prompts and the + frequency stated on the consent screen untrue. The first link goes out alongside interventions and the deadline when the participant starts a study that declares an endpoint. - Constraints are `NetworkType.UNMETERED` — `CONNECTED` when `allow_metered` is true — and `requiresBatteryNotLow`, with exponential backoff from 1 minute. @@ -464,7 +484,7 @@ count, and a collector in `FAILED` or `BLOCKED_ACCESS` shows its own reason code not have its delay reset on every app start. - The worker acts in `RUNNING`, `PAUSED`, `COMPLETED`, and `WITHDRAWN`, and no-ops in every other state or when the active study is not the one the job was scheduled for. Finishing or withdrawing - cancels prompts and the deadline but leaves delivery running, so a study that has ended still + cancels interventions and the deadline but leaves delivery running, so a study that has ended still sends its undelivered tail. The chain is simply not renewed once `uploadDrained()` reports that a terminal study has nothing outstanding; deleting local data cancels it outright. - A failed run returns `Result.retry()` rather than `failure()`: the usual cause is a network or @@ -477,6 +497,10 @@ count, and a collector in `FAILED` or `BLOCKED_ACCESS` shows its own reason code validity window, or app-version floor fails. A build that pins signers additionally refuses every signer it does not list. - No dynamically downloaded collector, no parsing fallback, no legacy reader. +- The current shape remains schema v1. Earlier prompt-shaped v1 configurations are rejected; there + is no compatibility decoder or schema-version alias. +- The researcher-assigned ID is present only inside the signed configuration and encrypted bundle. + Upload routing exposes the random per-import instance ID but never the assigned ID. - No plaintext study file, no plaintext export scratch file, no secret key in a log. - Study data leaves the device only as an HPKE-wrapped bundle, and only to a destination the participant chose or to the endpoint the signed configuration names. No analytics, no crash @@ -515,6 +539,13 @@ a failure not masking a collection incident, an upload before collection startin finished study still delivering its backlog and then reporting itself drained, and a study without an `upload` block never contacting an endpoint. +Schedule tests cover calendar and active-time one-shots, intervals, daily local time across timezone +changes, restart reconstruction, terminal occurrences, and pause accounting. Runtime tests cover all +four survey question types, required/optional validation, stable IDs without labels, expiry, and +concurrent submission proving exactly one immutable event. Identity tests cover distinct import +instance IDs, assigned-ID persistence/export, upload-header exclusion, CLI bulk uniqueness, and +cross-language canonical bytes. + The budget and streaming decryption have their own export tests: a budget stopping at an event boundary with the receipt naming that boundary and the bundle declaring the window it actually holds, a snapshot with no budget sending everything it was asked for, and a one-byte change to a diff --git a/docs/threat-model.md b/docs/threat-model.md index 5c1084e..971633a 100644 --- a/docs/threat-model.md +++ b/docs/threat-model.md @@ -9,6 +9,8 @@ A reference description of the protections in the current release, the limitatio | Study events on the device | Confidentiality while the phone is out of the participant's hands | | The exported or uploaded bundle | Confidentiality in transit and at the destination | | The study configuration | Integrity — a participant gets exactly the study they consented to | +| Assigned and random participant codes | Confidentiality and controlled linkability to a research roster or import | +| Survey answers and intervention history | Atomicity, immutability, and truthful lifecycle interpretation | | The participant's control | That start, pause, withdrawal, and deletion mean what they say | | The scope of collection | That a study cannot collect beyond what it declared | @@ -36,9 +38,9 @@ One detail matters for review: the app does **not** request StrongBox and does * A configuration is Ed25519-signed inside an `ADCCFG01` envelope. The signing public key travels inside the signed bytes, in a mandatory `signer` block, so a configuration certifies itself and one published app can verify any researcher's study without a rebuild. -**What a signature proves is that the configuration is unchanged since it was signed.** It does not prove who wrote it. A verified configuration establishes that the collector set, their parameters, the duration, the consent text, the export key, and the `upload` block are exactly the bytes the signer produced — nothing has been edited between signing and import. It establishes nothing about the identity behind that key unless the build pins that signer; the limitation below covers what follows from that. +**What a signature proves is that the configuration is unchanged since it was signed.** It does not prove who wrote it. A verified configuration establishes that identity mode, collectors, localized surveys, intervention actions and triggers, duration, consent, export key, and `upload` block are exactly the bytes the signer produced. It establishes nothing about the identity behind that key unless the build pins that signer. -On import the app checks envelope framing and length bounds, decodes the configuration strictly, requires the declared `signer.key_id` to equal the envelope's signer key ID, verifies the signature over the canonical configuration bytes, and checks the validity window and minimum app version. Nothing decoded is acted on until the signature verifies. Canonicality is enforced by re-encoding the decoded configuration and requiring a byte-identical match, so reordered keys, altered whitespace, duplicate keys, and reformatted numbers are rejected. Every object has an exact required key set — unknown *and* missing keys both fail — and an unknown collector ID fails even if the collector is marked optional. +On import the app checks envelope framing and length bounds, decodes the configuration strictly, requires the declared `signer.key_id` to equal the envelope's signer key ID, verifies the signature over the canonical configuration bytes, and checks the validity window and minimum app version. Nothing decoded is acted on until the signature verifies. Canonicality is enforced by re-encoding the decoded configuration and requiring a byte-identical match, so reordered keys, altered whitespace, duplicate keys, and reformatted numbers are rejected. Every object has an exact required key set — unknown *and* missing keys both fail — and an unknown collector ID fails even if the collector is marked optional. The current shape deliberately remains schema v1; prompt-shaped older v1 documents fail instead of entering a compatibility branch. A build may additionally pin signers, as `CollectorApplication.TRUSTED_SIGNING_KEYS`. That map is empty in the shipped build, which therefore accepts any correctly signed configuration and reports the publisher as unverified to the participant. A non-empty map is strictly exclusive: only listed signers are accepted, and the pinned key overrides the one the configuration declares and must equal it, so a configuration cannot claim a pinned key ID while carrying a different key. `ConfigurationVerifier` returns both the configuration and whether its signer was pinned, and the consent screen renders that distinction. @@ -52,7 +54,15 @@ Collectors are selected by ID from a registry of modules compiled into the APK: One template hedges on purpose. The accelerometer entry reads "about N times per second **or more**", because Android treats a sampling period as a hint rather than a contract and a device is free to deliver faster than the study asked for — observed on a current emulator image at over ten times the requested rate. Stating the configured rate alone would understate what is recorded. -Every participant-facing string, this text included, lives in the app's resources and ships in English and Traditional Chinese. The interface follows the phone's system language and offers a picker that writes through Android's `LocaleManager`, which is the same store the system's per-app language screen edits. Researcher-supplied text — title, purpose, researcher name, contact, and the consent summary — is not translated. It renders exactly as signed, in the language it was written in, because a translation performed on the device would be text no signature covers. +Every participant-facing app string lives in resources and ships in English and Traditional Chinese. The interface follows Android's per-app language. Ordinary researcher prose — title, purpose, researcher name, contact, and consent summary — renders exactly as signed. Survey content is the explicit exception: every localized value and its signed default are inside the signed configuration, and selection never creates unsigned text. + +### Identity and survey integrity + +Every import mints a new random UUID, even when the same configuration is imported twice. A personalized configuration may also contain one opaque assigned code restricted to a small ASCII grammar. The consent screen distinguishes the two modes and shows the assigned code for comparison. Both codes live in encrypted metadata and exports; only the random per-import UUID is allowed onto the clear upload-routing surface. + +An intervention occurrence is keyed by a deterministic SHA-256 identity over its signed logical schedule position. Its durable state distinguishes scheduled, notification posted, opened, submitted, and expired; recovery and timezone reconciliation use that identity instead of generating a new occurrence. Survey submission validates stable question and option IDs, then uses an encrypted transaction journal to commit one event and the corresponding terminal metadata together. There is no draft store and no update path after submission. These controls prevent duplicate commits and partial durable answers; they do not prove that a participant saw a notification or personally supplied an answer. + +The configuration admits at most 512 lifetime occurrences. This is a security and reliability bound, not an authoring suggestion: retaining every terminal identity is what prevents an old logical firing from reappearing after recovery, and the bound keeps that set under the authenticated metadata limit instead of silently weakening idempotency. ### Data leaving the device @@ -70,9 +80,9 @@ What an upload endpoint therefore sees: | --- | --- | | That this install is participating, and when each delivery arrives | Any event content; the body is ciphertext only the researcher's HPKE private key opens | | How much data was collected, from the body size and the declared sequence range | Anything derived from the payload without that private key | -| The `experiment_id`, `configuration_id`, and the participant instance ID, sent in request headers | A name, account, device identifier, or advertising ID; none is collected or sent | +| The `experiment_id`, `configuration_id`, and random participant instance ID, sent in request headers | The assigned participant ID, survey content, event content, name, account, device identifier, or advertising ID | -The participant instance ID is a random UUID minted per install when a study is imported and kept in that study's metadata. An uploading study needs it because bundles from different participants otherwise arrive indistinguishable; a manual export carries that information out of band instead. It is pseudonymous, and it is disclosed on the consent screen. +The participant instance ID is a random UUID minted for every import and kept in that study's metadata. An uploading study needs it because encrypted chunks otherwise arrive indistinguishable. It is pseudonymous and disclosed on the consent screen. A researcher-assigned ID, when present, stays inside HPKE ciphertext and must not be copied into endpoint headers or logs. Transport is TLS: the endpoint must be `https://`, validated when the configuration is decoded, and `usesCleartextTraffic="false"` remains set, so a plaintext HTTP endpoint cannot be configured or reached. There is **no certificate pinning**. The connection trusts the device's system trust store, so an attacker holding a certificate that store accepts — an enterprise or otherwise installed CA, for example — can see the delivery metadata above and can substitute their own endpoint. They still cannot read a bundle. @@ -88,6 +98,8 @@ Entering `RUNNING` mints an admission epoch token that collectors must present, A storage write failure or exhausted quota force-closes the admission gate, records an incident code, and fail-closes the study to `PAUSED`. There is no ring buffer and no silent dropping of events, and reclaiming space is a different thing from either: it can only release events an endpoint has already confirmed receiving, so a quota that fills with nothing delivered stops the study rather than making room, and what was released is recorded in `retainedFromSequence`, declared in every bundle's `first_sequence_number`, and stated on the participant's dashboard. Nothing that has not reached the research team is ever discarded to free space. +Occurrence lifecycle events and survey submissions have a stronger two-record boundary: the encrypted `ADCTXN01` journal makes the event append and updated metadata recoverable as one idempotent commit. Recovery completes the exact pending transaction or recognizes it as already complete. No unverified fallback reconstructs a response from UI state. + A corrupt segment, index gap, AEAD failure, or missing key is a hard failure, and only an incomplete trailing frame in the final segment may be recovered. Event segments missing *below* the retained floor are a hard failure too, because a prefix that disappeared without being reclaimed is indistinguishable from one that was tampered away. Metadata claiming more events than are durable is rejected in favour of the durable count, and an export that cannot read its whole window to the boundary fails rather than producing a partial file. Missing required access keeps a study from reaching `READY`, and a foreground service that fails to start rolls the runtime back instead of collecting. A dataset is therefore either complete over the window it declares or absent, rather than quietly partial. **When an event payload is authenticated, and when it is not.** Opening a study decrypts no events. The sequence number is stored unencrypted at the front of each frame, so the framing, the segment index, and the contiguity of the sequence are checked from the plaintext headers, and the metadata — which holds each collector's last event — is verified by its own AES-GCM tag. This is what makes a large quota workable: the cost of opening a study is linear in the number of frames rather than in the bytes decrypted. The trade-off is direct. **An event payload's authentication tag is verified when that event is read, not when the study is opened.** Corruption or tampering inside an event body surfaces on export or upload, as a hard failure at that point, rather than at startup. Nothing is accepted unverified — a tampered event still cannot reach a bundle — but the detection is deferred, so a device holding a damaged log can look healthy until its data is next read. @@ -124,6 +136,10 @@ A build that pins its signers removes this exposure for the studies it accepts, **Inference from the data itself.** That location traces, keyboard touch dynamics, and app usage patterns can identify a person is a property of the data, not a defect in the software. Minimisation and consent are the controls. The keyboard collector cannot see text, but its timing and within-key position data are behaviourally distinctive; the [data dictionary](data-dictionary.md) states this per collector. +**Assigned IDs and survey responses are direct governance responsibilities.** An opaque assigned code can still be identifying to the team that holds its roster, and free-text survey answers can contain names or other sensitive details. Bulk personalization keeps codes out of filenames and logs, and transport keeps them out of headers, but decryption intentionally reveals them to the private-key holder. Ethics review should minimize free text, document the roster join and retention policy, and state that closing an unfinished survey stores no answer while submission is final. + +**Notification timing is not participation evidence.** WorkManager is inexact, devices can delay work, and `NOTIFICATION_POSTED` only records that Android accepted the post. An occurrence ID prevents duplicate logical delivery across recovery; it cannot prove visibility, attention, or who tapped. Analyses must keep scheduled, posted, opened, submitted, and expired as separate outcomes. + **Configuration replay and clock manipulation.** The signed envelope has no nonce and no device binding, so the same configuration can be imported on any number of devices until it expires. Validity is checked against the device wall clock, so a participant who moves their clock backwards can revive an expired configuration. Keep validity windows short; a multi-year window makes both worse. **No signer revocation.** There is no revocation list, rotation protocol, or kill switch at any layer. A leaked study signing key can mint configurations that any build with an empty anchor map accepts, and configurations already signed with it stay valid until they expire; a short validity window is the only control. Where a build does pin signers, that set is fixed and auditable at build time, and retiring one of those keys requires shipping a new APK. diff --git a/researcher-tools/examples/README.md b/researcher-tools/examples/README.md index 41f9b6d..c59f570 100644 --- a/researcher-tools/examples/README.md +++ b/researcher-tools/examples/README.md @@ -10,6 +10,8 @@ These exist so a debug build can exercise signing and export decryption end to end, and so the example configuration in the [researcher guide](../../docs/researcher-guide.md) is runnable. That is their only purpose. The signing public key is not stored separately: it travels inside the configuration, as `demo-study.json`'s `signer.public_key`. +The example is an anonymous v1 configuration with one localized short-answer survey and one one-time survey intervention. It deliberately has no researcher-assigned participant code. + **Never use them for a real study.** A study signed with these keys is not authentic, and an export encrypted to this HPKE key is readable by anyone who clones this repository. **A release build ships no demonstration study.** The signed envelope and the code that loads it live in the app's `debug` source set, so neither is compiled into the release APK: a released app can only run a study that a research team signed and handed to a participant. A debug build still offers the demo, which is what the instrumentation test drives. diff --git a/researcher-tools/examples/demo-study.json b/researcher-tools/examples/demo-study.json index d0b92a0..5110a1b 100644 --- a/researcher-tools/examples/demo-study.json +++ b/researcher-tools/examples/demo-study.json @@ -2,6 +2,7 @@ "schema_version": 1, "experiment_id": "modular-sensing-demo", "configuration_id": "demo-config-2026", + "assigned_participant_id": null, "issued_at": "2026-01-01T00:00:00Z", "expires_at": "2035-01-01T00:00:00Z", "minimum_app_version": 1, @@ -74,11 +75,53 @@ } } ], - "prompts": [ + "surveys": [ + { + "id": "demo-survey", + "title": { + "default": "Study check-in", + "translations": { + "zh-TW": "研究確認" + } + }, + "description": { + "default": "Tell us how the study is going.", + "translations": {} + }, + "questions": [ + { + "type": "short_text", + "id": "status-note", + "prompt": { + "default": "How is it going?", + "translations": {} + }, + "required": false, + "maximum_length": 500 + } + ] + } + ], + "interventions": [ { "id": "demo-check-in", - "delay_minutes": 60, - "message": "Please check that the study is still running as expected." + "action": { + "type": "survey", + "notification_title": "Study check-in", + "notification_message": "Please complete the study check-in.", + "survey_id": "demo-survey" + }, + "triggers": [ + { + "id": "after-one-hour", + "schedule": { + "type": "one_time", + "offset_minutes": 60, + "clock": "ACTIVE_RUNNING_TIME" + }, + "availability_minutes": 1440 + } + ] } ], "storage": { diff --git a/researcher-tools/src/main/kotlin/cool/linc/androiddatacollector/researcher/Main.kt b/researcher-tools/src/main/kotlin/cool/linc/androiddatacollector/researcher/Main.kt index 4748cba..b6bf288 100644 --- a/researcher-tools/src/main/kotlin/cool/linc/androiddatacollector/researcher/Main.kt +++ b/researcher-tools/src/main/kotlin/cool/linc/androiddatacollector/researcher/Main.kt @@ -8,6 +8,7 @@ import cool.linc.androiddatacollector.core.protocol.SignedConfigurationEnvelope import cool.linc.androiddatacollector.core.definition.StudyConfigurationCodec import java.nio.file.Files import java.nio.file.Path +import java.nio.file.StandardCopyOption import java.security.KeyFactory import java.security.KeyPairGenerator import java.security.Signature @@ -24,6 +25,7 @@ fun main(arguments: Array) { "hpke-keygen" -> hpkeKeygen(args) "canonicalize" -> canonicalize(args) "sign" -> sign(args) + "personalize" -> personalize(args) "check-config" -> checkConfig(args) "decrypt" -> decrypt(args) else -> throw IllegalArgumentException(usage()) @@ -43,45 +45,108 @@ private fun hpkeKeygen(args: Arguments) { } private fun canonicalize(args: Arguments) { - val canonical = StudyConfigurationCodec.canonicalize(Files.readAllBytes(args.path("--input"))) + var canonical = StudyConfigurationCodec.canonicalize(Files.readAllBytes(args.path("--input"))) + args.optionalValue("--assigned-participant-id")?.let { assignedId -> + canonical = StudyConfigurationCodec.encode( + StudyConfigurationCodec.decode(canonical).copy(assignedParticipantId = assignedId), + ) + } writeNew(args.path("--output"), canonical) } private fun sign(args: Arguments) { - val configurationBytes = Files.readAllBytes(args.path("--config")) - val configuration = StudyConfigurationCodec.decode(configurationBytes) + var configurationBytes = Files.readAllBytes(args.path("--config")) + var configuration = StudyConfigurationCodec.decode(configurationBytes) + args.optionalValue("--assigned-participant-id")?.let { assignedId -> + configuration = configuration.copy(assignedParticipantId = assignedId) + configurationBytes = StudyConfigurationCodec.encode(configuration) + } val keyId = args.value("--key-id") require(configuration.signer.keyId == keyId) { "Configuration declares signer '${configuration.signer.keyId}' but --key-id is '$keyId'" } - val privateKeyBytes = Base64.getDecoder().decode(Files.readString(args.path("--private")).trim()) - val privateKey = KeyFactory.getInstance("Ed25519").generatePrivate(PKCS8EncodedKeySpec(privateKeyBytes)) + val envelope = signEnvelope( + configurationBytes, + configuration.signer.publicKey, + keyId, + Files.readString(args.path("--private")), + ) + writeNew(args.path("--output"), envelope) + println("signed ${configuration.experimentId} ${configuration.configurationId}") + println("fingerprint ${configuration.signer.fingerprint}") +} + +/** + * Produces one canonical JSON and signed envelope per tab-separated + * `configuration_idassigned_participant_id` row. The assigned code never appears in a + * filename or command output; researchers keep the supplied mapping as the join table. + */ +private fun personalize(args: Arguments) { + val baseBytes = StudyConfigurationCodec.canonicalize(Files.readAllBytes(args.path("--config"))) + val base = StudyConfigurationCodec.decode(baseBytes) + val keyId = args.value("--key-id") + require(base.signer.keyId == keyId) { "Configuration signer does not match --key-id" } + val privateKey = Files.readString(args.path("--private")) + val assignments = Files.readAllLines(args.path("--mapping")).mapIndexed { index, line -> + require(line.isNotBlank()) { "Blank mapping row ${index + 1}" } + val fields = line.split('\t') + require(fields.size == 2) { "Mapping row ${index + 1} must contain exactly two tab-separated fields" } + base.copy(configurationId = fields[0], assignedParticipantId = fields[1]) + } + require(assignments.isNotEmpty()) { "Mapping is empty" } + require(assignments.map { it.configurationId }.distinct().size == assignments.size) { + "Duplicate configuration ID in mapping" + } + + val output = args.path("--output-dir") + require(!Files.exists(output)) { "Refusing to overwrite ${output.toAbsolutePath()}" } + output.parent?.let(Files::createDirectories) + val staging = Files.createTempDirectory(output.parent, ".adc-personalize-") + try { + assignments.forEach { configuration -> + val canonical = StudyConfigurationCodec.encode(configuration) + val envelope = signEnvelope(canonical, configuration.signer.publicKey, keyId, privateKey) + writeNew(staging.resolve("${configuration.configurationId}.json"), canonical) + writeNew(staging.resolve("${configuration.configurationId}.adccfg"), envelope) + } + Files.move(staging, output, StandardCopyOption.ATOMIC_MOVE) + } catch (failure: Throwable) { + deleteTree(staging) + throw failure + } + println("personalized ${assignments.size} configurations") +} + +private fun signEnvelope( + configurationBytes: ByteArray, + declaredPublicKey: String, + keyId: String, + privateKeyBase64: String, +): ByteArray { + val privateKey = KeyFactory.getInstance("Ed25519").generatePrivate( + PKCS8EncodedKeySpec(Base64.getDecoder().decode(privateKeyBase64.trim())), + ) val signature = Signature.getInstance("Ed25519").run { initSign(privateKey) update(configurationBytes) sign() } - // The configuration carries the public key participants will verify with, so a mismatch here - // would produce a file that signs cleanly and then fails on every device. Catch it now. val declaredKey = KeyFactory.getInstance("Ed25519").generatePublic( - X509EncodedKeySpec(Base64.getDecoder().decode(configuration.signer.publicKey)), + X509EncodedKeySpec(Base64.getDecoder().decode(declaredPublicKey)), ) - val selfCheck = Signature.getInstance("Ed25519").run { + require(Signature.getInstance("Ed25519").run { initVerify(declaredKey) update(configurationBytes) verify(signature) - } - require(selfCheck) { "signer.public_key in the configuration does not match --private" } - val envelope = SignedConfigurationCodec.encode( - SignedConfigurationEnvelope( - signerKeyId = keyId, - configurationBytes = configurationBytes, - signature = signature, - ), + }) { "signer.public_key in the configuration does not match --private" } + return SignedConfigurationCodec.encode( + SignedConfigurationEnvelope(keyId, configurationBytes, signature), ) - writeNew(args.path("--output"), envelope) - println("signed ${configuration.experimentId} ${configuration.configurationId}") - println("fingerprint ${configuration.signer.fingerprint}") +} + +private fun deleteTree(root: Path) { + if (!Files.exists(root)) return + Files.walk(root).use { paths -> paths.sorted(Comparator.reverseOrder()).forEach(Files::deleteIfExists) } } private fun checkConfig(args: Arguments) { @@ -154,8 +219,9 @@ private fun usage(): String = """ Commands: signing-keygen --private FILE --public FILE hpke-keygen --private FILE --public FILE - canonicalize --input FILE --output FILE - sign --config FILE --private FILE --key-id ID --output FILE + canonicalize --input FILE --output FILE [--assigned-participant-id ID] + sign --config FILE --private FILE --key-id ID --output FILE [--assigned-participant-id ID] + personalize --config FILE --mapping TSV --private FILE --key-id ID --output-dir DIRECTORY check-config --envelope FILE [--public FILE --key-id ID] [--app-version N] [--now ISO_INSTANT] decrypt --bundle FILE --private FILE --config FILE --output FILE """.trimIndent() diff --git a/web/CONTRACT.md b/web/CONTRACT.md index c6d149f..38c9f7a 100644 --- a/web/CONTRACT.md +++ b/web/CONTRACT.md @@ -27,8 +27,9 @@ guesses, and code that contradicts them produces a file the Android app rejects. leading zeros, no trailing `.0`. 4. **`tink_hpke_public_keyset`** is re-emitted from Gson's `JsonObject.toString()`: compact, no whitespace, keys in the order they appeared. Emit it in the same order the keyset was built in. -5. **Root key order is fixed** by `StudyConfigurationCodec.encode` and is not alphabetical. Take it - from that function, in order. +5. **Root key order is fixed** by `StudyConfigurationCodec.encode` and is not alphabetical. The v1 + shape includes `assigned_participant_id`, `surveys`, and `interventions`; the former prompt shape + is invalid and has no compatibility path. Take the complete order from that function. 6. **`upload: null`** encodes as `"upload":{}`. ## `src/lib/adc/canonical.ts` @@ -121,7 +122,7 @@ picture genuinely cannot carry the meaning. ## Tests `pnpm test` runs the unit suites, including `tests/compat.spec.ts`, which shells out to -`researcher-tools` — building it with Gradle if it is missing — and asserts byte for byte that this +`researcher-tools` — rebuilding its distribution before the suite — and asserts byte for byte that this encoder and the Kotlin one agree, then signs a study here and has `check-config` accept it. `pnpm e2e` is separate because it needs a build, a static server, and a browser. It drives the diff --git a/web/e2e/one-line.mjs b/web/e2e/one-line.mjs index 4afc812..6a9cb2e 100644 --- a/web/e2e/one-line.mjs +++ b/web/e2e/one-line.mjs @@ -60,7 +60,8 @@ const LOCALES = [ /** * The researcher page holds its step in component state, not in the URL, so a view is a route plus * whatever has to be clicked to reach it. `expand` opens the parts of the Study step that are - * behind a switch: seven collectors, a prompt, and scheduled delivery. Those reveal labels and + * behind a switch or add action: seven collectors, both intervention action types, and scheduled + * delivery. Those reveal labels and * hints the site wrote, so leaving them closed would inventory half the step. */ const VIEWS = [ @@ -431,7 +432,8 @@ async function expandStudy(page) { const one = switches.nth(i); if ((await one.getAttribute('aria-checked')) === 'false') await one.click(); } - await page.locator('[data-testid="prompt-add"]').click(); + await page.locator('[data-testid="intervention-add"]').click(); + await page.locator('[data-testid="survey-add"]').click(); const delivery = page.locator('#delivery [role="switch"]').first(); if ((await delivery.getAttribute('aria-checked')) === 'false') await delivery.click(); await page.waitForTimeout(500); diff --git a/web/e2e/units.mjs b/web/e2e/units.mjs index 2eb1eef..115e91c 100644 --- a/web/e2e/units.mjs +++ b/web/e2e/units.mjs @@ -15,7 +15,7 @@ * Exit code: non-zero if any control on `/researcher/` shows a storage-unit number. * * It drives the researcher page the way a person would — reach the Study step, switch on all seven - * collectors, add a prompt, switch on delivery — so that every control the site has is mounted, and + * collectors, add an intervention, switch on delivery — so that every control the site has is mounted, and * then reads what is actually on screen. It does this in both locales, and it does it again after * clicking every preset chip on every control, so the assertions cover the values the page itself * advertises rather than only the ones it opens on. @@ -424,7 +424,7 @@ async function openEverything(page) { if (state !== 'true') throw new Error(`collector ${i} would not switch on`); } - await page.locator('[data-testid="prompt-add"]').click(); + await page.locator('[data-testid="intervention-add"]').click(); const delivery = page.locator('#delivery [role="switch"]').first(); if ((await delivery.getAttribute('aria-checked')) === 'false') await delivery.click(); await page.waitForTimeout(400); diff --git a/web/src/lib/adc/canonical.ts b/web/src/lib/adc/canonical.ts index 09ae575..fdda3dd 100644 --- a/web/src/lib/adc/canonical.ts +++ b/web/src/lib/adc/canonical.ts @@ -13,9 +13,13 @@ import type { CollectorConfig, + ChoiceOption, + InterventionConfig, + LocalizedText, NetworkTransport, - PromptConfig, StudyConfiguration, + SurveyDefinition, + SurveyQuestion, TinkKeyset, UploadConfig } from './types'; @@ -340,12 +344,42 @@ function collectorConfig(collector: CollectorConfig): string { } } -function encodePrompt(prompt: PromptConfig): string { - return ( - `{"id":${quoted(prompt.id)}` + - `,"delay_minutes":${integer(prompt.delay_minutes)}` + - `,"message":${quoted(prompt.message)}}` - ); +function localized(text: LocalizedText): string { + const translations = Object.entries(text.translations).sort(([a], [b]) => (a < b ? -1 : a > b ? 1 : 0)); + return `{"default":${quoted(text.default)},"translations":{${translations.map( + ([language, value]) => `${quoted(language)}:${quoted(value)}` + ).join(',')}}}`; +} + +function choices(options: ChoiceOption[]): string { + return `[${options.map((option) => `{"id":${quoted(option.id)},"label":${localized(option.label)}}`).join(',')}]`; +} + +function question(value: SurveyQuestion): string { + const common = `"type":${quoted(value.type)},"id":${quoted(value.id)},"prompt":${localized(value.prompt)},"required":${boolean(value.required)}`; + switch (value.type) { + case 'short_text': return `{${common},"maximum_length":${integer(value.maximum_length)}}`; + case 'scale': return `{${common},"minimum":${integer(value.minimum)},"maximum":${integer(value.maximum)},"minimum_label":${localized(value.minimum_label)},"maximum_label":${localized(value.maximum_label)}}`; + case 'single_choice': return `{${common},"options":${choices(value.options)}}`; + case 'multiple_choice': return `{${common},"options":${choices(value.options)},"minimum_selections":${integer(value.minimum_selections)},"maximum_selections":${integer(value.maximum_selections)}}`; + } +} + +function survey(value: SurveyDefinition): string { + return `{"id":${quoted(value.id)},"title":${localized(value.title)},"description":${localized(value.description)},"questions":[${value.questions.map(question).join(',')}]}`; +} + +function intervention(value: InterventionConfig): string { + const action = `{"type":${quoted(value.action.type)},"notification_title":${quoted(value.action.notification_title)},"notification_message":${quoted(value.action.notification_message)}${value.action.type === 'survey' ? `,"survey_id":${quoted(value.action.survey_id)}` : ''}}`; + const triggers = value.triggers.map((trigger) => { + const schedule = trigger.schedule.type === 'one_time' + ? `{"type":"one_time","offset_minutes":${integer(trigger.schedule.offset_minutes)},"clock":${quoted(trigger.schedule.clock)}}` + : trigger.schedule.type === 'interval' + ? `{"type":"interval","start_offset_minutes":${integer(trigger.schedule.start_offset_minutes)},"interval_minutes":${integer(trigger.schedule.interval_minutes)},"clock":${quoted(trigger.schedule.clock)}}` + : `{"type":"daily_local","local_time":${quoted(trigger.schedule.local_time)}}`; + return `{"id":${quoted(trigger.id)},"schedule":${schedule},"availability_minutes":${integer(trigger.availability_minutes)}}`; + }); + return `{"id":${quoted(value.id)},"action":${action},"triggers":[${triggers.join(',')}]}`; } /** The exact string `researcher-tools canonicalize` writes, root key order included. */ @@ -355,6 +389,7 @@ export function canonicalize(configuration: StudyConfiguration): string { `"schema_version":${integer(configuration.schema_version)}` + `,"experiment_id":${quoted(configuration.experiment_id)}` + `,"configuration_id":${quoted(configuration.configuration_id)}` + + `,"assigned_participant_id":${configuration.assigned_participant_id === null ? 'null' : quoted(configuration.assigned_participant_id)}` + `,"issued_at":${instantText(configuration.issued_at)}` + `,"expires_at":${instantText(configuration.expires_at)}` + `,"minimum_app_version":${integer(configuration.minimum_app_version)}` + @@ -366,7 +401,8 @@ export function canonicalize(configuration: StudyConfiguration): string { `,"consent":{"document_version":${quoted(configuration.consent.document_version)}` + `,"summary":${quoted(configuration.consent.summary)}}` + `,"collectors":[${configuration.collectors.map(encodeCollector).join(',')}]` + - `,"prompts":[${configuration.prompts.map(encodePrompt).join(',')}]` + + `,"surveys":[${configuration.surveys.map(survey).join(',')}]` + + `,"interventions":[${configuration.interventions.map(intervention).join(',')}]` + `,"storage":{"maximum_local_bytes":${integer(configuration.storage.maximum_local_bytes)}}` + `,"signer":{"key_id":${quoted(configuration.signer.key_id)}` + `,"public_key":${quoted(configuration.signer.public_key)}}` + diff --git a/web/src/lib/adc/schema.ts b/web/src/lib/adc/schema.ts index 8199a39..b80251e 100644 --- a/web/src/lib/adc/schema.ts +++ b/web/src/lib/adc/schema.ts @@ -12,8 +12,10 @@ import { canonicalBytes, formatInstant, keysetJson, parseInstant, type Instant } import { isUsableHpkePublicKeyset } from './tink'; import { BOUNDS, + ASSIGNED_PARTICIPANT_ID_PATTERN, DEFAULT_MINIMUM_APP_VERSION, ID_PATTERN, + MAXIMUM_INTERVENTION_OCCURRENCES, MAXIMUM_CONFIGURATION_BYTES, MAXIMUM_LOCAL_BYTES, MINIMUM_LOCAL_BYTES, @@ -22,6 +24,8 @@ import { UPLOAD_MINIMUM_INTERVAL_MINUTES, type CollectorConfig, type CollectorId, + type LocalizedText, + type SurveyQuestion, type StudyConfiguration } from './types'; @@ -42,7 +46,11 @@ export type IssueCode = | 'document_too_large' | 'signer_missing' | 'export_key_missing' - | 'keyset_unusable'; + | 'keyset_unusable' + | 'language_tag' + | 'unknown_reference' + | 'selection_bounds' + | 'schedule_bounds'; export interface Issue { /** Dotted path into the document, `collectors.2.config.interval_millis`. Empty is the document. */ @@ -73,6 +81,10 @@ export function validate(configuration: StudyConfiguration): Issue[] { } identifier(issues, 'experiment_id', configuration.experiment_id); identifier(issues, 'configuration_id', configuration.configuration_id); + if (configuration.assigned_participant_id !== null && + !ASSIGNED_PARTICIPANT_ID_PATTERN.test(configuration.assigned_participant_id)) { + issues.push({ path: 'assigned_participant_id', code: 'id_format' }); + } const issued = instant(issues, 'issued_at', configuration.issued_at); const expires = instant(issues, 'expires_at', configuration.expires_at); @@ -105,15 +117,54 @@ export function validate(configuration: StudyConfiguration): Issue[] { collectorConfig(issues, `${path}.config`, collector); }); - const promptIds = new Set(); - configuration.prompts.forEach((prompt, index) => { - const path = `prompts.${index}`; - identifier(issues, `${path}.id`, prompt.id); - if (promptIds.has(prompt.id)) issues.push({ path: `${path}.id`, code: 'duplicate_id' }); - promptIds.add(prompt.id); - integer(issues, `${path}.delay_minutes`, prompt.delay_minutes, BOUNDS.promptDelayMinutes); - text(issues, `${path}.message`, prompt.message, BOUNDS.promptMessage); + const surveyIds = new Set(); + configuration.surveys.forEach((survey, index) => { + const path = `surveys.${index}`; + identifier(issues, `${path}.id`, survey.id); + if (surveyIds.has(survey.id)) issues.push({ path: `${path}.id`, code: 'duplicate_id' }); + surveyIds.add(survey.id); + localized(issues, `${path}.title`, survey.title); + localized(issues, `${path}.description`, survey.description); + if (survey.questions.length < 1 || survey.questions.length > 100) { + issues.push(range(`${path}.questions`, [1, 100])); + } + const questionIds = new Set(); + survey.questions.forEach((question, questionIndex) => { + const questionPath = `${path}.questions.${questionIndex}`; + identifier(issues, `${questionPath}.id`, question.id); + if (questionIds.has(question.id)) issues.push({ path: `${questionPath}.id`, code: 'duplicate_id' }); + questionIds.add(question.id); + surveyQuestion(issues, questionPath, question); + }); + }); + + const interventionIds = new Set(); + const triggerIds = new Set(); + let maximumOccurrences = 0; + configuration.interventions.forEach((intervention, index) => { + const path = `interventions.${index}`; + identifier(issues, `${path}.id`, intervention.id); + if (interventionIds.has(intervention.id)) issues.push({ path: `${path}.id`, code: 'duplicate_id' }); + interventionIds.add(intervention.id); + text(issues, `${path}.action.notification_title`, intervention.action.notification_title, BOUNDS.notificationTitle); + text(issues, `${path}.action.notification_message`, intervention.action.notification_message, BOUNDS.notificationMessage); + if (intervention.action.type === 'survey' && !surveyIds.has(intervention.action.survey_id)) { + issues.push({ path: `${path}.action.survey_id`, code: 'unknown_reference' }); + } + if (intervention.triggers.length === 0) issues.push({ path: `${path}.triggers`, code: 'required' }); + intervention.triggers.forEach((trigger, triggerIndex) => { + const triggerPath = `${path}.triggers.${triggerIndex}`; + identifier(issues, `${triggerPath}.id`, trigger.id); + if (triggerIds.has(trigger.id)) issues.push({ path: `${triggerPath}.id`, code: 'duplicate_id' }); + triggerIds.add(trigger.id); + integer(issues, `${triggerPath}.availability_minutes`, trigger.availability_minutes, BOUNDS.availabilityMinutes); + validateSchedule(issues, `${triggerPath}.schedule`, trigger.schedule, configuration.duration_hours * 60); + maximumOccurrences += occurrenceCount(trigger.schedule, configuration.duration_hours * 60); + }); }); + if (maximumOccurrences > MAXIMUM_INTERVENTION_OCCURRENCES) { + issues.push({ path: 'interventions', code: 'schedule_bounds' }); + } integer(issues, 'storage.maximum_local_bytes', configuration.storage.maximum_local_bytes, [ MINIMUM_LOCAL_BYTES, @@ -166,7 +217,9 @@ export function emptyConfiguration(): StudyConfiguration { duration_hours: 24, consent: { document_version: '', summary: '' }, collectors: [], - prompts: [], + assigned_participant_id: null, + surveys: [], + interventions: [], storage: { maximum_local_bytes: DEFAULT_LOCAL_BYTES }, signer: { key_id: '', public_key: '' }, export: { researcher_key_id: '', tink_hpke_public_keyset: { primaryKeyId: 0, key: [] } }, @@ -234,6 +287,91 @@ function text(issues: Issue[], path: string, value: string, [min, max]: Bounds): } } +const BCP47 = /^[A-Za-z]{2,3}(?:-[A-Za-z0-9]{2,8})*$/; + +function localized(issues: Issue[], path: string, value: LocalizedText): void { + text(issues, `${path}.default`, value.default, BOUNDS.surveyText); + const languages = Object.keys(value.translations); + if (languages.length > 32) issues.push(range(`${path}.translations`, [0, 32])); + languages.forEach((language) => { + if (!BCP47.test(language)) issues.push({ path: `${path}.translations.${language}`, code: 'language_tag' }); + text(issues, `${path}.translations.${language}`, value.translations[language], BOUNDS.surveyText); + }); + if (new Set(languages.map((language) => language.toLowerCase())).size !== languages.length) { + issues.push({ path: `${path}.translations`, code: 'duplicate_id' }); + } +} + +function surveyQuestion(issues: Issue[], path: string, question: SurveyQuestion): void { + localized(issues, `${path}.prompt`, question.prompt); + if (question.type === 'short_text') { + integer(issues, `${path}.maximum_length`, question.maximum_length, BOUNDS.shortTextMaximumLength); + return; + } + if (question.type === 'scale') { + integer(issues, `${path}.minimum`, question.minimum, [-1_000, 1_000]); + integer(issues, `${path}.maximum`, question.maximum, [-1_000, 1_000]); + if (question.minimum >= question.maximum) issues.push({ path: `${path}.maximum`, code: 'window_order' }); + localized(issues, `${path}.minimum_label`, question.minimum_label); + localized(issues, `${path}.maximum_label`, question.maximum_label); + return; + } + if (question.options.length < 2 || question.options.length > 50) { + issues.push(range(`${path}.options`, [2, 50])); + } + const optionIds = new Set(); + question.options.forEach((option, index) => { + identifier(issues, `${path}.options.${index}.id`, option.id); + if (optionIds.has(option.id)) issues.push({ path: `${path}.options.${index}.id`, code: 'duplicate_id' }); + optionIds.add(option.id); + localized(issues, `${path}.options.${index}.label`, option.label); + }); + if (question.type === 'multiple_choice') { + integer(issues, `${path}.minimum_selections`, question.minimum_selections, [0, question.options.length]); + integer(issues, `${path}.maximum_selections`, question.maximum_selections, [1, question.options.length]); + if (question.minimum_selections > question.maximum_selections || + (question.required && question.minimum_selections === 0)) { + issues.push({ path: `${path}.maximum_selections`, code: 'selection_bounds' }); + } + } +} + +function validateSchedule( + issues: Issue[], + path: string, + schedule: StudyConfiguration['interventions'][number]['triggers'][number]['schedule'], + studyMinutes: number +): void { + if (schedule.type === 'daily_local') { + if (!/^(?:[01][0-9]|2[0-3]):[0-5][0-9]$/.test(schedule.local_time)) { + issues.push({ path: `${path}.local_time`, code: 'instant' }); + } + return; + } + const offset = schedule.type === 'one_time' ? schedule.offset_minutes : schedule.start_offset_minutes; + integer(issues, `${path}.${schedule.type === 'one_time' ? 'offset_minutes' : 'start_offset_minutes'}`, offset, [0, 525_599]); + if (offset >= studyMinutes) issues.push({ path, code: 'schedule_bounds' }); + if (schedule.type === 'interval') { + integer(issues, `${path}.interval_minutes`, schedule.interval_minutes, [1, 525_600]); + if (schedule.interval_minutes > 0 && Math.ceil((studyMinutes - offset) / schedule.interval_minutes) > 10_000) { + issues.push({ path, code: 'schedule_bounds' }); + } + } +} + +function occurrenceCount( + schedule: StudyConfiguration['interventions'][number]['triggers'][number]['schedule'], + studyMinutes: number +): number { + if (!Number.isInteger(studyMinutes) || studyMinutes <= 0) return 0; + if (schedule.type === 'one_time') return 1; + if (schedule.type === 'daily_local') return Math.ceil(studyMinutes / 1_440) + 1; + if (!Number.isInteger(schedule.start_offset_minutes) || !Number.isInteger(schedule.interval_minutes) || + schedule.start_offset_minutes < 0 || schedule.start_offset_minutes >= studyMinutes || + schedule.interval_minutes <= 0) return 0; + return Math.ceil((studyMinutes - schedule.start_offset_minutes) / schedule.interval_minutes); +} + function integer(issues: Issue[], path: string, value: number, bounds: Bounds): void { if (typeof value !== 'number' || !Number.isInteger(value)) issues.push({ path, code: 'integer' }); else if (value < bounds[0] || value > bounds[1]) issues.push(range(path, bounds)); diff --git a/web/src/lib/adc/types.ts b/web/src/lib/adc/types.ts index 5cb608e..334298f 100644 --- a/web/src/lib/adc/types.ts +++ b/web/src/lib/adc/types.ts @@ -13,6 +13,7 @@ */ export const SCHEMA_VERSION = 1; +export const MAXIMUM_INTERVENTION_OCCURRENCES = 512; /** * Pinned. The lowest `versionCode` the schema allows, and the only one this page authors — there is @@ -22,8 +23,9 @@ export const SCHEMA_VERSION = 1; */ export const DEFAULT_MINIMUM_APP_VERSION = 1; -/** `[a-z0-9][a-z0-9-]{2,63}` — experiment, configuration, prompt, signer, and researcher key IDs. */ +/** `[a-z0-9][a-z0-9-]{2,63}` — stable schema IDs. */ export const ID_PATTERN = /^[a-z0-9][a-z0-9-]{2,63}$/; +export const ASSIGNED_PARTICIPANT_ID_PATTERN = /^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$/; export const MINIMUM_LOCAL_BYTES = 8 * 1024 * 1024; export const MAXIMUM_LOCAL_BYTES = 8 * 1024 * 1024 * 1024; @@ -106,10 +108,55 @@ export type CollectorConfig = } | { id: 'keyboard_touch.v1'; required: boolean; config: { trajectory_sampling_hz: number } }; -export interface PromptConfig { +export interface LocalizedText { + default: string; + translations: Record; +} + +export interface ChoiceOption { + id: string; + label: LocalizedText; +} + +export type SurveyQuestion = + | { type: 'short_text'; id: string; prompt: LocalizedText; required: boolean; maximum_length: number } + | { + type: 'scale'; id: string; prompt: LocalizedText; required: boolean; + minimum: number; maximum: number; minimum_label: LocalizedText; maximum_label: LocalizedText; + } + | { type: 'single_choice'; id: string; prompt: LocalizedText; required: boolean; options: ChoiceOption[] } + | { + type: 'multiple_choice'; id: string; prompt: LocalizedText; required: boolean; + options: ChoiceOption[]; minimum_selections: number; maximum_selections: number; + }; + +export interface SurveyDefinition { + id: string; + title: LocalizedText; + description: LocalizedText; + questions: SurveyQuestion[]; +} + +export type RelativeClock = 'CALENDAR_TIME' | 'ACTIVE_RUNNING_TIME'; +export type InterventionSchedule = + | { type: 'one_time'; offset_minutes: number; clock: RelativeClock } + | { type: 'interval'; start_offset_minutes: number; interval_minutes: number; clock: RelativeClock } + | { type: 'daily_local'; local_time: string }; + +export interface InterventionTrigger { + id: string; + schedule: InterventionSchedule; + availability_minutes: number; +} + +export type InterventionAction = + | { type: 'notification'; notification_title: string; notification_message: string } + | { type: 'survey'; notification_title: string; notification_message: string; survey_id: string }; + +export interface InterventionConfig { id: string; - delay_minutes: number; - message: string; + action: InterventionAction; + triggers: InterventionTrigger[]; } export interface UploadConfig { @@ -133,6 +180,7 @@ export interface StudyConfiguration { schema_version: number; experiment_id: string; configuration_id: string; + assigned_participant_id: string | null; /** ISO-8601 instant, exactly as `Instant.toString()` renders it. */ issued_at: string; expires_at: string; @@ -143,7 +191,8 @@ export interface StudyConfiguration { duration_hours: number; consent: { document_version: string; summary: string }; collectors: CollectorConfig[]; - prompts: PromptConfig[]; + surveys: SurveyDefinition[]; + interventions: InterventionConfig[]; storage: { maximum_local_bytes: number }; signer: { key_id: string; public_key: string }; export: { researcher_key_id: string; tink_hpke_public_keyset: TinkKeyset }; @@ -163,8 +212,11 @@ export const BOUNDS = { // `require(minimumAppVersion > 0)` on a Kotlin `Int`, so the ceiling is the Int's, not "any // positive number": `requireInt` throws above it and the file is refused before the bound is read. minimumAppVersion: [1, 2_147_483_647], - promptDelayMinutes: [1, 525_600], - promptMessage: [1, 500], + notificationTitle: [1, 120], + notificationMessage: [1, 500], + availabilityMinutes: [1, 525_600], + surveyText: [1, 2_000], + shortTextMaximumLength: [1, 4_000], signerPublicKey: [32, 1_024], uploadEndpoint: [8, 2_048], samplingPeriodUs: [5_000, 1_000_000], diff --git a/web/src/lib/i18n/en.ts b/web/src/lib/i18n/en.ts index 8df07b7..d01656f 100644 --- a/web/src/lib/i18n/en.ts +++ b/web/src/lib/i18n/en.ts @@ -26,7 +26,6 @@ export const en: Messages = { download: 'Download', copy: 'Copy', importDraft: 'Import JSON', - addPrompt: 'Add prompt', back: 'Back', next: 'Next', confirmSaved: 'I have the file', @@ -36,6 +35,43 @@ export const en: Messages = { cancel: 'Cancel' }, + intervention: { + title: 'Interventions and surveys', + empty: 'No scheduled activities.', + notificationTiming: 'Android notification timing is best effort, not an exact wall-clock instant.', + anonymous: 'Anonymous / pseudonymous', + personalized: 'Personalized', + assignedId: 'Assigned participant code', + addNotification: 'Add notification', + addSurvey: 'Add survey', + addQuestion: 'Add question', + addTrigger: 'Add schedule', + survey: 'Survey', + surveyTitle: 'Survey title', + surveyDescription: 'Survey description', + question: 'Question ID', + questionType: 'Question type', + prompt: 'Question text', + required: 'Required', + maximumLength: 'Maximum characters', + scaleBounds: 'Scale minimum / maximum', + endpointLabels: 'Endpoint labels', + options: 'Options (stable ID | label, one per line)', + selectionBounds: 'Minimum / maximum selections', + notificationTitle: 'Notification title', + notificationMessage: 'Notification message', + trigger: 'Schedule ID', + scheduleType: 'Schedule type', + clock: 'Relative clock', + offset: 'Offset in minutes', + interval: 'Interval in minutes', + localTime: 'Local time', + availability: 'Available for minutes', + types: { shortText: 'Short text', scale: 'Numeric scale', singleChoice: 'Single choice', multipleChoice: 'Multiple choice' }, + schedules: { oneTime: 'One time', interval: 'Recurring interval', dailyLocal: 'Daily local time' }, + clocks: { calendar: 'Calendar time (pauses included)', active: 'Running time (pauses excluded)' } + }, + control: { language: 'Language', details: 'Details', @@ -106,9 +142,6 @@ export const en: Messages = { displacement: 'Minimum displacement', priority: 'Priority', trajectoryRate: 'Trajectory sampling', - promptId: 'Prompt ID', - promptDelay: 'Delay', - promptMessage: 'Message', upload: 'Scheduled upload', endpoint: 'Endpoint', uploadInterval: 'Interval', @@ -139,7 +172,6 @@ export const en: Messages = { fastestInterval: 'Never longer than the interval.', batchDelay: 'Higher means fewer, larger deliveries.', priority: 'Both need precise location. High accuracy uses GPS.', - promptDelay: 'From the first start. Delivery is inexact.', endpoint: 'https, and yours to run.', allowMetered: 'Off means Wi-Fi only. Mobile data is a cost nobody agreed to.' } @@ -206,7 +238,11 @@ export const en: Messages = { `The whole configuration must stay under ${number.format(max)} bytes`, signer_missing: 'Generate the signing key first', export_key_missing: 'Generate the export key first', - keyset_unusable: 'Not a keyset the app can encrypt to. Generate or import the export key again' + keyset_unusable: 'Not a keyset the app can encrypt to. Generate or import the export key again', + language_tag: 'Use a valid BCP 47 language tag', + unknown_reference: 'Choose a survey defined in this configuration', + selection_bounds: 'Selection limits do not match this question', + schedule_bounds: 'This schedule is outside the study or creates too many occurrences' }, status: { @@ -217,7 +253,6 @@ export const en: Messages = { }, empty: { - prompts: 'None. Most studies need none.', files: 'Nothing to hand out until the configuration is signed.' }, @@ -245,8 +280,6 @@ export const en: Messages = { researcher: { title: 'Prepare a study', - beyond: - 'After the study ends. This prompt will never reach a participant who finishes on time.', lede: 'Keys, study, and signature, in this tab. Nothing leaves it.', how: { file: { @@ -305,7 +338,7 @@ export const en: Messages = { validity: { title: 'How long does this run?' }, collectors: { title: 'Data' }, consent: { title: 'Consent' }, - prompts: { title: 'Prompts' }, + interventions: { title: 'Interventions' }, delivery: { title: 'Delivery', note: 'With this off, data only leaves when they export it.' diff --git a/web/src/lib/i18n/types.ts b/web/src/lib/i18n/types.ts index c9618af..332ce5c 100644 --- a/web/src/lib/i18n/types.ts +++ b/web/src/lib/i18n/types.ts @@ -77,6 +77,10 @@ export interface IssueMessages { signer_missing: string; export_key_missing: string; keyset_unusable: string; + language_tag: string; + unknown_reference: string; + selection_bounds: string; + schedule_bounds: string; } export type IssueCode = keyof IssueMessages; @@ -103,7 +107,6 @@ export interface Messages { download: string; copy: string; importDraft: string; - addPrompt: string; back: string; next: string; /** The reader asserting a downloaded private key reached their disk. */ @@ -115,6 +118,43 @@ export interface Messages { cancel: string; }; + intervention: { + title: string; + empty: string; + notificationTiming: string; + anonymous: string; + personalized: string; + assignedId: string; + addNotification: string; + addSurvey: string; + addQuestion: string; + addTrigger: string; + survey: string; + surveyTitle: string; + surveyDescription: string; + question: string; + questionType: string; + prompt: string; + required: string; + maximumLength: string; + scaleBounds: string; + endpointLabels: string; + options: string; + selectionBounds: string; + notificationTitle: string; + notificationMessage: string; + trigger: string; + scheduleType: string; + clock: string; + offset: string; + interval: string; + localTime: string; + availability: string; + types: { shortText: string; scale: string; singleChoice: string; multipleChoice: string }; + schedules: { oneTime: string; interval: string; dailyLocal: string }; + clocks: { calendar: string; active: string }; + }; + /** Accessible names for controls that carry no text. Visible verbs live in `action`. */ control: { language: string; @@ -194,9 +234,6 @@ export interface Messages { displacement: string; priority: string; trajectoryRate: string; - promptId: string; - promptDelay: string; - promptMessage: string; upload: string; endpoint: string; uploadInterval: string; @@ -232,7 +269,6 @@ export interface Messages { fastestInterval: string; batchDelay: string; priority: string; - promptDelay: string; endpoint: string; allowMetered: string; }; @@ -255,7 +291,6 @@ export interface Messages { }; empty: { - prompts: string; files: string; }; @@ -279,8 +314,6 @@ export interface Messages { researcher: { title: string; lede: string; - /** A prompt scheduled past the study's own duration. Legal, and it will never fire. */ - beyond: string; how: { file: Passage; keys: Passage; @@ -312,7 +345,7 @@ export interface Messages { validity: SectionTitle; collectors: SectionTitle; consent: SectionTitle; - prompts: SectionTitle; + interventions: SectionTitle; /** The one note left: it says what happens with the toggle off, which is the toggle's * meaning and not advice about filling the section in. */ delivery: Section; diff --git a/web/src/lib/i18n/zh-TW.ts b/web/src/lib/i18n/zh-TW.ts index ee69e13..bcae74a 100644 --- a/web/src/lib/i18n/zh-TW.ts +++ b/web/src/lib/i18n/zh-TW.ts @@ -27,7 +27,6 @@ export const zhTW: Messages = { download: '下載', copy: '複製', importDraft: '匯入 JSON', - addPrompt: '新增提示通知', back: '上一步', next: '下一步', confirmSaved: '檔案已存好', @@ -37,6 +36,43 @@ export const zhTW: Messages = { cancel: '取消' }, + intervention: { + title: '介入活動與問卷', + empty: '沒有排定的活動。', + notificationTiming: 'Android 會盡力依排程發出通知，但不保證精準的時刻。', + anonymous: '匿名／假名', + personalized: '個人化', + assignedId: '指定參與者代碼', + addNotification: '新增通知', + addSurvey: '新增問卷', + addQuestion: '新增題目', + addTrigger: '新增排程', + survey: '問卷', + surveyTitle: '問卷標題', + surveyDescription: '問卷說明', + question: '題目 ID', + questionType: '題型', + prompt: '題目文字', + required: '必填', + maximumLength: '最多字元數', + scaleBounds: '量尺最小值／最大值', + endpointLabels: '兩端標籤', + options: '選項（穩定 ID | 標籤，每行一項）', + selectionBounds: '最少／最多選項數', + notificationTitle: '通知標題', + notificationMessage: '通知訊息', + trigger: '排程 ID', + scheduleType: '排程類型', + clock: '相對時間算法', + offset: '延後分鐘數', + interval: '間隔分鐘數', + localTime: '本地時間', + availability: '可填寫分鐘數', + types: { shortText: '簡短文字', scale: '數字量尺', singleChoice: '單選', multipleChoice: '複選' }, + schedules: { oneTime: '單次', interval: '固定間隔', dailyLocal: '每日本地時間' }, + clocks: { calendar: '日曆時間（包含暫停）', active: '收集中時間（排除暫停）' } + }, + control: { language: '語言', details: '詳細資訊', @@ -106,9 +142,6 @@ export const zhTW: Messages = { displacement: '最小位移', priority: '定位模式', trajectoryRate: '軌跡取樣率', - promptId: '提示通知 ID', - promptDelay: '延遲', - promptMessage: '訊息', upload: '自動傳送', endpoint: '接收端點', uploadInterval: '傳送間隔', @@ -137,7 +170,6 @@ export const zhTW: Messages = { fastestInterval: '不能比定位間隔長。', batchDelay: '設得越大，送出的次數越少，一次送的越多。', priority: '兩種模式都需要精確位置。高精確度會用 GPS。', - promptDelay: '從第一次開始算起。送達時間不精確。', endpoint: '必須是 https，而且由你自己營運。', allowMetered: '關閉時只走 Wi-Fi。行動網路要花錢，沒有人同意過要付。' } @@ -203,7 +235,11 @@ export const zhTW: Messages = { document_too_large: ({ max }) => `整份設定必須小於 ${number.format(max)} 個位元組`, signer_missing: '請先產生簽章金鑰', export_key_missing: '請先產生匯出金鑰', - keyset_unusable: 'App 無法用這組金鑰加密。請重新產生匯出金鑰，或改匯入一組' + keyset_unusable: 'App 無法用這組金鑰加密。請重新產生匯出金鑰，或改匯入一組', + language_tag: '請使用有效的 BCP 47 語言標籤', + unknown_reference: '請選擇這份設定中已定義的問卷', + selection_bounds: '選取數量限制與這題不相容', + schedule_bounds: '排程超出研究期間，或產生過多次活動' }, status: { @@ -214,7 +250,6 @@ export const zhTW: Messages = { }, empty: { - prompts: '沒有提示通知，多數研究也不需要。', files: '要先簽署設定檔，才會有檔案可以交付。' }, @@ -242,7 +277,6 @@ export const zhTW: Messages = { researcher: { title: '準備一份研究', - beyond: '這則提示排在研究結束之後，準時完成的參與者不會收到。', lede: '金鑰、設定、簽章都在這個分頁裡，不會外流。', how: { file: { @@ -299,7 +333,7 @@ export const zhTW: Messages = { validity: { title: '這個研究會進行多久？' }, collectors: { title: '資料來源' }, consent: { title: '知情同意' }, - prompts: { title: '提示通知' }, + interventions: { title: '介入活動' }, delivery: { title: '自動傳送', note: '關閉時，資料要等參與者自己匯出才會離開手機。' diff --git a/web/src/lib/ui/Disclosure.svelte b/web/src/lib/ui/Disclosure.svelte index 5c8cbf5..a9bacfd 100644 --- a/web/src/lib/ui/Disclosure.svelte +++ b/web/src/lib/ui/Disclosure.svelte @@ -3,7 +3,7 @@ * rotates rather than swapping glyphs, so the control's identity survives the state change. */ import Icon from './Icon.svelte'; import type { IconRef } from './icons'; - import type { Snippet } from 'svelte'; + import { untrack, type Snippet } from 'svelte'; interface Props { label: string; @@ -16,7 +16,7 @@ let { label, icon, open = false, trailing, testid, children }: Props = $props(); - let expanded = $state(open); + let expanded = $state(untrack(() => open)); const uid = $props.id(); diff --git a/web/src/lib/ui/surfaces.css b/web/src/lib/ui/surfaces.css index fe795d8..f568243 100644 --- a/web/src/lib/ui/surfaces.css +++ b/web/src/lib/ui/surfaces.css @@ -1140,78 +1140,6 @@ button.fingerprint--plaque:hover { font-variant-numeric: tabular-nums; } -/* ---- prompts ------------------------------------------------------------------------------ */ - -.prompt { - display: grid; - grid-template-columns: 1fr; - gap: var(--sp-5); - padding: var(--sp-6); - background: var(--surface); - border: var(--line-hair) solid var(--rule); - border-radius: var(--r-panel); -} - -@media (min-width: 720px) { - .prompt { - grid-template-columns: 14rem 1fr auto; - align-items: start; - } -} - -.prompt__timeline { - position: relative; - block-size: 44px; - margin-block-end: var(--sp-5); -} - -.prompt__axis { - position: absolute; - inset-block-start: 21px; - inset-inline: 0; - block-size: var(--line-solid); - background: var(--rule); -} - -.prompt__tick { - position: absolute; - inset-block-start: 14px; - inline-size: 16px; - block-size: 16px; - margin-inline-start: -8px; - border-radius: var(--r-pill); - background: var(--accent-ink); - border: var(--line-solid) solid var(--surface); - transition: inset-inline-start var(--motion-panel); -} - -/* Past the end of the axis. The ring is a second channel, because a tick that is only a different - colour says nothing to a reader who cannot separate the two — and this one means a prompt that - will never fire. */ -.prompt__tick[data-beyond='true'] { - background: var(--caution-ink); - box-shadow: 0 0 0 2px var(--surface), 0 0 0 4px var(--caution-ink); -} - -.addtile { - display: grid; - place-items: center; - min-block-size: 88px; - border: var(--line-hair) dashed var(--rule); - border-radius: var(--r-panel); - color: var(--ink-faint); - transition: - border-color var(--motion-state), - color var(--motion-state), - background-color var(--motion-state); -} - -.addtile:hover { - border-color: var(--accent-ink); - color: var(--accent-ink); - background: var(--accent-wash); -} - /* ---- signature receipt -------------------------------------------------------------------- */ .receipt { diff --git a/web/src/routes/researcher/+page.svelte b/web/src/routes/researcher/+page.svelte index 8e4ea9c..a85845a 100644 --- a/web/src/routes/researcher/+page.svelte +++ b/web/src/routes/researcher/+page.svelte @@ -127,28 +127,7 @@ advisory: (path) => advisoryFor(path) }); - /** - * One thing that is legal and still worth knowing, so it is not an `Issue`: `validate` mirrors - * the device's own rules exactly, and a rule the device does not have must never block a - * signature the device would accept. - * - * A prompt past the end of the study is the one the timeline could only say in colour — the tick - * turned `--caution` and nothing else on the page mentioned it, so a researcher could ship a - * prompt that will never fire for anyone who finishes on time. - * - * `configuration_id` used to have one too, telling a researcher to change the id by hand after an - * edit. The id is derived from the document now, so the advice is the code's job; `status.stale` - * on the sign step is what still has to be said, and it is said there. - */ - function advisoryFor(path: string): string | null { - const prompt = /^prompts\.(\d+)\.delay_minutes$/.exec(path); - if (prompt) { - const delay = draft.configuration.prompts[Number(prompt[1])]?.delay_minutes; - const span = draft.configuration.duration_hours * 60; - if (typeof delay === 'number' && delay > span) return m.researcher.beyond; - } - return null; - } + const advisoryFor = (_path: string): null => null; /** * Counts are live from the first thing the researcher does, and silent before it. A page opened diff --git a/web/src/routes/researcher/InterventionEditor.svelte b/web/src/routes/researcher/InterventionEditor.svelte new file mode 100644 index 0000000..158eeea --- /dev/null +++ b/web/src/routes/researcher/InterventionEditor.svelte @@ -0,0 +1,223 @@ + + +
+ + {#if configuration.assigned_participant_id !== null} + + {/if} +
+ +{#if configuration.interventions.length === 0} +

{copy.empty}

+{/if} +

{copy.notificationTiming}

+ +{#each configuration.surveys as survey, surveyIndex (survey.id)} +
+ {copy.survey} · {survey.id} +
+ + + + + + +
+ {#each survey.questions as question, questionIndex (question.id)} +
+
+ + + + +
+ + {#if question.type === 'short_text'} + + {:else if question.type === 'scale'} +
+ + +
+ {:else} + + {#if question.type === 'multiple_choice'} + + {/if} + {/if} + +
+ {/each} + +
+{/each} + +{#each configuration.interventions as intervention, interventionIndex (intervention.id)} +
+ {intervention.action.type === 'survey' ? copy.survey : copy.addNotification} · {intervention.id} +
+ + {#if intervention.action.type === 'survey'} + + {/if} + + +
+ {#each intervention.triggers as trigger, triggerIndex (trigger.id)} +
+ + + + {#if trigger.schedule.type === 'daily_local'} + + {:else} + + {#if trigger.schedule.type === 'one_time'} + + {:else} + + {/if} + {#if trigger.schedule.type === 'interval'}{/if} + {/if} + +
+ {/each} +
+
+{/each} + +
+ + diff --git a/web/src/routes/researcher/PromptRow.svelte b/web/src/routes/researcher/PromptRow.svelte deleted file mode 100644 index e9bc190..0000000 --- a/web/src/routes/researcher/PromptRow.svelte +++ /dev/null @@ -1,66 +0,0 @@ - - -
- (prompt.id = value)} - /> - - (prompt.message = value)} - /> - - - -
- (prompt.delay_minutes = value)} - /> -
-
- - diff --git a/web/src/routes/researcher/PromptTimeline.svelte b/web/src/routes/researcher/PromptTimeline.svelte deleted file mode 100644 index e7a14af..0000000 --- a/web/src/routes/researcher/PromptTimeline.svelte +++ /dev/null @@ -1,33 +0,0 @@ - - - diff --git a/web/src/routes/researcher/StepStudy.svelte b/web/src/routes/researcher/StepStudy.svelte index e18a566..2b3c3a0 100644 --- a/web/src/routes/researcher/StepStudy.svelte +++ b/web/src/routes/researcher/StepStudy.svelte @@ -23,8 +23,7 @@ import TextField from '$lib/ui/TextField.svelte'; import ToggleField from '$lib/ui/ToggleField.svelte'; import CollectorCard from './CollectorCard.svelte'; - import PromptRow from './PromptRow.svelte'; - import PromptTimeline from './PromptTimeline.svelte'; + import InterventionEditor from './InterventionEditor.svelte'; import QuotaMeter from './QuotaMeter.svelte'; import WindowStrip from './WindowStrip.svelte'; import { COLLECTOR_ORDER, type Draft } from './draft.svelte'; @@ -255,44 +254,8 @@ -
- {#if configuration.prompts.length === 0} - - {:else} - - - - {#each configuration.prompts as prompt, index (index)} - draft.removePrompt(index)} - /> - {/each} - {/if} - - +
+
diff --git a/web/src/routes/researcher/draft.svelte.ts b/web/src/routes/researcher/draft.svelte.ts index 684a8a2..c05a85c 100644 --- a/web/src/routes/researcher/draft.svelte.ts +++ b/web/src/routes/researcher/draft.svelte.ts @@ -48,7 +48,8 @@ import { ID_PATTERN, type CollectorConfig, type CollectorId, - type PromptConfig, + type InterventionConfig, + type SurveyDefinition, type StudyConfiguration } from '$lib/adc/types'; import type { StepState } from '$lib/ui/types'; @@ -88,7 +89,9 @@ function studyStarted(configuration: StudyConfiguration): boolean { configuration.consent.document_version !== '' || configuration.consent.summary !== '' || configuration.collectors.length > 0 || - configuration.prompts.length > 0 || + configuration.surveys.length > 0 || + configuration.interventions.length > 0 || + configuration.assigned_participant_id !== null || configuration.upload !== null ); } @@ -166,7 +169,7 @@ export function createDraft() { /** * What is validated, canonicalised, signed, and downloaded. Never the editable object: the spread - * is shallow, so `collectors` and `prompts` pass through by reference as the `$state` proxies they + * is shallow, so collection and intervention arrays pass through as the `$state` proxies they * are and stay reactive. * * `signer` and `export` are the exception — they are replaced by fresh literals above, so that @@ -457,12 +460,24 @@ export function createDraft() { if (index >= 0) configuration.collectors.splice(index, 1); }, - addPrompt(prompt: PromptConfig) { - configuration.prompts.push(prompt); + addSurvey(survey: SurveyDefinition) { + configuration.surveys.push(survey); }, - removePrompt(index: number) { - configuration.prompts.splice(index, 1); + removeSurvey(index: number) { + const removed = configuration.surveys[index]?.id; + configuration.surveys.splice(index, 1); + configuration.interventions = configuration.interventions.filter( + (item) => item.action.type !== 'survey' || item.action.survey_id !== removed + ); + }, + + addIntervention(intervention: InterventionConfig) { + configuration.interventions.push(intervention); + }, + + removeIntervention(index: number) { + configuration.interventions.splice(index, 1); }, generateSigning, diff --git a/web/src/routes/researcher/labels.ts b/web/src/routes/researcher/labels.ts index 2778d98..4122561 100644 --- a/web/src/routes/researcher/labels.ts +++ b/web/src/routes/researcher/labels.ts @@ -84,15 +84,12 @@ export function fieldLabel(m: Messages, path: string): string { return label.priority; case 'trajectory_sampling_hz': return label.trajectoryRate; - case 'delay_minutes': - return label.promptDelay; - case 'message': - return label.promptMessage; case 'collectors': return m.researcher.study.section.collectors.title; } - if (segments[0] === 'prompts' && tail === 'id') return label.promptId; + if (segments[0] === 'interventions') return m.researcher.study.section.interventions.title; + if (segments[0] === 'surveys') return m.intervention.survey; // The whole document, and anything a future rule invents: the path itself, never nothing. return path || m.researcher.sign.canonical; } diff --git a/web/src/routes/researcher/parse.ts b/web/src/routes/researcher/parse.ts index a59fdf9..45c0ab4 100644 --- a/web/src/routes/researcher/parse.ts +++ b/web/src/routes/researcher/parse.ts @@ -3,8 +3,8 @@ * * `lib/adc` writes these three and never reads them, because the app and the CLI are the readers. * The editor is the fourth, and the cross-language workflow depends on it — one signed - * configuration per language means opening the first one, changing the prose, and issuing a new - * `configuration_id` under the same signer. + * configuration variant means opening the first one, changing the signed content, and issuing a + * new `configuration_id` under the same signer. * * Nothing here judges a *value*. `schema.ts` refuses a document; this only decides whether the bytes * were a study configuration at all, and fills the shape so the editor always has something to draw. @@ -18,18 +18,18 @@ * quietly dropped or a priority quietly changed is worse than a study that will not open. */ -import { defaultCollector, emptyConfiguration } from '$lib/adc/schema'; import { - BOUNDS, - DEFAULT_MINIMUM_APP_VERSION, isCollectorId, isLocationPriority, isNetworkTransport, MAXIMUM_CONFIGURATION_BYTES, - NETWORK_TRANSPORTS, type CollectorConfig, + type InterventionConfig, + type InterventionSchedule, + type LocalizedText, type NetworkTransport, - type PromptConfig, + type SurveyDefinition, + type SurveyQuestion, type StudyConfiguration } from '$lib/adc/types'; @@ -78,9 +78,8 @@ export function isEnvelope(bytes: Uint8Array): boolean { } /** - * A structural read, field by field, defaulting anything absent. Unknown keys are dropped rather - * than carried: the encoder emits a fixed set in a fixed order, so a key it would not write is a - * key that cannot survive a round trip anyway. + * A closed-world structural read, field by field. Unknown, absent, or mistyped fields are refused + * exactly as the Android codec refuses them. The editor never invents replacement study content. * * `tink_hpke_public_keyset` is the exception and is kept exactly as parsed, property order * included — the canonicaliser re-emits it in the order it was built, so re-ordering it here would @@ -97,8 +96,8 @@ export function parseConfiguration(bytes: Uint8Array): StudyConfiguration { if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) throw new Error('parse_shape'); const raw = parsed as Record; if (!('experiment_id' in raw) && !('collectors' in raw)) throw new Error('parse_shape'); + requireExactKeys(raw, ROOT_KEYS); - const base = emptyConfiguration(); const researcher = object(raw.researcher); const consent = object(raw.consent); const storage = object(raw.storage); @@ -106,75 +105,94 @@ export function parseConfiguration(bytes: Uint8Array): StudyConfiguration { const exported = object(raw.export); const upload = object(raw.upload); + requireExactKeys(researcher, ['name', 'contact']); + requireExactKeys(consent, ['document_version', 'summary']); + requireExactKeys(storage, ['maximum_local_bytes']); + requireExactKeys(signer, ['key_id', 'public_key']); + requireExactKeys(exported, ['researcher_key_id', 'tink_hpke_public_keyset']); + if (Object.keys(upload).length > 0) requireExactKeys(upload, ['endpoint', 'interval_minutes', 'allow_metered']); + return { - schema_version: numeric(raw.schema_version, base.schema_version), - experiment_id: string(raw.experiment_id, ''), - configuration_id: string(raw.configuration_id, ''), - issued_at: string(raw.issued_at, base.issued_at), - expires_at: string(raw.expires_at, base.expires_at), + schema_version: numeric(raw.schema_version), + experiment_id: string(raw.experiment_id), + configuration_id: string(raw.configuration_id), + assigned_participant_id: nullableString(raw.assigned_participant_id), + issued_at: string(raw.issued_at), + expires_at: string(raw.expires_at), minimum_app_version: appVersion(raw.minimum_app_version), - title: string(raw.title, ''), + title: string(raw.title), researcher: { - name: string(researcher.name, ''), - contact: string(researcher.contact, '') + name: string(researcher.name), + contact: string(researcher.contact) }, - purpose: string(raw.purpose, ''), - duration_hours: numeric(raw.duration_hours, base.duration_hours), + purpose: string(raw.purpose), + duration_hours: numeric(raw.duration_hours), consent: { - document_version: string(consent.document_version, ''), - summary: string(consent.summary, '') + document_version: string(consent.document_version), + summary: string(consent.summary) }, - collectors: Array.isArray(raw.collectors) ? raw.collectors.map(collector) : [], - prompts: Array.isArray(raw.prompts) ? raw.prompts.map(prompt) : [], + collectors: array(raw.collectors).map(collector), + surveys: array(raw.surveys).map(survey), + interventions: array(raw.interventions).map(intervention), storage: { - maximum_local_bytes: numeric(storage.maximum_local_bytes, base.storage.maximum_local_bytes) + maximum_local_bytes: numeric(storage.maximum_local_bytes) }, signer: { - key_id: string(signer.key_id, ''), - public_key: string(signer.public_key, '') + key_id: string(signer.key_id), + public_key: string(signer.public_key) }, export: { - researcher_key_id: string(exported.researcher_key_id, ''), + researcher_key_id: string(exported.researcher_key_id), tink_hpke_public_keyset: isKeysetShaped(exported.tink_hpke_public_keyset) ? (exported.tink_hpke_public_keyset as StudyConfiguration['export']['tink_hpke_public_keyset']) - : base.export.tink_hpke_public_keyset + : fail('parse_keyset') }, // `{}` is how an absent upload block is written, so an empty object is "no", not "malformed". upload: - typeof upload.endpoint === 'string' + Object.keys(upload).length > 0 ? { - endpoint: upload.endpoint, - interval_minutes: numeric(upload.interval_minutes, 60), - allow_metered: upload.allow_metered === true + endpoint: string(upload.endpoint), + interval_minutes: numeric(upload.interval_minutes), + allow_metered: boolean(upload.allow_metered) } : null }; } +const ROOT_KEYS = [ + 'schema_version', 'experiment_id', 'configuration_id', 'assigned_participant_id', 'issued_at', + 'expires_at', 'minimum_app_version', 'title', 'researcher', 'purpose', 'duration_hours', 'consent', + 'collectors', 'surveys', 'interventions', 'storage', 'signer', 'export', 'upload' +] as const; + +function requireExactKeys(value: Record, expected: readonly string[]): void { + const actual = Object.keys(value); + if (actual.length !== expected.length || expected.some((key) => !(key in value))) { + throw new Error('parse_keys'); + } +} + const object = (value: unknown): Record => - value && typeof value === 'object' && !Array.isArray(value) ? (value as Record) : {}; + value && typeof value === 'object' && !Array.isArray(value) + ? (value as Record) + : fail('parse_object'); -const string = (value: unknown, fallback: string): string => - typeof value === 'string' ? value : fallback; +const string = (value: unknown): string => typeof value === 'string' ? value : fail('parse_string'); +const nullableString = (value: unknown): string | null => + value === null || typeof value === 'string' ? value : fail('parse_string'); +const boolean = (value: unknown): boolean => typeof value === 'boolean' ? value : fail('parse_boolean'); +const array = (value: unknown): unknown[] => Array.isArray(value) ? value : fail('parse_array'); -const numeric = (value: unknown, fallback: number): number => - typeof value === 'number' && Number.isFinite(value) ? value : fallback; +const numeric = (value: unknown): number => + typeof value === 'number' && Number.isFinite(value) ? value : fail('parse_number'); + +function fail(message: string): never { throw new Error(message); } /** - * The one number that is clamped rather than carried through, because the page has no control for - * it: an out-of-range floor read out of a file would raise an issue on a path with nowhere to land. - * - * A legal one survives, deliberately. A file `researcher-tools` wrote with `minimum_app_version: 7` - * round-trips here unchanged — silently widening a compatibility floor somebody set on purpose - * would be an edit to a signed document that nobody asked for. + * Kept as a named seam because this field has no editor control. Its value is never clamped: + * `validate` reports an illegal floor, and a legal value round-trips byte for byte. */ -const appVersion = (value: unknown): number => - typeof value === 'number' && - Number.isInteger(value) && - value >= BOUNDS.minimumAppVersion[0] && - value <= BOUNDS.minimumAppVersion[1] - ? value - : DEFAULT_MINIMUM_APP_VERSION; +const appVersion = (value: unknown): number => numeric(value); /** * Whatever it turns out to be, it is kept property-for-property: the canonicaliser re-emits this @@ -186,96 +204,82 @@ function isKeysetShaped(value: unknown): boolean { } /** - * One collector, with every field of the declared type. The defaults stand in for anything missing - * or of the wrong type; a value of the right type is carried through even when it is out of range, - * because a repaired number is a number the researcher never chose and `validate` is what says so. + * One collector with the codec's exact closed-world shape. Values of the right type are carried + * through even when out of range so `validate` can point at them; absent or mistyped structure is + * refused because inventing a signed value would change the study. */ function collector(raw: unknown): CollectorConfig { const source = object(raw); + requireExactKeys(source, ['id', 'required', 'config']); if (!isCollectorId(source.id)) throw new Error('parse_collector'); - const base = defaultCollector(source.id); const config = object(source.config); - const required = source.required === undefined ? base.required : source.required === true; + const required = boolean(source.required); - switch (base.id) { + switch (source.id) { case 'app_lifecycle.v1': - return { ...base, required }; + requireExactKeys(config, []); + return { id: source.id, required, config: {} }; case 'accelerometer.v1': + requireExactKeys(config, ['sampling_period_us', 'maximum_report_latency_us']); return { - ...base, + id: source.id, required, config: { - sampling_period_us: numeric(config.sampling_period_us, base.config.sampling_period_us), - maximum_report_latency_us: numeric( - config.maximum_report_latency_us, - base.config.maximum_report_latency_us - ) + sampling_period_us: numeric(config.sampling_period_us), + maximum_report_latency_us: numeric(config.maximum_report_latency_us) } }; case 'network_state.v1': + requireExactKeys(config, ['include_bandwidth_estimates']); return { - ...base, + id: source.id, required, - config: { include_bandwidth_estimates: config.include_bandwidth_estimates === true } + config: { include_bandwidth_estimates: boolean(config.include_bandwidth_estimates) } }; case 'network_usage.v1': + requireExactKeys(config, ['transports', 'poll_interval_minutes']); return { - ...base, + id: source.id, required, config: { transports: transports(config.transports), - poll_interval_minutes: numeric( - config.poll_interval_minutes, - base.config.poll_interval_minutes - ) + poll_interval_minutes: numeric(config.poll_interval_minutes) } }; case 'usage_events.v1': + requireExactKeys(config, ['poll_interval_minutes']); return { - ...base, + id: source.id, required, - config: { - poll_interval_minutes: numeric( - config.poll_interval_minutes, - base.config.poll_interval_minutes - ) - } + config: { poll_interval_minutes: numeric(config.poll_interval_minutes) } }; case 'location.v1': { - const priority = config.priority ?? base.config.priority; + requireExactKeys(config, [ + 'interval_millis', 'minimum_interval_millis', 'maximum_batch_delay_millis', + 'minimum_displacement_meters', 'priority' + ]); + const priority = config.priority; if (!isLocationPriority(priority)) throw new Error('parse_collector'); return { - ...base, + id: source.id, required, config: { - interval_millis: numeric(config.interval_millis, base.config.interval_millis), - minimum_interval_millis: numeric( - config.minimum_interval_millis, - base.config.minimum_interval_millis - ), - maximum_batch_delay_millis: numeric( - config.maximum_batch_delay_millis, - base.config.maximum_batch_delay_millis - ), - minimum_displacement_meters: numeric( - config.minimum_displacement_meters, - base.config.minimum_displacement_meters - ), + interval_millis: numeric(config.interval_millis), + minimum_interval_millis: numeric(config.minimum_interval_millis), + maximum_batch_delay_millis: numeric(config.maximum_batch_delay_millis), + minimum_displacement_meters: numeric(config.minimum_displacement_meters), priority } }; } - case 'keyboard_touch.v1': + case 'keyboard_touch.v1': { + requireExactKeys(config, ['trajectory_sampling_hz']); return { - ...base, + id: source.id, required, - config: { - trajectory_sampling_hz: numeric( - config.trajectory_sampling_hz, - base.config.trajectory_sampling_hz - ) - } + config: { trajectory_sampling_hz: numeric(config.trajectory_sampling_hz) } }; + } } } @@ -289,7 +293,6 @@ function collector(raw: unknown): CollectorConfig { * same problem. */ function transports(raw: unknown): NetworkTransport[] { - if (raw === undefined) return [...NETWORK_TRANSPORTS]; if (!Array.isArray(raw)) throw new Error('parse_collector'); const named: NetworkTransport[] = []; for (const entry of raw) { @@ -299,11 +302,125 @@ function transports(raw: unknown): NetworkTransport[] { return named; } -function prompt(raw: unknown): PromptConfig { +function localized(raw: unknown): LocalizedText { + const source = object(raw); + requireExactKeys(source, ['default', 'translations']); + const translations = object(source.translations); + return { + default: string(source.default), + translations: Object.fromEntries(Object.entries(translations).map(([key, value]) => [key, string(value)])) + }; +} + +function survey(raw: unknown): SurveyDefinition { const source = object(raw); + requireExactKeys(source, ['id', 'title', 'description', 'questions']); return { - id: string(source.id, ''), - delay_minutes: numeric(source.delay_minutes, 0), - message: string(source.message, '') + id: string(source.id), + title: localized(source.title), + description: localized(source.description), + questions: array(source.questions).map(question) + }; +} + +function question(raw: unknown): SurveyQuestion { + const source = object(raw); + const commonKeys = ['type', 'id', 'prompt', 'required']; + const common = { + id: string(source.id), + prompt: localized(source.prompt), + required: boolean(source.required) + }; + switch (source.type) { + case 'short_text': + requireExactKeys(source, [...commonKeys, 'maximum_length']); + return { type: 'short_text', ...common, maximum_length: numeric(source.maximum_length) }; + case 'scale': { + requireExactKeys(source, [...commonKeys, 'minimum', 'maximum', 'minimum_label', 'maximum_label']); + return { + type: 'scale', ...common, minimum: numeric(source.minimum), maximum: numeric(source.maximum), + minimum_label: localized(source.minimum_label), maximum_label: localized(source.maximum_label) + }; + } + case 'single_choice': + requireExactKeys(source, [...commonKeys, 'options']); + return { type: 'single_choice', ...common, options: choices(source.options) }; + case 'multiple_choice': { + requireExactKeys(source, [...commonKeys, 'options', 'minimum_selections', 'maximum_selections']); + return { + type: 'multiple_choice', ...common, options: choices(source.options), + minimum_selections: numeric(source.minimum_selections), + maximum_selections: numeric(source.maximum_selections) + }; + } + default: throw new Error('parse_question'); + } +} + +function choices(raw: unknown) { + return array(raw).map((value) => { + const choice = object(value); + requireExactKeys(choice, ['id', 'label']); + return { id: string(choice.id), label: localized(choice.label) }; + }); +} + +function intervention(raw: unknown): InterventionConfig { + const source = object(raw); + requireExactKeys(source, ['id', 'action', 'triggers']); + const action = object(source.action); + const actionType = action.type; + if (actionType !== 'notification' && actionType !== 'survey') throw new Error('parse_action'); + let parsedAction: InterventionConfig['action']; + if (actionType === 'survey') { + requireExactKeys(action, ['type', 'notification_title', 'notification_message', 'survey_id']); + parsedAction = { + type: 'survey', + notification_title: string(action.notification_title), + notification_message: string(action.notification_message), + survey_id: string(action.survey_id) + }; + } else { + requireExactKeys(action, ['type', 'notification_title', 'notification_message']); + parsedAction = { + type: 'notification', + notification_title: string(action.notification_title), + notification_message: string(action.notification_message) + }; + } + return { + id: string(source.id), + action: parsedAction, + triggers: array(source.triggers).map((rawTrigger) => { + const trigger = object(rawTrigger); + requireExactKeys(trigger, ['id', 'schedule', 'availability_minutes']); + const schedule = object(trigger.schedule); + const type = schedule.type; + if (type !== 'one_time' && type !== 'interval' && type !== 'daily_local') throw new Error('parse_schedule'); + let parsedSchedule: InterventionSchedule; + if (type === 'daily_local') { + requireExactKeys(schedule, ['type', 'local_time']); + parsedSchedule = { type, local_time: string(schedule.local_time) }; + } else { + const clock = schedule.clock; + if (clock !== 'CALENDAR_TIME' && clock !== 'ACTIVE_RUNNING_TIME') throw new Error('parse_schedule'); + if (type === 'one_time') { + requireExactKeys(schedule, ['type', 'offset_minutes', 'clock']); + parsedSchedule = { type, offset_minutes: numeric(schedule.offset_minutes), clock }; + } else { + requireExactKeys(schedule, ['type', 'start_offset_minutes', 'interval_minutes', 'clock']); + parsedSchedule = { + type, + start_offset_minutes: numeric(schedule.start_offset_minutes), + interval_minutes: numeric(schedule.interval_minutes), + clock + }; + } + } + return { + id: string(trigger.id), schedule: parsedSchedule, + availability_minutes: numeric(trigger.availability_minutes) + }; + }) }; } diff --git a/web/src/routes/researcher/presets.ts b/web/src/routes/researcher/presets.ts index e736363..797f0e1 100644 --- a/web/src/routes/researcher/presets.ts +++ b/web/src/routes/researcher/presets.ts @@ -44,7 +44,6 @@ export const PRESETS = { maximum_local_bytes: [256, 512, 1_024, 2_048, 4_096, 8_192], // 1, 3, 7, 14 and 28 days, in the unit the schema counts: how long one participant runs. duration_hours: [24, 72, 168, 336, 672], - delay_minutes: [60, 360, 1_440, 4_320, 10_080], upload_interval_minutes: [15, 60, 360, 1_440, 10_080] } as const satisfies Record; @@ -80,11 +79,6 @@ export const LADDERS = { 2_160, 4_320, 8_760 ], // 8 MiB → 8 GiB. Every rung is a size somebody would say out loud, 512 MiB included. - // 1 min → 1 year, the prompt's own delay from the participant's start. - delay_minutes: [ - 1, 5, 10, 15, 30, 60, 120, 180, 240, 360, 720, 1_440, 2_880, 4_320, 5_760, 7_200, 10_080, - 14_400, 20_160, 30_240, 40_320, 60_480, 129_600, 259_200, 525_600 - ], // 1 min → 7 days. upload_interval_minutes: [ 1, 5, 15, 30, 60, 120, 180, 360, 720, 1_440, 2_880, 4_320, 7_200, 10_080 diff --git a/web/src/routes/researcher/scales.ts b/web/src/routes/researcher/scales.ts index 3477ba6..64e6b94 100644 --- a/web/src/routes/researcher/scales.ts +++ b/web/src/routes/researcher/scales.ts @@ -74,7 +74,6 @@ export type ScaleKey = | 'trajectory_sampling_hz' | 'duration_hours' | 'maximum_local_bytes' - | 'delay_minutes' | 'upload_interval_minutes'; /** Shape B, where control space is stored space and the ladder is the reachable set. */ @@ -230,9 +229,6 @@ export function scales(m: Messages, u: Units): Record { format: exactBinary }, - // 1 min → 1 year. 12 hours used to land on 715 min, 2 days on 2 891. - delay_minutes: laddered(LADDERS.delay_minutes, PRESETS.delay_minutes, u.minutes), - // 1 min → 7 days. 12 hours used to land on 722 min. upload_interval_minutes: laddered( LADDERS.upload_interval_minutes, @@ -256,6 +252,5 @@ export const SCALE_BOUNDS: Record = { trajectory_sampling_hz: BOUNDS.trajectorySamplingHz, duration_hours: BOUNDS.durationHours, maximum_local_bytes: [MINIMUM_LOCAL_BYTES, MAXIMUM_LOCAL_BYTES], - delay_minutes: BOUNDS.promptDelayMinutes, upload_interval_minutes: [UPLOAD_MINIMUM_INTERVAL_MINUTES, UPLOAD_MAXIMUM_INTERVAL_MINUTES] }; diff --git a/web/src/routes/researcher/steps.ts b/web/src/routes/researcher/steps.ts index ea65734..c0e4aef 100644 --- a/web/src/routes/researcher/steps.ts +++ b/web/src/routes/researcher/steps.ts @@ -40,7 +40,9 @@ export const STEPS: readonly StepDefinition[] = [ 'duration_hours', 'consent', 'collectors', - 'prompts', + 'assigned_participant_id', + 'surveys', + 'interventions', 'storage', 'upload' ] @@ -67,7 +69,7 @@ const EXACT = new Map([ /** * Longest-prefix match on segment boundaries. `validate` emits `collectors.2.config.interval_millis` - * and `prompts.0.id`, so the first segment decides. A path nothing claims — including the empty + * and `interventions.0.id`, so the first segment decides. A path nothing claims — including the empty * path `validate` uses for the whole document — lands on `sign`, where the issue list is: an issue * with no home must still be visible somewhere. */ diff --git a/web/tests/canonical.spec.ts b/web/tests/canonical.spec.ts index fd70b14..f96cd02 100644 --- a/web/tests/canonical.spec.ts +++ b/web/tests/canonical.spec.ts @@ -48,6 +48,7 @@ const demoStudy: StudyConfiguration = { schema_version: 1, experiment_id: 'modular-sensing-demo', configuration_id: 'demo-config-2026', + assigned_participant_id: null, issued_at: '2026-01-01T00:00:00Z', expires_at: '2035-01-01T00:00:00Z', minimum_app_version: 1, @@ -87,11 +88,38 @@ const demoStudy: StudyConfiguration = { }, { id: 'keyboard_touch.v1', required: false, config: { trajectory_sampling_hz: 60 } } ], - prompts: [ + surveys: [ + { + id: 'demo-survey', + title: { default: 'Study check-in', translations: { 'zh-TW': '研究確認' } }, + description: { default: 'Tell us how the study is going.', translations: {} }, + questions: [ + { + type: 'short_text', + id: 'status-note', + prompt: { default: 'How is it going?', translations: {} }, + required: false, + maximum_length: 500 + } + ] + } + ], + interventions: [ { id: 'demo-check-in', - delay_minutes: 60, - message: 'Please check that the study is still running as expected.' + action: { + type: 'survey', + notification_title: 'Study check-in', + notification_message: 'Please complete the study check-in.', + survey_id: 'demo-survey' + }, + triggers: [ + { + id: 'after-one-hour', + schedule: { type: 'one_time', offset_minutes: 60, clock: 'ACTIVE_RUNNING_TIME' }, + availability_minutes: 1440 + } + ] } ], storage: { maximum_local_bytes: 16777216 }, @@ -214,6 +242,7 @@ describe('canonicalize', () => { '{"schema_version":1' + ',"experiment_id":"modular-sensing-demo"' + ',"configuration_id":"demo-config-2026"' + + ',"assigned_participant_id":null' + ',"issued_at":"2026-01-01T00:00:00Z"' + ',"expires_at":"2035-01-01T00:00:00Z"' + ',"minimum_app_version":1' + @@ -234,7 +263,8 @@ describe('canonicalize', () => { ',{"id":"location.v1","required":false,"config":{"interval_millis":10000,"minimum_interval_millis":5000,"maximum_batch_delay_millis":30000,"minimum_displacement_meters":5.0,"priority":"BALANCED"}}' + ',{"id":"keyboard_touch.v1","required":false,"config":{"trajectory_sampling_hz":60}}' + ']' + - ',"prompts":[{"id":"demo-check-in","delay_minutes":60,"message":"Please check that the study is still running as expected."}]' + + ',"surveys":[{"id":"demo-survey","title":{"default":"Study check-in","translations":{"zh-TW":"研究確認"}},"description":{"default":"Tell us how the study is going.","translations":{}},"questions":[{"type":"short_text","id":"status-note","prompt":{"default":"How is it going?","translations":{}},"required":false,"maximum_length":500}]}]' + + ',"interventions":[{"id":"demo-check-in","action":{"type":"survey","notification_title":"Study check-in","notification_message":"Please complete the study check-in.","survey_id":"demo-survey"},"triggers":[{"id":"after-one-hour","schedule":{"type":"one_time","offset_minutes":60,"clock":"ACTIVE_RUNNING_TIME"},"availability_minutes":1440}]}]' + ',"storage":{"maximum_local_bytes":16777216}' + ',"signer":{"key_id":"demo-signer-2026","public_key":"' + DEMO_PUBLIC_KEY + @@ -253,7 +283,9 @@ describe('canonicalize', () => { title: '心理韌性研究 "2026"', purpose: 'Line one\nline two\ttabbed', researcher: { name: 'Lin\\Chen', contact: 'lab@example.invalid' }, - prompts: [], + assigned_participant_id: 'Cohort_A-017', + surveys: [], + interventions: [], collectors: [{ id: 'app_lifecycle.v1', required: true, config: {} }], upload: { endpoint: 'https://intake.example.invalid/v1/bundles?study=1', @@ -265,7 +297,9 @@ describe('canonicalize', () => { expect(canonical).toContain('"title":"心理韌性研究 \\"2026\\""'); expect(canonical).toContain('"purpose":"Line one\\nline two\\ttabbed"'); expect(canonical).toContain('"name":"Lin\\\\Chen"'); - expect(canonical).toContain('"prompts":[]'); + expect(canonical).toContain('"assigned_participant_id":"Cohort_A-017"'); + expect(canonical).toContain('"surveys":[]'); + expect(canonical).toContain('"interventions":[]'); expect(canonical).toContain( '"upload":{"endpoint":"https://intake.example.invalid/v1/bundles?study=1","interval_minutes":720,"allow_metered":false}' ); @@ -328,7 +362,8 @@ describe('canonicalBytes', () => { const configuration = { ...demoStudy, title: '研究' }; const bytes = canonicalBytes(configuration); expect(bytes).toBeInstanceOf(Uint8Array); - expect(bytes.length).toBe(canonicalize(configuration).length + 4); + expect(bytes).toEqual(new TextEncoder().encode(canonicalize(configuration))); + expect(bytes.length).toBeGreaterThan(canonicalize(configuration).length); expect(new TextDecoder().decode(bytes)).toBe(canonicalize(configuration)); }); }); diff --git a/web/tests/compat.spec.ts b/web/tests/compat.spec.ts index bde1452..40d2840 100644 --- a/web/tests/compat.spec.ts +++ b/web/tests/compat.spec.ts @@ -16,7 +16,7 @@ * @vitest-environment node */ -import { execFileSync } from 'node:child_process'; +import { execFile, execFileSync } from 'node:child_process'; import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { dirname, join } from 'node:path'; @@ -26,8 +26,10 @@ import { canonicalBytes, canonicalize } from '../src/lib/adc/canonical'; import { encodeEnvelope } from '../src/lib/adc/envelope'; import { fingerprint, generateSigningKeyPair, sign, verify } from '../src/lib/adc/crypto'; import { generateHpkeKeyset } from '../src/lib/adc/tink'; +import { decodeEnvelope } from '../src/routes/researcher/parse'; import type { CollectorConfig, + InterventionConfig, NetworkTransport, StudyConfiguration, TinkKeyset @@ -39,13 +41,18 @@ const CLI = join(REPOSITORY, 'researcher-tools/build/install/researcher-tools/bi let workspace = ''; let sequence = 0; -beforeAll(() => { - if (!existsSync(CLI)) { - execFileSync(join(REPOSITORY, 'gradlew'), [':researcher-tools:installDist'], { - cwd: REPOSITORY, - stdio: 'inherit' - }); - } +beforeAll(async () => { + await new Promise((resolve, reject) => { + execFile( + join(REPOSITORY, 'gradlew'), + [':researcher-tools:installDist'], + { cwd: REPOSITORY }, + (error, stdout, stderr) => { + if (!error) resolve(); + else reject(new Error(`researcher-tools build failed\n${stderr || stdout || error.message}`)); + } + ); + }); workspace = mkdtempSync(join(tmpdir(), 'adc-compat-')); }, 600_000); @@ -127,6 +134,7 @@ const BASE: StudyConfiguration = { schema_version: 1, experiment_id: 'compat-harness', configuration_id: 'compat-case-001', + assigned_participant_id: null, issued_at: '2026-01-01T00:00:00Z', expires_at: '2035-01-01T00:00:00Z', minimum_app_version: 1, @@ -136,7 +144,8 @@ const BASE: StudyConfiguration = { duration_hours: 24, consent: { document_version: 'harness-1', summary: 'A fixture. Nothing is collected.' }, collectors: [{ id: 'app_lifecycle.v1', required: true, config: {} }], - prompts: [], + surveys: [], + interventions: [], storage: { maximum_local_bytes: 16 * 1024 * 1024 }, signer: { key_id: 'compat-signer', public_key: DEMO_PUBLIC_KEY }, export: { researcher_key_id: 'compat-hpke', tink_hpke_public_keyset: DEMO_KEYSET }, @@ -147,6 +156,23 @@ function study(overrides: Partial): StudyConfiguration { return { ...BASE, ...overrides }; } +function notification( + id: string, + offsetMinutes: number, + message: string, + availabilityMinutes = 1_440 +): InterventionConfig { + return { + id, + action: { type: 'notification', notification_title: 'Study notice', notification_message: message }, + triggers: [{ + id: `${id.slice(0, 56)}-trigger`, + schedule: { type: 'one_time', offset_minutes: offsetMinutes, clock: 'CALENDAR_TIME' }, + availability_minutes: availabilityMinutes + }] + }; +} + function everyCollector(required: boolean): CollectorConfig[] { return [ { id: 'app_lifecycle.v1', required, config: {} }, @@ -236,6 +262,7 @@ const DEMO: StudyConfiguration = { schema_version: 1, experiment_id: 'modular-sensing-demo', configuration_id: 'demo-config-2026', + assigned_participant_id: null, issued_at: '2026-01-01T00:00:00Z', expires_at: '2035-01-01T00:00:00Z', minimum_app_version: 1, @@ -279,13 +306,8 @@ const DEMO: StudyConfiguration = { }, { id: 'keyboard_touch.v1', required: false, config: { trajectory_sampling_hz: 60 } } ], - prompts: [ - { - id: 'demo-check-in', - delay_minutes: 60, - message: 'Please check that the study is still running as expected.' - } - ], + surveys: [], + interventions: [notification('demo-check-in', 60, 'Please check that the study is still running as expected.')], storage: { maximum_local_bytes: 16_777_216 }, signer: { key_id: 'demo-signer-2026', public_key: DEMO_PUBLIC_KEY }, export: { researcher_key_id: 'demo-hpke-2026', tink_hpke_public_keyset: DEMO_KEYSET }, @@ -304,12 +326,12 @@ const DISPLACEMENTS = [ const CASES: Array<{ name: string; configuration: StudyConfiguration }> = [ { name: 'the demonstration study the CLI ships', configuration: DEMO }, { - name: 'every collector, all required, with upload and prompts', + name: 'every collector, all required, with upload and interventions', configuration: study({ collectors: everyCollector(true), - prompts: [ - { id: 'check-in-one', delay_minutes: 60, message: 'Still running?' }, - { id: 'check-in-two', delay_minutes: 1_440, message: '研究の確認 😀' } + interventions: [ + notification('check-in-one', 60, 'Still running?'), + notification('check-in-two', 1_200, '研究の確認 😀') ], upload: { endpoint: 'https://uploads.example.invalid/v1', @@ -319,8 +341,8 @@ const CASES: Array<{ name: string; configuration: StudyConfiguration }> = [ }) }, { - name: 'every collector, none required, no upload, no prompts', - configuration: study({ collectors: everyCollector(false), prompts: [], upload: null }) + name: 'every collector, none required, no upload, no interventions', + configuration: study({ collectors: everyCollector(false), interventions: [], upload: null }) }, ...everyCollector(true).map((collector) => ({ @@ -344,7 +366,7 @@ const CASES: Array<{ name: string; configuration: StudyConfiguration }> = [ researcher: { name: '林\t"Lin"\\研究員 😀', contact: 'mail@example.invalid\n<研究>' }, purpose: NASTY, consent: { document_version: 'v1\\"研究"', summary: NASTY }, - prompts: [{ id: 'nasty-prompt', delay_minutes: 5, message: NASTY.slice(0, 200) }], + interventions: [notification('nasty-notice', 5, NASTY.slice(0, 500))], upload: { endpoint: 'https://uploads.example.invalid/v1?a=b&c=d#研究', interval_minutes: 15, @@ -406,7 +428,7 @@ const CASES: Array<{ name: string; configuration: StudyConfiguration }> = [ }, { id: 'keyboard_touch.v1', required: false, config: { trajectory_sampling_hz: 1 } } ], - prompts: [{ id: 'a-b', delay_minutes: 1, message: 'M' }], + interventions: [notification('a-b', 0, 'M', 1)], upload: { endpoint: 'https://a', interval_minutes: 1, allow_metered: false } }) }, @@ -448,7 +470,7 @@ const CASES: Array<{ name: string; configuration: StudyConfiguration }> = [ }, { id: 'keyboard_touch.v1', required: true, config: { trajectory_sampling_hz: 120 } } ], - prompts: [{ id: `p${'-q8'.repeat(21)}`, delay_minutes: 525_600, message: 'M'.repeat(500) }], + interventions: [notification(`p${'-q8'.repeat(21)}`, 525_599, 'M'.repeat(500), 525_600)], upload: { endpoint: `https://e.invalid/${'a'.repeat(2_030)}`, interval_minutes: 10_080, @@ -462,14 +484,22 @@ const CASES: Array<{ name: string; configuration: StudyConfiguration }> = [ { name: 'transports: written in the other order', configuration: usage(['wifi', 'mobile']) }, { name: 'transports: repeated', configuration: usage(['wifi', 'wifi']) }, - { name: 'prompts: absent', configuration: study({ prompts: [] }) }, + { name: 'interventions: absent', configuration: study({ interventions: [] }) }, { - name: 'prompts: three of them, at both delay bounds', + name: 'interventions: one-time, interval, and local daily schedules', configuration: study({ - prompts: [ - { id: 'first-prompt', delay_minutes: 1, message: 'A' }, - { id: 'second-prompt', delay_minutes: 262_800, message: '請確認研究仍在執行 😀' }, - { id: 'third-prompt', delay_minutes: 525_600, message: 'Z'.repeat(500) } + interventions: [ + notification('first-notice', 0, 'A'), + { + id: 'interval-notice', + action: { type: 'notification', notification_title: 'Check in', notification_message: 'B' }, + triggers: [{ id: 'every-six-hours', schedule: { type: 'interval', start_offset_minutes: 60, interval_minutes: 360, clock: 'ACTIVE_RUNNING_TIME' }, availability_minutes: 120 }] + }, + { + id: 'daily-notice', + action: { type: 'notification', notification_title: 'Check in', notification_message: '請確認研究仍在執行 😀' }, + triggers: [{ id: 'local-evening', schedule: { type: 'daily_local', local_time: '20:30' }, availability_minutes: 720 }] + } ] }) }, @@ -587,7 +617,7 @@ describe('a browser-made .adccfg is one the app accepts', () => { title: '瀏覽器簽署的研究 😀 "end to end"', consent: { document_version: 'e2e-1', summary: NASTY }, collectors: everyCollector(true), - prompts: [{ id: 'e2e-prompt', delay_minutes: 30, message: '請確認 😀' }], + interventions: [notification('e2e-notice', 30, '請確認 😀')], signer: { key_id: keyId, public_key: signing.publicX509Base64 }, export: { researcher_key_id: 'browser-hpke', tink_hpke_public_keyset: hpke.publicKeyset }, upload: { @@ -655,6 +685,68 @@ describe('a browser-made .adccfg is one the app accepts', () => { 120_000 ); + it( + 'bulk-personalizes unique configurations without putting assigned codes in filenames', + () => { + const signing = generateSigningKeyPair(); + const directory = scratch(); + const config = study({ + configuration_id: 'personalization-template', + signer: { key_id: 'personalize-signer', public_key: signing.publicX509Base64 } + }); + const configPath = join(directory, 'template.json'); + const privatePath = join(directory, 'signer.key'); + const mappingPath = join(directory, 'mapping.tsv'); + const output = join(directory, 'artifacts'); + writeFileSync(configPath, canonicalBytes(config)); + writeFileSync(privatePath, signing.privatePkcs8Base64, 'utf8'); + writeFileSync(mappingPath, 'arm-a-config\tAssigned_A-017\narm-b-config\tAssigned_B-018\n', 'utf8'); + + const canonicalWithId = join(directory, 'personalized.json'); + runCli( + 'canonicalize', '--input', configPath, '--output', canonicalWithId, + '--assigned-participant-id', 'Assigned_C-019' + ); + expect(JSON.parse(readFileSync(canonicalWithId, 'utf8')).assigned_participant_id).toBe('Assigned_C-019'); + + const signedWithId = join(directory, 'personalized.adccfg'); + runCli( + 'sign', '--config', configPath, '--private', privatePath, + '--key-id', 'personalize-signer', '--output', signedWithId, + '--assigned-participant-id', 'Assigned_D-020' + ); + const signedConfiguration = JSON.parse( + new TextDecoder().decode(decodeEnvelope(readFileSync(signedWithId)).configurationBytes) + ); + expect(signedConfiguration.assigned_participant_id).toBe('Assigned_D-020'); + + expect(runCli( + 'personalize', '--config', configPath, '--mapping', mappingPath, + '--private', privatePath, '--key-id', 'personalize-signer', '--output-dir', output + )).toContain('personalized 2 configurations'); + const first = JSON.parse(readFileSync(join(output, 'arm-a-config.json'), 'utf8')); + const second = JSON.parse(readFileSync(join(output, 'arm-b-config.json'), 'utf8')); + expect(first.assigned_participant_id).toBe('Assigned_A-017'); + expect(second.assigned_participant_id).toBe('Assigned_B-018'); + expect(first.configuration_id).not.toBe(second.configuration_id); + expect(existsSync(join(output, 'Assigned_A-017.adccfg'))).toBe(false); + expect(runCli( + 'check-config', '--envelope', join(output, 'arm-a-config.adccfg'), + '--now', '2026-06-01T00:00:00Z' + )).toContain('valid compat-harness arm-a-config'); + + const duplicateMapping = join(directory, 'duplicate.tsv'); + writeFileSync(duplicateMapping, 'same-config\tAssigned_A\nsame-config\tAssigned_B\n', 'utf8'); + expect(() => runCli( + 'personalize', '--config', configPath, '--mapping', duplicateMapping, + '--private', privatePath, '--key-id', 'personalize-signer', + '--output-dir', join(directory, 'duplicate-output') + )).toThrow(/Duplicate configuration ID/); + expect(existsSync(join(directory, 'duplicate-output'))).toBe(false); + }, + 120_000 + ); + /** * `JSON.stringify(-0)` is `0`, so a negative zero cannot reach the CLI through an ordinary JSON * file and cannot be a case in the table above. It can still reach a number input, and both ends diff --git a/web/tests/hostile.spec.ts b/web/tests/hostile.spec.ts index 97d54dd..4a75cc4 100644 --- a/web/tests/hostile.spec.ts +++ b/web/tests/hostile.spec.ts @@ -1,6 +1,6 @@ import { describe, expect, it } from 'vitest'; import demoStudyJson from '../../researcher-tools/examples/demo-study.json?raw'; -import { canonicalize, parseInstant } from '$lib/adc/canonical'; +import { parseInstant } from '$lib/adc/canonical'; import { validate } from '$lib/adc/schema'; import { isUsableHpkePublicKeyset } from '$lib/adc/tink'; import { parseConfiguration } from '../src/routes/researcher/parse'; @@ -58,13 +58,10 @@ describe('a collector the codec cannot write', () => { }); }); - /** `"collectors":[{"id":"accelerometer.v1","required":true}]` used to throw out of `validate`. */ - it('fills a missing config instead of throwing out of validate', () => { - const configuration = load((document) => { + it('refuses a missing collector config instead of inventing defaults', () => { + refuses((document) => { document.collectors = [{ id: 'accelerometer.v1', required: true }]; }); - expect(codes(configuration)).toEqual([]); - expect(canonicalize(configuration)).toContain('"sampling_period_us":'); }); it('survives a collector that is a bare string', () => { @@ -125,12 +122,46 @@ describe('location priority', () => { }); }); -describe('prompts', () => { - it('fills a prompt with no id instead of throwing', () => { +describe('legacy prompts', () => { + it('rejects the removed prompt shape instead of silently migrating it', () => { + refuses((document) => { + delete document.interventions; + document.prompts = [{ id: 'old-prompt', delay_minutes: 5, message: 'hi' }]; + }); + }); +}); + +describe('closed-world v1 shape', () => { + it('refuses unknown nested fields instead of dropping them before signing', () => { + refuses((document) => { + (document.researcher as Record).legacy_name = 'discard me'; + }); + }); + + it('refuses a mistyped scalar instead of replacing it with a default', () => { + refuses((document) => { + document.duration_hours = '24'; + }); + }); + + it.each(['', 'contains space', 'é', 'a'.repeat(65)])('reports an invalid assigned ID %j', (id) => { + const configuration = load((document) => { document.assigned_participant_id = id; }); + expect(codes(configuration)).toContain('id_format'); + }); + + it('reports a schedule whose durable lifetime occurrence set exceeds the metadata bound', () => { const configuration = load((document) => { - document.prompts = [{ delay_minutes: 5, message: 'hi' }]; + document.interventions = [{ + id: 'too-frequent', + action: { type: 'notification', notification_title: 'Check-in', notification_message: 'Check in now.' }, + triggers: [{ + id: 'every-minute', + schedule: { type: 'interval', start_offset_minutes: 0, interval_minutes: 1, clock: 'CALENDAR_TIME' }, + availability_minutes: 5 + }] + }]; }); - expect(codes(configuration)).toContain('required'); + expect(codes(configuration)).toContain('schedule_bounds'); }); }); diff --git a/web/tests/ids.spec.ts b/web/tests/ids.spec.ts index 425261c..aa114df 100644 Binary files a/web/tests/ids.spec.ts and b/web/tests/ids.spec.ts differ diff --git a/web/tests/node.d.ts b/web/tests/node.d.ts index 17c2b1c..670a24d 100644 --- a/web/tests/node.d.ts +++ b/web/tests/node.d.ts @@ -4,11 +4,17 @@ * That test shells out to the `researcher-tools` CLI, which is the only way to compare this * encoder against the one the Android app runs. The project has no `@types/node` — nothing it * ships touches Node — and adding one for a single test would put a dependency in everyone's - * install for a file that never reaches the browser. These are the five modules and eleven + * install for a file that never reaches the browser. These are the five modules and twelve * functions that test calls, and nothing else. */ declare module 'node:child_process' { + export function execFile( + command: string, + args: readonly string[], + options: { cwd?: string }, + callback: (error: Error | null, stdout: string, stderr: string) => void + ): void; export function execFileSync( file: string, args: readonly string[], diff --git a/web/tests/researcher.spec.ts b/web/tests/researcher.spec.ts index 8b68088..a5e0642 100644 --- a/web/tests/researcher.spec.ts +++ b/web/tests/researcher.spec.ts @@ -131,7 +131,7 @@ describe('paths', () => { it('routes every issue a document can raise to a named step', () => { for (const path of [ 'experiment_id', 'consent.summary', 'collectors.2.config.interval_millis', - 'prompts.0.id', 'storage.maximum_local_bytes', 'upload.endpoint', + 'interventions.0.id', 'surveys.0.questions.0.id', 'storage.maximum_local_bytes', 'upload.endpoint', 'signer.key_id', 'export.tink_hpke_public_keyset', '' ]) { expect(['keys', 'study', 'sign', 'files']).toContain(stepForPath(path)); @@ -154,7 +154,11 @@ describe('issue messages', () => { { id: 'network_usage.v1', required: false, config: { transports: [], poll_interval_minutes: 0 } }, { id: 'location.v1', required: false, config: { interval_millis: 1000, minimum_interval_millis: 3000, maximum_batch_delay_millis: 0, minimum_displacement_meters: -1, priority: 'BALANCED' } } ]; - configuration.prompts = [{ id: 'a', delay_minutes: 0, message: '' }, { id: 'a', delay_minutes: 1, message: 'x' }]; + configuration.interventions = [{ + id: 'a', + action: { type: 'notification', notification_title: '', notification_message: '' }, + triggers: [{ id: 'b', schedule: { type: 'one_time', offset_minutes: -1, clock: 'CALENDAR_TIME' }, availability_minutes: 0 }] + }]; configuration.upload = { endpoint: 'http://x', interval_minutes: 0, allow_metered: false }; configuration.expires_at = configuration.issued_at; for (const m of [en, zhTW]) { diff --git a/web/tests/scales.spec.ts b/web/tests/scales.spec.ts index 2cca85d..cbdd000 100644 --- a/web/tests/scales.spec.ts +++ b/web/tests/scales.spec.ts @@ -133,7 +133,6 @@ for (const [name, catalogue, locale] of CATALOGUES) { maximum_batch_delay_millis: [600_000], duration_hours: [504, 2_160, 4_320], maximum_local_bytes: [512 * 1_024 * 1_024], - delay_minutes: [720, 2_880], upload_interval_minutes: [720] }; // In stored units, because that is what the file carries and what was measured.