Implement Constellation + Asterism services for RCS on Google Messages#3388
Implement Constellation + Asterism services for RCS on Google Messages#3388Lyapsus wants to merge 87 commits into
Conversation
- Add AIDL interfaces for Constellation (IConstellationApiService, IConstellationCallbacks) - Add SafeParcelable classes for Constellation requests/responses - Implement ConstellationService and ConstellationServiceImpl - Add Ts43Client scaffolding for EAP-AKA authentication - Register service in AndroidManifest - Enable linting in build.gradle
- Refactor Ts43Client to decouple from Android dependencies for testing - Add Ts43ClientTest to verify EAP-AKA challenge parsing - Fix off-by-one error in EAP packet parsing - Update ConstellationServiceImpl to use Ts43Client - Add required permissions to AndroidManifest - Enable unit tests in build.gradle
Major changes: - Implemented GoogleConstellationClient with real Constellation API calls - Fixed DroidGuard classloader caching to prevent Shared library already opened errors - Added support for GetConsent and Sync requests with proper proto structure - Implemented ClientCredentials with ECDSA signature generation - Added TelephonyInfoContainer with Gaia IDs for field 20 - Fixed include_asterism_consents field position (moved to field 8) - Added required params (policy_id, calling_api, calling_package) to GetConsent - Fixed SIMAssociation structure with proper field mapping - Added params field to Verification message (field 6) - Implemented proper DeviceId with android_id fields - Added support for real DroidGuard tokens (~43k chars) instead of fallback - Created AsterismService for consent management - Fixed SSL provider loading in conscrypt Known issues: - Still getting INVALID_ARGUMENT - likely due to OAuth token project mismatch - OAuth tokens from project 745476177629 but Constellation expects 496232013492 Refs: microg#2994
…nment
Constellation parity with stock GMS (bevw.java):
- mapExceptionToStatusCode: gRPC→Status(500x) mapping with stock parity
- decideVerificationOutcome: enforce VERIFIED↔non-empty-token invariant
- extractVerificationMethodFromJwt + mapVerificationMethodString: JWT-based method
- Error callback sends null response on non-success (P0.1 fix, bevw.java:53,83)
- Remove stub-token scaffold (generateStubToken, EntitlementResult.stub)
Multi-SIM correctness:
- findMatchingVerifiedNumber: phone-match before firstOrNull at 4 GPNV sites
- Pending verification phone matching in Proceed path
- State priority: hasNone guarded by !hasPending (no OTP short-circuit)
PII sanitization:
- OTP SMS: log sender+length only, mask code to G-***XX
- Verification response: IMSI/MSISDN redacted to ***last4
- OTP regex tightened to G-(\d{6}) only (no bare 6-digit match)
Also: FID computation (cdqp.b parity), 23 unit tests, stale TODO cleanup
… DG now passes Stock .unstable loads ZERO native .so from GmsCore/lib/. loadStockNativeLibs() added 32 entries to dl_iterate_phdr that stock never has (306 system .so vs our 338). This was the microg#1 detection signal causing tachyon_registration PERMISSION_DENIED. - Remove loadStockNativeLibs() call (32 extra .so gone) - Remove libgcore_jni.so loading (no longer needed) - Remove link_map unlink (nothing to hide) - Fix CACHE_FOLDER_NAME "cache_dg" → "dg_cache" (stock uses app_dg_cache/) - Fix vmKey hex to uppercase (stock uses uppercase DG cache path) Verified: pm clear + fresh start → tachyon REGISTERED_WITH_PREKEYS → E2EE RCS message sent and received on microG for the first time.
Full working state with tachyon_registration passing. Includes: - S220 3-line DG fix (loadStockNativeLibs removed, cache dir, vmKey uppercase) - DgSpoofContext PM proxy (S213-S215) - DgIntrospect native helper declarations (S213) - DgClassLoaders (S212-S214) - StockGmsData permissions list (S215) - Asterism service (S205) - Constellation client improvements (S207-S220) - Chimera service proxy fixes - Permission renames in AndroidManifest - force_manual_msisdn test setting - All debugging/logging additions This is the safety checkpoint before stripping dead code for bounty PR.
…InstrumentedContext Zero-caller functions proven unnecessary by S220 tachyon breakthrough: - loadStockNativeLibs(): loaded 32 .so stock never loads (was THE detection signal) - mmapStockApk(): created non-stock artifact SUSFS had to hide - getEnrichedClassLoader() + StockFirstClassLoader: stock-first ordering gave zero benefit - spoofContextClassLoader(): never actually called (originalClassLoader always null) - DgIntrospect native declarations: loadNativeLibOnly, nativeDlopenLazy, nativeHideSelfFromLinkMap, nativeHookDlIteratePhdr, tryLoadNativeHooks, loadAndHideNativeLib, unhookNative, InstrumentedContext - Removed loadNativeLibOnly() call from NetworkHandleProxyFactory.createHandleProxy() All removed code required libgcore_jni.so (native helper) which won't exist on locked-bootloader microG ROMs. Active code paths unchanged. -448 lines. Build verified.
…h hybrid APK) Targets stock GMS classes not present in normal microG APK — all 4 Class.forName calls throw ClassNotFoundException. Only useful with augment-apk-with-stock-dex.sh hybrid APK (proven irrelevant for DG quality in S206). Tested: pm clear + full DG cache wipe → auto-provisioned → ConfiguredState → tachyon REGISTERED_WITH_PREKEYS → E2EE MLS PROVISIONED. Zero regressions.
Reads pif.prop from filesDir or /data/adb/ — neither exists on locked-BL microG ROMs. Falls back to real Build.* values which is the correct behavior (DG reads native __system_property_get which returns real device props; Java-level mismatch would be a detection signal, not a fix). Tested: pm clear + full DG/constellation wipe → ConfiguredState → tachyon REGISTERED_WITH_PREKEYS. Zero regressions.
Resolves 4 merge conflicts: - HandleProxyFactory.kt: keep dg_cache fix + named DgVmClassLoader - LoginActivity.java: keep real DroidGuard token generation - PhenotypeService.kt: keep both RCS flags and Photos flag - WorkAccountAuthenticator.kt: take upstream refactor
- Replace ad-hoc HMAC-SHA1 key derivation with FIPS 186-2 PRF (RFC 4187 Section 7). The old deriveKey() produced wrong K_aut, causing AT_MAC verification failure on TS.43 servers. - Fix SIM response parsing: 3GPP TS 31.102 returns sequential length-value pairs (DB [len] [RES] [len] [CK] [len] [IK]), not nested TLV with inner tags. Old parser failed to extract RES/CK/IK from real SIM responses. - Fix AT_RES length byte order: was writing [bits_low][0x00] instead of big-endian [0x00][bits_low] per RFC 4187 Section 10.3. - Accept server-provided eap_aka_realm for NAI derivation, falling back to default 3GPP TS 23.003 realm. - Remove dead LengthInfo/readLength code from old TLV parser.
- HandleProxy: log VM constructor calls and failures with exception
class name and message (was completely silent on init failure)
- HandleProxy: log run() result size and init() result via android.util.Log
(was only via DgIntrospect which requires opt-in)
- HandleProxyFactory: log class cache hit/miss/fresh-load with vmKey
- HandleProxyFactory: log cache directory details on validation failure
- NetworkHandleProxyFactory: log DG availability check failure with
guidance ("check microG Settings > SafetyNet > DroidGuard enabled")
- NetworkHandleProxyFactory: log DB cache hit/miss for ALL flows
(was only constellation_verify)
- NetworkHandleProxyFactory: add vmKey and path details to cache write
failure exception (was empty IllegalStateException)
Replace HTTP 401/WWW-Authenticate challenge-response with the standard
GSMA TS.43 v5.0+ JSON body relay protocol:
Phase 1 - EAP-AKA auth:
GET with EAP_ID param -> server returns {"eap-relay-packet": "base64"}
-> SIM auth -> POST {"eap-relay-packet": "response"} back
-> server returns {"Token": {"token": "..."}} (up to 3 rounds)
Phase 2 - ODSA request (handled by caller):
GET with token param -> server returns phone number or temp token
Key changes:
- Accept entitlement_url + eap_aka_realm from Ts43Challenge proto
(server-provided URL, not CarrierConfig guesswork)
- Session cookies preserved across EAP rounds (required by many servers)
- Token extraction handles both JSON (Token.token) and OMA DM XML formats
- Remove old handleEapAkaChallenge (HTTP 401 nonce parsing)
- Wire handleTs43Challenge to pass server URL and realm to Ts43Client
Adversarial review found 3 remaining bugs in the TS.43 flow: 1. FIPS 186-2 PRF carry initialization: spec says XKEY = (1 + XKEY + w_i) mod 2^b - the +1 was missing. carry=0 -> carry=1 (produces correct key material from iteration 2+) 2. Session cookies not applied to POST requests in EAP relay loop. HttpURLConnection sends headers on getOutputStream(), so cookies set at loop top were too late for the POST created at loop bottom. Fix: apply cookies immediately after creating POST connection. 3. Missing ODSA Phase 2: auth token is intermediate, not the final result. After EAP-AKA auth, must make GET with token param to get the actual phone number / temp token response body. Added performOdsaRequest() for Phase 2, falls back to auth token if ODSA fails. Route response to correct proto field based on client_challenge vs server_challenge.
Codex steelman found 2 more bugs: 1. EAP relay loop dropped the final POST response: old for-loop sent POST at bottom, exited without reading response. Restructured to while-loop that always reads response before deciding next step. Extracted applyCookies/collectCookies helpers. 2. MK derivation used String.length() (char count) instead of UTF-8 byte length for identity. For ASCII NAIs these are equal, but non-ASCII realms would produce wrong MK. Fixed to use getBytes(UTF-8) consistently. Also: thread eapAkaRealm all the way through processEapPacket into generateEapAkaResponse for correct key derivation with server-provided realms. Make processEapPacket package-private for testability. Updated Ts43ClientTest to match new API: - Test processEapPacket directly (verifies EAP response format, AT_RES byte order, AT_MAC presence) - Test SIM response format (3GPP TS 31.102 sequential LV) - Test FIPS 186-2 PRF determinism (same inputs = same outputs) - Test EntitlementResult type behavior - Removed old handleEapAkaChallenge test (method removed)
S220 "tachyon wall broken" was actually stock GMS running from /data/app/ (higher versionCode override). The code removal was never tested on real microG. Restoring: - loadStockNativeLibs, mmapStockApk, getEnrichedClassLoader - StockFirstClassLoader, spoofContextClassLoader - forceLoadSkippedDex, spoofBuildFields - InstrumentedContext, native dlopen/hook declarations - DgIntrospect capture infrastructure S222 DG diagnostic logging preserved in merge resolution.
Asterism (svc 199) AIDL interfaces and SafeParcelable types for consent management. Service implementation already existed but lacked the formal AIDL contract. Skip loading microG's bundled libconscrypt_gmscore_jni.so on Android 10+ where system Conscrypt (/apex/com.android.conscrypt/) provides equivalent TLS. The bundled native lib ABORTs in JNI_OnLoad on some devices (Samsung Android 12), killing the host process and breaking RCS provisioning mid-flow.
gmsVersion 25.19.31 -> 26.02.33 to match stock GMS. DG server
may provide different bytecodes for different clientVersions.
Replace {{cl}} placeholder with stock CL number 858744110 in
GMSCORE_VERSION and IID registration versionName. The placeholder
was a clear non-stock fingerprint in User-Agent headers.
ProviderInstallerImpl: wraps system Conscrypt as GmsCore_OpenSSL on
Android 10+ instead of returning null. Messages expects this provider
name and crashes ("Unable to find a default SSL provider") without it.
VersionUtil/CryptAuth: replace BuildConfig.VERSION_NAME (20.47.14) and
literal {{cl}} with stock GMS values (26.02.33/858744110). The DG DB
cache ID now matches stock format, ensuring correct bytecode selection.
…GGABLE VersionUtil: hardcode versionCode=260233029 (was 260233059 from buildType offset) and buildType=190400 (was 190408 for 480dpi). Both match stock GMS 26.02.33, ensuring correct DG bytecode selection from server. DgSpoofContext: intercept getInstallerPackageName to return "com.android.vending" for GMS package. Mask FLAG_DEBUGGABLE and ensure FLAG_SYSTEM + FLAG_UPDATED_SYSTEM_APP set on spoofed ApplicationInfo. Result: token still 19.7K, PERMISSION_DENIED unchanged. The tachyon wall is not caused by version mismatches, installer info, or debug flags.
NetworkHandleProxyFactory used BuildConfig.VERSION_CODE from the DG core module (v20.47.14) instead of VersionUtil.versionCode (stock 260233029). Server returned different bytecodes (81741B vs 80125B/81519B) but token still 19.7K, still PERMISSION_DENIED. DG process identity is the gate.
DG captures getClass().getName() on each classloader in the chain, which goes into encrypted telemetry (field 2). Our custom DgVmClassLoader showed "com.google.android.gms.droidguard.DgVmClassLoader" - a non-stock name that fingerprints microG. Stock GMS uses plain "dalvik.system.DexClassLoader". DgVmClassLoader was a DexClassLoader subclass with zero behavior (only DgIntrospect.log debug calls). Replacing it produces the stock-identical classloader chain: DexClassLoader -> PathClassLoader -> BootClassLoader. Tested: tachyon_registration passes (REGISTERED_WITH_PREKEYS) with E2EE on two SIMs (Spusu 232/17 + Georg 232/12). Classloader chain confirmed via HandleProxyFactory log. Token 19,645-19,694 bytes. Caveat: simultaneous androidId change (data wipe) makes it impossible to isolate whether this fix or the fresh identity broke through. Both could be contributing factors.
Three-way test with identical androidId (4375908543810217853): 1. DexClassLoader -> REGISTERED_WITH_PREKEYS 2. DgVmClassLoader -> PERMISSION_DENIED 3. DexClassLoader -> REGISTERED_WITH_PREKEYS The classloader class name in DG encrypted telemetry (field 2) was the tachyon detection signal. "com.google.android.gms.droidguard.DgVmClassLoader" fingerprinted microG. Stock uses "dalvik.system.DexClassLoader". This one-line change breaks the tachyon wall that blocked RCS messaging on microG since S207 (2026-03-30).
|
(FWIW, commit dates are worth nothing if the commits were not public, I can push commits that “show” I started working on this feature 10 years ago if I want…) |
|
Fair. Well, I think I still possess transcripts - tho also pretty forgeable if one is brave enough. Server-side conversations history maybe? |
Split sync retry/response analysis and the pending challenge proceed loop out of GoogleConstellationClient so the remaining client code mostly coordinates protocol stages. Preserve the verified Tachyon and E2EE behavior while making the control flow easier to review and evolve.
Route the downstream consent, sync, proceed, and fallback stages through a shared call context so verifyPhoneNumber reads more like stage orchestration and less like a long chain of unrelated locals. Keep the proven runtime behavior unchanged while reducing review overhead in the main client.
Move the remaining call-context assembly and client credential signing out of the verifyPhoneNumber body so the main client reads as staged orchestration instead of mixed setup mechanics. Keep the verified Tachyon and end-to-end RCS behavior unchanged while tightening the review surface.
Stop treating policy ids as phone numbers, fail explicitly when no Gaia token is available instead of sending a fake placeholder, and derive the Constellation version fields from the module build version instead of local literals. Keep the verified Tachyon and end-to-end RCS behavior intact while removing review-time mismatches.
Drop reverse-engineering and historical comment clutter from the Constellation/Asterism codepaths, keep only the comments that still explain non-obvious behavior, and clear the branch-owned warnings with narrow compatibility fixes.
|
Is this pull request ready for review? Given you continue to push AI modifications, I'm changing the status to Draft. Please undraft once you decided to stop the AI from doing all sorts of random stuff. |
|
@mar-v-in I think I should have told something some time ago. I'm really sorry, I think it won't be an exaggeration to tell I was and am sincerely afraid to. |
DroidGuardInitReply.createFromParcel returned null when pfd/object were null, causing DroidGuardApiClient.openHandle to fall back to init(flow) with null Bundle -- losing the DroidGuardResultsRequest context (clientVersion, appArchitecture). Fix: always return non-null reply. AppCertManager device_key request had multiple issues vs stock GMS: - Content-Type was application/octet-stream (stock: application/x-protobuf) - Token field fetched real GCM token (stock: literal "missing_token") - Session ID hex used toString(16) producing negative hex for negative longs -- mismatches uint64 proto wire encoding. Fix: Long.toHexString - Headers: app was package name (stock: random UUID), User-Agent was set (stock: not sent), gmsversion/gmscoreFlow were missing - Removed duplicate lowercase content-type header Stock class identified as ajoq.java (not afuv.java as previously assumed). Headers verified against bddx.d/c methods. gmscoreFlow=3 corresponds to AUTH_NETWORK_REQUEST_APP_CERT_REQ enum. Note: device_key endpoint (android.googleapis.com/auth/devicekey) still returns HTTP 400 -- it requires GMS proprietary HTTP stack auth (adyl/adyk engine) that Volley cannot replicate. These fixes are correct protocol-level changes but insufficient alone.
…caller logging DroidGuardHandleImpl: trigger file + 3s sleep for ptrace attachment (tachyon only). PhenotypeService: improved .pb marker creation with proper path handling. ConstellationSyncFlow: simplified no-DG retry logic. AsterismService/RcsService: caller UID logging for debug.
…wrong) HTTP 200, 145 bytes with keyCert. The endpoint does NOT require GMS proprietary HTTP auth — it accepts standard Volley POST. S247 failures were from re-testing already-used androidIds (device_key is one-shot per androidId). Added logging: DG result size, HTTP status+body, success confirmation. Fresh-install device_key blocker is SOLVED.
Stock GMS (bewt.smali lines 9518-9605) places raw DG snapshot output in DeviceSignals field 1 (droidguard_result) and cached server-processed ARfb tokens in field 2 (droidguard_token). Our code always used field 2 for both, causing the server to misinterpret raw DG as a cached token. Add isCachedArfb parameter to buildClientInfo/buildSyncRequest to route tokens to the correct proto field based on their origin.
Leftover seccomp trace daemon trigger code from DG research. Wrote to /data/data/.../dg_seccomp_active on every DG init and slept 3s for tachyon flows when trace script was present.
AsterismServiceImpl: move IID token registration off the binder thread to avoid ANR when cache is cold (RSA keygen + FCM HTTP can take 3-8s). Callbacks are already oneway so the caller does not block on the response. Ts43Client: use TelephonyManager as primary MCC+MNC source instead of hardcoding 2-digit MNC from IMSI prefix. Fixes 3-digit MNC carriers (T-Mobile US 311260, Verizon, DT). ProviderInstallerImpl: try-with-resources for ZipFile to prevent fd leak if copyInputStream throws (pre-Android 10).
Challenge: fields 3-9 renumbered to match GMS hyza.java MessageInfo decode. MO=3, CarrierID=4, expiry=5 (TimestampPair), MT=6, Registered=7, FlashCall=8. Removed nonexistent field 9 and group_id field 4. ChallengeResponse: RegisteredSMS 4->6, FlashCall 5->7. MOChallenge: sms 2->4, polling_intervals 4->5. CarrierIDChallenge: all fields renumbered (1,2,3 -> 3,5,6). CarrierIDChallengeResponse: 1,2 -> 3,4. Ts43Challenge: client_challenge 5->4, server_challenge 8->5. Ts43Integrator enum: removed spurious UNKNOWN=0, all values shifted to match GMS bfpv.java switch statement. next_sync_time: ServerTimestamp -> TimestampPair in SyncResponse, GetConsentResponse, ProceedResponse (matches GMS hzdu class). All field numbers independently verified via raw MessageInfo byte decode of hyza.java constructor string.
ConstellationServiceImpl: move verifyPhoneNumber off binder thread to prevent ANR on Jibe UPI carriers (5-15s gRPC calls). Uses ExecutorService worker thread, same pattern as AsterismServiceImpl. Add 4 MicroGRcs INFO lines for remote diagnostics: - spatula present/absent (device_key status) - consent state + ARfb cache status - sync path (with-DG vs no-DG-fallback) - sync result (VERIFIED/PENDING/NONE + reason)
Messages checks PEv3 before PEv2 on initial RCS enable. PEv3 returns code 3000 (no config) before PEv2 can write config, requiring users to toggle RCS off/on twice. Stock GMS sends com.google.android.ims.library.phenotype.UPDATE after Phenotype sync, which Messages receives via dnok.java and routes to scheduleProvisioningForEligibleSims -- starting PEv2 directly. Send this broadcast every 15s for up to 5 min after serving RCS Phenotype flags, matching the stock trigger path. Stops early if Messages is uninstalled.
Debug APKs have FLAG_DEBUGGABLE which is a DG detection signal. No tester should use them. Halves CI time from ~25 min to ~12 min.
External tester (locked-BL, CrDroid GSI, XL Axiata Indonesia) gets VERIFIED from constellation sync but GPNV fails with "could not verify iid_token". The cached/seeded IID token from the ROM's pre-existing microG is rejected by the server. Fix: when GPNV returns this specific error, invalidate the cached IID token + RSA key pair, force fresh GCM registration, and retry GPNV once with the new token.
Two bugs in the initial retry implementation: - invalidateIidToken didn't clear appid.xml, so getOrRegisterIidToken would re-discover the same stale token from the ROM's pre-existing microG data and re-seed it - Registration path didn't persist iid_source, making it impossible to distinguish freshly registered tokens from unknown cached ones
The GPNV retry logic (67160ce) only covered the post-VERIFIED path. The NONE-state fallback GPNV had the same "could not verify iid_token" failure but no retry -- tester's logs confirmed it fires in the NONE path after server consumed the VERIFIED state from the first test.
Previous commits (67160ce, 8a34e0d) only added GPNV IID retry to the post-VERIFIED path. External tester's second test hit the NONE-state path instead (server consumed VERIFIED from first test), where the same iid_token rejection had no retry. Changes: - Add IID invalidation + re-registration retry to all 4 GPNV call sites: post-VERIFIED, NONE-state, sync-error fallback, post-Proceed - Auto-clear DG/ARfb cache on NONE reason=0 (ambiguous -- could be consumed VERIFIED or stale ARfb). Not cleared for known reasons (throttled, denied) where fresh DG won't help. - VERIFIED + GPNV failure now returns 5002 (retryable) instead of falling through to Status(8) non-retryable
Clear appid.xml before constellation_iid.xml. The race Codex flagged (concurrent getOrRegisterIidToken re-seeding from appid between the two clears) can't fire in practice -- binder thread uses SENDER_CONSTELLATION, retry uses SENDER_READ_ONLY -- but the safer ordering costs nothing.
Store the checkin androidId when generating the EC key pair. On each invocation, compare with current androidId. If they differ (e.g. APK update triggered fresh checkin), clear the orphaned key pair -- it was verified under the old identity and can't be used with the new one. This handles the case where a build update causes a fresh checkin (new androidId) but the old EC key persists, producing NONE reason=0 because the server doesn't recognize the old key with the new identity.
EC keys generated by builds without identity tracking have no stored ec_key_android_id. If such a key produces NONE reason=0 (+ GPNV also failed), clear it -- the key is likely orphaned from a previous androidId and can't be used with the current identity. The next attempt generates a fresh key bound to the current androidId. Only fires once per upgrade: after the first NONE clears the untracked key, the replacement key has ec_key_android_id set, so the check never triggers again.
New MicroGRcs lines visible via `adb logcat | grep MicroGRcs`: - iid/riid source (seeded, cached, registered, instance-id, random) - EC key state (existing/fresh, tracked/untracked, acked) - sync DG type (cached-arfb, raw-dg, none) - GPNV failure message on all 4 call sites
Summary
Implements the two GMS services Google Messages requires for RCS:
phonedeviceverification-pa.googleapis.comAlso fixes several DroidGuard issues that prevented tachyon transport registration.
(hopefully) closes #2994
This is the initial working implementation -- I obviously plan to clean up the code, reduce the diff, and address review feedback until everyone is happy with it.
What works
End result: E2EE RCS messages sent and received on microG. Tested on multiple SIMs of Spusu (232/17) and Georg (232/12) Austrian Jibe UPI carriers (Google-handled phone verification -- most of Europe, many worldwide). Google Messages v302730063 (2026-03-10 build).
Testing limitations
Not tested:
Ts43Client.java(EAP-AKA + FIPS 186-2 PRF + ODSA Phase 2). Sadly no TS.43 SIM available at hand for end-to-end testing so far, therefore testing would be especially appreciated.Attribution and tooling
This work was developed independently since at least 18 January 2026 based on commits dates, with specific components adopted from parallel efforts:
MoSmsVerifierported from PRs Implementplay-services-constellation#3359-3361.gmsVersion >= 25.19.31requirement was their discovery.