General development branch merging many open PRs to simplify testing - #1081
Open
billdenney wants to merge 91 commits into
Open
General development branch merging many open PRs to simplify testing#1081billdenney wants to merge 91 commits into
billdenney wants to merge 91 commits into
Conversation
Agent-Logs-Url: https://github.com/nlmixr2/rxode2/sessions/88a8ceec-0923-4854-8b6e-df279393fe37 Co-authored-by: mattfidler <514778+mattfidler@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…ting, and pre-solve warning - rxControl() gains oomFile, oomChunkSize, oomParallel parameters with validation - rxSolve.default() routes to .rxSolveOom() when oomFile is set - Pre-solve warning fires when estimated memory exceeds 90% of free RAM - rxData.cpp OOM error messages now suggest oomFile option - .rxOomChunkSize() helper computes safe chunk size from free RAM estimate Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…-backed event tables - .rxSolveOom(): main OOM solve loop; serialize then solve each chunk - Correct seed arithmetic: baseSeed + cumSub (no core scaling) - rxSolveOom S3 class: print, as.data.frame, $, nrow methods - rxEventTableFile(): file-backed event table for parquet/csv/fst/rds - rxSolveChunked(): user-facing wrapper with chunkSize/seed/parallel args - .rxOomChunkSize(): auto chunk size from rxMemoryEstimate free RAM - Adds arrow, mirai, fst to Suggests in DESCRIPTION - test-oom.R: four rxTest-wrapped test cases Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…dim.rxSolveOom - dim.rxSolveOom registered so nrow() dispatches correctly via dim() - @param entries for three new rxControl parameters - Regenerated NAMESPACE and Rd files Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ted by injected cmt() observable (nlmixr2est#685)
…ow.rxSolveOom Fixes the CI failures this PR (#1052) introduces on the dev-2026-06 integration branch: - rxValidate.R: preserve/restore the rxode2 RNG seed around each rxTest({}) block. test-oom.R calls rxSetSeed() and leaked useRxSeed=TRUE into later test files, breaking set.seed()-based reproducibility (test-random, test-rxRmvn). - rxOom.R: arrow::concat_tables() takes tables as individual ... arguments, not a list -> 'Invalid R object for std::shared_ptr<arrow::Table>'. Splice with do.call(). Also drop the redundant exported nrow.rxSolveOom (nrow() is not generic; dim.rxSolveOom already makes nrow() work) which tripped an undocumented-code-object WARNING. - rxsolve.R: skip the always-on OOM memory pre-check when subject*study or output-row counts overflow INT_MAX, so the dedicated 'too large' C guards (issue #999 segfault protection) report the right diagnostic instead of the 'use file=' suggestion. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
#1061 added single=FALSE to rxSolve()'s formals but not its roxygen, which produced a codoc mismatch WARNING (argument in code not in docs). Add the @param single description and the matching rxSolve.Rd usage/arguments entry. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
# Conflicts: # man/rxSolve.Rd
…/id) Both features never passed their own PR CI and need core changes (persistent self-referential lhs accumulators for #1011; per-subject seeding/id consistency for #1014). Back them out of this integration branch so its CI is green; feature completion stays with the PRs. - #1011: revert _max/_min NA handling in rxode2_model_shared.c; rm test-cmax-dynamic.R - #1014: revert .addSimId in rxsolve.R, confint.R id/sim.id handling, NEWS entry; rm test-sim-id-consistency.R Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…when absent
- as_arrow_table.rxSolveOom: arrow::read_parquet() returns a tibble by
default, but arrow::concat_tables() requires Arrow Tables ('Invalid R
object for std::shared_ptr<arrow::Table>'). Read with as_data_frame=FALSE.
- test-oom.R: arrow::as_arrow_dataset is not a generic in all arrow versions;
skip the as_arrow_dataset test when it is absent (matching the guarded
.s3register in rxode-options.R).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ndex pkgdown build_reference_index() errors when documented/exported topics are missing from _pkgdown.yml. The OOM PR added rxEventTableFile and rxSolveChunked (with Rd) but did not index them. Add them under Event Tables & Dosing and Solving & Simulation respectively. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
arrow has no as_arrow_dataset generic (it provides open_dataset()), so the as_arrow_dataset.rxSolveOom S3 method had nothing to dispatch from: the arrow:: call errored and the method/.s3register were unreachable dead code. Define an rxode2-owned as_arrow_dataset generic (exported + documented), register the rxSolveOom method to it, drop the phantom arrow .s3register, index it in _pkgdown.yml, and have test-oom.R call as_arrow_dataset(chnk) directly so the lazy-Dataset path is actually exercised (no longer skipped). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
# Conflicts: # NAMESPACE # R/rxsolve.R # man/rxSolve.Rd
The parallel=N path (rxSolve(..., file=, parallel=N)) failed for any model,
surfacing only as 'replacement has length zero' (the masked error when a
daemon task returns an errorValue). Three real defects:
- The mirai_map worker was declared function(.i) but mirai_map passes .args
as named arguments, so every chunk errored with 'unused arguments'. Declare
the .args (.modelObj, .chunkEvList, .chunkIdsList, .chunkParamsList, .inits,
.fwdCtlArgs, .mainTmp) as formals.
- .modelObj was referenced in the worker but never passed in .args.
- Each daemon wrote its parquet/rds chunk to its own session tempdir, which is
removed when the daemon shuts down, so the manifest pointed at deleted files
('Failed to open local file'). Write to the parent process's tempdir.
Also surface real daemon errors instead of masking them as a replacement-length
error, and add a regression test asserting the parallel solve matches the
serial chunked solve exactly (same pre-drawn etas).
Fixes the pkgdown render failure of vignettes/articles/rxode2-mirai-parallel.Rmd.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
# Conflicts: # R/rxsolve.R # man/rxSolve.Rd
The custom-mirai example collected results with lapply(tasks, `[]`), but
`[]` is not a function ('object `[]` not found'), which broke the pkgdown
render. mirai tasks are collected with m[], i.e. `[`, so use lapply(tasks, `[`).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ening
The R 4.5.3 (oldrel-1) CI job segfaulted intermittently during the test
suite (`rxode2.useLinCmt=FALSE` pass). The crash surfaced at an innocent
allocation (`.rxTimeId`, with `paste0(... width = -2147483648)` warnings
of a clobbered heap) -- the crash site is unrelated to the corruption
site, the hallmark of a heap-buffer-overflow elsewhere in the
shared-process suite.
AddressSanitizer over the full suite (R 4.5.3) pinned the corruptor to
the main ODE solve path:
ERROR: AddressSanitizer: heap-buffer-overflow ... READ of size 4
#0 syncIdx inst/include/rxode2parseHandleEvid.h:232
#1 handleEvid1 inst/include/rxode2parseHandleEvid.h:675
#2 ind_liblsoda0 src/par_solve.cpp:3964 ...
0 bytes after a 16-byte region allocated by rxAllocInd (idose, ndoses+1 ints)
`handle_evid()` does an unconditional `ind->ixds++` after handling a
dose, and `handleEvid1()` ignores `syncIdx()`'s return value, so a failed
re-sync can leave `ixds == ndoses` and the next dose pushes it past the
end. `syncIdx()` then dereferenced `ind->idose[ind->ixds]` *before*
bounds-checking `ixds`, reading one past the `ndoses+1`-element `idose`
array. The stray value is normally discarded by the subsequent re-sync
(solve results are unchanged), but the OOB read corrupts the heap on some
toolchains -> the oldrel-1 segfault.
Fix: short-circuit `syncIdx()` when `ixds` is out of `[0, ndoses)` so it
re-syncs instead of reading past the array. Verified with ASAN: the
overflow is gone and the full suite runs clean. Regression test
`tests/testthat/test-syncidx-dose-index-oob.R` reproduces the exact OOB
(modeled-duration + lag-time model dosed into two compartments; ASAN
aborts before the fix, clean after).
Also fixes three further out-of-bounds accesses found during the same
ASAN investigation (memory-safety hardening; none changes results, and
none is the segfault corruptor -- they are reads into local/guarded
buffers or an unreachable branch):
- cvPost.cpp `rcvC1`: 1x1 `omega` branch indexed `ret(1,1)` on a 0x0
matrix before sizing it -> `ret.set_size(1,1)`.
- linCmt.h `linCmtStan2ssInf8`: oral 2-cmt SS both-rates-non-positive
branch wrote `ret(3,0)` on a 3-row Eigen vector -> `ret(2,0)`.
- etTran.cpp: `combineDvid` read element [1] of a length-1 logical -> [0].
- rxDerived.cpp `derived1`: recycled length-1 pointer incremented past
its one-element buffer -> guarded.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
# Conflicts: # NEWS.md # src/cvPost.cpp # src/etTran.cpp # src/linCmt.h # src/rxDerived.cpp
…nto dev-2026-06 # Conflicts: # NEWS.md # src/cvPost.cpp # src/etTran.cpp # src/linCmt.h # src/rxDerived.cpp
… state
The automatic linCmt() conversion (rxSolve(useLinCmt=TRUE), the default in
5.1.3) detected any 1-3 compartment linear topology and folded every ODE
state into an analytic linCmt() solution. linCmt() exposes only the central
concentration, through the output line, so when another model line referenced
one of the folded states as a value -- a peripheral or metabolite observable
such as `Cp <- periph / vp`, or a second endpoint keyed to that state -- the
conversion dropped the coupled state and demoted it to a required input
parameter. The default solve then aborted with:
The following parameter(s) are required for solving: <state>
Passing useLinCmt=FALSE forced the full ODE solve and worked, but the default
must not break these models.
.odeToLinDetect() now returns NULL (no conversion) whenever a compartment
state is referenced as a *value* by any model line other than the ODE
equations and the single central output line. Compartment-position arguments
of f/rate/dur/alag modifiers and adaptive-dosing calls (bolus/infuse/...) are
not value references -- they are rewritten to the standard linCmt compartment
names -- so ordinary convertible models, including those with in-model dosing,
still linearize.
Regression tests cover the coupled-peripheral-observable, observed-metabolite,
and multi-endpoint (fetus) cases, plus the boundary where an observable derives
from the central output variable (still converts). NEWS updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Brings the 9 already-merged branches now on main into the PR #1081 branch so GitHub recomputes the diff against main's current tip. Collapses the PR diff from 65 files (+4154/-279) to 45 files (+3721/-131), leaving only the changes unique to dev-2026-06. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.