Estimating RR and RD when max censoring time > max event time

[cegepi]

Hello (again),

Really enjoy using this package and thankful for your efforts with adjustedCurves! I've noticed that adjusted_curve_diff and adjusted_curve_ratio will give NA values for the quantities they estimate if the times argument has a value is beyond when the last event occurred. As long as at least one individual is still under follow-up, one should (ideally) be able to get estimates for these values. For instance running:

adjusted_curve_ratio(adjsurv, times=1825, group_1="1", group_2="0", conf_int=TRUE)

will produce NAs for the ratio because the last event was at day 1450. However, the longest time to censoring is 2000, so the risk ratio should be constant between 1450 and 2000 (but still calculatable at day 2000). This will happen anytime the max censoring time is past the max event time. The current workaround is to: 1) Find the last event time 2) calculate the RR at that event time, 3) find the last follow-up date, 4) carry the RR forward to the last follow-up date.

Not sure if this an issue or enhancement, but would be awesome when possible to look into or add this feature. :)

Thanks again!

Charley

[RobinDenz1]

Hello Charley,

thank you for your continued interest in this package.

I am aware of this NA value issue. I would like to point out first that both adjustedsurv() and adjustedcif() work similarly: if you enter values in times that are larger than the largest observed event time you will get either a warning or an error (I am honestly not sure without looking it up).

This is a deliberate choice, because I noticed in my simulation studies that estimates beyond the last observed event time tend to be biased for most adjustment methods. Unfortunately I do not have this specific simulation code anymore so I can't prove this to you right now, but I am still hesitant to allow the sort of "extrapolation" you are suggesting.

I will think about it. If you have any solid arguments for it I might consider allowing it using an optional argument that is FALSE by default. Either way, I will unfortunately not be working on this package in the next few weeks due to other responsibilities.

Thanks and best wishes,
Robin

[cegepi]

Thank you Robin, for your quick reply and helpful insights. That is interesting about the bias. I am working in the pharmacoepidemiology claims database world and it is quite common to have people with censoring dates (end of insurance enrollment) that extend beyond the last outcome (e.g., diagnosis code) date. I wasn't aware about the bias, but would think the confidence intervals getting super wide would be informative that maybe one shouldn't carry their risk estimate out that far (say to the very last censoring time in the sample). However, if a large number of people were still under observation and outcome-free I would feel comfortable extending risk to that point.

Totally understand about timing of package updates/maintenance; it seems like a lot of work outside of other demands! A couple more things I'd like to point out for whenever they can be examined:

1. adjustedsurv: I noticed that the upper confidence limit for survival can sometimes be slightly greater than 1.0 during early parts of follow-up. For instance, I have a survival point estimate of 0.9930792 with the 95% CI (0.9801613, 1.0059971).
2. adjusted_curve_ratio: results sometimes give RR that are incorrect. RD seems to be spot on. For example, the point estimate from just taking the ratio of the adjsurv output risks would be 1.12, but the adjusted_curve_ratio will say 1.15 at that time. I also got a RR of 0.67 with a negative UCL (0.18, -3.76)

Thanks and sorry I am a pest on this page :)

All the best,

Charley

[RobinDenz1]

Don't call yourself a pest!
I appreciate your input alot, since you are one of the few people opening issues who seem to have actually read the dochnentation.

On your first point: yes that can happen with some methods when using the normal approximation to obtain confidence intervals. You can sidestep this by setting force_bounds=TRUE in the adjustedsurv() call though.

Your second point is more serious. If this indeed happens it is a bug that I need to fix. Could you perhaps provide me a reproducible example of this issue?

Many thanks!
Robin

[cegepi]

Hi Robin,

Thanks for the response! Hmm I'm working with a claims database that is not open access so in order to make a reprex I would probably need to do some advanced stuff (plasmode simulation?) to make a sharable dataset; truth be told, not sure how to do that. The adjustedcif function gave me a cumulative incidence of 4.5 (1.7, 7.4) for drug A and 5.4 (0, 11.1) for drug B at time 360 (see attached flextableAdjcif.png). The risk difference (A versus B) at time 360 was correct with -0.9 (-7.3, 5.5) and the point estimate of the risk ratio was correct at 0.833. But the confidence limits for the RR were (0.245, -13.5), see attached flextableRR.png. Any blank spaces were NaN in R. So the lower limit seems plausible, but obviously something was going on with the UCL being negative and that large. The corresponding file for the RR output is also attached (flextableRR.html) Note, this does not always happen with adjusted_curve_ratio. I've noticed it tends to happen once I am doing any analysis on a smaller subgroup with fewer events.

Two things: 1) could still be user error somewhere on my part. 2) I definitely realize without a toy dataset in hand it may be difficult to see if true bug.

Thanks! Charley

RR output:

Risk output

[RobinDenz1]

Thank you for sharing this. It's good to know that the only issue seems to be the confidence intervals. The reason for this is the approach I used to calculate them. The package relies on the method given by Fieller (1954), which uses only the group-specific survival probabilities and their associated standard errors to calculate the confidence interval of their ratio. This is convenient from a computational perspective (these values are already there, no further calculations required), but it can lead to some bad estimates, which is known in the literature.

Since the calculations are based on bootstrapping anyways, another option would be to calculate the ratio in each bootstrap sample separately and to then use some bootstrap confidence interval (such as a percentile confidence interval) to obtain the relevant confidence interval. This, however, is computationally more expensive and requires a bit more code. I will probably add this as an option in future versions for cases like yours.