Suppose we are planning a drug development program testing the superiority of an experimental treatment over a control treatment. Our drug development program consists of an exploratory phase II trial which is, in case of promising results, followed by a confirmatory phase III trial.
The drugdevelopR package enables us to optimally plan such programs using a utility-maximizing approach. To get a brief introduction, we presented a very basic example on how the package works in Introduction to planning phase II and phase III trials with drugdevelopR. As we only discussed the most important results of the function in the introduction, we now want to interpret the rest of the output.
We are in the same setting as in the introduction, i.e. we suppose we are developing a new tumor treatment, exper. The patient variable that we want to investigate is the difference in tumor width between the one-year visit and baseline. This is a normally distributed outcome variable.
The parameters we insert into the function
optimum_normal are the same parameters we also inserted in
the basic setting. Thus, for more information on the input values we
refer to the Introduction.
We start by loading the drugdevelopR package.
Then we insert the input values into the function
optimal_normal:
res <- optimal_normal(Delta1 = 0.625, fixed = TRUE, # treatment effect
n2min = 20, n2max = 400, # sample size region
stepn2 = 4, # sample size step size
kappamin = 0.02, kappamax = 0.2, # threshold region
stepkappa = 0.02, # threshold step size
c2 = 0.675, c3 = 0.72, # maximal total trial costs
c02 = 15, c03 = 20, # maximal per-patient costs
b1 = 3000, b2 = 8000, b3 = 10000, # gains for patients
alpha = 0.025, # significance level
beta = 0.1, # 1 - power
Delta2 = NULL, w = NULL, in1 = NULL, in2 = NULL,
a = NULL,b = NULL) # setting all unneeded parameters to NULLAfter setting all these input parameters and running the function, let’s take a look at the output of the program.
res
#> Optimization result:
#> Utility: 2946.07
#> Sample size:
#> phase II: 92, phase III: 192, total: 284
#> Probability to go to phase III: 1
#> Total cost:
#> phase II: 77, phase III: 158, cost constraint: Inf
#> Fixed cost:
#> phase II: 15, phase III: 20
#> Variable cost per patient:
#> phase II: 0.675, phase III: 0.72
#> Effect size categories (expected gains):
#> small: 0 (3000), medium: 0.5 (8000), large: 0.8 (10000)
#> Success probability: 0.85
#> Joint probability of success and observed effect of size ... in phase III:
#> small: 0.72, medium: 0.12, large: 0
#> Significance level: 0.025
#> Targeted power: 0.9
#> Decision rule threshold: 0.06 [Kappa]
#> Assumed true effect: 0.625 [Delta]
#> Treatment effect offset between phase II and III: 0 [gamma]The program returns a total of thirteen values and the input values. We will now take a closer look and explain all output values with the necessary detail:
res$u is the expected utility of the program for the
optimal sample size and threshold value. In our case it amounts to
2946.07, i.e. we have an expected utility of 294 607 000$. As the
function maximizes the expected utility, this is the highest expected
utility one can receive given the input parameters.res$Kappa is the optimal threshold value for the
go/no-go decision rule. We see that we need a treatment effect of more
than 0.06 in phase II in order to proceed to phase III. If in phase II
we get a treatment effect that is below 0.06, the program is deemed
unsuccessful and no phase III trial will be conducted. If we are above
the threshold value, we proceed to phase III.res$n2 is the optimal sample size for phase II and
res$n3 the resulting sample size for phase III. We see that
the optimal scenario requires 92 participants in phase II and 192
participants in phase III. res$n is the sum of
res$n2 and res$n3 and refers to the total
number of participants. Note, that one can set a maximum number of
participants, leading to constrained optimization. For more information,
see the article about further
input parameters.res$pgo is the probability to go to phase III, which
coincides with the probability that the observed treatment effect is
above the optimal threshold value. In this case, we get a probability
which is close to one and hence rounded to 1. This means that we can
expect a treatment effect of above 0.06 in phase II and hence, the trial
will proceed to phase III almost certainly.res$sProg is the probability that the whole program is
successful, i.e. that we proceed to phase III and observe a positive
treatment effect in phase III. In our case the probability of a
successful program is 0.85, so a trial with these input parameters will
be successful 85% of the time. The probability of a successful program
is further subdivided in probabilities for small, medium or large
treatment effect. The probability of a small treatment effect
res$sProg1 is 0.72 in our case, for a medium treatment
effect the probability res$sProg2 is 0.12 and for a large
treatment effect the probability res&$sProg3 is 0. The
treatment effects correspond to the benefit categories, so with a
probability of 0.72 we receive a benefit of 300,000,000$ and so on. Due
to rounding, the probabilities may not add up exactly to the overall
probability of a successful program. Note that one can set a minimum
probability of successful program, leading to constrained optimization.
For more information, see the article about further
input parameters.res$K2 in
phase II and res$K3 in phase III. The costs in phase II are
calculated as c02 + n2 ⋅ c2
and the costs in phase II are calculated as (c03 + n3 ⋅ c3) ⋅ pgo,
i.e. in phase III the expected costs are returned. In our setting, we
have costs of 77 in 10^5$ (i.e. 7,200,000$) in phase II and costs of 158
in 10^5$(i.e. 13,900,000) in phase III. Note, that one can set a maximum
cost limit for K = K2 + K3
, leading to constrained optimization. For more information, see the
article about further
input parameters.This tutorial explains how we can interpret all of the output generated by our program.
For more information on how to use the package, see: