This vignette accompanies development version 0.99.2. The benchmark
covers selected configurations, and the validation study uses 100
datasets per metric with 99 resampling draws. These small simulations
provide diagnostics; they do not establish general statistical
calibration. The local study observed 24% peak false positives, 10% peak
detection, 74.9% transfer interval coverage, and 43% drift interval
coverage in the tested scenarios. These results limit inferential use of
these methods. Report sources are scripts/benchmark.R and
scripts/validate.R.
Experimental physics data typically come with explicit measurement
uncertainties (one sigma per data point), correlated noise, and a
parametric model the analyst believes in. This is different from
biological data, where measurement uncertainty is rarely known and the
model is rarely a tight parametric form. The crossStatR functions in
this vignette (sci_regression(),
kinetic_model_compare(), noise_characterize(),
parameter_uncertainty()) are designed for that setting. The
example works through a simulated exponential-decay experiment with
correlated noise and known per-point uncertainties.
The example simulates a first-order decay
A * exp(-k * t) with A = 5,
k = 0.3, sampled at 60 time points. Each measurement has a
known 1-sigma uncertainty that varies with time, and the noise is
autocorrelated (AR(1) with rho = 0.4) so that conventional
OLS standard errors will be wrong.
set.seed(808)
t <- seq(0, 10, length.out = 60)
A_true <- 5; k_true <- 0.3
truth <- A_true * exp(-k_true * t)
sigma <- 0.05 + 0.02 * t # heteroscedastic 1-sigma
# AR(1) noise correlated in time
eps <- as.numeric(stats::filter(rnorm(length(t), sd = sigma),
filter = 0.4, method = "recursive"))
y <- truth + eps
df <- data.frame(t = t, y = y, sigma = sigma)ggplot(df, aes(x = t, y = y)) +
geom_point() +
geom_errorbar(aes(ymin = y - sigma, ymax = y + sigma), width = 0.05) +
geom_line(aes(y = truth), colour = "tomato", linetype = "dashed") +
labs(x = "Time (s)", y = "Signal", colour = NULL) +
theme_bw()Simulated exponential decay with heteroscedastic 1-sigma error bars. The noise is AR(1)-correlated across time.
When the per-point measurement uncertainty is known, weighted least
squares with w_i = 1 / sigma_i^2 is the BLUE.
sci_regression() with
weight_type = "precision" does exactly this:
fit_wls <- sci_regression(y ~ t, data = df,
weights = 1 / df$sigma^2,
weight_type = "precision")
print(fit_wls)
#> <crossStatR::regression> Regression: wls
#> term estimate se statistic p_value ci_low ci_high
#> (Intercept) 4.3183396 0.09932416 43.47723 5.811444e-46 4.1195207 4.5171585
#> t -0.5560631 0.02869300 -19.37974 5.164061e-27 -0.6134984 -0.4986278
#>
#> RMSE = 0.7251, R^2 = 0.8662The fit is a linear model on t, not on the decay form,
so the coefficients are intercept and slope. The point of this section
is the weighting machinery. For the actual decay form the example needs
nonlinear regression, which the example fits below.
For the nonlinear decay model y = A * exp(-k * t):
fit_nls <- sci_regression(y ~ A * exp(-k * t),
data = df,
nonlinear = TRUE,
start = list(A = 4, k = 0.4))
#> Waiting for profiling to be done...
print(fit_nls)
#> <crossStatR::regression> Regression: nls
#> term estimate se statistic p_value ci_low ci_high
#> A 4.9475637 0.105884481 46.72605 9.968576e-48 4.7347923 5.1641759
#> k 0.2903047 0.009317014 31.15856 6.481212e-38 0.2716535 0.3099605
#>
#> RMSE = 0.2420, R^2 = NAThe standard errors reported by nls assume independent
noise. With the AR(1) noise they will be too small, and the confidence
intervals will under-cover. The example addresses this below with
parameter_uncertainty().
noise_characterize() decomposes the signal into a smooth
trend (a polynomial of degree degree) and a residual
series, then reports the residual standard deviation, the
autocorrelation at lags 1 through 10, a heteroscedasticity test, a
periodogram, and signal-dependent noise bins. The example uses the
difference method here, which is appropriate when the signal is smooth
relative to the sampling rate.
nc <- noise_characterize(t, y, method = "residual", degree = 4L)
print(nc)
#> <crossStatR::noise_characterize> Polynomial regression (degree 4) residuals
#> noise_sd = 0.2023
#> Autocorrelation (lags 1-10): 0.507 0.177 -0.063 -0.251 -0.281 -0.218 -0.239 -0.137 0.007 0.163
#> heteroscedasticity r = -0.285 (p = 0.0275638)
#> dominant frequency = 0.4609
#>
#> Signal-dependent noise:
#> bin mean_signal sd_noise n
#> 1 0.2945173 0.38332813 6
#> 2 0.6537315 0.34684976 6
#> 3 0.7617225 0.19968706 6
#> 4 0.8007891 0.21022302 6
#> 5 0.9104527 0.14507407 6
#> 6 1.1874568 0.11052060 6
#> 7 1.6857673 0.06189307 6
#> 8 2.4165724 0.11056875 6
#> 9 3.3482826 0.08963082 6
#> 10 4.4065307 0.12587243 6The autocorrelation reported at lag 1 should be near
0.4, which is the AR(1) coefficient used in the simulation.
If it exceeds 2 / sqrt(n) (about 0.26 here) the
independence assumption is questionable, and any inferential procedure
that assumes independent residuals will give misleading standard
errors.
Real decay data are often ambiguous between a single first-order
decay and a biexponential. kinetic_model_compare() fits
both, reports AIC, BIC, RSS, parameter estimates with CIs, and an
identifiability diagnostic based on the condition number of the
sensitivity matrix.
kmc <- kinetic_model_compare(t, y)
print(kmc)
#> <crossStatR::kinetic_model_compare> best model: biexp
#>
#> Model comparison:
#> model AIC BIC RSS n_params condition_number
#> first_order -166.23541 -162.04672 3.515170 2 16.005684
#> second_order -120.95117 -116.76248 7.476951 2 41.179904
#> biexp -167.78475 -159.40737 3.204637 4 190.875385
#> saturation 43.11267 47.30136 115.148475 2 1.973019
#> identifiable delta_AIC
#> TRUE 1.549344
#> TRUE 46.833585
#> TRUE 0.000000
#> TRUE 210.897420The best model by AIC is the first-order decay here, which is the truth. But AIC deltas below 2 should be read as “models are essentially indistinguishable”, not as “the simpler model is correct”. With sparse data the biexponential will fit nearly as well as the first-order decay by setting one of its amplitudes near zero, and the condition number of the biexponential’s sensitivity matrix will be large. The example does not perform a likelihood-ratio test automatically because detecting nesting from a black-box function is not reliable; if the analyst knows the nesting, compute the LRT manually from the reported RSS values.
The decay constant k is the parameter of interest. Its
CI from nls assumes independent Gaussian noise. The example
compares three uncertainty propagation methods via
parameter_uncertainty(): the delta method (linearisation,
fast, asymptotically exact), Monte Carlo from the parameter distribution
(captures nonlinearity, assumes the parameter estimate is approximately
Gaussian), and a nonparametric bootstrap (refits on resampled data,
makes the fewest distributional assumptions).
# Extract the nls fit and its covariance
nls_fit <- fit_nls$model
params <- stats::coef(nls_fit)
covmat <- stats::vcov(nls_fit)
g <- function(p) p[2] # extract k
# Delta method
r_delta <- parameter_uncertainty(params, covmat, g, method = "delta")
print(r_delta)
#> <crossStatR::parameter_uncertainty> method = delta
#> estimate = 0.2903
#> se = 0.009317
#> 95% CI: [[0.2502, 0.3304]]
#> (Delta method; linearised at the supplied params.)
# Monte Carlo
r_mc <- parameter_uncertainty(params, covmat, g, method = "monte_carlo",
n_sim = 1000, seed = 1)
print(r_mc)
#> <crossStatR::parameter_uncertainty> method = monte_carlo
#> estimate = 0.2905
#> se = 0.009697
#> 95% CI: [[0.2709, 0.309]]
# Bootstrap (refits on resampled (t, y, sigma) rows)
refit_fn <- function(d) {
fit <- try(stats::nls(y ~ A * exp(-k * t), data = d,
start = list(A = 4, k = 0.4)),
silent = TRUE)
if (inherits(fit, "try-error")) return(rep(NA_real_, 2))
stats::coef(fit)
}
r_boot <- parameter_uncertainty(params, covmat, g, method = "bootstrap",
n_sim = 200, seed = 1,
original_data = list(data = df, refit = refit_fn))
print(r_boot)
#> <crossStatR::parameter_uncertainty> method = bootstrap
#> estimate = 0.2878
#> se = 0.009599
#> 95% CI: [[0.2671, 0.3053]]The bootstrap interval is wider than the delta-method interval here because the AR(1) correlation in the noise is absorbed into the bootstrap resampling distribution. The example uses the bootstrap interval more than the delta-method interval for this dataset, because the delta method’s independence assumption is violated.
For highly nonlinear g(theta) the delta method can be
severely biased; always compare against Monte Carlo. The Monte Carlo CI
assumes the parameter estimate is approximately Gaussian; for small
samples this is optimistic. The bootstrap requires the original data and
a working refit function, and it does not currently report the
refit-failure rate.
Aggressive smoothing before fitting the decay model shrinks the
apparent amplitude and biases the rate constant. To see this, smooth the
raw data with adaptive_smoothing() and refit the nonlinear
model:
sm <- adaptive_smoothing(t, y, method = "spline")
df_smooth <- data.frame(t = t, y = sm$smoothed)
fit_smooth <- try(stats::nls(y ~ A * exp(-k * t), data = df_smooth,
start = list(A = 4, k = 0.4)),
silent = TRUE)
if (!inherits(fit_smooth, "try-error")) {
cat("Smoothed fit: A =", round(coef(fit_smooth)[1], 3),
" k =", round(coef(fit_smooth)[2], 3), "\n")
}
#> Smoothed fit: A = 4.948 k = 0.29
cat("True: A =", A_true, " k =", k_true, "\n")
#> True: A = 5 k = 0.3The fitted k from the smoothed data is biased relative
to the truth, and the standard error from nls is too small
because smoothing introduces correlations that the nls
asymptotic variance does not account for. The documentation recommends
fitting models directly on the raw noisy data whenever possible, and
letting the likelihood handle the noise. Smooth only for visualisation,
not for inference.
For experimental physics data: supply per-point measurement
uncertainties to sci_regression() with
weight_type = "precision". Check
noise_characterize() for autocorrelation; if it is present,
do not trust the nls asymptotic standard errors and use the
bootstrap via parameter_uncertainty(). Compare competing
kinetic models with kinetic_model_compare() but read small
AIC deltas as non-identifiability, not as support for the simpler model.
Do not smooth before fitting; the bias is real and propagates.
Seber, G. A. F. and Wild, C. J. (1989). Nonlinear Regression. Wiley. Source.
Oehlert, G. W. (1992). A Note on the Delta Method. The American Statistician, 46, 27-29. Source.
Box, G. E. P., Jenkins, G. M. and Reinsel, G. C. (2008). Time Series Analysis: Forecasting and Control. 4th ed. Wiley. Source.