Next: neldermead()
Up: Mathematical Macros Help File
Previous: moorepenrose()
Contents
Usage:
levmar(b,x,y,f,param [,deriv:deriv,crit:crvec,active:active,\
maxit:itmax,minit:itmin,print:T])
or
levmar(b,x,y,param ,resid:res [,deriv:deriv,crit:crvec,active:active,\
maxit:itmax,minit:itmin,print:T])
b REAL vector of starting values for coefficients
x REAL variable. Without 'resid:res' a vector or matrix
with nrows(x) = nrows(y); with 'resid:res' nrows(x) =
norows(y) is not required
y REAL vector of data to be fit
f Macro: fit <- f(b,x,param) returns a vector of fitted values
with length nrows(x) = nrows(y); not allowed with 'resid:res'
param vector or structure of additional parameters for f or NULL
res Macro: res(b, x, y, param) computes a vector of residuals of
length nrows(y). 'resid:res' is required when argument f is
omitted and is not allowed when f is an argument.
crvect vector(numsig, nsigsq, delta), 3 criteria for convergence
deriv optional macro: deriv(b,x,y,param,j) computes derivative of
f(b,x,param) or -res(b,x,y,param) with respect to b[j]
active LOGICAL vector the same length as b
itmax integer >= 0, maximum number of iterations permitted (default
= 30)
itmin 0 <= minimum <= itmax = number of iterations performed (default
= 1)
print if T, partial results are printed on each iteration
Returned value is structure(coefs:b_hat,hessian:hes,jacobian:jac,
gradient:g,rss:Rssmin,residuals:resids,nobs:n,iter:niter,
iconv:convflag)
|
Keywords:
nonlinear least squares, minimize
Introduction
levmar() uses an analogue of the Levenberg-Marquardt and Gauss algorithms
to minimize a sum of squares. Specifically, the criterion minimized is
sum(resids^2) where resids is computed as
resids <- y - f(b, x, param)
or as
resids <- res(b, x, y, param)
where f() or res() is a macro provided by the user
You can supply a macro to compute derivatives with respect to the elements
of b analytically or rely on difference-based numerical differentiation.
levmar() is intended for primarily use in higher level macros such as
nlreg() (non-linear regression) and arima() (estimation of ARIMA time
series models).
levmar() is based on a Fortran program for nonlinear least squares of Ken
Brown. See below for references.
Usage and arguments
levmar(b,x,y,f,param) uses an iterative algorithm to find a REAL vector
b_hat which minimizes Rss = sum((y - f(b_hat,x,param))^2), a sum of
squared residuals.
levmar(b,x,y,param, resid:res) does the same except the quantity minimized
is Rss = sum(res(b, x, y, param)^2).
In these usages, derivatives are computed by differences.
levmar(b,x,y,f,param, deriv:der) and levmar(b,x,y,param,resid:res,
deriv:der) do the same, except derivatives are computed by macro der()
instead of by differencing.
The arguments to levmar() are as follows
b REAL vector of starting values for b_hat
x REAL variable. When f is an argument, x must be a vector or
matrix with nrows(x) = nrows(y). When 'resid:res' is an
argument, nrows(x) is not required; if x is not used, it should
be 0
y REAL vector of data to be fit
param vector, matrix or structure of additional fixed parameters or
NULL
f macro called as fit <- f(b,x,param). fit is a vector of
containing nrows(x) = nrows(y) function values
res macro called as r <- res(b, x, y, param). r is a vector
of nrows(y) residuals
der macro called as derivs_j <- der(x,y,param,j). derivs_j is
vector containing the nrows(y) derivatives with respect to
b[j] of the elements of f(b,x,param) or -res(b,x,y,param).
Return value
levmar() returns structure(coefs:b_hat,hessian:hes,jacobian:jac,
gradient:g,rss:Rssmin,residuals:resids,nobs:n,iter:niter,iconv:convflag)
where component values are as follows:
b_hat REAL vector which minimizes Rss
hes jac' %*% jac, an approximation to the Hessian matrix H,
where H[i,j] = 2nd order partial derivative of Rss/2
with respect to b_hat[i] and. b_hat[j]
jac the nrows(y) by nrows(b) Jacobian matrix; -jac[,j] = vector
of partial derivatives of the elements of the residual
vector with respect to b_hat[j].
g the nrows(b) REAL gradient vector with g[j] = partial
derivative of Rss/2 with respect to b_hat[j]. g should be
close to a vector of zeros.
Rssmin the minimized value of Rss
resids vector of residuals of length nrows(y)
n positive integer = nrows(y) = nrows(resids) = nrows(jac)
niter positive integer = the number of iterations
conflag Convergence status flag; 0 = not converged, 1 = met relative
change in b_hat criterion, 2 = met relative change in Rss
criterion, 3 = met norm of gradient vector criterion, 4 =
failed to reduce Rss on a step of the iteration.
When any parameters are inactive as specified by keyword 'active' (see
below), jac is nrows(y) by p, hes is p by p and g has length p where p =
number of active parameters.
Keyword phrase arguments
There are several of keywords that can be used to control the iteration
and hold parameters at fixed values.
Keyword Phrase Value and Explanation
crit:crvec vector(numsig, nsigsq, delta), 3 criteria for
convergence (default = vector(8,5,-1)
numsig = desired number of significant digits in
the elements of b_hat (conflag = 1 when met)
nsigsq = desired number of significant digits in the
minimized Rss (conflag = 2 when met)
delta is a threshold for ||g||. Iteration is stopped
when ||g|| <= delta (conflag = 3 when met)
A negative criterion is not used
active:act LOGICAL vector the same length as b. b[j]
"participates" in the iteration only if act[j] is
True (default = rep(T,length(b))). When act[j] is
False, b_hat[j] remains at the starting value
maxit:itmax Non-negative integer specifying the maximum number of
iterations (default = 30). When itmax = 0, no iterations
are done and the quantities returned are computed at
the starting value b
minit:itmin Non-negative integer < itmax specifies the mininum
number of iterations
print:T When T, partial results are printed on each iteration
Iteration stops when (i) itmax is exceeded, (ii) any of the three
convergence criteria are satisfied, or (iii) when there has been no
reduction in Rss on an iteration after 10 halvings of the initial step
size.
Convergence criteria
There are 3 possible convergence criteria, at least one of which must be
enabled. levmar() terminates iteration when at least itmin iterations
have been completed and any convergence criterion is satisfied or when
itmax iterations have been completed.
The criteria are specified by optional argument crit:vec, where vec is
vector(numsig [, nsigsq [, delta]]) with length <= 3. The default values
for numsig, nsiqsq and delta are 8, 5 and -1, respectively.
A negative value for a criterion means it is not active.
numsig Desired number of significant digits in every active
coefficient. Specifically, the criterion is satisfied if,
for every active coefficient b[j], the change d_j satisfies
abs(d_j) < 10^-numsig*max(.5,abs(b_j)) where b_j is the
updated value of b[j]. Component 'iconv' of the return value
is 1 when satisfied.
nsigsq Desired number of significant digits in Rss = the residual
sum of squares. Specifically, the criterion is satisfied
when abs(Rss_new - Rss_old) < 10^-nsigsq*max(.5,Rss_new).
Component 'iconv' of the return value is 2 when satisfied.
delta Desired maximum norm ||g|| of the gradient vector g.
Specifically, the criterion is satisfied when
sqrt(sum(g^2)) <= delta. Component 'iconv' of the return
value is 3 when satisfied.
For numsig and nsigsq, the returned coefficients are the values updated
on that iteration. For delta, the returned coefficients are the values
found on the previous iteration.
Criteria are checked in the order delta, numsig and nsigsq.
Example:
Fit the function b1 + b2*b3^x to data from Snedecor and Cochran with
starting values b1 = b2 = 40 and b3 = 1.
Cmd> x <- vector(0,1,2,3,4,5)
Cmd> y <- vector(57.5, 45.7, 38.7, 35.3, 33.1, 32.2)
Cmd> func <- macro("@b <- $1; @b[1]+@b[2]*@b[3]^($2)", dollars:T)
Cmd> startVal <- vector(40,40,1) # starting values for iteration
Cmd> stuff <- levmar(startVal,x,y,func,NULL); stuff
component: coefs
(1) 30.724 26.821 0.55184
component: hessian
(1,1) 6 2.1683 111.39
(2,1) 2.1683 1.4367 30.242
(3,1) 111.39 30.242 2675.8
component: jacobian
(1,1) 1 1 0
(2,1) 1 0.55184 26.821
(3,1) 1 0.30453 29.602
(4,1) 1 0.16805 24.503
(5,1) 1 0.092736 18.029
(6,1) 1 0.051176 12.436
component: gradient
(1) 5.5896e-08 -8.4145e-09 2.4503e-05
component: rss
(1) 0.097248
component: residuals
(1) -0.044919 0.17523 -0.19159 0.068869 -0.11115
(6) 0.10356
component: nobs
(1) 6
component: iter
(1) 7
component: iconv
(1) 1
Cmd> # compute MSE and approximate standard errors
Cmd> mse <- stuff$rss/(stuff$nobs - length(stuff$coefs)); mse
(1) 0.032416
Cmd> sqrt(mse*diag(solve(stuff$hessian)))
(1) 0.23099 0.2577 0.008448
Cmd> # don't iterate over coefficient 1
Cmd> startVal1 <- vector(30,40,1)
Cmd> levmar(startVal1,x,y,func,NULL,active:vector(F,T,T))
component: coefs
(1) 30 27.418 0.57447
component: hessian
(1,1) 1.4907 34.492
(2,1) 34.492 3136.2
component: jacobian
(1,1) 1 0
(2,1) 0.57447 27.418
(3,1) 0.33002 31.502
(4,1) 0.18959 27.145
(5,1) 0.10891 20.792
(6,1) 0.062567 14.931
component: gradient
(1) 3.5552e-09 0.00067866
component: rss
(1) 0.38885
component: residuals
(1) 0.08214 -0.05082 -0.34842 0.10193 0.11385
(6) 0.48454
component: nobs
(1) 6
component: iter
(1) 7
component: iconv
(1) 1
References
For information on the algorithm, see K M Brown and J E Dennis,
Derivative free analogues of the Levenberg-Marquardt and Gauss
algorithms for nonlinear least squares approximation, Numerische
Mathematik, Vol. 18, pp. 289-297 (1972), and K M Brown, Computer
oriented methods for fitting tabular data in the linear and nonlinear
least squares sense, Technical Report No. 72-13, University of
Minnesota Department of Computer and Information Sciences.
Gary Oehlert
2005-08-12