Next: neldermead() Up: Mathematical Macros Help File Previous: moorepenrose()   Contents

levmar()

Usage:
levmar(b,x,y,f,param [,deriv:deriv,crit:crvec,active:active,\
  maxit:itmax,minit:itmin,print:T])
or
levmar(b,x,y,param ,resid:res [,deriv:deriv,crit:crvec,active:active,\
  maxit:itmax,minit:itmin,print:T])
  b        REAL vector of starting values for coefficients
  x        REAL variable.   Without 'resid:res'  a vector or matrix
           with nrows(x) = nrows(y); with 'resid:res' nrows(x) =
           norows(y) is not required
  y        REAL vector of data to be fit
  f        Macro:  fit <- f(b,x,param) returns a vector of fitted values
           with length nrows(x) = nrows(y); not allowed with 'resid:res'
  param    vector or structure of additional parameters for f or NULL
  res      Macro: res(b, x, y, param) computes a vector of residuals of
           length nrows(y).  'resid:res' is required when argument f is
           omitted and is not allowed when f is an argument.
  crvect   vector(numsig, nsigsq, delta), 3 criteria for convergence
  deriv    optional macro: deriv(b,x,y,param,j) computes derivative of
           f(b,x,param) or -res(b,x,y,param)  with respect to b[j]
  active   LOGICAL vector the same length as b
  itmax    integer >= 0, maximum number of iterations permitted (default
           = 30)
  itmin    0 <= minimum <= itmax = number of iterations performed (default
           = 1)
  print    if T, partial results are printed on each iteration

  Returned value is structure(coefs:b_hat,hessian:hes,jacobian:jac,
    gradient:g,rss:Rssmin,residuals:resids,nobs:n,iter:niter,
    iconv:convflag)



Keywords: nonlinear least squares, minimize
                              Introduction
levmar() uses an analogue of the Levenberg-Marquardt and Gauss algorithms
to minimize a sum of squares.  Specifically, the criterion minimized is
sum(resids^2) where resids is computed as
    resids <- y - f(b, x, param)
or as
    resids <- res(b, x, y, param)
where f() or res() is a macro provided by the user

You can supply a macro to compute derivatives with respect to the elements
of b analytically or rely on difference-based numerical differentiation.

levmar() is intended for primarily use in higher level macros such as
nlreg() (non-linear regression) and arima() (estimation of ARIMA time
series models).

levmar() is based on a Fortran program for nonlinear least squares of Ken
Brown.  See below for references.

                           Usage and arguments
levmar(b,x,y,f,param) uses an iterative algorithm to find a REAL vector
b_hat which minimizes Rss = sum((y - f(b_hat,x,param))^2), a sum of
squared residuals.

levmar(b,x,y,param, resid:res) does the same except the quantity minimized
is Rss = sum(res(b, x, y, param)^2).

In these usages, derivatives are computed by differences.

levmar(b,x,y,f,param, deriv:der) and levmar(b,x,y,param,resid:res,
deriv:der) do the same, except derivatives are computed by macro der()
instead of by differencing.

The arguments to levmar() are as follows
  b        REAL vector of starting values for b_hat
  x        REAL variable.  When f is an argument, x must be a vector or
           matrix with nrows(x) = nrows(y).  When 'resid:res' is an
           argument, nrows(x) is not required; if x is not used, it should
           be 0
  y        REAL vector of data to be fit
  param    vector, matrix or structure of additional fixed parameters or
           NULL
  f        macro called as fit <- f(b,x,param).  fit is a vector of
           containing nrows(x) = nrows(y) function values
  res      macro called as r <- res(b, x, y, param).  r is a vector
           of nrows(y) residuals
  der      macro called as derivs_j <- der(x,y,param,j).  derivs_j is
           vector containing the nrows(y) derivatives with respect to
           b[j] of the elements of f(b,x,param) or -res(b,x,y,param).

                              Return value
levmar() returns structure(coefs:b_hat,hessian:hes,jacobian:jac,
gradient:g,rss:Rssmin,residuals:resids,nobs:n,iter:niter,iconv:convflag)
where component values are as follows:

  b_hat     REAL vector which minimizes Rss
  hes       jac' %*% jac, an approximation to the Hessian matrix H,
            where H[i,j] = 2nd order partial derivative of Rss/2
            with respect to b_hat[i] and. b_hat[j]
  jac       the nrows(y) by nrows(b) Jacobian matrix; -jac[,j] = vector
            of partial derivatives of the elements of the residual
            vector with respect to b_hat[j].
  g         the nrows(b) REAL gradient vector with g[j] = partial
            derivative of Rss/2 with respect to b_hat[j].  g should be
            close to a vector of zeros.
  Rssmin    the minimized value of Rss
  resids    vector of residuals of length nrows(y)
  n         positive integer = nrows(y) = nrows(resids) = nrows(jac)
  niter     positive integer = the number of iterations
  conflag   Convergence status flag; 0 = not converged, 1 = met relative
            change in b_hat criterion, 2 = met relative change in Rss
            criterion, 3 = met norm of gradient vector criterion, 4 =
            failed to reduce Rss on a step of the iteration.

When any parameters are inactive as specified by keyword 'active' (see
below), jac is nrows(y) by p, hes is p by p and g has length p where p =
number of active parameters.

                        Keyword phrase arguments
There are several of keywords that can be used to control the iteration
and hold parameters at fixed values.

  Keyword Phrase   Value and Explanation
  crit:crvec       vector(numsig, nsigsq, delta), 3 criteria for
                   convergence (default = vector(8,5,-1)
                   numsig = desired number of significant digits in
                   the elements of b_hat (conflag = 1 when met)
                   nsigsq = desired number of significant digits in the
                   minimized Rss (conflag = 2 when met)
                   delta is a threshold for ||g||.  Iteration is stopped
                   when ||g|| <= delta (conflag = 3 when met)
                   A negative criterion is not used
  active:act       LOGICAL vector the same length as b.  b[j]
                   "participates" in the iteration only if act[j] is
                   True (default = rep(T,length(b))).  When act[j] is
                   False, b_hat[j] remains at the starting value
  maxit:itmax      Non-negative integer specifying the maximum number of
                   iterations (default = 30).  When itmax = 0, no iterations
                   are done and the quantities returned are computed at
                   the starting value b
  minit:itmin      Non-negative integer < itmax specifies the mininum
                   number of iterations
  print:T          When T, partial results are printed on each iteration

Iteration stops when (i) itmax is exceeded, (ii) any of the three
convergence criteria are satisfied, or (iii) when there has been no
reduction in Rss on an iteration after 10 halvings of the initial step
size.

                          Convergence criteria
There are 3 possible convergence criteria, at least one of which must be
enabled.  levmar() terminates iteration when at least itmin iterations
have been completed and any convergence criterion is satisfied or when
itmax iterations have been completed.

The criteria are specified by optional argument crit:vec, where vec is
vector(numsig [, nsigsq [, delta]]) with length <= 3.  The default values
for numsig, nsiqsq and delta are 8, 5 and -1, respectively.

A negative value for a criterion means it is not active.

  numsig   Desired number of significant digits in every active
           coefficient.  Specifically, the criterion is satisfied if,
           for every active coefficient b[j], the change d_j satisfies
           abs(d_j) < 10^-numsig*max(.5,abs(b_j)) where b_j is the
           updated value of b[j].  Component 'iconv' of the return value
           is 1 when satisfied.

  nsigsq   Desired number of significant digits in Rss = the residual
           sum of squares.  Specifically, the criterion is satisfied
           when abs(Rss_new - Rss_old) < 10^-nsigsq*max(.5,Rss_new).
           Component 'iconv' of the return value is 2 when satisfied.

  delta    Desired maximum norm ||g|| of the gradient vector g.
           Specifically, the criterion is satisfied when
           sqrt(sum(g^2)) <= delta.  Component 'iconv' of the return
           value is 3 when satisfied.

For numsig and nsigsq, the returned coefficients are the values updated
on that iteration.  For delta, the returned coefficients are the values
found on the previous iteration.

Criteria are checked in the order delta, numsig and nsigsq.

Example:
Fit the function b1 + b2*b3^x to data from Snedecor and Cochran with
starting values b1 = b2 = 40 and b3 = 1.

  Cmd> x <- vector(0,1,2,3,4,5)

  Cmd> y <- vector(57.5, 45.7, 38.7, 35.3, 33.1, 32.2)

  Cmd> func <- macro("@b <- $1; @b[1]+@b[2]*@b[3]^($2)", dollars:T)

  Cmd> startVal <- vector(40,40,1) # starting values for iteration

  Cmd> stuff <- levmar(startVal,x,y,func,NULL); stuff
  component: coefs
  (1)      30.724      26.821     0.55184
  component: hessian
  (1,1)           6      2.1683      111.39
  (2,1)      2.1683      1.4367      30.242
  (3,1)      111.39      30.242      2675.8
  component: jacobian
  (1,1)           1           1           0
  (2,1)           1     0.55184      26.821
  (3,1)           1     0.30453      29.602
  (4,1)           1     0.16805      24.503
  (5,1)           1    0.092736      18.029
  (6,1)           1    0.051176      12.436
  component: gradient
  (1)  5.5896e-08 -8.4145e-09  2.4503e-05
  component: rss
  (1)    0.097248
  component: residuals
  (1)   -0.044919     0.17523    -0.19159    0.068869    -0.11115
  (6)     0.10356
  component: nobs
  (1)           6
  component: iter
  (1)           7
  component: iconv
  (1)           1

  Cmd> # compute MSE and approximate standard errors

  Cmd> mse <- stuff$rss/(stuff$nobs - length(stuff$coefs)); mse
  (1)     0.032416

  Cmd> sqrt(mse*diag(solve(stuff$hessian)))
  (1)      0.23099       0.2577     0.008448

  Cmd> # don't iterate over coefficient 1

  Cmd> startVal1 <- vector(30,40,1)

  Cmd> levmar(startVal1,x,y,func,NULL,active:vector(F,T,T))
  component: coefs
  (1)          30      27.418     0.57447
  component: hessian
  (1,1)      1.4907      34.492
  (2,1)      34.492      3136.2
  component: jacobian
  (1,1)           1           0
  (2,1)     0.57447      27.418
  (3,1)     0.33002      31.502
  (4,1)     0.18959      27.145
  (5,1)     0.10891      20.792
  (6,1)    0.062567      14.931
  component: gradient
  (1)  3.5552e-09  0.00067866
  component: rss
  (1)     0.38885
  component: residuals
  (1)     0.08214    -0.05082    -0.34842     0.10193     0.11385
  (6)     0.48454
  component: nobs
  (1)           6
  component: iter
  (1)           7
  component: iconv
  (1)           1

                               References
For information on the algorithm, see K M Brown and J E Dennis,
Derivative free analogues of the Levenberg-Marquardt and Gauss
algorithms for nonlinear least squares approximation, Numerische
Mathematik, Vol. 18, pp. 289-297 (1972), and K M Brown, Computer
oriented methods for fitting tabular data in the linear and nonlinear
least squares sense, Technical Report No. 72-13, University of
Minnesota Department of Computer and Information Sciences.


Gary Oehlert 2006-01-30