Next: predlimits() Up: Regression Macros Help File Previous: estimlimits()   Contents

nlreg()

Usage:
nlreg(b,x,y,f,param [,deriv:deriv,crit:vec,active:active,maxit:itmax,\
  minit:itmin, print:T, keep:T, quiet:T])
nlreg(b,x,y param ,resid:res [,deriv:deriv,crit:vec,active:active,\
  maxit:itmax,minit:itmin,print:T,keep:T, quiet:T])
 b      REAL vector of starting values for the iterative fitting
 x      REAL matrix; with argument f, nrows(x) = nrows(y); with
        'resid:res' nrows(x) = nrows(y) can be omitted; if x is not
        used by res(), it should be 0
 y      REAL vector
 f      macro: f(b,x,param) computes a vector of fitted values for y
        (not allowed with 'resid:res'; required without
        'resid:res')
 param  a vector or structure of additional parameters for f() or NULL
 res    macro: res(b,x,y [,param]) computes a vector of residuals from
        a function defined or directly referenced in res() (not allowed
        when f is an argument; required when f is not an argument).
 deriv  optional macro: deriv(b,x,y,param,j) computes derivative of
        f(x,b,param) or -res(b,x,y) with respect to b[j], returning a
        vector the same length as y
 active LOGICAL vector the same length as b specifies which parameters
        are included in iteration
 vec    vector(numsig, nsigsq, delta): 3 criteria for convergence
        numsig = number of digits of accuracy in coefficients
        nsiqsq = number of digits of accuracy in residual SS
        delta  = threshhold for norm of gradient
 itmin  the minimum number >= 0 of iterations performed
 itmax  the maximum number >= itmin of iterations allowed
 print  If T, partial results printed at each iteration
 keep   If T, nlreg() returns the structure returned by levmar()
        with components, coefs, hessian, jacobian, gradient,
        rss, residuals, nobs, iter, iconv plus component edf
 quiet  If F (default, unless keep:T), no summary results are printed.
        quiet:T is illegal without keep:T



Keywords: nonlinear fitting, regression
                              Introduction
nlreg() is a macro for carrying out nonlinear least squares regression.
It uses macro levmar() to do the actual minimization.  The function
fitted and/or residuals from the function are specified by macros in the
argument list.  Derivatives may be computed by a macro.  If no macro to
compute derivatives is provided, derivatives are computed numerically by
differencing.

                           Usage and arguments
 nlreg(b,x,y,f,param [,deriv:deriv,crit:vec,active:active,maxit:itmax,\
       minit:itmin,print:T, keep:T, quiet:T])
or
 nlreg(b,x,y, param ,resid:res [,deriv:deriv, crit:vec,\
       active:active,maxit:itmax,minit:itmin,print:T, keep:T, quiet:T])

  b      REAL vector of starting values for the iterative fitting
  x      REAL matrix; when f is an argument, nrows(x) = nrows(y) is
         required; when 'resid:res' is an argument, nrows(x) !=
         nrows(y) is allowed; if res() does not use x, x should be 0
  y      REAL vector
  f      macro; f(b,x,param) computes a vector with length nrows(y) of
         fitted values for y (not allowed with 'resid:res')
  param  a vector or structure of additional parameters for func or NULL
  res    macro; res(b,x,y [,param]) computes a vector of residuals from
         a function defined or directly referenced in res().  f must not
         be an argument when resid:res is an argument.  Conversely,
         when resid:res is not an argument, f is a required  argument.
         res() must return a vector of length nrows(y) and need not use
         x.
  deriv  optional macro: deriv(b,x,y,param,j) computes derivative of
         f(x,b,param) or -res(b,x,y,param) with respect to b[j],
         returning a vector the same length as y
  active LOGICAL vector the same length as b.  active[i] = F means b[i]
         is kept constant and does not participate in iteration
  vec    vector(numsig, nsigsq, delta), 3 criteria for convergence
         numsig = number of digits of accuracy in coefficients [8]
         nsiqsq = number of digits of accuracy in residual SS [5]
         delta  = norm of gradient threshhold [-1]
  itmin  the minimum number >= 0 of iterations performed
  itmax  the maximum number >= itmin of iterations allowed
  print  If T, partial results printed at each iteration
  keep   If T, nlreg() returns the structure returned by levmar() with
         components, coefs, hessian, jacobian, gradient, rss, residuals,
         nobs, iter, iconv plus component edf (error degrees of freedom)
  quiet  If F (default, unless keep:T), no summary results are
         printed.  quiet:T is illegal without keep:T

                             Value returned
With 'keep:T', nlreg() returns structure(coefs:b_hat,hessian:hes,
jacobian:jac,gradient:g,rss:Rssmin,residuals:resids,nobs:n,iter:niter,
iconv:convflag,edf:errordf).  The components are the same as returned by
levmar() with the addition of 'edf'.

The component values are as follows:

  b_hat     REAL vector which minimizes Rss
  hes       jac' %*% jac, an approximation to the Hessian matrix H,
            where H[i,j] = 2nd order partial derivative of Rss/2
            with respect to b_hat[i] and. b_hat[j]
  jac       the nrows(y) by nrows(b) Jacobian matrix; -jac[,j] = vector
            of partial derivatives of the elements of the residual
            vector with respect to b_hat[j].
  g         the nrows(b) REAL gradient vector with g[j] = partial
            derivative of Rss/2 with respect to b_hat[j].  g should be
            close to a vector of zeros at convergence
  Rssmin    the minimized value of Rss
  resids    vector of residuals of length nrows(y)
  n         positive integer = nrows(y) = nrows(resids) = nrows(jac)
  niter     positive integer = the number of iterations
  conflag   Convergence status flag; 0 = not converged, 1 = met relative
            change in b_hat criterion, 2 = met relative change in Rss
            criterion, 3 = met norm of gradient vector criterion, 4 =
            failed to reduce Rss on a step of the iteration.
  edf       Error degrees of freedom = nrows(y) - length(b_hat).

When any parameters are inactive as specified by keyword 'active' (see
below), jac is nrows(y) by p, hes is p by p, g has length p and edf =
nrows(y) - p, where p = number of active parameters.

                        Keyword phrase arguments
There are several of keywords that can be used to control the iteration
and hold parameters at fixed values.

  Keyword Phrase   Value and Explanation
  crit:crvec       vector(numsig, nsigsq, delta), 3 criteria for
                   convergence (default = vector(8,5,-1).  See below.
  active:act       LOGICAL vector the same length as b.  b[j]
                   "participates" in the iteration only if act[j] is
                   True (default = rep(T,length(b))).  When act[j] is
                   False, b_hat[j] remains at the starting value
  maxit:itmax      Non-negative integer specifying the maximum number of
                   iterations (default = 30).  When itmax = 0, no iterations
                   are done and the quantities returned are computed at
                   the starting value b
  minit:itmin      Non-negative integer < itmax specifies the mininum
                   number of iterations
  print:T          When T, partial results are printed on each iteration

Iteration stops when (i) itmax is exceeded, (ii) any of the three
convergence criteria are satisfied, or (iii) when there has been no
reduction in Rss on an iteration after 10 halvings of the initial step
size.

                          Use of argument param
One use for param might be to provide a choice between several functions
to be fit.  For example, the code for macro f might be the following:

  @b <- $1; @x <- $2; @p <- $3
  @val <- @b[1]
  for(@i,1,@p){
      @P <- @b[3*@i+1]
      @val <- @val + @b[3*@i-1]*cos(@x/@P,cycles:T) +\
        @b[3*@i]*sin(@x/@P,cycles:T)
  }
  @val # return

Here is the equivalent code for a macro res():

  @b <- $1; @x <- $2; @y <- $3; @p <- $4
  @res <- @y - @b[1]
  for(@i,1,@p){
      @P <- @b[3*@i+1]
      @res <- @res - @b[3*@i-1]*cos(@x/@P,cycles:T) -\
        @b[3*@i]*sin(@x/@P,cycles:T)
  }
  @res # return

Either might be used in fitting a curve with p cosine components with
unknown periods, amplitudes or phases, with the number of terms
specified by params.

                          Convergence criteria
There are 3 possible convergence criteria, at least one of which must be
enabled.  nlreg() terminates iteration when at least itmin iterations
have been completed and any convergence criterion is satisfied or when
itmax iterations have been completed.

The criteria are specified by optional argument crit:vec, where vec is
vector(numsig [, nsigsq [, delta]]) with length <= 3.  The default values
for numsig, nsiqsq and delta are 8, 5 and -1, respectively.

A negative value for a criterion means it is not active.

  numsig   Desired number of significant digits in every active
           coefficient.  Specifically, the criterion is satisfied if,
           for every active coefficient b[j], the change d_j satisfies
           abs(d_j) < 10^-numsig*max(.5,abs(b_j)) where b_j is the
           updated value of b[j].  Component 'iconv' of the return value
           is 1 when satisfied.

  nsigsq   Desired number of significant digits in Rss = the residual
           sum of squares.  Specifically, the criterion is satisfied
           when abs(Rss_new - Rss_old) < 10^-nsigsq*max(.5,Rss_new).
           Component 'iconv' of the return value is 2 when satisfied.

  delta    Desired maximum norm ||g|| of the gradient vector g.
           Specifically, the criterion is satisfied when
           sqrt(sum(g^2)) <= delta.  Component 'iconv' of the return
           value is 3 when satisfied.

For numsig and nsigsq, the returned coefficients are the values updated
on that iteration.  For delta, the returned coefficients are the values
found on the previous iteration.

Criteria are checked in the order delta, numsig and nsigsq.

                          Test macros and data
Included in this file are the following four macros used in debugging
and testing nlreg():

 linear()    Linear function f <- x %*% b
 asymptot()  f = b[1] + b[2]*(b[3])^x & derivatives
 DandSFunc() f = b[1] + (.49 - b[1])*exp(-b[2]*(x - 8)) & derivatives
 testfun()   f = b[1]+b[2]*x+b[3]*x^2+b[4]*sin(b[5]*x)+b[6]*exp(-b[7]*x)

In addition, the following data sets are included in this file:
 DandS_T10.2  Data from Draper and Smith to be used with DandSFunc()
 SandCT19.8.1 Data from Snedecor and Cochran to be used with
              asymptot()
Example:
Fit the function b1 + b2*b3^x to data from Snedecor and Cochran with
starting values b1 = b2 = 40 and b3 = 1.

The data are also available in data set SandCT19.8.1 and the macro is
available as macro asymptot(), both in this file.

  Cmd> x <- vector(0, 1, 2, 3, 4, 5)

  Cmd> y <- vector(57.5, 45.7, 38.7, 35.3, 33.1, 32.2)

  Cmd> f <- macro("@b <- $1; @b[1]+@b[2]*@b[3]^($2)", dollars:T)

  Cmd> startVal <- vector(40,40,1) # starting values for iteration

  Cmd> nlreg(startVal,x,y,f)
             Coef      StdErr           t     P Value
  B 1      30.724     0.23099      133.01  9.3704e-07
  B 2      26.821      0.2577      104.08  1.9555e-06
  B 3     0.55184    0.008448      65.322  7.9056e-06
  ---------------------------------------------------
  N: 6, MSE: 0.032416, DF: 3
  Converged with relative change in all coefs < 1e-05 in 7 iterations

  Cmd> resfunc <- macro("($3) - f($1,$2)") # computes residuals

  Cmd> nlreg(startVal,x,y,resid:resfunc) # identical
             Coef      StdErr           t     P Value
  B 1      30.724     0.23099      133.01  9.3704e-07
  B 2      26.821      0.2577      104.08  1.9555e-06
  B 3     0.55184    0.008448      65.322  7.9056e-06
  ---------------------------------------------------
  N: 6, MSE: 0.032416, DF: 3
  Converged with relative change in all coefs < 1e-05 in 7 iterations

  Cmd> stuff <- nlreg(startVal,x,y,f,keep:T)

  Cmd> compnames(stuff)
   (1) "coefs"
   (2) "hessian"
   (3) "jacobian"
   (4) "gradient"
   (5) "rss"
   (6) "residuals"
   (7) "nobs"
   (8) "iter"
   (9) "iconv"
  (10) "edf"

  Cmd> stuff[vector(1,2,5,8,9,10)]
  component: coefs
  (1)       30.724       26.821      0.55184
  component: hessian
  (1,1)            6       2.1683       111.39
  (2,1)       2.1683       1.4367       30.242
  (3,1)       111.39       30.242       2675.8
  component: rss
  (1)     0.097248
  component: iter
  (1)            7
  component: iconv
  (1)            1
  component: edf
  (1)            3

  Cmd> # compute approximate standard errors

  Cmd> sqrt((stuff$rss/stuff$edf)*diag(solve(stuff$hessian)))
  (1)      0.23099       0.2577     0.008448

                             Cross reference
See also levmar().


Gary Oehlert 2006-01-30