.. module:: statsmodels.base.optimizer .. currentmodule:: statsmodels.base.optimizer

Optimization

statsmodels uses three types of algorithms for the estimation of the parameters of a model.

Basic linear models such as :ref:WLS and OLS <regression> are directly estimated using appropriate linear algebra.
:ref:RLM <rlm> and :ref:GLM <glm>, use iteratively re-weighted least squares. However, you can optionally select one of the scipy optimizers discussed below.
For all other models, we use optimizers <https://docs.scipy.org/doc/scipy/reference/optimize.html>_ from scipy <https://docs.scipy.org/doc/scipy/reference/index.html>_.

Where practical, certain models allow for the optional selection of a scipy optimizer. A particular scipy optimizer might be default or an option. Depending on the model and the data, choosing an appropriate scipy optimizer enables avoidance of a local minima, fitting models in less time, or fitting a model with less memory.

statsmodels supports the following optimizers along with keyword arguments associated with that specific optimizer:

newton - Newton-Raphson iteration. While not directly from scipy, we consider it an optimizer because only the score and hessian are required.

tol : float Relative error in params acceptable for convergence.
nm - scipy's fmin_nm

xtol : float Relative error in params acceptable for convergence ftol : float Relative error in loglike(params) acceptable for convergence maxfun : int Maximum number of function evaluations to make.

bfgs - Broyden–Fletcher–Goldfarb–Shanno optimization, scipy's fmin_bfgs.

gtol : float
    Stop when norm of gradient is less than gtol.
norm : float
    Order of norm (np.inf is max, -np.inf is min)
epsilon
    If fprime is approximated, use this value for the step
    size. Only relevant if LikelihoodModel.score is None.

lbfgs - A more memory-efficient (limited memory) implementation of bfgs. Scipy's fmin_l_bfgs_b.

m : int
    The maximum number of variable metric corrections used to
    define the limited memory matrix. (The limited memory BFGS
    method does not store the full hessian but uses this many
    terms in an approximation to it.)
pgtol : float
    The iteration will stop when
    ``max{|proj g_i | i = 1, ..., n} <= pgtol`` where pg_i is
    the i-th component of the projected gradient.
factr : float
    The iteration stops when
    ``(f^k - f^{k+1})/max{|f^k|,|f^{k+1}|,1} <= factr * eps``,
    where eps is the machine precision, which is automatically
    generated by the code. Typical values for factr are: 1e12
    for low accuracy; 1e7 for moderate accuracy; 10.0 for
    extremely high accuracy. See Notes for relationship to
    ftol, which is exposed (instead of factr) by the
    scipy.optimize.minimize interface to L-BFGS-B.
maxfun : int
    Maximum number of iterations.
epsilon : float
    Step size used when approx_grad is True, for numerically
    calculating the gradient
approx_grad : bool
    Whether to approximate the gradient numerically (in which
    case func returns only the function value).

cg - Conjugate gradient optimization. Scipy's fmin_cg.

gtol : float
    Stop when norm of gradient is less than gtol.
norm : float
    Order of norm (np.inf is max, -np.inf is min)
epsilon : float
    If fprime is approximated, use this value for the step
    size. Can be scalar or vector.  Only relevant if
    Likelihoodmodel.score is None.

ncg - Newton conjugate gradient. Scipy's fmin_ncg.

fhess_p : callable f'(x, \*args)
    Function which computes the Hessian of f times an arbitrary
    vector, p.  Should only be supplied if
    LikelihoodModel.hessian is None.
avextol : float
    Stop when the average relative error in the minimizer
    falls below this amount.
epsilon : float or ndarray
    If fhess is approximated, use this value for the step size.
    Only relevant if Likelihoodmodel.hessian is None.

powell - Powell's method. Scipy's fmin_powell.

xtol : float
    Line-search error tolerance
ftol : float
    Relative error in loglike(params) for acceptable for
    convergence.
maxfun : int
    Maximum number of function evaluations to make.
start_direc : ndarray
    Initial direction set.

basinhopping - Basin hopping. This is part of scipy's basinhopping tools.

niter : integer
    The number of basin hopping iterations.
niter_success : integer
    Stop the run if the global minimum candidate remains the
    same for this number of iterations.
T : float
    The "temperature" parameter for the accept or reject
    criterion. Higher "temperatures" mean that larger jumps
    in function value will be accepted. For best results
    `T` should be comparable to the separation (in function
    value) between local minima.
stepsize : float
    Initial step size for use in the random displacement.
interval : integer
    The interval for how often to update the `stepsize`.
minimizer : dict
    Extra keyword arguments to be passed to the minimizer
    `scipy.optimize.minimize()`, for example 'method' - the
    minimization method (e.g. 'L-BFGS-B'), or 'tol' - the
    tolerance for termination. Other arguments are mapped from
    explicit argument of `fit`:
    - `args` <- `fargs`
    - `jac` <- `score`
    - `hess` <- `hess`

minimize - Allows the use of any scipy optimizer.

min_method : str, optional Name of minimization method to use. Any method specific arguments can be passed directly. For a list of methods and their arguments, see documentation of scipy.optimize.minimize. If no method is specified, then BFGS is used.

Model Class

Generally, there is no need for an end-user to directly call these functions and classes. However, we provide the class because the different optimization techniques have unique keyword arguments that may be useful to the user.

.. autosummary:: :toctree: generated/

Optimizer _fit_newton _fit_bfgs _fit_lbfgs _fit_nm _fit_cg _fit_ncg _fit_powell _fit_basinhopping