pysr3.linear.models module

class pysr3.linear.models.LinearCADModel(lam: float = 0.0, rho: float = 1.0, tol_solver: float = 1e-05, max_iter_solver: int = 1000, stepping: str = 'line-search', logger_keys: Set = ('converged',), fixed_step_len=None, prior=None, **kwargs)

Bases: SimpleLinearModel

Initializes the model

Parameters:

  • lam (float) – strength of CAD regularizer

  • rho (float) – cut-off amplitude above which the coefficients are not penalized

  • tol_solver (float) – tolerance for the stop criterion of PGD solver

  • max_iter_solver (int) – maximal number of iterations for PGD solver

  • stepping (str) – step-size policy for PGD. Can be either “line-search” or “fixed”

  • logger_keys (List[str]) – list of keys for the parameters that the logger should track

  • fixed_step_len (float) – step-size for PGD algorithm. If “linear-search” is used for stepping then the algorithm uses this value as the maximal step possible. Use this parameter if you know the Lipschitz-smoothness constant L for your problem as fixed_step_len=1/L.

  • prior (Optional[Prior]) – an instance of Prior class. If None then a non-informative prior is used.

  • kwargs – for passing debugging info

instantiate()
class pysr3.linear.models.LinearCADModelSR3(lam: float = 0.0, rho: float = 1.0, el: float = 1.0, tol_solver: float = 1e-05, max_iter_solver: int = 1000, stepping: str = 'line-search', logger_keys: Set = ('converged',), fixed_step_len=None, prior=None, practical=False, **kwargs)

Bases: SimpleLinearModelSR3

Initializes the model

Parameters:

  • lam (float) – strength of CAD regularizer

  • rho (float) – cut-off amplitude above which the coefficients are not penalized

  • el (float) – constant for SR3 relaxation. Bigger values correspond to tighter relaxation.

  • tol_solver (float) – tolerance for the stop criterion of PGD solver

  • max_iter_solver (int) – maximal number of iterations for PGD solver

  • stepping (str) – step-size policy for PGD. Can be either “line-search” or “fixed”

  • logger_keys (List[str]) – list of keys for the parameters that the logger should track

  • fixed_step_len (float) – step-size for PGD algorithm. If “linear-search” is used for stepping then the algorithm uses this value as the maximal step possible. Use this parameter if you know the Lipschitz-smoothness constant L for your problem as fixed_step_len=1/L.

  • prior (Optional[Prior]) – an instance of Prior class. If None then a non-informative prior is used.

  • kwargs – for passing debugging info

instantiate()
class pysr3.linear.models.LinearL0Model(nnz: int | None = None, tol_solver: float = 1e-05, max_iter_solver: int = 1000, stepping: str = 'line-search', logger_keys: Set = ('converged',), fixed_step_len=None, prior=None, **kwargs)

Bases: SimpleLinearModel

Initializes the model

Parameters:

  • nnz (int) – number of non-zero features to select

  • tol_solver (float) – tolerance for the stop criterion of PGD solver

  • max_iter_solver (int) – maximal number of iterations for PGD solver

  • stepping (str) – step-size policy for PGD. Can be either “line-search” or “fixed”

  • logger_keys (List[str]) – list of keys for the parameters that the logger should track

  • fixed_step_len (float) – step-size for PGD algorithm. If “linear-search” is used for stepping then the algorithm uses this value as the maximal step possible. Use this parameter if you know the Lipschitz-smoothness constant L for your problem as fixed_step_len=1/L.

  • prior (Optional[Prior]) – an instance of Prior class. If None then a non-informative prior is used.

  • kwargs – for passing debugging info

instantiate()
class pysr3.linear.models.LinearL0ModelSR3(nnz: int | None = None, el: float = 1.0, tol_solver: float = 1e-05, max_iter_solver: int = 1000, stepping: str = 'line-search', logger_keys: Set = ('converged',), fixed_step_len=None, prior=None, practical=False, **kwargs)

Bases: SimpleLinearModelSR3

Initializes the model

Parameters:

  • nnz (int) – number of non-zero features that we want to select

  • el (float) – constant for SR3 relaxation. Bigger values correspond to tighter relaxation.

  • tol_solver (float) – tolerance for the stop criterion of PGD solver

  • max_iter_solver (int) – maximal number of iterations for PGD solver

  • stepping (str) – step-size policy for PGD. Can be either “line-search” or “fixed”

  • logger_keys (List[str]) – list of keys for the parameters that the logger should track

  • fixed_step_len (float) – step-size for PGD algorithm. If “linear-search” is used for stepping then the algorithm uses this value as the maximal step possible. Use this parameter if you know the Lipschitz-smoothness constant L for your problem as fixed_step_len=1/L.

  • prior (Optional[Prior]) – an instance of Prior class. If None then a non-informative prior is used.

  • kwargs – for passing debugging info

instantiate()
class pysr3.linear.models.LinearL1Model(lam: float = 0, tol_solver: float = 1e-05, max_iter_solver: int = 1000, stepping: str = 'line-search', logger_keys: Set = ('converged',), fixed_step_len=None, prior=None, **kwargs)

Bases: SimpleLinearModel

Initializes the model

Parameters:

  • lam (float) – strength of LASSO prior

  • tol_solver (float) – tolerance for the stop criterion of PGD solver

  • max_iter_solver (int) – maximal number of iterations for PGD solver

  • stepping (str) – step-size policy for PGD. Can be either “line-search” or “fixed”

  • logger_keys (List[str]) – list of keys for the parameters that the logger should track

  • fixed_step_len (float) – step-size for PGD algorithm. If “linear-search” is used for stepping then the algorithm uses this value as the maximal step possible. Use this parameter if you know the Lipschitz-smoothness constant L for your problem as fixed_step_len=1/L.

  • prior (Optional[Prior]) – an instance of Prior class. If None then a non-informative prior is used.

  • kwargs – for passing debugging info

instantiate()
class pysr3.linear.models.LinearL1ModelSR3(lam: float = 0.0, el: float = 1.0, tol_solver: float = 1e-05, max_iter_solver: int = 1000, stepping: str = 'line-search', logger_keys: Set = ('converged',), fixed_step_len=None, prior=None, practical=False, **kwargs)

Bases: SimpleLinearModelSR3

Initializes the model

Parameters:

  • lam (float) – strength of LASSO regularizer

  • el (float) – constant for SR3 relaxation. Bigger values correspond to tighter relaxation.

  • tol_solver (float) – tolerance for the stop criterion of PGD solver

  • max_iter_solver (int) – maximal number of iterations for PGD solver

  • stepping (str) – step-size policy for PGD. Can be either “line-search” or “fixed”

  • logger_keys (List[str]) – list of keys for the parameters that the logger should track

  • fixed_step_len (float) – step-size for PGD algorithm. If “linear-search” is used for stepping then the algorithm uses this value as the maximal step possible. Use this parameter if you know the Lipschitz-smoothness constant L for your problem as fixed_step_len=1/L.

  • prior (Optional[Prior]) – an instance of Prior class. If None then a non-informative prior is used.

  • kwargs – for passing debugging info

instantiate()
class pysr3.linear.models.LinearModel(logger_keys=None)

Bases: BaseEstimator, RegressorMixin

Initializes a linear model.

Parameters:

logger_keys (Tuple[str]) – Set of values that the logger is supposed to log

check_is_fitted()

Checks if the model was fitted before. Throws an error otherwise.

Returns:

None

fit(x: ndarray, y: ndarray, initial_parameters: dict | None = None, warm_start=False, regularization_weights=None, **kwargs)

Fits a Linear Model to the given data.

Parameters:

  • x (np.ndarray) – Data

  • y (np.ndarray) – Answers, real-valued array.

  • initial_parameters (np.ndarray) –

    Dict with possible fields:

    • ‘x0’ : np.ndarray, shape = [n],
      Initial estimate of model’s coefficients. If None then it defaults to an all-ones vector.

  • warm_start (bool, default is False) – Whether to use previous parameters as initial ones. Overrides initial_parameters if given. Throws NotFittedError if set to True when not fitted.

  • kwargs – Not used currently, left here for passing debugging parameters.

Returns:

self (LinearLMESparseModel) – Fitted regression model.

fit_problem(problem: LinearProblem, initial_parameters: dict | None = None, warm_start=False, regularization_weights=None, **kwargs)

Fits the model to a provided problem

Parameters:

  • problem (LinearProblem) – an instance of LinearProblem that contains all data-dependent information

  • initial_parameters (np.ndarray) –

    Dict with possible fields:

    • ‘x0’ : np.ndarray, shape = [n],
      Initial estimate of fixed effects. If None then it defaults to an all-ones vector.

  • warm_start (bool, default is False) – Whether to use previous parameters as initial ones. Overrides initial_parameters if given. Throws NotFittedError if set to True when not fitted.

  • kwargs – Not used currently, left here for passing debugging parameters.

Returns:

self

instantiate() → Tuple[LinearOracle | None, Regularizer | None, PGDSolver | None]
predict(x, **kwargs)

Makes a prediction if .fit(X, y) was called before and throws an error otherwise.

Parameters:

  • x (np.ndarray) – Data matrix. Should have the same format as the data which was used for fitting the model: the number of columns and the columns’ labels should be the same. It may contain new groups, in which case the prediction will be formed using the fixed effects only.

  • columns_labels (Optional[List[int]]) – List of column labels. There shall be only one column of group labels and answers STDs, and overall n columns with fixed effects (1 or 3) and k columns of random effects (2 or 3).

    • 1 : fixed effect

    • 2 : random effect

    • 3 : both fixed and random,

    • 0 : groups labels

    • 4 : answers standard deviations

Returns:

y (np.ndarray) – Models predictions.

predict_problem(problem, **kwargs)

Makes a prediction if .fit was called before and throws an error otherwise.

Parameters:

  • problem (LMEProblem) – An instance of LMEProblem. Should have the same format as the data which was used for fitting the model. It may contain new groups, in which case the prediction will be formed using the fixed effects only.

  • kwargs – for passing debugging parameters

Returns:

y (np.ndarray) – Models predictions.

class pysr3.linear.models.LinearSCADModel(lam: float = 0.0, rho: float = 3.7, sigma: float = 1.0, tol_solver: float = 1e-05, max_iter_solver: int = 1000, stepping: str = 'line-search', logger_keys: Set = ('converged',), fixed_step_len=None, prior=None, **kwargs)

Bases: SimpleLinearModel

Initializes the model

Parameters:

  • lam (float) – strength of SCAD regularizer

  • rho (float, rho > 1) – first knot of the SCAD spline

  • sigma (float,) – a positive constant such that sigma*rho is the second knot of the SCAD spline

  • tol_solver (float) – tolerance for the stop criterion of PGD solver

  • max_iter_solver (int) – maximal number of iterations for PGD solver

  • stepping (str) – step-size policy for PGD. Can be either “line-search” or “fixed”

  • logger_keys (List[str]) – list of keys for the parameters that the logger should track

  • fixed_step_len (float) – step-size for PGD algorithm. If “linear-search” is used for stepping then the algorithm uses this value as the maximal step possible. Use this parameter if you know the Lipschitz-smoothness constant L for your problem as fixed_step_len=1/L.

  • prior (Optional[Prior]) – an instance of Prior class. If None then a non-informative prior is used.

  • kwargs – for passing debugging info

instantiate()
class pysr3.linear.models.LinearSCADModelSR3(lam: float = 0.0, rho: float = 2.0, sigma: float = 1.0, el: float = 1.0, tol_solver: float = 1e-05, max_iter_solver: int = 1000, stepping: str = 'line-search', logger_keys: Set = ('converged',), fixed_step_len=None, prior=None, practical=False, **kwargs)

Bases: SimpleLinearModelSR3

Initializes the model

Parameters:

  • lam (float) – strength of SCAD regularizer

  • rho (float, rho > 1) – first knot of the SCAD spline

  • sigma (float,) – a positive constant such that sigma*rho is the second knot of the SCAD spline

  • el (float) – constant for SR3 relaxation. Bigger values correspond to tighter relaxation.

  • tol_solver (float) – tolerance for the stop criterion of PGD solver

  • max_iter_solver (int) – maximal number of iterations for PGD solver

  • stepping (str) – step-size policy for PGD. Can be either “line-search” or “fixed”

  • logger_keys (List[str]) – list of keys for the parameters that the logger should track

  • fixed_step_len (float) – step-size for PGD algorithm. If “linear-search” is used for stepping then the algorithm uses this value as the maximal step possible. Use this parameter if you know the Lipschitz-smoothness constant L for your problem as fixed_step_len=1/L.

  • prior (Optional[Prior]) – an instance of Prior class. If None then a non-informative prior is used.

  • kwargs – for passing debugging info

instantiate()
class pysr3.linear.models.SimpleLinearModel(tol_solver: float = 1e-05, max_iter_solver: int = 1000, stepping: str = 'line-search', logger_keys: Set = ('converged',), fixed_step_len=None, prior=None, **kwargs)

Bases: LinearModel

Initializes the model

Parameters:

  • tol_solver (float) – tolerance for the stop criterion of PGD solver

  • max_iter_solver (int) – maximal number of iterations for PGD solver

  • stepping (str) – step-size policy for PGD. Can be either “line-search” or “fixed”

  • logger_keys (List[str]) – list of keys for the parameters that the logger should track

  • fixed_step_len (float) – step-size for PGD algorithm. If “linear-search” is used for stepping then the algorithm uses this value as the maximal step possible. Use this parameter if you know the Lipschitz-smoothness constant L for your problem as fixed_step_len=1/L.

  • prior (Optional[Prior]) – an instance of Prior class. If None then a non-informative prior is used.

  • kwargs – for passing debugging info

get_information_criterion(x, y, ic='bic')
instantiate()
class pysr3.linear.models.SimpleLinearModelSR3(el: float = 1.0, tol_solver: float = 1e-05, max_iter_solver: int = 1000, stepping: str = 'line-search', logger_keys: Set = ('converged',), fixed_step_len=None, prior=None, practical=False, **kwargs)

Bases: LinearModel

Initializes the model

Parameters:

  • el (float) – constant for SR3 relaxation. Bigger values correspond to tighter relaxation.

  • tol_solver (float) – tolerance for the stop criterion of PGD solver

  • max_iter_solver (int) – maximal number of iterations for PGD solver

  • stepping (str) – step-size policy for PGD. Can be either “line-search” or “fixed”

  • logger_keys (List[str]) – list of keys for the parameters that the logger should track

  • fixed_step_len (float) – step-size for PGD algorithm. If “linear-search” is used for stepping then the algorithm uses this value as the maximal step possible. Use this parameter if you know the Lipschitz-smoothness constant L for your problem as fixed_step_len=1/L.

  • prior (Optional[Prior]) – an instance of Prior class. If None then a non-informative prior is used.

  • practical (bool) – whether to use a direct efficient value function evaluation method

  • kwargs – for passing debugging info

get_information_criterion(x, y, ic='bic')
instantiate()