pysr3.regularizers module

Various regularizers (L0, LASSO, CAD, SCAD, etc)

class pysr3.regularizers.CADRegularizer(rho, lam)

Bases: Regularizer

Implement Clipped Absolute Deviation (CAD) regularizer

Creates CAD regularizer.

Parameters:

  • rho (float) – constant that prevents values larger than it from being penalized.

  • lam (float) – strength of the regularizer

forget()

Unlinks all problem-dependent information from the regularizer.

Returns:

None

instantiate(weights=None, **kwargs)

Attach regularization weights

Parameters:

weights (ndarray) – individual weights for the regularizer’s coordinates.

Returns:

None

prox(x, alpha)

Return the value of the proximal operator evaluated at the point x and the step parameter alpha.

Parameters:

  • x (ndarray) – point.

  • alpha – step parameter.

Returns:

result of the application of the proximal operator to x

value(x)

Returns the value for the regularizer at the point x

Parameters:

x (ndarray) – point

Returns:

the value of the regularizer

class pysr3.regularizers.DummyRegularizer

Bases: Regularizer

Fake regularizer that has no effect.

prox(x, alpha)

Return the value of the proximal operator evaluated at the point x and the step parameter alpha.

Parameters:

  • x (ndarray) – point.

  • alpha – step parameter.

Returns:

result of the application of the proximal operator to x

value(x)

Returns the value for the regularizer at the point x

Parameters:

x (ndarray) – point

Returns:

the value of the regularizer

class pysr3.regularizers.ElasticRegularizer(eps=0, other_regularizer: Regularizer | None = None)

Bases: Regularizer

instantiate(weights, **kwargs)

Attaches weights to the regularizer.

Parameters:

kwargs – whatever is needed for the regularizer to work

Returns:

None

prox(x, alpha)

Return the value of the proximal operator evaluated at the point x and the step parameter alpha.

Parameters:

  • x (ndarray) – point.

  • alpha – step parameter.

Returns:

result of the application of the proximal operator to x

value(x)

Returns the value for the regularizer at the point x

Parameters:

x (ndarray) – point

Returns:

the value of the regularizer

class pysr3.regularizers.L0Regularizer(nnz=None)

Bases: Regularizer

Implements an L0-type regularizer, where the desired number of non-zero coordinates for features is given

Create the regularizer.

Parameters:

nnz (int) – desired number of non-zero features

forget()

Unlinks all problem-dependent information from the regularizer.

Returns:

None

instantiate(weights, **kwargs)

Attaches weights to the regularizer.

Parameters:

weights – regularization weights

Returns:

None

prox(x, alpha)

Return the value of the proximal operator evaluated at the point x and the step parameter alpha.

Parameters:

  • x (ndarray) – point.

  • alpha – step parameter.

Returns:

result of the application of the proximal operator to x

value(x)

Returns the value for the regularizer at the point x

Parameters:

x (ndarray) – point

Returns:

the value of the regularizer

class pysr3.regularizers.L0RegularizerLME(nnz_tbeta=None, nnz_tgamma=None, independent_beta_and_gamma=False, oracle: LinearLMEOracle | None = None)

Bases: Regularizer

Implements an L0-type regularizer, where the desired number of non-zero coordinates for fixed and random effects is given

Create the regularizer.

Parameters:

  • nnz_tbeta (int) – desired number of non-zero fixed effects

  • nnz_tgamma (int) – desired number of non-zero random effects

  • independent_beta_and_gamma (bool) – If true then we only can set an element of gamma as non-zero when the respective element of beta is non-zero too.

  • oracle (LinearLMEOracle) – class that encompasses the information about the problem

forget()

Unlinks all problem-dependent information from the regularizer.

Returns:

None

instantiate(weights, **kwargs)

Attaches weights to the regularizer.

Parameters:

weights – regularization weights

Returns:

None

optimal_tbeta(beta: ndarray)

Returns tbeta which minimizes the loss function with all other variables fixed.

It is a projection of beta on the sparse subspace with no more than k elements, which can be constructed by taking largest k elements from beta and setting the rest to be 0.

Parameters:

beta (np.ndarray, shape = [n]) – Vector of estimates of fixed effects.

Returns:

tbeta (np.ndarray, shape = [n]) – Minimizer of the loss function w.r.t tbeta with other arguments fixed.

optimal_tgamma(tbeta, gamma)

Returns tgamma which minimizes the loss function with all other variables fixed.

It is a projection of gamma on the sparse subspace with no more than nnz_gamma elements, which can be constructed by taking largest nnz_gamma elements from gamma and setting the rest to be 0. In addition, it preserves that for all the elements where tbeta = 0 it implies that tgamma = 0 as well.

Parameters:

  • tbeta (np.ndarray, shape = [n]) – Vector of (nnz_beta)-sparse estimates for fixed parameters.

  • gamma (np.ndarray, shape = [k]) – Vector of covariance estimates of random effects.

Returns:

tgamma (np.ndarray, shape = [k]) – Minimizer of the loss function w.r.t tgamma with other arguments fixed.

prox(x, alpha)

Return the value of the proximal operator evaluated at the point x and the step parameter alpha.

Parameters:

  • x (ndarray) – point.

  • alpha – step parameter.

Returns:

result of the application of the proximal operator to x

value(x)

Returns the value for the regularizer at the point x

Parameters:

x (ndarray) – point

Returns:

the value of the regularizer

class pysr3.regularizers.L1Regularizer(lam)

Bases: Regularizer

Implements an L1-regularizer, a.k.a. LASSO. N.B. Adaptive LASSO is implemented by providing custom weights.

Creates LASSO regularizer

Parameters:

lam (float) – strength of the regularizer

forget()

Unlinks all problem-dependent information from the regularizer.

Returns:

None

instantiate(weights, **kwargs)

Attach regularization weights

Parameters:

weights (ndarray) – individual weights for the regularizer’s coordinates.

Returns:

None

prox(x, alpha)

Return the value of the proximal operator evaluated at the point x and the step parameter alpha.

Parameters:

  • x (ndarray) – point.

  • alpha – step parameter.

Returns:

result of the application of the proximal operator to x

value(x)

Returns the value for the regularizer at the point x

Parameters:

x (ndarray) – point

Returns:

the value of the regularizer

class pysr3.regularizers.PositiveQuadrantRegularizer(other_regularizer: Regularizer | None = None)

Bases: Regularizer

instantiate(weights, oracle=None, **kwargs)

Attaches weights to the regularizer.

Parameters:

kwargs – whatever is needed for the regularizer to work

Returns:

None

prox(x, alpha)

Return the value of the proximal operator evaluated at the point x and the step parameter alpha.

Parameters:

  • x (ndarray) – point.

  • alpha – step parameter.

Returns:

result of the application of the proximal operator to x

value(x)

Returns the value for the regularizer at the point x

Parameters:

x (ndarray) – point

Returns:

the value of the regularizer

class pysr3.regularizers.Regularizer

Bases: object

Template class for regularizers

forget()

Unlinks all problem-dependent information from the regularizer.

Returns:

None

instantiate(**kwargs)

Attaches weights to the regularizer.

Parameters:

kwargs – whatever is needed for the regularizer to work

Returns:

None

prox(x, alpha)

Return the value of the proximal operator evaluated at the point x and the step parameter alpha.

Parameters:

  • x (ndarray) – point.

  • alpha – step parameter.

Returns:

result of the application of the proximal operator to x

value(x) → float

Returns the value for the regularizer at the point x

Parameters:

x (ndarray) – point

Returns:

the value of the regularizer

class pysr3.regularizers.SCADRegularizer(rho, sigma, lam)

Bases: Regularizer

Implements Smoothly Clipped Absolute Deviations (SCAD) regularizer.

Creates SCAD regularizer

Parameters:

  • rho (float, rho > 1) – first knot of the spline

  • sigma (float, sigma > 1) – sigma*rho is the second knot of the spline

  • lam (float, lambda > 1) – strength of the regularizer

forget()

Unlinks all problem-dependent information from the regularizer.

Returns:

None

instantiate(weights=None, **kwargs)

Attach regularization weights

Parameters:

weights (ndarray) – individual weights for the regularizer’s coordinates.

Returns:

None

prox(x, alpha)

Return the value of the proximal operator evaluated at the point x and the step parameter alpha.

Parameters:

  • x (ndarray) – point.

  • alpha – step parameter.

Returns:

result of the application of the proximal operator to x

value(x)

Returns the value for the regularizer at the point x

Parameters:

x (ndarray) – point

Returns:

the value of the regularizer