pysr3.linear.oracles module

class pysr3.linear.oracles.LinearOracle(problem: LinearProblem | None = None, prior: Prior | None = None)

Bases: object

Implements a supplementary class that abstracts model. That is, it takes a problem and provides losses and gradients with respect to the parameters of the model.

It separates the model form the optimization routine for better code patterns. The solver takes an oracle and optimizes its loss using its gradient, but it does not know which model it optimizes. The oracle, in its turn, has no idea how the solution for its model will be obtained.

Initializes LinearOracle

Parameters:

  • problem (LinearProblem, optional) – an instance of LinearProblem containing the data

  • prior (Prior) – an instance of Prior for the models’ coefficients, if needed. See the docs for pysr3.priors module.

aic(x)

Calculates Akaike information criterion (AIC)

Parameters:

x (ndarray (num_features, )) – models parameters

Returns:

aic (float) – AIC

bic(x)

Calculates Bayess information criterion (BIC)

Parameters:

x (ndarray (num_features, )) – models parameters

Returns:

bic (float) – BIC

forget()

Detaches the problem from the oracle

gradient(x)

Calculates gradient of Gaussian negative log-likelihood with respect to the given set of models parameters x.

Parameters:

x (ndarray (num_features, )) – models parameters

Returns:

gradient (ndarray (num_features, )) – gradient

gradient_value_function(x)

Calculates gradient of the value function with respect to the given set of models parameters x. It’s the same as normal gradient if the oracle does not implement an SR3 relaxation.

Parameters:

x (ndarray (num_features, )) – models parameters

Returns:

gradient (ndarray (num_features, )) – gradient

hessian(x)

Calculates Hessian of Gaussian negative log-likelihood with respect to the given set of models parameters x.

Parameters:

x (ndarray (num_features, )) – models parameters

Returns:

hessian (ndarray (num_features, num_features)) – Hessian

instantiate(problem)

Attaches the given problem to the oracle

Parameters:

problem (LinearProblem) – instance of the problem

Returns:

oracle (LinearOracle) – oracle for this problem

loss(x)

Calculates Gaussian negative log-likelihood for the given set of models parameters x.

Parameters:

x (ndarray (num_features, )) – models parameters

Returns:

loss (float) – loss value

value_function(x)

Calculates value function for the given set of models parameters x. It’s the same as loss if the oracle does not implement an SR3 relaxation.

Parameters:

x (ndarray (num_features, )) – models parameters

Returns:

loss (float) – loss value

class pysr3.linear.oracles.LinearOracleSR3(problem: LinearProblem | None = None, lam: float = 1, practical: bool = False, prior: Prior | None = None)

Bases: object

Implements a supplementary class that abstracts SR3-model. That is, it takes a problem and provides losses and gradients with respect to the parameters of the model.

It separates the model form the optimization routine for better code patterns. The solver takes an oracle and optimizes its loss using its gradient, but it does not know which model it optimizes. The oracle, in its turn, has no idea how the solution for its model will be obtained.

Instantiates an oracle

Parameters:

  • problem (LinearProblem, optional) – an instance of LinearProblem containing the data

  • prior (Prior) – an instance of Prior for the models’ coefficients, if needed. See the docs for pysr3.priors module.

  • lam (float) – coefficient for the strength SR3-relaxation. It’s NOT the same as the regularization (sparsity) coefficient. See the paper for more details.

  • practical (bool) – whether to use an optimization method that is much faster than the default.

aic(x)

Calculates Akaike information criterion (AIC)

Parameters:

x (ndarray (num_features, )) – models parameters

Returns:

aic (float) – AIC

bic(x)

Calculates Bayess information criterion (BIC)

Parameters:

x (ndarray (num_features, )) – models parameters

Returns:

bic (float) – BIC

find_optimal_parameters(x0, regularizer=None, tol: float = 0.0001, max_iter: int = 1000, **kwargs)

Implements a “practical” optimization scheme that works faster than just a standard gradient descent. This function is meant to be called by pysr3.solvers.FakePGDSolver

Parameters:

  • x0 (ndarray (num_features, )) – starting point for the optimization

  • regularizer (Regularizer) – regularizer that implements sparsifiction prior.

  • tol (float) – tolerance for the solver

  • max_iter (int) – maximum number of iterations

  • kwargs – other keyword arguments

Returns:

x (ndarray (num_features, )) – the optimal solution

forget()

Detaches the problem from the oracle

gradient_value_function(x)

Calculates gradient of the value function with respect to the given set of models parameters x. It’s the same as normal gradient if the oracle does not implement an SR3 relaxation.

Parameters:

x (ndarray (num_features, )) – models parameters

Returns:

gradient (ndarray (num_features, )) – gradient

instantiate(problem)

Attaches the given problem to the oracle

Parameters:

problem (LinearProblem) – instance of the problem

Returns:

oracle (LinearOracleSR3) – oracle for this problem

loss(x, w)

Calculates Gaussian negative log-likelihood of SR3 relaxation for the given set of models parameters x.

Parameters:

  • x (ndarray (num_features, )) – models parameters

  • w (ndarray (num_features, )) – dual (relaxed) parameters that SR3-relaxation introduces

Returns:

loss (float) – loss value

value_function(x)

Calculates value function for the given set of models parameters x.

Parameters:

x (ndarray (num_features, )) – models parameters

Returns:

loss (float) – loss value