Regularization¶
If there is one model that has a misfit that equals the desired tolerance, then there are infinitely many other models which can fit to the same degree. The challenge is to find that model which has the desired characteristics and is compatible with a priori information. A single model can be selected from an infinite ensemble by measuring the length, or norm, of each model. Then a smallest, or sometimes largest, member can be isolated. Our goal is to design a norm that embodies our prior knowledge and, when minimized, yields a realistic candidate for the solution of our problem. The norm can penalize variation from a reference model, spatial derivatives of the model, or some combination of these.
Tikhonov Regularization¶
Here we will define regularization of a model, m, in general however, this should be thought of as (mm_ref) but otherwise it is exactly the same:
Our discrete gradient operator works on cell centers and gives the derivative on the cell faces, which is not where we want to be evaluating this integral. We need to average the values back to the cellcenters before we integrate. To avoid null spaces, we square first and then average. In 2D with ij notation it looks like this:
If we let D_1 be the derivative matrix in the x direction
Where d_1 is the one dimensional derivative:
Recall that this is really a just point wise multiplication, or a diagonal matrix times a vector. When we multiply by something in a diagonal we can interchange and it gives the same results (i.e. it is point wise)
and the transpose also is true (but the sizes have to make sense...):
So R(m) can simplify to:
We will define W_x as:
And then W as a tall matrix of all of the different regularization terms:
Then we can write
The API¶

class
SimPEG.Regularization.
SimpleSmall
(mesh=None, **kwargs)[source]¶ Simple Small regularization  L2 regularization on the difference between a model and a reference model. Cell weights may be included. This does not include a volume contribution.
\[r(m) = \frac{1}{2}(\mathbf{m}  \mathbf{m_ref})^ op \mathbf{W}^T \mathbf{W} (\mathbf{m}  \mathbf{m_{ref}})\]where \(\mathbf{m}\) is the model, \(\mathbf{m_{ref}}\) is a reference model and \(\mathbf{W}\) is a weighting matrix (default Identity). If cell weights are provided, then it is
diag(np.sqrt(cell_weights))
)Optional Inputs
param BaseMesh mesh: SimPEG mesh param int nP: number of parameters param IdentityMap mapping: regularization mapping, takes the model from model space to the space you want to regularize in param numpy.ndarray mref: reference model param numpy.ndarray indActive: active cell indices for reducing the size of differential operators in the definition of a regularization mesh param numpy.ndarray cell_weights: cell weights Required Properties:
 cell_weights (
Array
): regularization weights applied at cell centers, a list or numpy array of <type ‘float’> with shape (*)  indActive (
Array
): indices of active cells in the mesh, a list or numpy array of <type ‘bool’>, <type ‘int’> with shape (*)  mapping (
IdentityMap
): mapping which is applied to model in the regularization, an instance of IdentityMap, Default: IdentityMap(,)  mref (
Array
): reference model, a numpy, Zero or Identity array of <type ‘float’>, <type ‘int’> with shape (*)  regmesh (
RegularizationMesh
): regularization mesh, an instance of RegularizationMesh

W
¶ Weighting matrix
 cell_weights (

class
SimPEG.Regularization.
SimpleSmoothDeriv
(mesh, orientation='x', **kwargs)[source]¶ Base Simple Smooth Regularization. This base class regularizes on the first spatial derivative, not considering length scales, in the provided orientation
Optional Inputs
param BaseMesh mesh: SimPEG mesh param int nP: number of parameters param IdentityMap mapping: regularization mapping, takes the model from model space to the space you want to regularize in param numpy.ndarray mref: reference model param numpy.ndarray indActive: active cell indices for reducing the size of differential operators in the definition of a regularization mesh param numpy.ndarray cell_weights: cell weights param bool mrefInSmooth: include the reference model in the smoothness computation? (eg. look at Deriv of m (False) or Deriv of (mmref) (True)) param numpy.ndarray cell_weights: vector of cell weights (applied in all terms) Required Properties:
 cell_weights (
Array
): regularization weights applied at cell centers, a list or numpy array of <type ‘float’> with shape (*)  indActive (
Array
): indices of active cells in the mesh, a list or numpy array of <type ‘bool’>, <type ‘int’> with shape (*)  mapping (
IdentityMap
): mapping which is applied to model in the regularization, an instance of IdentityMap, Default: IdentityMap(,)  mref (
Array
): reference model, a numpy, Zero or Identity array of <type ‘float’>, <type ‘int’> with shape (*)  mrefInSmooth (
Bool
): include mref in the smoothness calculation?, a boolean, Default: False  regmesh (
RegularizationMesh
): regularization mesh, an instance of RegularizationMesh

mrefInSmooth
¶ include mref in the smoothness calculation?

W
¶ Weighting matrix that takes the first spatial difference (no length scales considered) in the specified orientation
 cell_weights (

class
SimPEG.Regularization.
Simple
(mesh, alpha_s=1.0, alpha_x=1.0, alpha_y=1.0, alpha_z=1.0, **kwargs)[source]¶ Simple regularization that does not include length scales in the derivatives.
\[r(\mathbf{m}) = \alpha_s \phi_s + \alpha_x \phi_x + \alpha_y \phi_y + \alpha_z \phi_z\]where:
 \(\phi_s\) is a
SimPEG.Regularization.Small
instance  \(\phi_x\) is a
SimPEG.Regularization.SimpleSmoothDeriv
instance, withorientation='x'
 \(\phi_y\) is a
SimPEG.Regularization.SimpleSmoothDeriv
instance, withorientation='y'
 \(\phi_z\) is a
SimPEG.Regularization.SimpleSmoothDeriv
instance, withorientation='z'
Required Inputs
param BaseMesh mesh: a SimPEG mesh Optional Inputs
param IdentityMap mapping: regularization mapping, takes the model from model space to the space you want to regularize in param numpy.ndarray mref: reference model param numpy.ndarray indActive: active cell indices for reducing the size of differential operators in the definition of a regularization mesh param numpy.ndarray cell_weights: cell weights param bool mrefInSmooth: include the reference model in the smoothness computation? (eg. look at Deriv of m (False) or Deriv of (mmref) (True)) param numpy.ndarray cell_weights: vector of cell weights (applied in all terms) Weighting Parameters
param float alpha_s: weighting on the smallness (default 1.) param float alpha_x: weighting on the xsmoothness (default 1.) param float alpha_y: weighting on the ysmoothness (default 1.) param float alpha_z: weighting on the zsmoothness(default 1.) Required Properties:
 alpha_s (
Float
): smallness weight, a float, Zero or Identity  alpha_x (
Float
): weight for the first xderivative, a float, Zero or Identity  alpha_xx (
Float
): weight for the second xderivative, a float, Zero or Identity  alpha_y (
Float
): weight for the first yderivative, a float, Zero or Identity  alpha_yy (
Float
): weight for the second yderivative, a float, Zero or Identity  alpha_z (
Float
): weight for the first zderivative, a float, Zero or Identity  alpha_zz (
Float
): weight for the second zderivative, a float, Zero or Identity  cell_weights (
Array
): regularization weights applied at cell centers, a list or numpy array of <type ‘float’> with shape (*)  indActive (
Array
): indices of active cells in the mesh, a list or numpy array of <type ‘bool’>, <type ‘int’> with shape (*)  mapping (
IdentityMap
): mapping which is applied to model in the regularization, an instance of IdentityMap, Default: IdentityMap(,)  mref (
Array
): reference model, a numpy, Zero or Identity array of <type ‘float’>, <type ‘int’> with shape (*)  mrefInSmooth (
Bool
): include mref in the smoothness calculation?, a boolean, Default: False  regmesh (
RegularizationMesh
): regularization mesh, an instance of RegularizationMesh
 \(\phi_s\) is a

class
SimPEG.Regularization.
Small
(mesh=None, **kwargs)[source]¶ Small regularization  L2 regularization on the difference between a model and a reference model. Cell weights may be included. A volume contribution is included
\[r(m) = \frac{1}{2}(\mathbf{m}  \mathbf{m_ref})^ op \mathbf{W}^T \mathbf{W} (\mathbf{m}  \mathbf{m_{ref}})\]where \(\mathbf{m}\) is the model, \(\mathbf{m_{ref}}\) is a reference model and \(\mathbf{W}\) is a weighting matrix (default
diag(np.sqrt(vol))
. If cell weights are provided, then it isdiag(np.sqrt(vol * cell_weights))
)Optional Inputs
param BaseMesh mesh: SimPEG mesh param int nP: number of parameters param IdentityMap mapping: regularization mapping, takes the model from model space to the space you want to regularize in param numpy.ndarray mref: reference model param numpy.ndarray indActive: active cell indices for reducing the size of differential operators in the definition of a regularization mesh param numpy.ndarray cell_weights: cell weights Required Properties:
 cell_weights (
Array
): regularization weights applied at cell centers, a list or numpy array of <type ‘float’> with shape (*)  indActive (
Array
): indices of active cells in the mesh, a list or numpy array of <type ‘bool’>, <type ‘int’> with shape (*)  mapping (
IdentityMap
): mapping which is applied to model in the regularization, an instance of IdentityMap, Default: IdentityMap(,)  mref (
Array
): reference model, a numpy, Zero or Identity array of <type ‘float’>, <type ‘int’> with shape (*)  regmesh (
RegularizationMesh
): regularization mesh, an instance of RegularizationMesh

W
¶ Weighting matrix
 cell_weights (

class
SimPEG.Regularization.
SmoothDeriv
(mesh, orientation='x', **kwargs)[source]¶ Base Smooth Regularization. This base class regularizes on the first spatial derivative in the provided orientation
Optional Inputs
param BaseMesh mesh: SimPEG mesh param int nP: number of parameters param IdentityMap mapping: regularization mapping, takes the model from model space to the space you want to regularize in param numpy.ndarray mref: reference model param numpy.ndarray indActive: active cell indices for reducing the size of differential operators in the definition of a regularization mesh param numpy.ndarray cell_weights: cell weights param bool mrefInSmooth: include the reference model in the smoothness computation? (eg. look at Deriv of m (False) or Deriv of (mmref) (True)) param numpy.ndarray cell_weights: vector of cell weights (applied in all terms) Required Properties:
 cell_weights (
Array
): regularization weights applied at cell centers, a list or numpy array of <type ‘float’> with shape (*)  indActive (
Array
): indices of active cells in the mesh, a list or numpy array of <type ‘bool’>, <type ‘int’> with shape (*)  mapping (
IdentityMap
): mapping which is applied to model in the regularization, an instance of IdentityMap, Default: IdentityMap(,)  mref (
Array
): reference model, a numpy, Zero or Identity array of <type ‘float’>, <type ‘int’> with shape (*)  mrefInSmooth (
Bool
): include mref in the smoothness calculation?, a boolean, Default: False  regmesh (
RegularizationMesh
): regularization mesh, an instance of RegularizationMesh

mrefInSmooth
¶ include mref in the smoothness calculation?

W
¶ Weighting matrix that constructs the first spatial derivative stencil in the specified orientation
 cell_weights (

class
SimPEG.Regularization.
SmoothDeriv2
(mesh, orientation='x', **kwargs)[source]¶ Base Smooth Regularization. This base class regularizes on the second spatial derivative in the provided orientation
Optional Inputs
param BaseMesh mesh: SimPEG mesh param int nP: number of parameters param IdentityMap mapping: regularization mapping, takes the model from model space to the space you want to regularize in param numpy.ndarray mref: reference model param numpy.ndarray indActive: active cell indices for reducing the size of differential operators in the definition of a regularization mesh param numpy.ndarray cell_weights: cell weights param bool mrefInSmooth: include the reference model in the smoothness computation? (eg. look at Deriv of m (False) or Deriv of (mmref) (True)) param numpy.ndarray cell_weights: vector of cell weights (applied in all terms) Required Properties:
 cell_weights (
Array
): regularization weights applied at cell centers, a list or numpy array of <type ‘float’> with shape (*)  indActive (
Array
): indices of active cells in the mesh, a list or numpy array of <type ‘bool’>, <type ‘int’> with shape (*)  mapping (
IdentityMap
): mapping which is applied to model in the regularization, an instance of IdentityMap, Default: IdentityMap(,)  mref (
Array
): reference model, a numpy, Zero or Identity array of <type ‘float’>, <type ‘int’> with shape (*)  regmesh (
RegularizationMesh
): regularization mesh, an instance of RegularizationMesh

W
¶ Weighting matrix that takes the second spatial derivative in the specified orientation
 cell_weights (

class
SimPEG.Regularization.
Tikhonov
(mesh, alpha_s=1e06, alpha_x=1.0, alpha_y=1.0, alpha_z=1.0, alpha_xx=0.0, alpha_yy=0.0, alpha_zz=0.0, **kwargs)[source]¶ L2 Tikhonov regularization with both smallness and smoothness (first order derivative) contributions.
\[\phi_m(\mathbf{m}) = \alpha_s \ W_s (\mathbf{m}  \mathbf{m_{ref}} ) \^2 + \alpha_x \ W_x \frac{\partial}{\partial x} (\mathbf{m}  \mathbf{m_{ref}} ) \^2 + \alpha_y \ W_y \frac{\partial}{\partial y} (\mathbf{m}  \mathbf{m_{ref}} ) \^2 + \alpha_z \ W_z \frac{\partial}{\partial z} (\mathbf{m}  \mathbf{m_{ref}} ) \^2\]Note if the key word argument mrefInSmooth is False, then mref is not included in the smoothness contribution.
param BaseMesh mesh: SimPEG mesh param IdentityMap mapping: regularization mapping, takes the model from model space to the thing you want to regularize param numpy.ndarray indActive: active cell indices for reducing the size of differential operators in the definition of a regularization mesh param bool mrefInSmooth: (default = False) put mref in the smoothness component? param float alpha_s: (default 1e6) smallness weight param float alpha_x: (default 1) smoothness weight for first derivative in the xdirection param float alpha_y: (default 1) smoothness weight for first derivative in the ydirection param float alpha_z: (default 1) smoothness weight for first derivative in the zdirection param float alpha_xx: (default 1) smoothness weight for second derivative in the xdirection param float alpha_yy: (default 1) smoothness weight for second derivative in the ydirection param float alpha_zz: (default 1) smoothness weight for second derivative in the zdirection Required Properties:
 alpha_s (
Float
): smallness weight, a float, Zero or Identity  alpha_x (
Float
): weight for the first xderivative, a float, Zero or Identity  alpha_xx (
Float
): weight for the second xderivative, a float, Zero or Identity  alpha_y (
Float
): weight for the first yderivative, a float, Zero or Identity  alpha_yy (
Float
): weight for the second yderivative, a float, Zero or Identity  alpha_z (
Float
): weight for the first zderivative, a float, Zero or Identity  alpha_zz (
Float
): weight for the second zderivative, a float, Zero or Identity  cell_weights (
Array
): regularization weights applied at cell centers, a list or numpy array of <type ‘float’> with shape (*)  indActive (
Array
): indices of active cells in the mesh, a list or numpy array of <type ‘bool’>, <type ‘int’> with shape (*)  mapping (
IdentityMap
): mapping which is applied to model in the regularization, an instance of IdentityMap, Default: IdentityMap(,)  mref (
Array
): reference model, a numpy, Zero or Identity array of <type ‘float’>, <type ‘int’> with shape (*)  mrefInSmooth (
Bool
): include mref in the smoothness calculation?, a boolean, Default: False  regmesh (
RegularizationMesh
): regularization mesh, an instance of RegularizationMesh
 alpha_s (

class
SimPEG.Regularization.
SparseSmall
(mesh, norm=2, **kwargs)[source]¶ Sparse smallness regularization
Inputs
param int norm: norm on the smallness Required Properties:
 cell_weights (
Array
): regularization weights applied at cell centers, a list or numpy array of <type ‘float’> with shape (*)  epsilon (
Float
): Threshold value for the model norm, a float  gamma (
Float
): Model norm scaling to smooth out convergence, a float, Default: 1.0  indActive (
Array
): indices of active cells in the mesh, a list or numpy array of <type ‘bool’>, <type ‘int’> with shape (*)  mapping (
IdentityMap
): mapping which is applied to model in the regularization, an instance of IdentityMap, Default: IdentityMap(,)  model (
Array
): current model, a list or numpy array of <type ‘float’> with shape (*)  mref (
Array
): reference model, a numpy, Zero or Identity array of <type ‘float’>, <type ‘int’> with shape (*)  norm (
Float
): norm used, a float, Default: 2  regmesh (
RegularizationMesh
): regularization mesh, an instance of RegularizationMesh

f_m
¶

W
¶
 cell_weights (

class
SimPEG.Regularization.
SparseDeriv
(mesh, orientation='x', **kwargs)[source]¶  Base Class for sparse regularization on first spatial derivatives
Required Properties:
 cell_weights (
Array
): regularization weights applied at cell centers, a list or numpy array of <type ‘float’> with shape (*)  epsilon (
Float
): Threshold value for the model norm, a float  gamma (
Float
): Model norm scaling to smooth out convergence, a float, Default: 1.0  indActive (
Array
): indices of active cells in the mesh, a list or numpy array of <type ‘bool’>, <type ‘int’> with shape (*)  mapping (
IdentityMap
): mapping which is applied to model in the regularization, an instance of IdentityMap, Default: IdentityMap(,)  model (
Array
): current model, a list or numpy array of <type ‘float’> with shape (*)  mref (
Array
): reference model, a numpy, Zero or Identity array of <type ‘float’>, <type ‘int’> with shape (*)  mrefInSmooth (
Bool
): include mref in the smoothness calculation?, a boolean, Default: False  norm (
Float
): norm used, a float, Default: 2  regmesh (
RegularizationMesh
): regularization mesh, an instance of RegularizationMesh

mrefInSmooth
¶ include mref in the smoothness calculation?

f_m
¶

cellDiffStencil
¶

W
¶
 cell_weights (

class
SimPEG.Regularization.
Sparse
(mesh, alpha_s=1.0, alpha_x=1.0, alpha_y=1.0, alpha_z=1.0, **kwargs)[source]¶ The regularization is:
\[R(m) = \frac{1}{2}\mathbf{(mm_\text{ref})^\top W^\top R^\top R W(mm_\text{ref})}\]where the IRLS weight
\[R = \eta TO FINISH LATER!!!\]So the derivative is straight forward:
\[R(m) = \mathbf{W^\top R^\top R W (mm_\text{ref})}\]The IRLS weights are recomputed after each beta solves. It is strongly recommended to do a few GaussNewton iterations before updating.
Required Properties:
 alpha_s (
Float
): smallness weight, a float, Zero or Identity  alpha_x (
Float
): weight for the first xderivative, a float, Zero or Identity  alpha_xx (
Float
): weight for the second xderivative, a float, Zero or Identity  alpha_y (
Float
): weight for the first yderivative, a float, Zero or Identity  alpha_yy (
Float
): weight for the second yderivative, a float, Zero or Identity  alpha_z (
Float
): weight for the first zderivative, a float, Zero or Identity  alpha_zz (
Float
): weight for the second zderivative, a float, Zero or Identity  cell_weights (
Array
): regularization weights applied at cell centers, a list or numpy array of <type ‘float’> with shape (*)  eps_p (
Float
): Threshold value for the model norm, a float, Default: 0.1  eps_q (
Float
): Threshold value for the model gradient norm, a float, Default: 0.1  gamma (
Float
): Model norm scaling to smooth out convergence, a float, Default: 1.0  indActive (
Array
): indices of active cells in the mesh, a list or numpy array of <type ‘bool’>, <type ‘int’> with shape (*)  mapping (
IdentityMap
): mapping which is applied to model in the regularization, an instance of IdentityMap, Default: IdentityMap(,)  model (
Array
): current model, a list or numpy array of <type ‘float’> with shape (*)  mref (
Array
): reference model, a numpy, Zero or Identity array of <type ‘float’>, <type ‘int’> with shape (*)  mrefInSmooth (
Bool
): include mref in the smoothness calculation?, a boolean, Default: False  norms (
Array
): Norms used to create the sparse regularization, a list or numpy array of <type ‘float’>, <type ‘int’> with shape (*), Default: [2.0, 2.0, 2.0, 2.0]  regmesh (
RegularizationMesh
): regularization mesh, an instance of RegularizationMesh

norms
¶ Norms used to create the sparse regularization

eps_p
¶ Threshold value for the model norm

eps_q
¶ Threshold value for the model gradient norm

model
¶ current model

gamma
¶ Model norm scaling to smooth out convergence
 alpha_s (

class
SimPEG.Regularization.
RegularizationMesh
(mesh, **kwargs)[source]¶ Regularization Mesh
This contains the operators used in the regularization. Note that these are not necessarily true differential operators, but are constructed from a SimPEG Mesh.
param BaseMesh mesh: problem mesh param numpy.array indActive: bool array, size nC, that is True where we have active cells. Used to reduce the operators so we regularize only on active cells Required Properties:
 indActive (
Array
): active indices in mesh, a list or numpy array of <type ‘bool’>, <type ‘int’> with shape (*)

indActive
¶ active indices in mesh

vol
¶ reduced volume vector
Return type: numpy.array Returns: reduced cell volume

Pac
¶ projection matrix that takes from the reduced space of active cells to full modelling space (ie. nC x nindActive)
Return type: scipy.sparse.csr_matrix Returns: active cell projection matrix

Pafx
¶ projection matrix that takes from the reduced space of active xfaces to full modelling space (ie. nFx x nindActive_Fx )
Return type: scipy.sparse.csr_matrix Returns: active facex projection matrix

Pafy
¶ projection matrix that takes from the reduced space of active yfaces to full modelling space (ie. nFy x nindActive_Fy )
Return type: scipy.sparse.csr_matrix Returns: active facey projection matrix

Pafz
¶ projection matrix that takes from the reduced space of active zfaces to full modelling space (ie. nFz x nindActive_Fz )
Return type: scipy.sparse.csr_matrix Returns: active facez projection matrix

aveFx2CC
¶ averaging from active cell centers to active xfaces
Return type: scipy.sparse.csr_matrix Returns: averaging from active cell centers to active xfaces

aveCC2Fx
¶ averaging from active xfaces to active cell centers
Return type: scipy.sparse.csr_matrix Returns: averaging matrix from active xfaces to active cell centers

aveFy2CC
¶ averaging from active cell centers to active yfaces
Return type: scipy.sparse.csr_matrix Returns: averaging from active cell centers to active yfaces

aveCC2Fy
¶ averaging from active yfaces to active cell centers
Return type: scipy.sparse.csr_matrix Returns: averaging matrix from active yfaces to active cell centers

aveFz2CC
¶ averaging from active cell centers to active zfaces
Return type: scipy.sparse.csr_matrix Returns: averaging from active cell centers to active zfaces

aveCC2Fz
¶ averaging from active zfaces to active cell centers
Return type: scipy.sparse.csr_matrix Returns: averaging matrix from active zfaces to active cell centers

cellDiffx
¶ cell centered difference in the xdirection
Return type: scipy.sparse.csr_matrix Returns: differencing matrix for active cells in the xdirection

cellDiffy
¶ cell centered difference in the ydirection
Return type: scipy.sparse.csr_matrix Returns: differencing matrix for active cells in the ydirection

cellDiffz
¶ cell centered difference in the zdirection
Return type: scipy.sparse.csr_matrix Returns: differencing matrix for active cells in the zdirection

faceDiffx
¶ xface differences
Return type: scipy.sparse.csr_matrix Returns: differencing matrix for active faces in the xdirection

faceDiffy
¶ yface differences
Return type: scipy.sparse.csr_matrix Returns: differencing matrix for active faces in the ydirection

faceDiffz
¶ zface differences
Return type: scipy.sparse.csr_matrix Returns: differencing matrix for active faces in the zdirection

cellDiffxStencil
¶ cell centered difference stencil (no cell lengths include) in the xdirection
Return type: scipy.sparse.csr_matrix Returns: differencing matrix for active cells in the xdirection

cellDiffyStencil
¶ cell centered difference stencil (no cell lengths include) in the ydirection
Return type: scipy.sparse.csr_matrix Returns: differencing matrix for active cells in the ydirection

cellDiffzStencil
¶ cell centered difference stencil (no cell lengths include) in the ydirection
Return type: scipy.sparse.csr_matrix Returns: differencing matrix for active cells in the ydirection
 indActive (