Neural Network initilization

statinf.ml.initializations.init_params(rows, cols, method='xavier', mean=0.0, std=1.0, tensor=True, seed=None)[source]

Initialize the weight and bias matrices based on probabilistic distribution.

Parameters
  • rows (int) – Size of the input, number of rows to be generated.

  • cols (int) – Size of the output, number of columns to be generated.

  • method (str, optional) – Distibution to use to generated the weights, defaults to ‘xavier’.

  • mean (float, optional) – Mean for the distribution to be generated, defaults to 0.

  • std (float, optional) – Standard deviation for the distribution to be generated, defaults to 1.

  • tensor (bool, optional) – Needs to return a theano friendly-format, defaults to True.

  • seed (int, optional) – Seed to be set for randomness, defaults to None.

Raises

ValueErrormethod needs to be ‘ones’, ‘zeros’, ‘uniform’, ‘xavier’ or ‘normal’, see below for details.

Method
  • Zeros: \(W_j = \vec{0}\)

  • Ones: \(W_j = \vec{1}\)

  • Uniform: \(W_j \sim \mathcal{U} _{\left[0, 1 \right)}\)

  • Xavier: \(W_j \sim \mathcal{U}\left[ -\frac{\sqrt{6}}{\sqrt{n_j + n_{j+1}}}, \frac{\sqrt{6}}{\sqrt{n_j + n_{j+1}}} \right]\)

  • Normal: \(W_j \sim \mathcal{N}(0, 1)\)

References
Returns

Weight or bias matrix for initiating ML optimization

Return type

numpy.array