Neural Network initilization

statinf.ml.initializations.init_params(rows, cols, method='xavier', mean=0.0, std=1.0, key=DeviceArray([4146024105, 967050713], dtype=uint32))[source]

Initialize the weight and bias matrices based on probabilistic distribution.

Parameters

• rows (int) – Size of the input, number of rows to be generated.

• cols (int) – Size of the output, number of columns to be generated.

• method (str, optional) – Distibution to use to generated the weights, defaults to ‘xavier’.

• mean (float, optional) – Mean for the distribution to be generated, defaults to 0.

• std (float, optional) – Standard deviation for the distribution to be generated, defaults to 1.

• tensor (bool, optional) – Needs to return a theano friendly-format, defaults to True.

• seed (int, optional) – Seed to be set for randomness, defaults to None.

Raises

ValueError – method needs to be ‘ones’, ‘zeros’, ‘uniform’, ‘xavier’ or ‘normal’, see below for details.

Method

• Zeros: \(W_j = \vec{0}\)

• Ones: \(W_j = \vec{1}\)

• Uniform: \(W_j \sim \mathcal{U} _{\left[0, 1 \right)}\)

• Xavier: \(W_j \sim \mathcal{U}\left[ -\frac{\sqrt{6}}{\sqrt{n_j + n_{j+1}}}, \frac{\sqrt{6}}{\sqrt{n_j + n_{j+1}}} \right]\)

• Normal: \(W_j \sim \mathcal{N}(0, 1)\)

References

• Neuneier, Ralph, and Hans Georg Zimmermann. “How to train neural networks” In Neural networks: tricks of the trade, pp. 373-423. Springer, Berlin, Heidelberg, 1998.

• Glorot, Xavier, and Yoshua Bengio. “Understanding the difficulty of training deep feedforward neural networks” In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp. 249-256. 2010.

Returns

Weight or bias matrix for initiating ML optimization

Return type

numpy.array