Model performance

Binary accuracy

class statinf.ml.performance.BinaryPerformance(y_true, y_pred)[source]

Bases: object

Gives detailed perfomance metrics for binary calssification models.

Parameters

y_true (numpy.ndarray) – Array of true targets.
y_pred (numpy.ndarray) – Array of predicted targets.

F1_score()[source]

F1-score

Formula: \[F_{1} = 2 \cdot \dfrac{precision \times recall}{precision + recall}\]
Returns: F1-score
Return type: float

accuracy()[source]

Binary accuracy of the model. Percentage of equal values between y_true and y_pred.

Formula: \[accuracy = \dfrac{TP + TN}{n}\]
Returns: Accuracy
Return type: float

confusion()[source]

Confusion matrix

Returns

Confusion matrix

	True 0	True 0
Predicted 0	\(TN\)	\(TN\)
Predicted 1	\(FP\)	\(TP\)

Return type

pandas.DataFrame

precision()[source]

Precision metric, proportion of actual 1 values amongst the ones predicted.

Formula: \[precision = \dfrac{TP}{TP + FP}\]
Returns: Precision
Return type: float

recall()[source]

Recall metric, proportion of values we predicted as one from the actuals ones.

Formula: \[recall = \dfrac{TP}{TP + FN}\]
Returns: Recall
Return type: float

statinf.ml.performance.mape(y_true, y_pred, weights=False)[source]

Computes the Mean Absolute Percentage Error (MAPE) or Weighted Mean Absolute Percentage Error (WMAPE).

Parameters

y_true (numpy.array) – Real values on which to compare.
y_pred (numpy.array) – Predicted values.
weights (bool) – Compute WMAPE.

Formula

\(MAPE(y, \hat{y}) = \dfrac{100}{n} \sum_{i=1}^{n} \dfrac{|y - \hat{y}|}{y}\)
\(WMAPE(y, \hat{y}) = 100 \dfrac{\sum_{i=1}^{n} {\dfrac{|y - \hat{y}|}{y}} \times y}{\sum_{i=1}^{n} y}\)

Returns: Mean Absolute Percentage Error as percentage.
Return type: float

statinf.ml.performance.mean_squared_error(y_true, y_pred, root=False)[source]

Mean Squared Error

Parameters

y_true (numpy.ndarray) – Real values on which to compare.
y_pred (numpy.ndarray) – Predicted values.
root (bool, optional) – Return Root Mean Squared Error (RMSE), defaults to False.

Formula

\(loss = \dfrac{1}{m} \times \sum_{i=1}^{m} (y_i - \hat{y}_i)^2\)

References

Friedman, J., Hastie, T. and Tibshirani, R., 2001. The elements of statistical learning. Ch. 2, pp. 24.

Returns

Mean Squared Error or its root.

Return type

float

Example

from statinf.ml.performance import BinaryPerformance

## Load the performance class
perf = BinaryPerformance(y_true=data[Y], y_pred=predicted)

## Model accuracy
print(perf.accuracy())
## Confusion matrix
print(perf.confusion())