metrics
ADLStream.evaluation.metrics
Metrics factory.
accuracy(y, o)
¶
Accuracy score.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
y |
np.array |
Real values. |
required |
o |
np.array |
Predictions. |
required |
Returns:
Type | Description |
---|---|
float |
Mean accuracy. |
Source code in ADLStream/evaluation/metrics.py
def accuracy(y, o):
"""Accuracy score.
Args:
y (np.array): Real values.
o (np.array): Predictions.
Returns:
float: Mean accuracy.
"""
y, o = _preprocess_predictions(y, o)
return np.mean(y == o)
auc(y, o)
¶
Area under curve (AUC).
TODO Implement AUC
Parameters:
Name | Type | Description | Default |
---|---|---|---|
y |
np.array |
Real values. |
required |
o |
np.array |
Predictions. |
required |
Returns:
Type | Description |
---|---|
float |
AUC. |
Source code in ADLStream/evaluation/metrics.py
def auc(y, o):
"""Area under curve (AUC).
TODO Implement AUC
Args:
y (np.array): Real values.
o (np.array): Predictions.
Returns:
float: AUC.
"""
raise NotImplementedError
evaluate(metric, y, o, **kwargs)
¶
Compute a specific loss function given expected output and predicted output.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
metric |
str |
Loss function to use. |
required |
y |
list or np.array |
Target data. |
required |
o |
list or np.array |
Predictions. |
required |
Returns:
Type | Description |
---|---|
float |
loss |
Source code in ADLStream/evaluation/metrics.py
def evaluate(metric, y, o, **kwargs):
"""Compute a specific loss function given expected output and predicted output.
Args:
metric (str): Loss function to use.
y (list or np.array): Target data.
o (list or np.array): Predictions.
Returns:
float: loss
"""
assert metric.upper() in METRICS, "Metric {} not supported.".format(metric)
o = np.asarray(o) if type(o) == type([]) else o
y = np.asarray(y).reshape(o.shape) if type(y) == type([]) else y
return METRICS[metric.upper()](y, o, **kwargs)
kappa(y, o)
¶
Cohen kappa score.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
y |
np.array |
Real values. |
required |
o |
np.array |
Predictions. |
required |
Returns:
Type | Description |
---|---|
float |
Kappa. |
Source code in ADLStream/evaluation/metrics.py
def kappa(y, o):
"""Cohen kappa score.
Args:
y (np.array): Real values.
o (np.array): Predictions.
Returns:
float: Kappa.
"""
confusion = _confusion_matrix(y, o)
n_classes = confusion.shape[0]
if n_classes < 2:
return 1
sum0 = np.sum(confusion, axis=0)
sum1 = np.sum(confusion, axis=1)
expected = np.outer(sum0, sum1) / np.sum(sum0)
w_mat = np.ones([n_classes, n_classes], dtype=np.int)
w_mat.flat[:: n_classes + 1] = 0
k = np.sum(w_mat * confusion) / np.sum(w_mat * expected)
return 1 - k
mae(y, o)
¶
Mean absolute error (MAE).
Parameters:
Name | Type | Description | Default |
---|---|---|---|
y |
np.array |
Real values. |
required |
o |
np.array |
Predictions. |
required |
Returns:
Type | Description |
---|---|
float |
MAE. |
Source code in ADLStream/evaluation/metrics.py
def mae(y, o):
"""Mean absolute error (MAE).
Args:
y (np.array): Real values.
o (np.array): Predictions.
Returns:
float: MAE.
"""
return np.mean(np.abs(o - y))
mape(y, o)
¶
Mean Absolute Percentage Error (MAPE).
TODO Implement MAPE
Parameters:
Name | Type | Description | Default |
---|---|---|---|
y |
np.array |
Real values. |
required |
o |
np.array |
Predictions. |
required |
Returns:
Type | Description |
---|---|
float |
MAPE. |
Source code in ADLStream/evaluation/metrics.py
def mape(y, o):
"""Mean Absolute Percentage Error (MAPE).
TODO Implement MAPE
Args:
y (np.array): Real values.
o (np.array): Predictions.
Returns:
float: MAPE.
"""
raise NotImplementedError
wape(y, o)
¶
Weighted Absolute Percentage Error (WAPE).
TODO Implement WAPE
Parameters:
Name | Type | Description | Default |
---|---|---|---|
y |
np.array |
Real values. |
required |
o |
np.array |
Predictions. |
required |
Returns:
Type | Description |
---|---|
float |
WAPE. |
Source code in ADLStream/evaluation/metrics.py
def wape(y, o):
"""Weighted Absolute Percentage Error (WAPE).
TODO Implement WAPE
Args:
y (np.array): Real values.
o (np.array): Predictions.
Returns:
float: WAPE.
"""
raise NotImplementedError