Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update the metric module #702

Open
nudles opened this issue May 17, 2020 · 4 comments
Open

Update the metric module #702

nudles opened this issue May 17, 2020 · 4 comments

Comments

@nudles
Copy link
Member

nudles commented May 17, 2020

It would be better to implement a metric as a function in the metric.py as metric typically has not states. Therefore, no need to make it as a class. e.g,

def accuracy(y_pred, y_true):
   """compute the accuracy.

     Args:
        y_pred(numpy array or tensor): each value is a label index
        y_true(numpy array or tensor): each value is a label index
   """
      check shape match
      convert y_pred and y_true to np array     
      return np.sum(y_pred== y_true) / y_true.shape[0]

Refer to https://keras.io/api/metrics/

@nudles
Copy link
Member Author

nudles commented May 19, 2020

Or we can convert SINGA Tensor to numpy array and use sklearn.metrics to compute the metrics https://scikit-learn.org/stable/modules/model_evaluation.html.

@chrishkchris
Copy link
Contributor

chrishkchris commented May 19, 2020

When we use this accuracy:
y_true (each value is a label index) is provided directly by the dataset
y_pred (each value is a label index) needs to be obtained by applying max function to the network output (i.e. logit) BEFORE the softmaxCrossEntropy

so we need a max function to turn the logits into pred label index?

@nudles
Copy link
Member Author

nudles commented Jun 18, 2020

I suggest to use sklearn's functions to evaluate the metrics.
We can them remove the metric.py module.

@chrishkchris
Copy link
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants