Hi there,I would like to better understand how the non-Binary ROC curves are computed.I'm trying to replicate realist in matlab, but results are very different from the one obtained with bayesialab.Here is what I'm currently doing:1- use batch inference from the database and produce a CSV with the poster probability of each datapoint.2 - for each row find the the state exhibiting the max posterior probability and use that as a score3 - select all prediction for one of the many target state, (i.e. a row of the confusion matrix) and compare them with their real state.4 - compute the roc curve (i.e. the ratio between true positive and false positive at a given threshold (i.e a value of PP)).I guess I'm doing something wrong somewhere but I'm not sure where :? .Any help would be appreciatedThanksAndrea
Quote 0 0
Dan
Hi Andrea,Here is the workflow:[*]Batch Inference to compute the posterior probabilities[*]Sort the data from the highest to the lowest posterior probabilities of your positive state[*]Let P be the actual number of positive states in your data set[*]Let S be the size of your data set[*]Let P_i be the sum of the True Positives when processing the first i sorted instances[*]Let N_i be the sum of the False Positives when processing the first i sorted instances[*]TPR_i = P_i/P[*]FPR_i = N_i/(S-P)Hope this helps,Dan
Quote 0 0