Home › Forums › AWS › AWS Certified Machine Learning – Specialty › Unsure of a question in ML exam (changing the cost function) › Reply To: Unsure of a question in ML exam (changing the cost function)
-
“The model can simply predict all to be negative (99.9 accurate) and have 10 false negatives.”
>> I understand that you’ve put it in simple terms but for the sake of others, I’d say that the prediction will favor the majority class but the predictions do not necessarily be distributed in True Negatives and False Negatives only.
“But if the test data is 5000 positive and 5000 negative the accuracy will fall to 50%. In that case, we should minimize FN and should be more tolerant of false positives.(Even if it guessed positive and it is not – it is ok,because real data might contain much much more)”
>> Both FN and FP reduces the accuracy of the model. In this case, we won’t know how many are FN or FP unless we have the prediction results. But for brevity’s sake, let’s consider the following prediction results:
TN = 400,000
TP = 75,000
FP = 10,000
FN = 15,000
Let’s say that for every predicted false positive, the company loses $5. The company does not lose anything for FN. We shouldn’t be more tolerant of false positives because it has more impact in terms of costs.