Summarize Evaluation Model

machine learning

supervised learning

evaluation

Key Concepts: TP, FP, FN, TN

For a given class (say “Road”):

• True positives (TP): Predicted positive and are actually positive
• False positives (FP): Predicted positive and are actually negative
• True negatives (TN): Predicted negative and are actually negative
• False negatives (FN): Predicted negative and are actually positive

Metrics Explained (with intuition)

Accuracy

Meaning: Fraction of correctly classified pixels over all pixels.

Formula: Accuracy= (TP+TN)/(TP+TN+FP+FN)

Interpretation:

• How many pixels the model got right overall.
• Can be misleading if some classes dominate (e.g. background pixels).

Layman Example: If 95% of your image is background and the model always predicts “background,” accuracy will be 95%, even if it misses all buildings or roads.

---

Precision

Meaning: Of all pixels predicted as this class, how many are actually correct? and it focuses on the false positives.

Formula: Precision=TP​/(TP+FP)

Interpretation:

• High precision means few false alarms.
• The model predicts a class only when it is confident.

Layman Example: If you predict 100 “road” pixels and 90 are truly roads, precision = 0.9.

---

Recall (Sensitivity)

Meaning: Of all true pixels of this class, how many did the model correctly find? and it focuses on the false negatives.

Formula: Recall=TP/(TP+FN​)

Interpretation:

• High recall means most real objects are detected.
• May include more incorrect predictions.

Layman Example: If there are 100 true “road” pixels and you correctly predict 90 of them, recall = 0.9.

---

F1 Score

Meaning: Balanced measure combining precision and recall.

Formula: F1 = 2×(Precision×Recall)/(Precision+Recall)​

Interpretation:

• High only when both precision and recall are high.
• Drops sharply if either precision or recall is low.
• Useful as a single overall metric. Good overall measure when you need both accuracy and completeness.

Layman Example:

• Precision = 0.9, Recall = 0.9 → F1 = 0.9
• Precision = 0.9, Recall = 0.5 → F1 = 0.64 — big drop.

---

IoU (Intersection over Union)

Meaning: How much overlap is there between predicted and actual pixels for a class?

Formula: IoU = TP/(TP+FP+FN)​

Interpretation:

• Compares overlap between predicted mask and ground truth mask.
• IoU is the overlap area ÷ combined area.
• Higher IoU → better overlap → more accurate mask.

Layman Example: If your model’s “road” mask overlaps 80% with the true “road” mask, IoU = 0.8.

---

mIoU (Mean Intersection over Union)

Meaning: The average IoU across all classes. Formula: 𝑚𝐼𝑜𝑈=1𝑁∑𝑖=1𝑁𝐼𝑜𝑈𝑖

Interpretation:

• Compute IoU separately for each class (road, building, vegetation, etc.).
• Take the average across all classes.
• Standard benchmark metric in semantic segmentation.

Layman Example: If IoU for road = 0.8, vegetation = 0.7, building = 0.6 → mIoU = 0.7.

---

Dice Coefficient (a.k.a. F1 Score for segmentation)

Meaning: Measures how well the predicted mask overlaps the true mask, giving more weight to overlap than IoU.

Formula: Dice=2TP/(2TP+FP+FN)

Interpretation:

• Equivalent to F1 score at the pixel level.
• More sensitive to overlap than IoU.
• Especially effective for small or imbalanced objects.

Layman Example:

• If predicted and true masks mostly overlap, Dice is close to 1.
• If they barely overlap, Dice approaches 0.

More Deep explain for Metrics