Repository navigation
fix(metrics): return 1.0 for zero-entropy labels in HCVScore - #534
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #534 +/- ##
===========================================
+ Coverage 43.97% 87.06% +43.08%
===========================================
Files 85 100 +15
Lines 7281 9915 +2634
===========================================
+ Hits 3202 8632 +5430
+ Misses 4079 1283 -2796 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Thanks for the fix. The root cause is correct: Checked
Requested changes / questions Follow-ups, not blocking
Once 1–3 are addressed, this looks good to merge from my side. |
|
Cheers for merging @Mec-iS. The lint failure isn't from this PR - it's clippy's I'd only put a tolerance on the single-class case because that's where I'd seen the noise, which isn't much of a reason. I've done your clamp, but the way sklearn does it (MI comes back as 0 straight away if either side only has one label) rather than with an epsilon. That makes those zeros exact by construction, so the test checks them with Opened #535 for |
|
ok thanks. no problem, this will go all in v0.8.0 |
Checklist
Current behaviour
When a clustering puts every point in one cluster (DBSCAN with a big enough
epswill),HCVScoregives NaN for completeness and v-measure, where they should be 1.0 and 0.0. A ground truth with only one class goes the other way, with homogeneity coming out asinfand v-measure as NaN again. Both come down to dividing by an entropy of 0. Theunwrap_or(0f64)looks like it's there for that, butentropy()always returnsSome, so it never kicks in.New expected behaviour
Zero entropy scores 1.0, which is what scikit-learn does (
MI / entropy_C if entropy_C else 1.0), and whatcluster_hcv_single_cluster_no_panicintests_edge_cases.rsexpects too, though that file isn't compiled at the moment. With a single class the mutual information comes out at about 1e-16 rather than exactly 0, so completeness ends up at 1e-16 and not 0.0. I've left that alone, and the test allows for it.Change logs
Changed
HCVScorescores a zero entropy as 1.0 instead of NaN orinf