Dear members of the BirdVox discussion group,
First of all, let me wish you all a Happy New Year 2021!
It is my pleasure to announce the release of the fifth version of
the BirdVoxDetect software. I thank the β-testers of BirdVoxDetect
v0.4 and everyone who provided feedback since my last email from
November 2020, announcing the v0.4 series. Special thanks to
β-tester Phil Taylor who helped me chase bugs and corner cases for
BirdVoxDetect v0.5 :)
Below my signature is a detailed description of what has changed since BirdVoxDetect v0.4.
You may already install the β version of BirdVoxDetect v0.5 via the Python Package Index
pip install birdvoxdetect==0.5.0b1
Or from the source code: https://github.com/BirdVox/birdvoxdetect
We are actively looking for beta-testers, so please contact me on this discussion group or via the GitHub issue tracker to report feedback.
Once the most urgent feedback is addressed, we'll release BirdVoxDetect v0.5 in its "stable" form.
Sincerely,
Vincent.
I. Hierarchical consistency
In BirdVoxDetect v0.4, we had introduced a new species
classifier, named TaxoNet. The gist of TaxoNet is to perform
predictions at multiple levels of the avian taxonomy: order,
family, and species.
Our numerical experiments on 300 hours of annotated audio have
demonstrated that TaxoNet outperforms the classifier of
BirdVoxDetect v0.3 (see previous release notes for details).
However, we have noted that the output of TaxoNet is occasionally
inconsistent across levels. For example, the "family" level may
return "Turdidae" (Thrushes) while the "species" level would
return "WTSP", i.e. White-Throated Sparrow.
Although infrequent, this lack of consistency may be confusing to
users. To address it, we have implemented a new algorithm for
hierarchical classification which ensures consistency by design:
see https://github.com/BirdVox/birdvoxdetect/pull/73
The key idea is to query TaxoNet in a coarse-to-fine fashion,
while progressively narrowing down the list of candidate species
corresponding to the detected flight call. Thanks to Justin
Salamon for proposing this elegant solution.
We are currently in the process of evaluating the impact of hierarchical consistency on species classification at the full-season scale. We expect this impact to be neutral or slightly favorable. The main reason behind implementing hierarchical consistency wasn't so much classification accuracy than the principle of least surprise from a user's perspective.
II. TaxoNet confidence
We have taken the opportunity of rewriting the classify_species function (see #73) to enrich the information that is stored in the BirdVoxDetect checklist. In addition to textual labels for order, family, and species, we have added three new columns, corresponding to "order confidence", "family confidence" , and "species confidence".
As an example, here's the typical output of BirdVoxDetect
v0.5.0b1 with a file from the test suite:
Time (hh:mm:ss) |
Detection confidence (%) |
Order |
Order confidence (%) |
Family |
Family confidence (%) |
Species (4-letter code) |
Species confidence (%) |
00:00:08.78 |
70.14839172363280 |
Passeriforme |
92.61552095413210 |
Turdidae |
98.76762628555300 |
SWTH |
99.93199110031130 |
Compare with v0.4.1:
Time (hh:mm:ss) Species (4-letter code) Family Order Confidence (%)
00:00:08.78 SWTH Turdidae Passeriforme 70.14839
These values of confidence all range between 50% and 100% and provide insight on the prediction of TaxoNet. Users should be warned, however, that the percentages are not calibrated, and thus do not come with any probabilistic interpretation. Rather, they are simply the output values of the sigmoid units in TaxoNet. This is unlike "Detection confidence (%)", which is carefully calibrated to match the precision of the detector (on a precision--recall curve).
Do not hesitate to contact me for more details about these new columns in the checklist. I'd be happy to help.
--
You received this message because you are subscribed to the Google Groups "birdvox" group.
To unsubscribe from this group and stop receiving emails from it, send an email to birdvox+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/birdvox/ca991f15-5d37-876a-2f1b-4ecfbb73c8bc%40ls2n.fr.
For more options, visit https://groups.google.com/d/optout.