Introduction This is a follow-up post on my previous one [1] for debugging Computer Vision Image Classification models. The code snippets here are the primary modification to the initial one mentioned in [1] and [2]. Every Image Classification model has misclassifications of False Positives and False Negatives. However, to judge its…

In previous posts, it was discussed that Animal Camera Trap data has some data challenges like Illumination, Motion Blur, Small region of interest, Occlusion, Camouflage, Perspective, Weather Conditions, Camera Malfunctions, Temporal Changes, and Non-Animal Variability [1]. These are the images where the model had to resort to putting high feature…

Mega-Detector is a prestigious Microsoft product for object detection [1], well documented, and used by many animal-analytics (and other applications) companies [2]. Its latest version “MDv5” is even pre-trained on Animal CameraTrap data. There are multiple data challenges regarding Animal Camera Trap data like Illumination, Motion Blur, Small region of…

Decision analysis — This community generally works with discrete actions, possibly discrete random outcomes, but often features complex utility functions and the handling of risk. Problems are relatively small. • Stochastic search (derivative based) — This field is centered on the basic problem minx EF(x, W) where x is continuous…

Model Reliability in production is getting attention from AI leaders like Google and Facebook. Different terminologies are used to describe a model’s ability to generalize. In-Distribution (ID) Generalization We start with the simplest and most popular In-Distribution Generalization where a dataset is divided into Train/Val/Test sets and then performance measured on the Test…

There is no OOD (Out-Of-Distribution) benchmark dataset for Wildlife datasets like iNauralist, iWildCam, and their alikes. Thus image classifiers could take shortcuts by learning environments instead of animals [1]. CIFAR 10 Vs. CIFAR 100 are benchmark datasets for OOD performance evaluation. iNaturalist re-design analogous to CIFAR10Vs.100 is presented in this…

A high-performing model in Training/Validation/Test could still fail in production. Good “In-Distribution” performance is not enough to guarantee model’s reliability. I recommend that model’s performance must be evaluated on Out-Of-Distribution (OOD) and Domain Adaptation. Out-Of-Distribution (OOD ) Performance CIFAR 10 Vs. CIFAR 100 is the most known benchmark dataset for OOD performance evaluation. I…

State-of-the-art algorithms show excellent performance ONLY when tested at the same Camera Trap location where they were trained but generalization to new locations is poor [1]. In this post, image clustering is utilized to discover/unveil different environments in camera trap data. Terra-Incognita 6GB sample camera trap data CCT20 Benchmark subset…

A high-performing model in Training/Validation/Test could still fail in production. Image Classification Deep Learning models have been noticed to have reliability issues [1], [2], and [3]. As in [1], the argument of focus here is “Deep neural networks can be unreliable in the real world when the training set does…