TAPe Achieves 98% Object Detection Accuracy on COCO with Just 2% Labeled Data
The TAPe approach achieved 98% accuracy on just 2% of the COCO dataset (around 2,400 images). False positives were reduced to 30 per 1,500 images. Transitioning from centroids to bounding boxes significantly improved visual detection precision.
TAPe (Theory of Active Perception) transforms images into structured patches with defined relationships, replacing raw pixels with compact vectors. This enables efficient application of ML techniques—clustering, classification, and detection—without heavy self-supervised pretraining.
Optimizing TAPe Data Collection
The key experiment used inverted pyramids: focusing on the central TAPe patch while expanding scale toward image edges. This boosted accuracy to 98%.
Tested parameters included:
- Number of prototypes per class (optimal: 2).
- Balance between background and object patches (twice as many background patches for better discrimination).
Too much background leads to overfitting on background; too little causes missed objects.
Prototype and Background Balance in the Model
For each class, two prototypes are optimal—providing diversity without overfitting. More prototypes blur embedding clusters.
Background-to-object balance:
| Ratio | Accuracy | False Positives |
|-------|----------|------------------|
| 1:1 | 92% | High |
| 2:1 (background) | 98% | Low |
| 4:1 (background) | 95% | Overfitting on background |
Makes sense: background is less variable, so more examples are needed for reliable identification.
From Centroids to Bounding Boxes
Earlier detection focused on centroids (average accuracy: 72%). Now, the model generates bounding boxes around objects. Visual results are promising, with quantitative evaluation ongoing.
Experiments on "views" (scanning):
- Corners + center of the image are sufficient for detection without classification.
- Textured classes (zebra, piano, boat) achieve 93–94% success due to rich texture cues.
- Weak classes: forks (small size), people (high pose/viewpoint variability).
Variability is addressed via augmentation based on TAPe logic.
Integrating TAPe with Embeddings
TAPe patches act as natural fixed-size tokens, unlike pixel-based methods. An image becomes a sequence of patches with predefined structural relationships.
This simplifies tasks like iBOT: attention works without global mechanisms or gradient descent on raw data.
Traditional embedding training involves two stages:
- Spatial separation (divergence of vectors).
- Convergence within a single class.
TAPe replaces both stages with a single training process on structured objects. Enables expressing TAPe operations directly in embedding space for hybrid tasks.
Key Takeaways
- 98% accuracy on 2% of COCO without classification—thanks to inverted pyramids and balanced background-to-object ratio.
- Bounding boxes instead of centroids: average accuracy rose from 72% to up to 94% on textured classes.
- Two prototypes per class + 2:1 background-to-object ratio = optimal configuration.
- False positives reduced to 30 per 1,500 images.
- TAPe patches as tokens simplify attention without global gradients.
Overall, the experimental scope confirms TAPe’s effectiveness in resource-constrained computer vision scenarios.
— Editorial Team
No comments yet.