What gives 74% classification accuracy in TAPe on COCO?

This is the result on 5k val images without ImageNet pretraining, close to SOTA (79%). Achieved on synthetic data with 82% reconstruction.

Why transformers are not suitable for TAPe?

They distort pre-existing patch dependencies through attention, slow down convergence and require gradient descent.

How does TAPe generate synthetic data?

According to internal structure rules, without pixels, ensuring realistic dependencies without manual labeling.

TAPe embeddings: 74% on COCO without transformers

In the sixth day of TAPe diary, experiments on training embeddings on synthetic data with 74% COCO classification accuracy are described. 82% patch reconstruction achieved. Transformer problems identified and plans to transition to TAPe-native mechanisms.

By Sudonull Editorial Team • Published 2026-04-14 • Updated 2026-04-14

TAPe Diary, Day 6: Synthetic Embeddings and Ditching Transformers

TAPe breaks images into stable, structured features with predefined relationships. This yields a compact vector representation used for self-supervised learning and 80-class COCO classification.

Two Core Training Tasks

The model trains simultaneously on two tasks:

Embedding Generation: On synthetic TAPe data generated via internal rules. Unlike pixel-level generative models that fail to produce realistic object contours, this synthetic approach solves data scarcity.
Classification: Recognizing patches (single or multiple) as one of 80 COCO classes.

Synthetic data is created without manual labeling, preserving structural dependencies.

Compact COCO Dataset for Fast Convergence

Training uses 5K COCO validation images: 3.5K for training, 1.5K for validation. The val set is more challenging than the full train set (100K+ images) and ensures complete class coverage.

TAPe achieves fast convergence thanks to its extracted structure. Unlike YOLO, which requires ImageNet pretraining and 100K+ COCO images, TAPe works in reverse—starting small and scaling up.

Results: 82% Reconstruction and 74% Classification

Patch Reconstruction: 82% conditional accuracy—sufficient for downstream tasks.
Classification: 74% top-1 accuracy across 80 COCO classes, close to SOTA (79%).

This is just the first pass—optimization potential remains high.

Transformer Problems in TAPe Architecture

Current architecture uses standard transformers for patch interactions—a temporary setup for experiments. Key drawbacks:

Data Distortion: Attention (with three weight matrices QKV) imposes artificial interactive dependencies, disrupting TAPe’s inherent structure.
Slow Convergence: Full COCO training takes significant time.
Gradient Descent Dependency: Required for transformers, complicating optimization (RL alternatives aren’t ideal).

Transformers are useful for scale testing but not suitable for final TAPe design.

Moving Toward TAPe-Optimized Mechanisms

In TAPe, patch dependencies are predefined—attention is redundant. Next steps:

Replace transformers with native TAPe operations to preserve data structure.
Optimize without gradients, as done in earlier diary entries.
Scale to full COCO while maintaining efficiency (115K parameters vs. 2M+ in YOLO).

What Matters

74% accuracy on 5K COCO val without ImageNet pretraining.
Synthetic TAPe data eliminates need for pixel-level annotation.
82% reconstruction confirms embedding quality.
Dropping transformers: focus on TAPe structures to avoid distortion.
Efficiency: fast convergence on small datasets, low compute cost.

— Editorial Team

TAPe embeddings: 74% on COCO without transformers