# Reverse-Engineering TiinyAI Pocket Lab: The Real Architecture Behind the Marketing Hype
A researcher, using marketing images and renders of the TiinyAI Pocket Lab, figured out the device's actual hardware setup. The company claims it can run models up to 120B parameters at 20 tokens/s for $1299, but analysis reveals split memory and limited performance. The device connects via USB-C to a host and combines an SoC with an external NPU.
Identifying Key Components
The SoC in the Pocket Lab is the CIX P1 (CD8180) from CIX Technology. Specs:
- 12 Armv9.2 cores;
- Integrated NPU at 30 TOPS;
- Support for 128-bit LPDDR5X;
- PCIe Gen4 x4 via M.2;
- TSMC 6 nm process.
This chip handles general tasks and basic AI, and it's already used in mini-PCs. The discrete NPU at 160 TOPS is on an M.2 module with two chips, likely based on VeriSilicon's VIP9400 (80 TOPS per core). The software stack includes ACUITY and TIM-VX, which explains Tiiny's proprietary model format.
Total memory is 80 GB LPDDR5X, but it's split: 32 GB for the SoC and 48 GB for the dNPU. The pools are linked via PCIe Gen4 x4 (theoretical 8 GB/s, real-world 6–7 GB/s), creating a bottleneck for LLM inference. Local memory bandwidth is ~100 GB/s per side, but inter-component traffic is limited by the bus.
Performance Issues and Marketing Claims
The claimed 190 TOPS is the sum of 30 TOPS (SoC) + 160 TOPS (dNPU). The GPT-OSS-120B model (INT4) doesn't fit in the 48 GB dNPU, requiring activation transfers over PCIe. The model file gpt_oss_120b_layer_27_36.q4_0_4x64.gguf points to static layer segmentation between SoC and NPU.
TiinyAI relies on PowerInfer for speculative decoding of "hot"/"cold" neurons and TurboSparse. The demoed models are MoE with low active density (3–8B parameters per token):
- Qwen3-Coder-Next 80B 3B MoE;
- GLM-4.7-Flash 30B ~3B MoE;
- Qwen3-30B-A3B 30B 3B MoE;
- GPT-OSS-20B 21B 3.6B MoE;
- Qwen3-8B 8B 8B.
Benchmarks use short contexts (32 tokens) for peak 20 tok/s. Under real workloads:
| Context | Tokens/s |
|---------|----------|
| 8K | 12 |
| 16K | 9 |
| 32K | 6 |
| 64K | 4.47 |
Time-to-first-token is high, which is critical for RAG and agents. Comparison: RTX 4060 Ti (~$400) delivers 70–90 tok/s on 3B MoE at 100K+ context vs. 6–12 tok/s on Pocket Lab at 8–32K.
Architecture and Operating Mode
Pocket Lab isn't a standalone PC—it's a USB-C peripheral with Linux on ARM SoC. The host handles UI, model loading, and networking; the device provides an OpenAI-compatible API via a virtual network adapter. The 1 TB NVMe SSD with AES-256 is standard, not "bank-grade".
Full configuration:
- SoC CIX P1 + 32 GB LPDDR5X;
- M.2 dNPU (2x VIP9400) + 48 GB LPDDR5X;
- PCIe Gen4 x4 bridge;
- 1 TB NVMe SSD;
- Proprietary software with Tiiny format.
Key Takeaways
- Split memory limits inference on large models due to PCIe bottleneck;
- Claimed 120B@20t/s isn't achievable for dense models; reality is low-activity MoE;
- Device depends on host, yet marketed as a "supercomputer";
- Components are real (CIX P1, VeriSilicon IP), but performance doesn't match the hype;
- Lack of transparency from the company: minimal info on team, China ties despite US registration.
— Editorial Team
No comments yet.