# AI Models Master Scientific Taste: 30B-Parameter Model Outperforms GPT-5.2 in Predicting Citability
Researchers from Fudan University and OpenMOSS have developed the Scientific Judge language model, capable of assessing the potential of scientific papers based on their abstracts. Instead of manual expert labeling, citation data was used as a natural indicator of value. The model with 30 billion parameters achieved 80.6% accuracy in pairwise abstract comparison, surpassing GPT-5.2 and Gemini 3 Pro.
A dataset of 700 thousand pairs of articles from 2.1 million arXiv publications was compiled. Pairs were formed based on criteria: published at the same time, same field. The task is to predict which article will receive more citations. This approach allows the model to simulate the 'voting' of the scientific community without subjective evaluations.
Generalization Beyond Training Data
Scientific Judge demonstrates strong generalizability. Trained on articles up to 2024, the model accurately predicts citability of 2025 publications. Training only on Computer Science materials transfers to physics, mathematics, and biology.
The skill of assessing citability is applied to conference reviewer scores like ICLR — accuracy reaches 87.7%. This indicates capture of universal features of 'quality' science, not domain-specific patterns.
- Cross-domain transferability: CS → physics, mathematics, biology.
- Temporal extrapolation: predictions for 2025 on data up to 2024.
- Application to reviews: 87.7% on ICLR datasets.
Scientific Thinker: Idea Generation with Self-Criticism
Based on Scientific Judge, the Scientific Thinker model was created for generating research continuations. Judge acts as an internal critic: from several ideas, it selects the most promising one, guiding Thinker's training.
SciThinker-30B outperforms the base model in 81.5% of cases and is comparable to GPT-5.2. The process includes:
- Generating several ideas based on the abstract.
- Evaluation by Judge on citation potential.
- Selection and fine-tuning on top ideas.
This self-improvement cycle allows the model to focus on high-potential directions without external feedback.
Limitations and Prospects
Citations are an imperfect proxy for scientific value: subject to biases, delays, and network effects. The ideas proposed by SciThinker have not undergone experimental validation. Nevertheless, the approach proves that 'scientific taste' can be extracted from aggregated community data.
Prospects include refining proxy metrics (e.g., combining with patents, implementations) and integration into review pipelines. For AI developers, this opens the way to creating specialized judges for other domains.
Key Takeaways
- Scientific Judge (30B) — 80.6% accuracy in pairwise ranking by future citations, better than GPT-5.2.
- Generalization: cross-domain and out-of-time predictions with preserved performance.
- SciThinker uses Judge for self-critique, achieving parity with top models in idea generation.
- Training on 700K pairs from arXiv — a cheap way to simulate expert taste.
- Limitation: citations as proxy require cautious application.
— Editorial Team
No comments yet.