LLMEval is a public research initiative from Fudan NLP Lab focused on building comprehensive, fair, and robust evaluation frameworks for large language models. Its research examines how models perform across more than 13 academic disciplines, medical applications, and demanding reasoning tasks. The project develops benchmarks, datasets, evaluation pipelines, research papers, and a leaderboard, and makes selected resources available through GitHub and other research platforms.

Key offerings include LLMEval-Fair, a large-scale, longitudinal benchmark using graduate-level questions and contamination-resistant evaluation methods; LLMEval-Med, a physician-validated benchmark for clinical knowledge, reasoning, safety, and text generation; and LLMEval-Logic, a Chinese logical reasoning benchmark featuring solver-verified answers and adversarially hardened questions. LLMEval also shares evaluation code and data to support reproducible research and comparisons among language models. Its published studies assess dozens of models and investigate fairness, robustness, data contamination, and medical reliability. Researchers and interested members of the public can explore its open resources and contact the team about collaboration.