From d32204a3cff064f13e41c05e11f2352597fe942a Mon Sep 17 00:00:00 2001 From: zwhe99 Date: Thu, 25 Sep 2025 13:02:27 +0800 Subject: [PATCH 1/3] update readme --- README.md | 39 ++------------------------------------- 1 file changed, 2 insertions(+), 37 deletions(-) diff --git a/README.md b/README.md index 38970bb..6dfab9c 100644 --- a/README.md +++ b/README.md @@ -11,27 +11,9 @@
- -[![Data](https://img.shields.io/badge/Data-4d5eff?style=for-the-badge&logo=huggingface&logoColor=ffffff&labelColor)](https://huggingface.co/datasets/zwhe99/DeepMath-103K) -[![Model](https://img.shields.io/badge/Model-4d5eff?style=for-the-badge&logo=huggingface&logoColor=ffffff&labelColor)](https://huggingface.co/collections/zwhe99/deepmath-6816e139b7f467f21a459a9a) -[![Github](https://img.shields.io/badge/Code-000000?style=for-the-badge&logo=github&logoColor=white)](https://github.com/zwhe99/DeepMath) -[![GitHub Stars](https://img.shields.io/github/stars/zwhe99/DeepMath?style=for-the-badge&logo=github&logoColor=white&label=Stars&color=000000)](https://github.com/zwhe99/DeepMath) -[![arXiv](https://img.shields.io/badge/arXiv-2504.11456-b31b1b.svg?style=for-the-badge)](https://arxiv.org/abs/2504.11456)
-## πŸ”₯ News -- **May 8, 2025**: We found that 48 samples contained hints that revealed the answers. The relevant questions have now been revised to remove the leaked answers. -- **April 14, 2025**: We release **`DeepMath-103K`**, a large-scale dataset featuring challenging, verifiable, and decontaminated math problems tailored for RL and SFT. We open source: - -## πŸ“¦ Resource - -- πŸ€— Training data: [`DeepMath-103K`](https://huggingface.co/datasets/zwhe99/DeepMath-103K) -- πŸ€— Model weights: [`DeepMath-Zero-7B`](https://huggingface.co/zwhe99/DeepMath-Zero-7B), [`DeepMath-Zero-Math-7B`](https://huggingface.co/zwhe99/DeepMath-Zero-Math-7B), [`DeepMath-1.5B`](https://huggingface.co/zwhe99/DeepMath-1.5B), [`DeepMath-Omn-1.5B`](https://huggingface.co/zwhe99/DeepMath-Omn-1.5B) -- πŸ’» Code: [`DeepMath`](https://github.com/zwhe99/DeepMath) -- πŸ“ Paper: [`arXiv:2504.11456`](https://arxiv.org/abs/2504.11456) - - ## πŸ“– Overview **`DeepMath-103K`** is meticulously curated to push the boundaries of mathematical reasoning in language models. Key features include: @@ -73,7 +55,7 @@ The problems in DeepMath-103K are novel and unique, whereas many existing datase ## πŸ“ŠMain Results -DeepMath serise models achieve many **SOTA** results on challenging math benchmarks: +DeepMath serise models achieve strong results on challenging math benchmarks:
@@ -85,8 +67,6 @@ DeepMath serise models achieve many **SOTA** results on challenging math benchma #### Environment Preparation ```shell -git clone --recurse-submodules https://github.com/zwhe99/DeepMath.git && cd DeepMath - conda create -y -n deepmath python=3.12.2 && conda activate deepmath pip3 install ray[default] pip3 install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/cu124 @@ -162,19 +142,4 @@ This work can not be done without the help of the following works: - **[verl](https://github.com/volcengine/verl)**: A very fast reinforcement learning framework. - **[Vivacem/MMIQC](https://huggingface.co/datasets/Vivacem/MMIQC)**: A mixture of question-response pairs extracted from Mathematics Stack Exchange pages. - **[TIGER-Lab/WebInstructSub](https://huggingface.co/datasets/TIGER-Lab/WebInstructSub)**: Instruction data from MathStackExchange and ScienceStackExchange. -- **[AI-MO/NuminaMath-CoT](https://huggingface.co/datasets/AI-MO/NuminaMath-CoT)**: Approximately 860k math problems. - - - -## πŸ“š Citation -```bibtex -@article{deepmath, - title={DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoning}, - author={He, Zhiwei and Liang, Tian and Xu, Jiahao and Liu, Qiuzhi and Chen, Xingyu and Wang, Yue and Song, Linfeng and Yu, Dian and Liang, Zhenwen and Wang, Wenxuan and Zhang, Zhuosheng and Wang, Rui and Tu, Zhaopeng and Mi, Haitao and Yu, Dong}, - year={2025}, - eprint={2504.11456}, - archivePrefix={arXiv}, - primaryClass={cs.CL}, - url={https://arxiv.org/abs/2504.11456}, -} -``` +- **[AI-MO/NuminaMath-CoT](https://huggingface.co/datasets/AI-MO/NuminaMath-CoT)**: Approximately 860k math problems. \ No newline at end of file From 5844c28da6db07a595e8a5772c06aaadd094afe0 Mon Sep 17 00:00:00 2001 From: zwhe99 Date: Thu, 25 Sep 2025 13:08:43 +0800 Subject: [PATCH 2/3] Integrate verl into the main repository --- verl | 1 - 1 file changed, 1 deletion(-) delete mode 160000 verl diff --git a/verl b/verl deleted file mode 160000 index 05dc0fc..0000000 --- a/verl +++ /dev/null @@ -1 +0,0 @@ -Subproject commit 05dc0fc0b238ecd53c9125f5d7ae43cfbef1f48e From 09a038b0046dbd0c0cf2cb45d0222e2d3c739129 Mon Sep 17 00:00:00 2001 From: zwhe99 Date: Thu, 25 Sep 2025 13:11:41 +0800 Subject: [PATCH 3/3] minor --- .gitmodules | 4 ---- 1 file changed, 4 deletions(-) delete mode 100644 .gitmodules diff --git a/.gitmodules b/.gitmodules deleted file mode 100644 index adeeb18..0000000 --- a/.gitmodules +++ /dev/null @@ -1,4 +0,0 @@ -[submodule "verl"] - path = verl - url = https://github.com/zwhe99/verl.git - branch = deepmath