TIMPS-Coder-7B: A Fine-Tuned Code Generation Model via SFT, GRPO, DPO, and Self-Play

Abstract

TIMPS-Coder-7B is a 7-billion-parameter code generation model produced by fine-tuning Qwen2.5-Coder-7B-Instruct through a four-stage post-training pipeline: supervised fine-tuning (SFT), Group Relative Policy Optimization (GRPO) with a tool-discipline reward, Direct Preference Optimization (DPO), and Self-Guided Self-Play (SGS). On MBPP+ pass@1 — the benchmark we can verify is free of train/test overlap — TIMPS-Coder-7B scores 73.3%, a 3.7 percentage-point improvement over its base model.

On HumanEval pass@1, TIMPS-Coder-7B scores 98.8% and on HumanEval+ pass@1 it scores 82.9%. We disclose in Section 6 that these two figures are affected by a data-contamination issue in our own SFT pipeline: the SFT stage trains directly on all 164 HumanEval test-set problems and their canonical solutions. We report both numbers for completeness but do not present them as a state-of-the-art claim, and we recommend MBPP+ as the primary indicator of the model's real capability. This paper documents the full training pipeline, the evaluation protocol, comparative results against nine other open code models, the contamination issue, and a separate known artifact in the raw MBPP metric — all surfaced transparently rather than omitted.

1. Introduction

Instruction-tuned code models in the 7B–9B parameter range are attractive because they run on a single consumer or prosumer GPU while remaining competitive with much larger models on standard code-generation benchmarks. Qwen2.5-Coder-7B-Instruct is one of the strongest open baselines in this class. TIMPS-Coder-7B starts from that baseline and applies a further round of post-training aimed at improving pass@1 correctness on function-level code generation tasks.

The goal of this paper is to give a complete, reproducible account of what was done — not just the headline numbers, and not just the flattering ones. Section 6 in particular documents a mistake in our own training-data pipeline rather than omitting it.

2. Base Model

Qwen2.5-Coder-7B-Instruct was selected as the base model because it was, prior to this work, among the strongest open 7B-class code models on HumanEval and MBPP, giving a high but improvable baseline to fine-tune against (86.6% HumanEval pass@1, 71.3% HumanEval+ pass@1).

3. Training Pipeline

TIMPS-Coder-7B was produced through four sequential post-training stages applied on top of the Qwen2.5-Coder-7B-Instruct checkpoint, all run on Kaggle Notebooks using a dual-T4 GPU accelerator (2×15GB, primary device used for training). LoRA rank was pinned at 64 (alpha = 64, dropout 0.05) across every stage — using Rank-Stabilized LoRA (RSLoRA) for RL-stage stability — targeting all linear projections in the transformer block (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj). Sequence length was 2048 tokens on T4.

3.1 Stage 1 — Supervised Fine-Tuning (SFT)

SFT warms the base model up on a THINK → INSPECT → ACT → VERIFY response format, teaching it the tool-use structure used throughout the rest of the pipeline before any preference-based optimization is applied.

• Dataset composition: a mixture of openai/openai_humaneval (164 problems, full test split), google-research-datasets/mbpp (train split, first 500 of 974 rows), princeton-nlp/SWE-bench_Verified (first 300 of ~500 rows), newfacade/LeetCodeDataset (streamed, capped at 500 rows), and WaltonFuture/agentic-sft-new (streamed, capped at 500 rows).
• LoRA configuration: rank 64, alpha 64, dropout 0.05, target modules listed above, RSLoRA enabled.
• Learning rate / schedule: 2e-5, cosine schedule, weight decay 0.01.
• Batch size: 1 per device, gradient accumulation 16 (effective batch 16) on T4.
• Steps: up to 750 (full run) or 400 (fast/smoke-test mode).
• Hardware: Kaggle T4×2 (one T4 as primary device), Unsloth for 2× training speed and gradient checkpointing.

Important caveat, expanded in Section 6: the HumanEval component of this dataset is the exact 164-problem test split used later for evaluation, paired with canonical solutions. The MBPP component uses only the disjoint train split, so it does not share this issue.

3.2 Stage 2 — Group Relative Policy Optimization (GRPO)

GRPO generates a group of sampled completions per prompt and updates the policy toward the relatively higher-scoring completions within that group, rather than against a separately trained reward model.

• Reward signal: a weighted combination of three components rather than pass rate alone — Correctness (weight 0.5, binary: did the code pass its tests), Tool Discipline (weight 0.3: did the model inspect the codebase via read_file / search_code / inspect_error before writing code), and Verification (weight 0.2: did the model run tests or a linter after writing code). This rewards the process of writing correct code, not just the outcome.
• Group size: 2 completions per prompt on T4 (up to 4 on higher-memory GPUs).
• Learning rate: 5e-6, KL penalty β = 0.04 against the reference policy.
• Batch size: 1 per device, gradient accumulation 8.
• Epochs: up to 3 (full run) or 1 (fast mode).

This stage is the one most directly responsible for the pass@1 gains, since it optimizes directly for functional correctness and process discipline rather than surface-level token likelihood.

3.3 Stage 3 — Direct Preference Optimization (DPO)

For each problem, two completions were generated with the GRPO-trained policy and scored using the same reward functions from Stage 2; the higher-scoring completion became the "chosen" example and the lower-scoring one "rejected."

• Preference pair construction: 2 completions per problem, scored and ranked by the Stage 2 reward functions.
• Dataset size: 200 preference pairs (full run) or 100 (fast mode).
• Training configuration: learning rate 1e-6, preference strength β = 0.1, batch size 1, gradient accumulation 4, 1 epoch. DPO uses a substantially lower learning rate than SFT/GRPO since it refines preferences on an already-competent policy rather than learning from scratch.

3.4 Stage 4 — Self-Guided Self-Play (SGS)

Adapted from Self-Guided Self-Play research originally developed for theorem proving, this stage uses three interacting roles — Solver (attempts problems), Conjecturer (generates harder sub-problems from ones the Solver already handles), and Guide (scores problem quality) — with our CodeQA sandbox's test suite standing in for the theorem prover's verifier.

• Rounds: 7 (pinned, independent of fast/full mode).
• Attempts per round: 2–4 solutions per target problem.
• Target problems: 30 (fast mode) or 60 (full mode), stratified across easy/medium/hard difficulty.
• Update rule: REINFORCE on a separate scoring pass (generation and scoring phases run sequentially, with GPU memory cleared between them, to fit a 7B model in 4-bit + LoRA on a 15GB T4).

Note this is meaningfully scaled down from the original theorem-proving paper (50 rounds × 8 attempts) due to the Kaggle session compute budget, so its marginal contribution here likely understates what the method can do with more compute.

3.5 Adapter Merging

LoRA adapters from all four stages were merged sequentially into the base Qwen2.5-Coder-7B-Instruct weights via PEFT's merge_and_unload, producing the single deployable checkpoint published on Hugging Face, with no added inference overhead versus the base model.

4. Evaluation Methodology

4.1 Benchmarks

All scores are pass@1: a single sample is generated per problem (temperature 0.2 via evalplus), with no retries or majority voting.

4.2 Comparison Set

TIMPS-Coder-7B was evaluated against nine other open models in the 3.8B–9.2B parameter range: Qwen2.5-Coder-7B (base, non-instruct), Qwen2.5-Coder-7B-Instruct, DeepSeek-Coder-7B-Instruct-v1.5, CodeLlama-7B-Instruct, CodeGemma-7B-it, StarCoder2-7B, Llama-3.1-8B-Instruct, Phi-3.5-mini-instruct (3.8B), and Gemma-2-9B-it. Comparison-model scores are drawn from their respective official technical reports and model cards.

Evaluation code is published in benchmark_evaluation.ipynb in the GitHub repository for independent reproduction — including the dataset-preparation notebook that produces the contamination described in Section 6.

5. Results

5.1 Improvement over base model

* Affected by training-data overlap — see Section 6.

5.2 Full comparison against 7B–9B code models

* TIMPS-Coder-7B's HumanEval and HumanEval+ scores are affected by training-data overlap (Section 6) and should not be read as a like-for-like comparison against models not trained on this data.
† See Section 7 — a separate evaluation-harness artifact, unrelated to the contamination issue.

On the one benchmark we can verify is uncontaminated, MBPP+, TIMPS-Coder-7B scores higher than every compared model except the non-instruct Qwen2.5-Coder-7B base (72.0%), which it exceeds.

6. Data Contamination Disclosure

This section is placed before the detailed results discussion because it materially changes how the headline HumanEval number should be read.

The SFT dataset-preparation step (documented in TIMPS-Coder-7B.ipynb) loads openai/openai_humaneval's test split in full — all 164 problems — and converts every one into a training example, using the benchmark's own canonical solution as the target completion, formatted in the THINK/ACT/VERIFY template used throughout the pipeline. These are the exact same 164 problems used later to compute the HumanEval pass@1 score in Section 5. HumanEval+ evaluates the identical 164 problems with an expanded test suite, so it inherits the same overlap.

This means the 98.8% HumanEval and 82.9% HumanEval+ scores are not a measurement of generalization to unseen problems and should not be presented as a state-of-the-art comparison against models that were not trained on this data. We report them for completeness — they are true measurements of what the released checkpoint does on those specific inputs — but we do not claim leaderboard status on this basis.

By contrast, the MBPP component of the SFT data uses only the train split (first 500 of 974 rows), while evaluation uses the separate, disjoint test split. We have no evidence of overlap between these splits, and MBPP+ (73.3% pass@1) is the result we consider representative of TIMPS-Coder-7B's actual capability.

Planned follow-up: re-run HumanEval-style evaluation on a decontaminated or held-out benchmark (a HumanEval variant excluded from training, or LiveCodeBench) and publish that result as an explicit update to this report rather than revise the numbers above silently.

7. Known Limitation: MBPP Raw Pass@1 Anomaly

Separately from the contamination issue above, TIMPS-Coder-7B's raw MBPP pass@1 score (5.4%) is not representative of its actual MBPP-level capability and stands in sharp contrast to its MBPP+ score (73.3%). We believe this is an evaluation-harness formatting mismatch: post-DPO/SGS, the model's output consistently follows the THINK/ACT/VERIFY format baked into training by design, and the strict (non-EvalPlus) MBPP harness appears to mis-parse this format when extracting the code block for execution, while the more permissive MBPP+ harness parses it correctly. We are treating MBPP+ as authoritative for MBPP-level performance and are investigating the harness discrepancy for a follow-up correction, rather than excluding the raw number.

8. Reproducibility

• Model weights: huggingface.co/sandeeprdy1729/TIMPS-Coder-7B
• Training notebook: TIMPS-Coder-7B.ipynb (includes the dataset-preparation code described in Section 6)
• Evaluation notebook: benchmark_evaluation.ipynb
• Repository: github.com/Sandeeprdy1729/TIMPS-Coder-7B

We encourage independent reproduction of these results, and welcome reports of discrepancies.

9. Limitations and Future Work

In priority order:

• The HumanEval/HumanEval+ training-data overlap in Section 6 is the most important open item and should be resolved before any further generalization claims are made about this model.
• The MBPP raw pass@1 harness artifact (Section 7) is unresolved.
• Per-stage ablation scores isolating the individual contribution of GRPO, DPO, and SGS are not yet measured from intermediate checkpoints.
• Evaluation is currently limited to Python — multilingual evaluation (HumanEval-X, MultiPL-E) is planned but not yet run.
• Agentic/multi-turn benchmarks such as SWE-bench are a natural next step given the tool-discipline reward already used in training, but have not yet been run as a standalone evaluation.
• Inference-speed benchmarking against comparison models has not yet been published.

Citation

Published by TIMPS. For questions about this model or the TIMPS coding agent ecosystem, contact [fill in].