tutor runs on gpt-4.1-mini, and primary-school "/" is read as decimal or fraction by rule

Two benchmark rounds (14 models on 30 problems, then gpt-4o-mini vs gpt-4.1-mini
on 102 problems under three prompt versions, hand-graded): gpt-4.1-mini scored
95-96/102 against 85-90 for gpt-4o-mini, at 0.12 vs 0.04 cents per answer.
The default now lives in settings so the deploy needs no .env edit.

The context note "in the book / is a decimal point, not a fraction" was written
for the upper-grade books but went to every grade, so gpt-4.1-mini computed
1/7 + 1/8 as 3.5. For primary grades the note now says / can be either, and a
reading rule sits next to the question itself (the same rule at the end of
the system prompt had no effect). Slash problems went from 11 to 14 of 17;
three are still wrong.

generate_answer now reports the provider's exact per-call cost, reasoning
tokens and whether the raw output broke the plain-text rule. The benchmark
fixes primary grades being sent with no book context, caches retrieval so
runs compare on identical context, and retries Qdrant timeouts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-09-26 02:36:54 +03:30
parent e0176ca0a6
commit 11bbe1c248
6 changed files with 147 additions and 20 deletions

View File

@@ -89,7 +89,7 @@ OPENAI_BASE_URL=https://api.gapgpt.app/v1
# تا ساده و بی‌اشتباه بمونه.
# OPENAI_BASE_URL=https://api.openai.com/v1
OPENAI_API_KEY_OPENAI_DIRECT=
OPENAI_CHAT_MODEL=gpt-4o-mini
OPENAI_CHAT_MODEL=gpt-4.1-mini
OPENAI_VISION_MODEL=gpt-4.1-mini
# این رو تغییر نده؛ باید با مدلی که تیمِ دیگه باهاش collectionها رو ساخته یکی باشه.
OPENAI_EMBEDDING_MODEL=text-embedding-3-small