tutor runs on gpt-4.1-mini, and primary-school "/" is read as decimal or fraction by rule

Two benchmark rounds (14 models on 30 problems, then gpt-4o-mini vs gpt-4.1-mini
on 102 problems under three prompt versions, hand-graded): gpt-4.1-mini scored
95-96/102 against 85-90 for gpt-4o-mini, at 0.12 vs 0.04 cents per answer.
The default now lives in settings so the deploy needs no .env edit.

The context note "in the book / is a decimal point, not a fraction" was written
for the upper-grade books but went to every grade, so gpt-4.1-mini computed
1/7 + 1/8 as 3.5. For primary grades the note now says / can be either, and a
reading rule sits next to the question itself (the same rule at the end of
the system prompt had no effect). Slash problems went from 11 to 14 of 17;
three are still wrong.

generate_answer now reports the provider's exact per-call cost, reasoning
tokens and whether the raw output broke the plain-text rule. The benchmark
fixes primary grades being sent with no book context, caches retrieval so
runs compare on identical context, and retries Qdrant timeouts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-09-26 02:36:54 +03:30
parent e0176ca0a6
commit 11bbe1c248
6 changed files with 147 additions and 20 deletions

View File

@@ -775,7 +775,10 @@ if ZARINPAL_REDIRECT_URL == '-':
# تست‌های موجود و اجرای بدونِ کلید دست‌نخورده کار می‌کنن).
OPENAI_API_KEY = os.environ.get('OPENAI_API_KEY', '')
OPENAI_BASE_URL = os.environ.get('OPENAI_BASE_URL', 'https://api.gapgpt.app/v1')
OPENAI_CHAT_MODEL = os.environ.get('OPENAI_CHAT_MODEL', 'gpt-4o-mini')
# gpt-4.1-mini از ۲۰۲۶-۰۹-۲۶ (تصمیمِ صاحبِ پروژه): در سنجشِ ۱۰۲ مسئله ۹۶ درست در برابرِ
# ۸۵–۹۰ِ gpt-4o-mini، با ۲٫۹ برابر هزینه (۰٫۱۲ سنت برای هر پاسخ). شرح: گزارش ۲۳ و
# docs/architecture.md. پیش‌فرض همین‌جاست تا دیپلوی به ویرایشِ دستیِ .env نیاز نداشته باشد.
OPENAI_CHAT_MODEL = os.environ.get('OPENAI_CHAT_MODEL', 'gpt-4.1-mini')
# اختیاری: مدلِ جدا فقط برای پایه‌های دهم تا دوازدهم (حسابان/مثلثات). خالی یعنی
# همه‌ی پایه‌ها OPENAI_CHAT_MODEL. انتخابش با `rag_model_benchmark` سنجیده می‌شود.
OPENAI_CHAT_MODEL_UPPER = os.environ.get('OPENAI_CHAT_MODEL_UPPER', '')