keep ingested chunks findable and structured

Two bugs made self-ingested content useless, both silently:

- _payload wrote `chapter_number` (int) while math retrieval filters on
  `chapter` (the full "فصل ۲: کسر" string), so anything we ingest could
  never match a search — no error, just permanently invisible content.
- ingest_pdf flattened the vision extraction into one text blob and threw
  away type/keywords/generalSolution/problem/solution. LessonChunk is the
  source of truth, so rebuilding meant paying for vision again — and the
  problem/solution pairs are the input for building lesson questions.

LessonChunk now stores those fields and _payload emits them under the same
keys the other team's collections use, so ours and theirs stay one shape.

Also fixes a regression from the whitenoise commit: the Manifest storage
variant raises when staticfiles.json is absent, which broke all 23 admin
rendering tests (Django forces DEBUG=False in tests, so USE_FINDERS is off)
and would have taken the admin panel down entirely the day DEBUG=0 ships
without a collectstatic run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-09-10 16:24:18 +03:30
parent f0e3e3c96c
commit 52d9694583
7 changed files with 235 additions and 3 deletions

View File

@@ -24,12 +24,28 @@ class LessonChunk(models.Model):
lesson_number = models.IntegerField()
chapter = models.IntegerField(null=True, blank=True)
# عنوانِ کاملِ فصل، دقیقاً همون رشته‌ای که در payloadِ تیمِ دیگه هست
# («فصل ۲: کسر»). retrieval برای ریاضی روی همین مقدار فیلتر می‌کنه، پس
# بدونِ این هر چیزی که ingest کنیم در جستجو پیدا نمی‌شه.
chapter_title = models.CharField(max_length=255, blank=True)
grade_title = models.CharField(max_length=100, blank=True)
order = models.PositiveIntegerField()
title = models.CharField(max_length=255)
original_text = models.TextField()
summary = models.TextField()
embedding_vector = models.JSONField(null=True, blank=True)
# --- ساختارِ استخراج‌شده از کتاب (همون قراردادِ ریاضیِ تیمِ دیگه) -------
# vision یک‌بار و با هزینه این‌ها رو درمی‌آره؛ اگه فقط متنِ صاف‌شده رو
# نگه داریم، برای هر بازسازی باید دوباره همون هزینه رو بدیم. ضمناً
# problem/solution ورودیِ مستقیمِ ساختِ سؤال‌های مسیرِ درس‌اند.
problem_type = models.CharField(max_length=255, blank=True)
keywords = models.JSONField(default=list, blank=True)
general_solution = models.TextField(blank=True)
problem = models.TextField(blank=True)
solution = models.TextField(blank=True)
collection = models.CharField(max_length=200, db_index=True)
qdrant_point_id = models.UUIDField(default=uuid.uuid4, editable=False)