new app — cleaned-up RAG pipeline (Qdrant retrieval + GPT), ported from Rag_backend

Self-contained app; wired into settings/urls in the next commit. Data
stays in the parallel team's live Qdrant — nothing is copied into our DB.

apps.py
  - RagConfig.
models.py
  - LessonChunk: DB-of-record for one summarized lesson chunk
    (original_text + summary + embedding_vector) mirrored as a single
    Qdrant point; loose lesson_id/grade_id ints, no cross-app FK.
    LessonChunkImage for future multimodal lessons.
migrations/0001_initial.py
  - initial tables for the two models.
services/clients.py
  - lazy, lru_cache'd OpenAI + Qdrant clients (Rag_backend builds them at
    import time and crashes without keys). RagNotConfigured raised when a
    key/lib is missing; Qdrant client uses check_compatibility=False for
    the 1.19-client / 1.17-server minor version gap.
services/config.py
  - rag_is_configured(), collection-name template, grade-title resolution
    (RAG_GRADE_TITLE_OVERRIDES -> content.Grade.title), retrieval top-k.
services/known_collections.py
  - exact live Qdrant catalog: 6 collection names, math chapter-number ->
    exact "فصل N: ..." string tables per (subject, grade), and the
    علوم/اجتماعی lesson-number -> title maps. filter_key_for_subject()
    picks `chapter` for math, `lesson_number` otherwise;
    resolve_math_chapter() maps a plain chapter number (Persian or Latin
    digits) to the stored string.
services/embedding.py
  - embed_text() pinned to OPENAI_EMBEDDING_MODEL (text-embedding-3-small,
    1536d) so vectors match the other team's existing collections.
services/chunking.py
  - split_text_into_chunks(), summarize_chunk() (gpt-4o-mini, JSON out);
    plain exceptions instead of ninja HttpError.
services/qdrant_store.py
  - ensure_collection(), upsert_points(), search(): the lesson filter
    matches the value both as str and native, since Qdrant stores
    lesson_number as int and chapter as string.
services/retrieval.py
  - resolve_target() -> (collection, filter_key, filter_value), reused by
    the cache key and debug block.
  - retrieve_context() -> (chunks, meta): embed -> Qdrant search scoped to
    one lesson/chapter -> top-k. Math payloads (no `summary`) fall back to
    generalSolution/problem/solution; a 404 collection returns empty.
services/generation.py
  - generate_answer() and stream_answer() (token iterator for SSE);
    context-grounded Persian prompt that explains from retrieved text
    rather than refusing.
services/pdf.py
  - textbook PDF -> page images -> gpt-4o vision problem-type extraction
    (adapted from Rag_backend mathapp); returns dicts, no on-disk JSON.
    pymupdf/pdf2image optional — missing libs raise RagNotConfigured.
services/ingest.py
  - ingest_text() / ingest_pdf(): chunk -> summarize -> embed -> persist
    LessonChunk + upsert Qdrant, in one transaction.
services/answer_cache.py
  - short-TTL cache of the final answer keyed by
    sha1(collection | filter_value | normalized-question).
services/tutor.py
  - thin answer_question() (retrieve -> generate) for direct callers.
serializers.py, urls.py, views.py
  - POST /api/rag/ingest/text, POST /api/rag/ingest/pdf (HasInternalServiceKey,
    throttle scope rag_ingest), GET /api/rag/chunks review list.
    RagProviderUnavailable -> clean 503.
admin.py
  - LessonChunk admin with an inline for its images.
management/commands/rag_catalog.py
  - read-only live Qdrant listing (--values, --json); never writes.
tests.py
  - mocked-client tests: ask-with-retrieval, ingest, SSE stream, answer
    cache, and rag_is_configured() gating.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

rag/services/chunking.py

@@ -0,0 +1,92 @@
+"""
+تقسیمِ متنِ درس به تکه‌های مستقل + خلاصه‌سازیِ آموزشیِ هر تکه.
+
+برداشتِ پاک‌سازی‌شده از subjects/services/chunker.py مخزنِ Rag_backend:
+- به‌جای HttpError نینجا، از استثناهای معمولی استفاده می‌کنیم (لایه‌ی view
+  در DRF تبدیلش می‌کنه).
+- کلاینت lazy است.
+"""
+import json
+
+from django.conf import settings
+
+from . import clients
+
+
+class ChunkingError(RuntimeError):
+    pass
+
+
+def split_text_into_chunks(text: str, chunk_size: int = 400, overlap: int = 100) -> list[str]:
+    if not text:
+        return []
+    chunks: list[str] = []
+    start = 0
+    text_length = len(text)
+    step = max(chunk_size - overlap, 1)
+    while start < text_length:
+        end = min(start + chunk_size, text_length)
+        chunks.append(text[start:end])
+        start += step
+    return chunks
+
+
+def _strip_code_fence(raw: str) -> str:
+    raw = raw.strip()
+    if raw.startswith('```json'):
+        raw = raw[len('```json'):]
+    elif raw.startswith('```'):
+        raw = raw[len('```'):]
+    if raw.endswith('```'):
+        raw = raw[:-3]
+    return raw.strip()
+
+
+def summarize_chunk(chunk_text: str) -> str:
+    """خلاصه‌ی مستقل و آموزشیِ یک تکه به فارسی."""
+    prompt = f"""
+    You are an expert Persian educational summarizer.
+    Summarize the following text clearly and independently.
+
+    - The summary must be self-contained.
+    - Capture all key educational points.
+    - Write in natural Persian (Farsi).
+
+    Text:
+    {chunk_text}
+
+    Return JSON only:
+    {{"summary": "خلاصه کامل این بخش در فارسی"}}
+    """
+    client = clients.get_openai_client()
+    response = client.chat.completions.create(
+        model=settings.OPENAI_CHAT_MODEL,
+        messages=[
+            {'role': 'system', 'content': 'You are a helpful assistant that always outputs valid JSON.'},
+            {'role': 'user', 'content': prompt},
+        ],
+        max_tokens=400,
+        temperature=0.4,
+        top_p=0.9,
+    )
+    raw = _strip_code_fence(response.choices[0].message.content or '')
+    try:
+        data = json.loads(raw)
+    except json.JSONDecodeError as exc:
+        raise ChunkingError(f'خروجیِ خلاصه‌سازی JSON معتبر نبود: {raw[:300]}') from exc
+    summary = data.get('summary')
+    if not summary:
+        raise ChunkingError(f'کلیدِ summary در خروجی نبود: {raw[:300]}')
+    return summary
+
+
+def chunk_and_summarize(text: str, chunk_size: int = 400, overlap: int = 100) -> list[dict]:
+    """[{order, original_text, summary}, ...]"""
+    result = []
+    for index, original in enumerate(split_text_into_chunks(text, chunk_size, overlap), start=1):
+        result.append({
+            'order': index,
+            'original_text': original,
+            'summary': summarize_chunk(original),
+        })
+    return result