change for ocr confidence

opus_submitter/API_USAGE.md

@@ -1,231 +0,0 @@
-# Opus Magnum Submission API Usage
-
-## Overview
-
-The API is built with Django Ninja and provides endpoints for managing puzzle submissions with OCR validation and S3 file storage.
-
-## Base URL
-- Development: `http://localhost:8000/api/`
-- API Documentation: `http://localhost:8000/api/docs/`
-
-## Authentication
-Most endpoints support both authenticated and anonymous submissions. Admin endpoints require staff permissions.
-
-## Environment Variables
-
-### Required for S3 Storage
-```bash
-USE_S3=true
-AWS_ACCESS_KEY_ID=your_access_key
-AWS_SECRET_ACCESS_KEY=your_secret_key
-AWS_STORAGE_BUCKET_NAME=your_bucket_name
-AWS_S3_REGION_NAME=us-east-1
-```
-
-### Optional
-```bash
-STEAM_API_KEY=your_steam_api_key
-```
-
-## API Endpoints
-
-### 1. Get Available Puzzles
-```http
-GET /api/submissions/puzzles
-```
-
-Response:
-```json
-[
-  {
-    "id": 1,
-    "steam_item_id": "3479143948",
-    "title": "P41-FLOC",
-    "author_name": "Flame Legrems",
-    "description": "A challenging puzzle...",
-    "tags": ["puzzle", "chemistry"],
-    "order_index": 0,
-    "steam_url": "https://steamcommunity.com/workshop/filedetails/?id=3479143948",
-    "created_at": "2025-05-29T11:19:24Z",
-    "updated_at": "2025-05-30T22:15:09Z"
-  }
-]
-```
-
-### 2. Create Submission
-```http
-POST /api/submissions/submissions
-Content-Type: multipart/form-data
-```
-
-Form Data:
-- `data`: JSON with submission data
-- `files`: Array of uploaded files
-
-Example data:
-```json
-{
-  "notes": "My best solutions so far",
-  "responses": [
-    {
-      "puzzle_id": 1,
-      "puzzle_name": "P41-FLOC",
-      "cost": "150",
-      "cycles": "89", 
-      "area": "12",
-      "needs_manual_validation": false,
-      "ocr_confidence_score": 0.95
-    }
-  ]
-}
-```
-
-Response:
-```json
-{
-  "id": "123e4567-e89b-12d3-a456-426614174000",
-  "user": null,
-  "notes": "My best solutions so far",
-  "responses": [
-    {
-      "id": 1,
-      "puzzle": 1,
-      "puzzle_name": "P41-FLOC",
-      "cost": "150",
-      "cycles": "89",
-      "area": "12",
-      "needs_manual_validation": false,
-      "files": [
-        {
-          "id": 1,
-          "original_filename": "solution.gif",
-          "file_size": 1024000,
-          "content_type": "image/gif",
-          "file_url": "https://bucket.s3.amazonaws.com/media/submissions/123.../file.gif",
-          "ocr_processed": false,
-          "created_at": "2025-10-29T00:00:00Z"
-        }
-      ],
-      "final_cost": "150",
-      "final_cycles": "89", 
-      "final_area": "12"
-    }
-  ],
-  "total_responses": 1,
-  "needs_validation": false,
-  "is_validated": false,
-  "created_at": "2025-10-29T00:00:00Z"
-}
-```
-
-### 3. List Submissions
-```http
-GET /api/submissions/submissions?limit=20&offset=0
-```
-
-### 4. Get Submission Details
-```http
-GET /api/submissions/submissions/{submission_id}
-```
-
-### 5. Admin: Validate Response (Staff Only)
-```http
-PUT /api/submissions/responses/{response_id}/validate
-Content-Type: application/json
-```
-
-Body:
-```json
-{
-  "validated_cost": "150",
-  "validated_cycles": "89",
-  "validated_area": "12"
-}
-```
-
-### 6. Admin: List Responses Needing Validation (Staff Only)
-```http
-GET /api/submissions/responses/needs-validation
-```
-
-### 7. Admin: Validate Entire Submission (Staff Only)
-```http
-POST /api/submissions/submissions/{submission_id}/validate
-```
-
-### 8. Get Statistics
-```http
-GET /api/submissions/stats
-```
-
-Response:
-```json
-{
-  "total_submissions": 150,
-  "total_responses": 300,
-  "needs_validation": 25,
-  "validated_submissions": 120,
-  "validation_rate": 0.8
-}
-```
-
-## OCR Validation Logic
-
-The system automatically flags responses for manual validation when:
-
-1. **Incomplete OCR Data**: Missing cost, cycles, or area values
-2. **Low Confidence**: OCR confidence score below threshold
-3. **Manual Flag**: Explicitly marked by frontend OCR processing
-
-### Manual Validation Workflow
-
-1. Admin views responses needing validation: `GET /responses/needs-validation`
-2. Admin reviews the uploaded files and OCR results
-3. Admin provides corrected values: `PUT /responses/{id}/validate`
-4. System updates `validated_*` fields and clears validation flag
-5. Optional: Mark entire submission as validated: `POST /submissions/{id}/validate`
-
-## File Storage
-
-- **Development**: Files stored locally in `media/submissions/`
-- **Production**: Files stored in S3 with path structure: `submissions/{submission_id}/{uuid}_{filename}`
-- **Supported Formats**: JPEG, PNG, GIF, MP4, WebM
-- **Size Limit**: 10MB per file
-
-## Error Handling
-
-The API returns standard HTTP status codes:
-
-- `200`: Success
-- `400`: Bad Request (validation errors)
-- `401`: Unauthorized
-- `403`: Forbidden (admin required)
-- `404`: Not Found
-- `500`: Internal Server Error
-
-Error Response Format:
-```json
-{
-  "detail": "Error message",
-  "code": "error_code"
-}
-```
-
-## Frontend Integration
-
-The Vue frontend should:
-
-1. Upload files with OCR data extracted client-side
-2. Group files by detected puzzle name
-3. Create one submission with multiple responses
-4. Handle validation flags and display admin feedback
-5. Show file upload progress and S3 URLs
-
-## Admin Interface
-
-Django Admin provides:
-
-- **Submission Management**: View, validate, and manage submissions
-- **Response Validation**: Bulk actions for validation workflow
-- **File Management**: View uploaded files and OCR data
-- **Statistics Dashboard**: Track validation rates and submission metrics

opus_submitter/src/components/AdminPanel.vue

@@ -48,14 +48,41 @@
                 </td>
                 <td>
                   <div class="text-sm space-y-1">
-                    <div>Cost: {{ response.cost || '-' }}</div>
+                    <div class="flex justify-between items-center">
-                    <div>Cycles: {{ response.cycles || '-' }}</div>
+                      <span>Cost: {{ response.cost || '-' }}</span>
-                    <div>Area: {{ response.area || '-' }}</div>
+                      <span 
+                        v-if="response.ocr_confidence_cost" 
+                        class="badge badge-xs"
+                        :class="getConfidenceBadgeClass(response.ocr_confidence_cost)"
+                      >
+                        {{ Math.round(response.ocr_confidence_cost * 100) }}%
+                      </span>
+                    </div>
+                    <div class="flex justify-between items-center">
+                      <span>Cycles: {{ response.cycles || '-' }}</span>
+                      <span 
+                        v-if="response.ocr_confidence_cycles" 
+                        class="badge badge-xs"
+                        :class="getConfidenceBadgeClass(response.ocr_confidence_cycles)"
+                      >
+                        {{ Math.round(response.ocr_confidence_cycles * 100) }}%
+                      </span>
+                    </div>
+                    <div class="flex justify-between items-center">
+                      <span>Area: {{ response.area || '-' }}</span>
+                      <span 
+                        v-if="response.ocr_confidence_area" 
+                        class="badge badge-xs"
+                        :class="getConfidenceBadgeClass(response.ocr_confidence_area)"
+                      >
+                        {{ Math.round(response.ocr_confidence_area * 100) }}%
+                      </span>
+                    </div>
                   </div>
                 </td>
                 <td>
                   <div class="badge badge-warning badge-sm">
-                    {{ response.ocr_confidence_score ? Math.round(response.ocr_confidence_score * 100) + '%' : 'Low' }}
+                    {{ getOverallConfidence(response) }}%
                   </div>
                 </td>
                 <td>
@@ -273,6 +300,26 @@ onMounted(() => {
   loadData()
 })
 
+// Helper functions for confidence display
+const getConfidenceBadgeClass = (confidence: number): string => {
+  if (confidence >= 0.8) return 'badge-success'
+  if (confidence >= 0.6) return 'badge-warning'
+  return 'badge-error'
+}
+
+const getOverallConfidence = (response: PuzzleResponse): number => {
+  const confidences = [
+    response.ocr_confidence_cost,
+    response.ocr_confidence_cycles,
+    response.ocr_confidence_area
+  ].filter(conf => conf !== undefined && conf !== null) as number[]
+  
+  if (confidences.length === 0) return 0
+  
+  const average = confidences.reduce((sum, conf) => sum + conf, 0) / confidences.length
+  return Math.round(average * 100)
+}
+
 // Expose refresh method
 defineExpose({
   refresh: loadData

opus_submitter/src/components/FileUpload.vue

@@ -83,7 +83,17 @@
               
               <div v-else-if="file.ocrData" class="mt-1 space-y-1">
                 <div class="text-xs flex items-center justify-between">
-                  <span class="font-medium text-success">✓ OCR Complete</span>
+                  <div class="flex items-center gap-2">
+                    <span class="font-medium text-success">✓ OCR Complete</span>
+                    <span 
+                      v-if="file.ocrData.confidence" 
+                      class="badge badge-xs"
+                      :class="getConfidenceBadgeClass(file.ocrData.confidence.overall)"
+                      :title="`Overall confidence: ${Math.round(file.ocrData.confidence.overall * 100)}%`"
+                    >
+                      {{ Math.round(file.ocrData.confidence.overall * 100) }}%
+                    </span>
+                  </div>
                   <button 
                     @click="retryOCR(file)" 
                     class="btn btn-xs btn-ghost"
@@ -95,15 +105,43 @@
                 <div class="text-xs space-y-1 bg-base-200 p-2 rounded">
                   <div v-if="file.ocrData.puzzle">
                     <strong>Puzzle:</strong> {{ file.ocrData.puzzle }}
+                    <span 
+                      v-if="file.ocrData.confidence?.puzzle" 
+                      class="ml-2 opacity-60"
+                      :title="`Puzzle confidence: ${Math.round(file.ocrData.confidence.puzzle * 100)}%`"
+                    >
+                      ({{ Math.round(file.ocrData.confidence.puzzle * 100) }}%)
+                    </span>
                   </div>
                   <div v-if="file.ocrData.cost">
                     <strong>Cost:</strong> {{ file.ocrData.cost }}
+                    <span 
+                      v-if="file.ocrData.confidence?.cost" 
+                      class="ml-2 opacity-60"
+                      :title="`Cost confidence: ${Math.round(file.ocrData.confidence.cost * 100)}%`"
+                    >
+                      ({{ Math.round(file.ocrData.confidence.cost * 100) }}%)
+                    </span>
                   </div>
                   <div v-if="file.ocrData.cycles">
                     <strong>Cycles:</strong> {{ file.ocrData.cycles }}
+                    <span 
+                      v-if="file.ocrData.confidence?.cycles" 
+                      class="ml-2 opacity-60"
+                      :title="`Cycles confidence: ${Math.round(file.ocrData.confidence.cycles * 100)}%`"
+                    >
+                      ({{ Math.round(file.ocrData.confidence.cycles * 100) }}%)
+                    </span>
                   </div>
                   <div v-if="file.ocrData.area">
                     <strong>Area:</strong> {{ file.ocrData.area }}
+                    <span 
+                      v-if="file.ocrData.confidence?.area" 
+                      class="ml-2 opacity-60"
+                      :title="`Area confidence: ${Math.round(file.ocrData.confidence.area * 100)}%`"
+                    >
+                      ({{ Math.round(file.ocrData.confidence.area * 100) }}%)
+                    </span>
                   </div>
                 </div>
               </div>
@@ -306,4 +344,10 @@ const processOCR = async (submissionFile: SubmissionFile) => {
 const retryOCR = (submissionFile: SubmissionFile) => {
   processOCR(submissionFile)
 }
+
+const getConfidenceBadgeClass = (confidence: number): string => {
+  if (confidence >= 0.8) return 'badge-success'
+  if (confidence >= 0.6) return 'badge-warning'
+  return 'badge-error'
+}
 </script>

opus_submitter/src/services/apiService.ts

@@ -122,7 +122,9 @@ export class ApiService {
         cycles?: string
         area?: string
         needs_manual_validation?: boolean
-        ocr_confidence_score?: number
+        ocr_confidence_cost?: number
+        ocr_confidence_cycles?: number
+        ocr_confidence_area?: number
       }>
     },
     files: File[]
@@ -268,7 +270,9 @@ export const submissionHelpers = {
           cycles: fileWithOCR?.ocrData?.cycles,
           area: fileWithOCR?.ocrData?.area,
           needs_manual_validation: needsValidation,
-          ocr_confidence_score: needsValidation ? 0.5 : 0.9 // Rough estimate
+          ocr_confidence_cost: fileWithOCR?.ocrData?.confidence?.cost || 0.0,
+          ocr_confidence_cycles: fileWithOCR?.ocrData?.confidence?.cycles || 0.0,
+          ocr_confidence_area: fileWithOCR?.ocrData?.confidence?.area || 0.0
         }
       })
 

opus_submitter/src/services/ocrService.ts

@@ -5,6 +5,13 @@ export interface OpusMagnumData {
   cost: string;
   cycles: string;
   area: string;
+  confidence: {
+    puzzle: number;
+    cost: number;
+    cycles: number;
+    area: number;
+    overall: number;
+  };
 }
 
 export interface OCRRegion {
@@ -64,6 +71,7 @@ export class OpusMagnumOCRService {
 
           // Extract text from each region
           const results: Partial<OpusMagnumData> = {};
+          const confidenceScores: Record<string, number> = {};
 
           for (const [key, region] of Object.entries(this.regions)) {
             const regionCanvas = document.createElement('canvas');
@@ -108,9 +116,12 @@ export class OpusMagnumOCRService {
             }
 
             // Perform OCR on the region
-            const { data: { text } } = await this.worker!.recognize(regionCanvas);
+            const { data: { text, confidence } } = await this.worker!.recognize(regionCanvas);
             let cleanText = text.trim();
             
+            // Store the confidence score for this field
+            confidenceScores[key] = confidence / 100; // Tesseract returns 0-100, we want 0-1
+
             // Post-process based on field type
             if (key === 'cost') {
               // Handle common OCR misreadings where G is read as 6
@@ -134,16 +145,29 @@ export class OpusMagnumOCRService {
               cleanText = this.findBestPuzzleMatch(cleanText);
             }
 
-            results[key as keyof OpusMagnumData] = cleanText;
+            (results as any)[key] = cleanText;
           }
 
           URL.revokeObjectURL(imageUrl);
           
+          // Calculate overall confidence as the average of all field confidences
+          const confidenceValues = Object.values(confidenceScores);
+          const overallConfidence = confidenceValues.length > 0 
+            ? confidenceValues.reduce((sum, conf) => sum + conf, 0) / confidenceValues.length 
+            : 0;
+          
           resolve({
             puzzle: results.puzzle || '',
             cost: results.cost || '',
             cycles: results.cycles || '',
-            area: results.area || ''
+            area: results.area || '',
+            confidence: {
+              puzzle: confidenceScores.puzzle || 0,
+              cost: confidenceScores.cost || 0,
+              cycles: confidenceScores.cycles || 0,
+              area: confidenceScores.area || 0,
+              overall: overallConfidence
+            }
           });
         } catch (error) {
           URL.revokeObjectURL(imageUrl);

opus_submitter/src/types/index.ts

@@ -29,6 +29,13 @@ export interface OpusMagnumData {
   cost: string
   cycles: string
   area: string
+  confidence: {
+    puzzle: number
+    cost: number
+    cycles: number
+    area: number
+    overall: number
+  }
 }
 
 export interface SubmissionFile {
@@ -49,7 +56,9 @@ export interface PuzzleResponse {
   cycles?: string
   area?: string
   needs_manual_validation?: boolean
-  ocr_confidence_score?: number
+  ocr_confidence_cost?: number
+  ocr_confidence_cycles?: number
+  ocr_confidence_area?: number
   validated_cost?: string
   validated_cycles?: string
   validated_area?: string

opus_submitter/static_source/vite/manifest.json

@@ -16,12 +16,12 @@
     "src": "node_modules/.pnpm/@mdi+font@7.4.47/node_modules/@mdi/font/fonts/materialdesignicons-webfont.woff2"
   },
   "src/main.ts": {
-    "file": "assets/main-CNlI4PW6.js",
+    "file": "assets/main-Cv9F8wz5.js",
     "name": "main",
     "src": "src/main.ts",
     "isEntry": true,
     "css": [
-      "assets/main-HDjkw-xK.css"
+      "assets/main-DWmJTLXa.css"
     ],
     "assets": [
       "assets/materialdesignicons-webfont-CSr8KVlo.eot",

opus_submitter/submissions/admin.py

@@ -160,7 +160,7 @@ class PuzzleResponseInline(admin.TabularInline):
     readonly_fields = ["created_at", "updated_at"]
     fields = [
         "puzzle", "puzzle_name", "cost", "cycles", "area",
-        "needs_manual_validation", "ocr_confidence_score"
+        "needs_manual_validation", "ocr_confidence_cost", "ocr_confidence_cycles", "ocr_confidence_area"
     ]
 
 
@@ -235,7 +235,7 @@ class PuzzleResponseAdmin(admin.ModelAdmin):
             "fields": ("submission", "puzzle", "puzzle_name")
         }),
         ("OCR Data", {
-            "fields": ("cost", "cycles", "area", "ocr_confidence_score")
+            "fields": ("cost", "cycles", "area", "ocr_confidence_cost", "ocr_confidence_cycles", "ocr_confidence_area")
         }),
         ("Validation", {
             "fields": (

opus_submitter/submissions/api.py

@@ -92,7 +92,9 @@ def create_submission(
                     cycles=response_data.cycles,
                     area=response_data.area,
                     needs_manual_validation=response_data.needs_manual_validation,
-                    ocr_confidence_score=response_data.ocr_confidence_score,
+                    ocr_confidence_cost=response_data.ocr_confidence_cost,
+                    ocr_confidence_cycles=response_data.ocr_confidence_cycles,
+                    ocr_confidence_area=response_data.ocr_confidence_area,
                 )
 
                 # Process files for this response

opus_submitter/submissions/migrations/0006_remove_puzzleresponse_ocr_confidence_score_and_more.py

@@ -0,0 +1,32 @@
+# Generated by Django 5.2.7 on 2025-10-30 10:46
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('submissions', '0005_alter_submission_notes'),
+    ]
+
+    operations = [
+        migrations.RemoveField(
+            model_name='puzzleresponse',
+            name='ocr_confidence_score',
+        ),
+        migrations.AddField(
+            model_name='puzzleresponse',
+            name='ocr_confidence_area',
+            field=models.FloatField(blank=True, help_text='OCR confidence score for area (0.0 to 1.0)', null=True),
+        ),
+        migrations.AddField(
+            model_name='puzzleresponse',
+            name='ocr_confidence_cost',
+            field=models.FloatField(blank=True, help_text='OCR confidence score for cost (0.0 to 1.0)', null=True),
+        ),
+        migrations.AddField(
+            model_name='puzzleresponse',
+            name='ocr_confidence_cycles',
+            field=models.FloatField(blank=True, help_text='OCR confidence score for cycles (0.0 to 1.0)', null=True),
+        ),
+    ]

opus_submitter/submissions/models.py

@@ -295,8 +295,15 @@ class PuzzleResponse(models.Model):
     needs_manual_validation = models.BooleanField(
         default=False, help_text="Whether OCR failed and manual validation is needed"
     )
-    ocr_confidence_score = models.FloatField(
+
-        null=True, blank=True, help_text="OCR confidence score (0.0 to 1.0)"
+    ocr_confidence_cost = models.FloatField(
+        null=True, blank=True, help_text="OCR confidence score for cost (0.0 to 1.0)"
+    )
+    ocr_confidence_cycles = models.FloatField(
+        null=True, blank=True, help_text="OCR confidence score for cycles (0.0 to 1.0)"
+    )
+    ocr_confidence_area = models.FloatField(
+        null=True, blank=True, help_text="OCR confidence score for area (0.0 to 1.0)"
     )
 
     # Manual validation overrides

opus_submitter/submissions/schemas.py

@@ -25,7 +25,9 @@ class PuzzleResponseIn(Schema):
     cycles: Optional[str] = None
     area: Optional[str] = None
     needs_manual_validation: bool = False
-    ocr_confidence_score: Optional[float] = None
+    ocr_confidence_cost: Optional[float] = None
+    ocr_confidence_cycles: Optional[float] = None
+    ocr_confidence_area: Optional[float] = None
 
 
 class SubmissionIn(Schema):
@@ -73,7 +75,9 @@ class PuzzleResponseOut(ModelSchema):
             "cycles",
             "area",
             "needs_manual_validation",
-            "ocr_confidence_score",
+            "ocr_confidence_cost",
+            "ocr_confidence_cycles",
+            "ocr_confidence_area",
             "validated_cost",
             "validated_cycles",
             "validated_area",