Legrems/opus-submitter
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

change for ocr confidence

Browse Source
This commit is contained in:
Loïc Gremaud 2025-10-30 12:01:49 +01:00
parent b5f31a8c72
commit 2260c7cc27
16 changed files with 237 additions and 295 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

231
opus_submitter/API_USAGE.md
View File

@ -1,231 +0,0 @@
# Opus Magnum Submission API Usage
## Overview
The API is built with Django Ninja and provides endpoints for managing puzzle submissions with OCR validation and S3 file storage.
## Base URL
- Development: `http://localhost:8000/api/`
- API Documentation: `http://localhost:8000/api/docs/`
## Authentication
Most endpoints support both authenticated and anonymous submissions. Admin endpoints require staff permissions.
## Environment Variables
### Required for S3 Storage
```bash
USE_S3=true
AWS_ACCESS_KEY_ID=your_access_key
AWS_SECRET_ACCESS_KEY=your_secret_key
AWS_STORAGE_BUCKET_NAME=your_bucket_name
AWS_S3_REGION_NAME=us-east-1
```
### Optional
```bash
STEAM_API_KEY=your_steam_api_key
```
## API Endpoints
### 1. Get Available Puzzles
```http
GET /api/submissions/puzzles
```
Response:
```json
[
{
"id": 1,
"steam_item_id": "3479143948",
"title": "P41-FLOC",
"author_name": "Flame Legrems",
"description": "A challenging puzzle...",
"tags": ["puzzle", "chemistry"],
"order_index": 0,
"steam_url": "https://steamcommunity.com/workshop/filedetails/?id=3479143948",
"created_at": "2025-05-29T11:19:24Z",
"updated_at": "2025-05-30T22:15:09Z"
}
]
```
### 2. Create Submission
```http
POST /api/submissions/submissions
Content-Type: multipart/form-data
```
Form Data:
- `data`: JSON with submission data
- `files`: Array of uploaded files
Example data:
```json
{
"notes": "My best solutions so far",
"responses": [
{
"puzzle_id": 1,
"puzzle_name": "P41-FLOC",
"cost": "150",
"cycles": "89",
"area": "12",
"needs_manual_validation": false,
"ocr_confidence_score": 0.95
}
]
}
```
Response:
```json
{
"id": "123e4567-e89b-12d3-a456-426614174000",
"user": null,
"notes": "My best solutions so far",
"responses": [
{
"id": 1,
"puzzle": 1,
"puzzle_name": "P41-FLOC",
"cost": "150",
"cycles": "89",
"area": "12",
"needs_manual_validation": false,
"files": [
{
"id": 1,
"original_filename": "solution.gif",
"file_size": 1024000,
"content_type": "image/gif",
"file_url": "https://bucket.s3.amazonaws.com/media/submissions/123.../file.gif",
"ocr_processed": false,
"created_at": "2025-10-29T00:00:00Z"
}
],
"final_cost": "150",
"final_cycles": "89",
"final_area": "12"
}
],
"total_responses": 1,
"needs_validation": false,
"is_validated": false,
"created_at": "2025-10-29T00:00:00Z"
}
```
### 3. List Submissions
```http
GET /api/submissions/submissions?limit=20&offset=0
```
### 4. Get Submission Details
```http
GET /api/submissions/submissions/{submission_id}
```
### 5. Admin: Validate Response (Staff Only)
```http
PUT /api/submissions/responses/{response_id}/validate
Content-Type: application/json
```
Body:
```json
{
"validated_cost": "150",
"validated_cycles": "89",
"validated_area": "12"
}
```
### 6. Admin: List Responses Needing Validation (Staff Only)
```http
GET /api/submissions/responses/needs-validation
```
### 7. Admin: Validate Entire Submission (Staff Only)
```http
POST /api/submissions/submissions/{submission_id}/validate
```
### 8. Get Statistics
```http
GET /api/submissions/stats
```
Response:
```json
{
"total_submissions": 150,
"total_responses": 300,
"needs_validation": 25,
"validated_submissions": 120,
"validation_rate": 0.8
}
```
## OCR Validation Logic
The system automatically flags responses for manual validation when:
1. **Incomplete OCR Data**: Missing cost, cycles, or area values
2. **Low Confidence**: OCR confidence score below threshold
3. **Manual Flag**: Explicitly marked by frontend OCR processing
### Manual Validation Workflow
1. Admin views responses needing validation: `GET /responses/needs-validation`
2. Admin reviews the uploaded files and OCR results
3. Admin provides corrected values: `PUT /responses/{id}/validate`
4. System updates `validated_*` fields and clears validation flag
5. Optional: Mark entire submission as validated: `POST /submissions/{id}/validate`
## File Storage
- **Development**: Files stored locally in `media/submissions/`
- **Production**: Files stored in S3 with path structure: `submissions/{submission_id}/{uuid}_{filename}`
- **Supported Formats**: JPEG, PNG, GIF, MP4, WebM
- **Size Limit**: 10MB per file
## Error Handling
The API returns standard HTTP status codes:
- `200`: Success
- `400`: Bad Request (validation errors)
- `401`: Unauthorized
- `403`: Forbidden (admin required)
- `404`: Not Found
- `500`: Internal Server Error
Error Response Format:
```json
{
"detail": "Error message",
"code": "error_code"
}
```
## Frontend Integration
The Vue frontend should:
1. Upload files with OCR data extracted client-side
2. Group files by detected puzzle name
3. Create one submission with multiple responses
4. Handle validation flags and display admin feedback
5. Show file upload progress and S3 URLs
## Admin Interface
Django Admin provides:
- **Submission Management**: View, validate, and manage submissions
- **Response Validation**: Bulk actions for validation workflow
- **File Management**: View uploaded files and OCR data
- **Statistics Dashboard**: Track validation rates and submission metrics

55
opus_submitter/src/components/AdminPanel.vue
View File

@ -48,14 +48,41 @@
</td>
<td>
<div class="text-sm space-y-1">
<div>Cost: {{ response.cost || '-' }}</div>
<div>Cycles: {{ response.cycles || '-' }}</div>
<div>Area: {{ response.area || '-' }}</div>
<div class="flex justify-between items-center">
<span>Cost: {{ response.cost || '-' }}</span>
<span
v-if="response.ocr_confidence_cost"
class="badge badge-xs"
:class="getConfidenceBadgeClass(response.ocr_confidence_cost)"
>
{{ Math.round(response.ocr_confidence_cost * 100) }}%
</span>
</div>
<div class="flex justify-between items-center">
<span>Cycles: {{ response.cycles || '-' }}</span>
<span
v-if="response.ocr_confidence_cycles"
class="badge badge-xs"
:class="getConfidenceBadgeClass(response.ocr_confidence_cycles)"
>
{{ Math.round(response.ocr_confidence_cycles * 100) }}%
</span>
</div>
<div class="flex justify-between items-center">
<span>Area: {{ response.area || '-' }}</span>
<span
v-if="response.ocr_confidence_area"
class="badge badge-xs"
:class="getConfidenceBadgeClass(response.ocr_confidence_area)"
>
{{ Math.round(response.ocr_confidence_area * 100) }}%
</span>
</div>
</div>
</td>
<td>
<div class="badge badge-warning badge-sm">
{{ response.ocr_confidence_score ? Math.round(response.ocr_confidence_score * 100) + '%' : 'Low' }}
{{ getOverallConfidence(response) }}%
</div>
</td>
<td>
@ -273,6 +300,26 @@ onMounted(() => {
loadData()
})
// Helper functions for confidence display
const getConfidenceBadgeClass = (confidence: number): string => {
if (confidence >= 0.8) return 'badge-success'
if (confidence >= 0.6) return 'badge-warning'
return 'badge-error'
}
const getOverallConfidence = (response: PuzzleResponse): number => {
const confidences = [
response.ocr_confidence_cost,
response.ocr_confidence_cycles,
response.ocr_confidence_area
].filter(conf => conf !== undefined && conf !== null) as number[]
if (confidences.length === 0) return 0
const average = confidences.reduce((sum, conf) => sum + conf, 0) / confidences.length
return Math.round(average * 100)
}
// Expose refresh method
defineExpose({
refresh: loadData

46
opus_submitter/src/components/FileUpload.vue
View File

@ -83,7 +83,17 @@
<div v-else-if="file.ocrData" class="mt-1 space-y-1">
<div class="text-xs flex items-center justify-between">
<span class="font-medium text-success">✓ OCR Complete</span>
<div class="flex items-center gap-2">
<span class="font-medium text-success">✓ OCR Complete</span>
<span
v-if="file.ocrData.confidence"
class="badge badge-xs"
:class="getConfidenceBadgeClass(file.ocrData.confidence.overall)"
:title="`Overall confidence: ${Math.round(file.ocrData.confidence.overall * 100)}%`"
>
{{ Math.round(file.ocrData.confidence.overall * 100) }}%
</span>
</div>
<button
@click="retryOCR(file)"
class="btn btn-xs btn-ghost"
@ -95,15 +105,43 @@
<div class="text-xs space-y-1 bg-base-200 p-2 rounded">
<div v-if="file.ocrData.puzzle">
<strong>Puzzle:</strong> {{ file.ocrData.puzzle }}
<span
v-if="file.ocrData.confidence?.puzzle"
class="ml-2 opacity-60"
:title="`Puzzle confidence: ${Math.round(file.ocrData.confidence.puzzle * 100)}%`"
>
({{ Math.round(file.ocrData.confidence.puzzle * 100) }}%)
</span>
</div>
<div v-if="file.ocrData.cost">
<strong>Cost:</strong> {{ file.ocrData.cost }}
<span
v-if="file.ocrData.confidence?.cost"
class="ml-2 opacity-60"
:title="`Cost confidence: ${Math.round(file.ocrData.confidence.cost * 100)}%`"
>
({{ Math.round(file.ocrData.confidence.cost * 100) }}%)
</span>
</div>
<div v-if="file.ocrData.cycles">
<strong>Cycles:</strong> {{ file.ocrData.cycles }}
<span
v-if="file.ocrData.confidence?.cycles"
class="ml-2 opacity-60"
:title="`Cycles confidence: ${Math.round(file.ocrData.confidence.cycles * 100)}%`"
>
({{ Math.round(file.ocrData.confidence.cycles * 100) }}%)
</span>
</div>
<div v-if="file.ocrData.area">
<strong>Area:</strong> {{ file.ocrData.area }}
<span
v-if="file.ocrData.confidence?.area"
class="ml-2 opacity-60"
:title="`Area confidence: ${Math.round(file.ocrData.confidence.area * 100)}%`"
>
({{ Math.round(file.ocrData.confidence.area * 100) }}%)
</span>
</div>
</div>
</div>
@ -306,4 +344,10 @@ const processOCR = async (submissionFile: SubmissionFile) => {
const retryOCR = (submissionFile: SubmissionFile) => {
processOCR(submissionFile)
}
const getConfidenceBadgeClass = (confidence: number): string => {
if (confidence >= 0.8) return 'badge-success'
if (confidence >= 0.6) return 'badge-warning'
return 'badge-error'
}
</script>

8
opus_submitter/src/services/apiService.ts
View File

@ -122,7 +122,9 @@ export class ApiService {
cycles?: string
area?: string
needs_manual_validation?: boolean
ocr_confidence_score?: number
ocr_confidence_cost?: number
ocr_confidence_cycles?: number
ocr_confidence_area?: number
}>
},
files: File[]
@ -268,7 +270,9 @@ export const submissionHelpers = {
cycles: fileWithOCR?.ocrData?.cycles,
area: fileWithOCR?.ocrData?.area,
needs_manual_validation: needsValidation,
ocr_confidence_score: needsValidation ? 0.5 : 0.9 // Rough estimate
ocr_confidence_cost: fileWithOCR?.ocrData?.confidence?.cost || 0.0,
ocr_confidence_cycles: fileWithOCR?.ocrData?.confidence?.cycles || 0.0,
ocr_confidence_area: fileWithOCR?.ocrData?.confidence?.area || 0.0
}
})

30
opus_submitter/src/services/ocrService.ts
View File

@ -5,6 +5,13 @@ export interface OpusMagnumData {
cost: string;
cycles: string;
area: string;
confidence: {
puzzle: number;
cost: number;
cycles: number;
area: number;
overall: number;
};
}
export interface OCRRegion {
@ -64,6 +71,7 @@ export class OpusMagnumOCRService {
// Extract text from each region
const results: Partial<OpusMagnumData> = {};
const confidenceScores: Record<string, number> = {};
for (const [key, region] of Object.entries(this.regions)) {
const regionCanvas = document.createElement('canvas');
@ -108,9 +116,12 @@ export class OpusMagnumOCRService {
}
// Perform OCR on the region
const { data: { text } } = await this.worker!.recognize(regionCanvas);
const { data: { text, confidence } } = await this.worker!.recognize(regionCanvas);
let cleanText = text.trim();
// Store the confidence score for this field
confidenceScores[key] = confidence / 100; // Tesseract returns 0-100, we want 0-1
// Post-process based on field type
if (key === 'cost') {
// Handle common OCR misreadings where G is read as 6
@ -134,16 +145,29 @@ export class OpusMagnumOCRService {
cleanText = this.findBestPuzzleMatch(cleanText);
}
results[key as keyof OpusMagnumData] = cleanText;
(results as any)[key] = cleanText;
}
URL.revokeObjectURL(imageUrl);
// Calculate overall confidence as the average of all field confidences
const confidenceValues = Object.values(confidenceScores);
const overallConfidence = confidenceValues.length > 0
? confidenceValues.reduce((sum, conf) => sum + conf, 0) / confidenceValues.length
: 0;
resolve({
puzzle: results.puzzle || '',
cost: results.cost || '',
cycles: results.cycles || '',
area: results.area || ''
area: results.area || '',
confidence: {
puzzle: confidenceScores.puzzle || 0,
cost: confidenceScores.cost || 0,
cycles: confidenceScores.cycles || 0,
area: confidenceScores.area || 0,
overall: overallConfidence
}
});
} catch (error) {
URL.revokeObjectURL(imageUrl);

11
opus_submitter/src/types/index.ts
View File

@ -29,6 +29,13 @@ export interface OpusMagnumData {
cost: string
cycles: string
area: string
confidence: {
puzzle: number
cost: number
cycles: number
area: number
overall: number
}
}
export interface SubmissionFile {
@ -49,7 +56,9 @@ export interface PuzzleResponse {
cycles?: string
area?: string
needs_manual_validation?: boolean
ocr_confidence_score?: number
ocr_confidence_cost?: number
ocr_confidence_cycles?: number
ocr_confidence_area?: number
validated_cost?: string
validated_cycles?: string
validated_area?: string

17
opus_submitter/static_source/vite/assets/main-CNlI4PW6.js
View File

File diff suppressed because one or more lines are too long

17
opus_submitter/static_source/vite/assets/main-Cv9F8wz5.js Normal file
View File

File diff suppressed because one or more lines are too long

2
opus_submitter/static_source/vite/assets/main-HDjkw-xK.css → opus_submitter/static_source/vite/assets/main-DWmJTLXa.css
View File

File diff suppressed because one or more lines are too long

4
opus_submitter/static_source/vite/manifest.json
View File

@ -16,12 +16,12 @@
"src": "node_modules/.pnpm/@mdi+font@7.4.47/node_modules/@mdi/font/fonts/materialdesignicons-webfont.woff2"
},
"src/main.ts": {
"file": "assets/main-CNlI4PW6.js",
"file": "assets/main-Cv9F8wz5.js",
"name": "main",
"src": "src/main.ts",
"isEntry": true,
"css": [
"assets/main-HDjkw-xK.css"
"assets/main-DWmJTLXa.css"
],
"assets": [
"assets/materialdesignicons-webfont-CSr8KVlo.eot",

4
opus_submitter/submissions/admin.py
View File

@ -160,7 +160,7 @@ class PuzzleResponseInline(admin.TabularInline):
readonly_fields = ["created_at", "updated_at"]
fields = [
"puzzle", "puzzle_name", "cost", "cycles", "area",
"needs_manual_validation", "ocr_confidence_score"
"needs_manual_validation", "ocr_confidence_cost", "ocr_confidence_cycles", "ocr_confidence_area"
]
@ -235,7 +235,7 @@ class PuzzleResponseAdmin(admin.ModelAdmin):
"fields": ("submission", "puzzle", "puzzle_name")
}),
("OCR Data", {
"fields": ("cost", "cycles", "area", "ocr_confidence_score")
"fields": ("cost", "cycles", "area", "ocr_confidence_cost", "ocr_confidence_cycles", "ocr_confidence_area")
}),
("Validation", {
"fields": (

4
opus_submitter/submissions/api.py
View File

@ -92,7 +92,9 @@ def create_submission(
cycles=response_data.cycles,
area=response_data.area,
needs_manual_validation=response_data.needs_manual_validation,
ocr_confidence_score=response_data.ocr_confidence_score,
ocr_confidence_cost=response_data.ocr_confidence_cost,
ocr_confidence_cycles=response_data.ocr_confidence_cycles,
ocr_confidence_area=response_data.ocr_confidence_area,
)
# Process files for this response

32
opus_submitter/submissions/migrations/0006_remove_puzzleresponse_ocr_confidence_score_and_more.py Normal file
View File

@ -0,0 +1,32 @@
# Generated by Django 5.2.7 on 2025-10-30 10:46
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('submissions', '0005_alter_submission_notes'),
]
operations = [
migrations.RemoveField(
model_name='puzzleresponse',
name='ocr_confidence_score',
),
migrations.AddField(
model_name='puzzleresponse',
name='ocr_confidence_area',
field=models.FloatField(blank=True, help_text='OCR confidence score for area (0.0 to 1.0)', null=True),
),
migrations.AddField(
model_name='puzzleresponse',
name='ocr_confidence_cost',
field=models.FloatField(blank=True, help_text='OCR confidence score for cost (0.0 to 1.0)', null=True),
),
migrations.AddField(
model_name='puzzleresponse',
name='ocr_confidence_cycles',
field=models.FloatField(blank=True, help_text='OCR confidence score for cycles (0.0 to 1.0)', null=True),
),
]

11
opus_submitter/submissions/models.py
View File

@ -295,8 +295,15 @@ class PuzzleResponse(models.Model):
needs_manual_validation = models.BooleanField(
default=False, help_text="Whether OCR failed and manual validation is needed"
)
ocr_confidence_score = models.FloatField(
null=True, blank=True, help_text="OCR confidence score (0.0 to 1.0)"
ocr_confidence_cost = models.FloatField(
null=True, blank=True, help_text="OCR confidence score for cost (0.0 to 1.0)"
)
ocr_confidence_cycles = models.FloatField(
null=True, blank=True, help_text="OCR confidence score for cycles (0.0 to 1.0)"
)
ocr_confidence_area = models.FloatField(
null=True, blank=True, help_text="OCR confidence score for area (0.0 to 1.0)"
)
# Manual validation overrides

8
opus_submitter/submissions/schemas.py
View File

@ -25,7 +25,9 @@ class PuzzleResponseIn(Schema):
cycles: Optional[str] = None
area: Optional[str] = None
needs_manual_validation: bool = False
ocr_confidence_score: Optional[float] = None
ocr_confidence_cost: Optional[float] = None
ocr_confidence_cycles: Optional[float] = None
ocr_confidence_area: Optional[float] = None
class SubmissionIn(Schema):
@ -73,7 +75,9 @@ class PuzzleResponseOut(ModelSchema):
"cycles",
"area",
"needs_manual_validation",
"ocr_confidence_score",
"ocr_confidence_cost",
"ocr_confidence_cycles",
"ocr_confidence_area",
"validated_cost",
"validated_cycles",
"validated_area",