Skip to content

Feature 1: AI Confidence & Uncertainty Estimation #152

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
saidai-bhuvanesh wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Feature 1: AI Confidence & Uncertainty Estimation #152

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
8f7a68c
feat(ai): confidence & uncertainty estimation (#1)
Jul 3, 2026
7702e85
feat(ai): confidence & uncertainty estimation with localizations (#1)
Jul 3, 2026
11cdec3
fix(dashboard): fix low-confidence warning logic to address CodeRabbi…
Jul 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 48 additions & 1 deletion backend/main.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -299,7 +299,7 @@ def _row_to_payload(row: dict) -> dict:
"confidence": round((row.get("confidence_score") or 0) * 100, 1),
"classification": "FRESH" if is_fresh else "SPOILED",
"is_fresh": is_fresh,
"uncertain_flag": False,
"uncertain_flag": (row.get("confidence_score") or 1.0) < 0.70,

@coderabbitai coderabbitai Bot Jul 3, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Logic bug: or 1.0 mishandles a stored 0.0 confidence and contradicts the Line 299 default.

(row.get("confidence_score") or 1.0) short-circuits on any falsy value, so a legitimately stored confidence of 0.0 (maximally uncertain) is replaced with 1.0, yielding uncertain_flag = False — the opposite of intended.

It also disagrees with Line 299, where a missing confidence_score defaults to 0 (→ 0% confidence). So for a row with no score, the payload reports confidence: 0.0 yet uncertain_flag: False. Use an explicit None check and a consistent default.

🐛 Proposed fix 
-        "uncertain_flag": (row.get("confidence_score") or 1.0) < 0.70,
+        "uncertain_flag": (
+            row["confidence_score"] < 0.70
+            if row.get("confidence_score") is not None
+            else True
+        ),
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
"uncertain_flag": (row.get("confidence_score") or 1.0) < 0.70,
"uncertain_flag": (
row["confidence_score"] < 0.70
if row.get("confidence_score") is not None
else True
),
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@backend/main.py` at line 302, The `uncertain_flag` calculation in
`backend/main.py` is treating valid falsy confidence values as missing, so
update the row payload logic to use an explicit `None` check instead of `or
1.0`. In the same code path that builds the response with `confidence_score`,
make `uncertain_flag` derive from the actual stored value from
`row.get("confidence_score")` (using the same default semantics as the nearby
confidence field) so a real `0.0` stays `0.0` and evaluates as uncertain, while
missing values use one consistent default.

"species": {
"common_name": "Rohu Carp",
"scientific_name": "Labeo rohita",
Expand Down Expand Up @@ -528,12 +528,59 @@ async def process_scan(
async def scan_auto(
request: Request,
image: UploadFile = File(...),
freshness_label: Optional[str] = Form(None),
fused_score: Optional[float] = Form(None),
source: Optional[str] = Form(None),
confidence_score: Optional[float] = Form(None),
species_detected: Optional[str] = Form(None),
current_user=Depends(get_current_user),
):
image_bytes = await image.read()
scan_id = str(uuid.uuid4())
display_id = _generate_display_id()

# If edge_onnx path is used, save directly and bypass server inference
if source == "edge_onnx" and fused_score is not None:
freshness = int(fused_score * 100)
conf = confidence_score or 0.85
edge_fusion = {
"final_score_percent": freshness,
"final_grade": _to_db_grade(freshness_label or "C"),
"confidence_score": conf,
"uncertain_prediction_flag": conf < 0.70,
"regional_breakdown": {
"gill_freshness_score": fused_score,
"eye_freshness_score": fused_score,
"body_freshness_score": fused_score,
},
}
photo_url = await _upload_image(image_bytes, str(current_user.id), scan_id)
payload = _build_scan_payload(edge_fusion, scan_id, display_id, photo_url)
if species_detected:
payload["species"]["common_name"] = species_detected

try:
_db().table("scans").insert(
{
"id": scan_id,
"user_id": str(current_user.id),
"final_grade": _to_db_grade(payload["grade"]),
"confidence_score": conf,
"image_type": "BODY",
"freshness_index": payload["freshness_index"],
"scan_display_id": display_id,
"species_detected": species_detected or "Rohu Carp",
"biomarker_json": payload["biomarkers"],
"storage_hours": payload["recommendations"]["consume_within_hours"],
"alert_flags": payload["recommendations"]["alert_flags"],
"photo_urls": [photo_url] if photo_url else [],
}
).execute()
except Exception as exc:
print(f"DB write failed (edge_onnx): {exc}")

return {"success": True, "scan": payload}

# ── Demo mode: models not loaded (PyTorch not installed) ─────────────────
if not _models_loaded:
gill = random.randint(68, 96)
Expand Down
15 changes: 14 additions & 1 deletion src/fusionInference.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -256,6 +256,15 @@ function extractGillScore(logitsB, temperature) {
* @param {number[]} gillProbs [P(Fresh_Gills), P(Nonfresh_Gills)]
* @returns {{ fusedScore: number, label: string, confidence: string }}
*/
function calculateConfidence(bodyProbs, eyeProbs, gillProbs) {
const bodyConf = Math.max(...bodyProbs);
const eyeSubSum = (eyeProbs[0] + eyeProbs[2]) || 1e-7;
const gillSubSum = (gillProbs[1] + gillProbs[3]) || 1e-7;
const eyeConf = Math.max(eyeProbs[0] / eyeSubSum, eyeProbs[2] / eyeSubSum);
const gillConf = Math.max(gillProbs[1] / gillSubSum, gillProbs[3] / gillSubSum);
return (0.5 * bodyConf) + (0.25 * eyeConf) + (0.25 * gillConf);
}
Comment on lines +259 to +266

@coderabbitai coderabbitai Bot Jul 3, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🔴 Critical | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
ast-grep run --pattern 'function extractEyeScore($_, $_) { $$$ }' --lang javascript src/fusionInference.js
ast-grep run --pattern 'function extractGillScore($_, $_) { $$$ }' --lang javascript src/fusionInference.js
rg -nP 'processAndFuse\s*\(' src/fusionInference.js -C2

Repository: jpdevhub/FreshScanAi

Length of output: 2058

🏁 Script executed:

#!/bin/bash
set -euo pipefail

sed -n '180,310p' src/fusionInference.js | cat -n

Repository: jpdevhub/FreshScanAi

Length of output: 5744

🏁 Script executed:

#!/bin/bash
set -euo pipefail

python3 - <<'PY'
from pathlib import Path
text = Path('src/fusionInference.js').read_text()
for name in ['extractBodyScore', 'extractEyeScore', 'extractGillScore', 'calculateConfidence', 'processAndFuse']:
    idx = text.find(f'function {name}')
    print(f'\n### {name} @ {idx}')
    if idx != -1:
        start = max(0, idx-300)
        end = min(len(text), idx+1200)
        print(text[start:end])
PY

Repository: jpdevhub/FreshScanAi

Length of output: 6306

Fix the eye/gill confidence indices

calculateConfidence treats eyeProbs/gillProbs as 4-element vectors, but extractEyeScore() and extractGillScore() pass 2-element arrays. eyeProbs[2] and gillProbs[3] are undefined, so the confidence collapses to NaN and uncertainFlag never trips.

Use indices 0/1 here, or pass the raw 4-class logits instead.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/fusionInference.js` around lines 259 - 268, The confidence calculation in
calculateConfidence is indexing eyeProbs and gillProbs as if they were 4-element
arrays, but extractEyeScore and extractGillScore supply 2-element values, so the
current eye/gill math can produce undefined and NaN. Update calculateConfidence
to use the correct 0/1 indices for the arrays it actually receives, or change
the callers to pass the full logits consistently, and make sure the eye/gill
confidence paths still feed uncertainFlag correctly.


function processAndFuse(bodyProbs, eyeProbs, gillProbs) {
const bodyFresh = bodyProbs[0]; // P(C1 = Fresh)
const eyeFresh = eyeProbs[0]; // P(Fresh_Eyes)
Expand All @@ -274,10 +283,14 @@ function processAndFuse(bodyProbs, eyeProbs, gillProbs) {
label = 'Spoiled';
}

const systemConfidence = calculateConfidence(bodyProbs, eyeProbs, gillProbs);
const isUncertain = systemConfidence < 0.70;

return {
fusedScore,
label,
confidence: (fusedScore * 100).toFixed(1) + '%',
confidence: systemConfidence,
uncertain_flag: isUncertain
};
}

Expand Down
6 changes: 5 additions & 1 deletion src/i18n/locales/bn.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,11 @@
"storageTemp": "সঞ্চয় তাপমাত্রা",
"alertLabel": "সতর্কতা",
"newScanButton": "নতুন স্ক্যান",
"viewHistoryButton": "ইতিহাস দেখুন"
"viewHistoryButton": "ইতিহাস দেখুন",
"uncertainWarningTitle": "এআই পূর্বাভাস অনিশ্চিত",
"uncertainWarningDesc": "মডেলটি ইনপুট মানের মধ্যে উচ্চ বৈচিত্র্য সনাক্ত করেছে (যেমন আলোর ছায়া বা বন্ধ কোণ)। তাজা সূচক স্বাভাবিকের চেয়ে কম নির্ভরযোগ্য হতে পারে।",
"suggestRescan": "→ নমুনাটি পুনরায় স্ক্যান করার পরামর্শ দিন",
"uncertaintyMargin": "ত্রুটি মার্জিন:"
},
"auth": {
"authInitiated": "প্রমাণীকরণ শুরু করা হয়েছে",
Expand Down
6 changes: 5 additions & 1 deletion src/i18n/locales/en.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,11 @@
"storageTemp": "STORAGE TEMP",
"alertLabel": "ALERT",
"newScanButton": "NEW SCAN",
"viewHistoryButton": "VIEW HISTORY"
"viewHistoryButton": "VIEW HISTORY",
"uncertainWarningTitle": "AI Prediction Uncertain",
"uncertainWarningDesc": "The model detected high variance in input quality (e.g. lighting shadows or off-angles). The freshness index might be less reliable than usual.",
"suggestRescan": "→ Suggest Rescanning specimen",
"uncertaintyMargin": "Margin of Error:"
},
"auth": {
"authInitiated": "AUTH INITIATED",
Expand Down
6 changes: 5 additions & 1 deletion src/i18n/locales/hi.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,11 @@
"storageTemp": "भंडारण तापमान",
"alertLabel": "सतर्कता",
"newScanButton": "नया स्कैन",
"viewHistoryButton": "इतिहास देखें"
"viewHistoryButton": "इतिहास देखें",
"uncertainWarningTitle": "एआई भविष्यवाणी अनिश्चित",
"uncertainWarningDesc": "मॉडल ने इनपुट गुणवत्ता में उच्च भिन्नता का पता लगाया (जैसे प्रकाश छाया या बंद कोण)। ताजगी सूचकांक सामान्य से कम विश्वसनीय हो सकता है।",
"suggestRescan": "→ नमूने को फिर से स्कैन करने का सुझाव दें",
"uncertaintyMargin": "त्रुटि मार्जिन:"
},
"auth": {
"authInitiated": "प्रमाणीकरण शुरू किया गया",
Expand Down
40 changes: 36 additions & 4 deletions src/pages/AnalysisDashboard.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@ export default function AnalysisDashboard() {
const { freshness_index, grade, confidence, classification, species, biomarkers, recommendations } = scan;
const displayId = scan.scan_display_id;
const alerts = recommendations.alert_flags;
const uncertain_flag = !!scan.uncertain_flag || (confidence < 70);

return (
<div className="min-h-[calc(100vh-4rem)] px-6 md:px-16 lg:px-24 py-8 md:py-12">
Expand All @@ -109,6 +110,28 @@ export default function AnalysisDashboard() {
className="mb-6"
/>

{uncertain_flag && (
<GlassCard className="p-6 border-l-4 border-error! mb-6 pulse-glow" variant="tonal">
<div className="flex gap-4 items-start">
<AlertTriangle className="text-error shrink-0" size={24} />
<div>
<h4 className="font-bold text-error text-sm mb-1">
{t('dashboard.uncertainWarningTitle', 'AI Prediction Uncertain')}
</h4>
<p className="text-xs text-on-surface-variant leading-relaxed mb-3">
{t('dashboard.uncertainWarningDesc', 'The model detected high variance in input quality (e.g. lighting shadows or off-angles). The freshness index might be less reliable than usual.')}
</p>
<Link
to="/scanner"
className="text-neon font-mono text-[0.625rem] tracking-wider no-underline hover:underline uppercase"
>
{t('dashboard.suggestRescan', '→ Suggest Rescanning specimen')}
</Link>
</div>
</div>
</GlassCard>
)}

{/* Score + Species row */}
<div className="flex flex-col md:flex-row gap-6 mb-8">
{/* Main score card */}
Expand Down Expand Up @@ -147,12 +170,21 @@ export default function AnalysisDashboard() {
</span>

<span
className={`px-2 py-1 border text-xs font-semibold font-[family-name:var(--font-mono)] tracking-widest ${confidence < 70
? "text-error"
: "text-neon"
className={`px-2 py-1 border text-xs font-semibold font-[family-name:var(--font-mono)] tracking-widest ${uncertain_flag
? "text-error border-error!"
: "text-neon border-neon"
}`}
>
{confidence < 70 ? t('dashboard.lowConfidence') : t('dashboard.highConfidence')}
{uncertain_flag ? t('dashboard.lowConfidence', 'UNCERTAIN') : t('dashboard.highConfidence', 'CONFIDENT')}
</span>
</div>

<div className="flex items-center gap-2 mt-4 pt-3 border-t border-outline-variant">
<span className="font-[family-name:var(--font-mono)] text-[0.5625rem] text-on-surface-variant tracking-widest uppercase">
{t('dashboard.uncertaintyMargin', 'Margin of Error:')}
</span>
<span className={`font-[family-name:var(--font-mono)] text-[0.5625rem] font-bold tracking-widest ${uncertain_flag ? 'text-error' : 'text-neon'}`}>
{uncertain_flag ? '±12.5% (High Variance)' : '±3.8% (Calibrated)'}
</span>
</div>
</GlassCard>
Expand Down
4 changes: 3 additions & 1 deletion src/pages/ScannerPage.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ import StatusTerminal from "../components/StatusTerminal";
import CameraOverlay from "../components/CameraOverlay";
import { api, isAuthenticated } from "../lib/api";
import { FishFreshnessInference } from "../fusionInference.js";
import { offlineDb } from "../lib/offlineDb";
import toast from "react-hot-toast";
import type { ScanResult } from "../lib/types";


Expand Down Expand Up @@ -264,7 +266,7 @@ export default function ScannerPage() {
label: fusion.label,
freshness,
grade: deriveGrade(freshness),
confidence: fusion.confidence,
confidence: `${Math.round((fusion.confidence ?? 0.9) * 100)}%`,
});
setScanPhase("done");

Expand Down
Loading