Document-Warehouse-Ver-2.0-POC
Mobile PDF Scan Upload Workfl
Overview
This document describes the workflow for a Next.js PWA mobile interface that allows users to scan vendor bills from their mobile device and upload them to S3, with AI-powered vendor name verification.
Scripts
INSERT INTO AISystemPrompts (ContextTable, PromptCategory,sysrolename, PromptText, ind)
VALUES
('DocUploadBill','DocUploadBill','BillCreate','Extract from PDF (do not rely only on exact labels, use context):
1. PartyName → Seller / Vendor / Supplier / Company / Billed By / From
OR the entity issuing the invoice (usually at top with address/GSTIN)
2. GSTIN → Any valid GST number (15-character format) found in seller details
3. TotalAmount → Final payable amount (largest or highlighted amount in bill, often at bottom)
Even if labels are missing or different, infer values based on position and context.
Return JSON:
{""PartyName"":"",""GSTIN"":"",""TotalAmount"":0.00}
Return only valid JSON. Do not return false or empty unless value truly not present.','U')
ALTER TABLE Users ADD DocUploadToken UNIQUEIDENTIFIER NULL, DocUploadInfo NVARCHAR(MAX) NULL;
Program.cs changes
builder.Services.AddScoped<IDocUploadRepository, DocUploadRepository>();
builder.Services.AddScoped<IDocUploadService, DocUploadService>();Context
Currently, BillDetails.razor allows PDF uploads via desktop. The goal is to extend this with a mobile scanning capability.
Current Desktop Flow
User Selects PDF → Validate Size (≤5MB) → Generate S3 Key
→ Request Pre-signed URL → Upload to S3 via PUT
→ Update Bill Record → Return to Details PageS3 Path Pattern
s3://erpcrystal-docs/MFGReports/Docs/{dbname}/Bill/{YearBillNo}.pdfSystem Architecture
Current ERPCrystal MFG Architecture
┌─────────────────────────────────────────────────────────────────┐
│ ERPCRYSTAL MFG │
├─────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────┐ ┌─────────────────┐ ┌──────────────┐│
│ │ WEB (Blazor) │◀──▶│ API │◀──▶│ MODELS ││
│ │ 150+ Razor │ │ 80+ Controllers│ │ 100+ Models ││
│ │ Pages │ │ │ │ ││
│ └────────┬────────┘ └────────┬────────┘ └──────────────┘│
│ │ │ │
│ │ ┌──────────────────────────────────────┐ │
│ └───▶│ AWS S3 (erpcrystal-docs) │ │
│ │ MFGReports/Docs/{dbname}/Bill/ │ │
│ └──────────────────────────────────────┘ │
│ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ EXISTING AI/OCR INFRASTRUCTURE │ │
│ │ • PdfExtractorService (CloudFront endpoint) │ │
│ │ • AIInsightsService (AWS Bedrock Nova Lite) │ │
│ └─────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────┘Key Models
PartyMst (Vendor)
public class PartyMst {
public string PartyId { get; set; }
public string PartyTypeCode { get; set; } // "C"=Customer, "V"=Vendor
public string PartyName { get; set; }
public string PartyShortName { get; set; }
public string PartyGstin { get; set; }
// ... 40+ fields
}BillMain
public class BillMain {
public int Id { get; set; }
public string BillNo { get; set; }
public string PartyId { get; set; }
public string PartyName { get; set; }
public string PartyShortName { get; set; }
public string YearBillNo { get; set; } // Composite key: {yearlabel}{billno}
// ... 50+ fields
}Existing Services
S3 Integration
- Configuration:
appsettings.jsonAWS section - Bucket:
erpcrystal-docs - Region:
ap-south-1 - CloudFront:
d2da2gtk8otj3i.cloudfront.net
AI Services
- PdfExtractorService: Sends PDF to CloudFront OCR endpoint
- AIInsightsService: Uses AWS Bedrock Nova Lite (
amazon.nova-lite-v1:0)
Proposed Mobile Workflow
Architecture Diagram
┌─────────────────────────────────────────────────────────────────────────────┐
│ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ NEXT.JS PWA (Mobile) │ │
│ │ ┌───────────┐ ┌───────────┐ ┌───────────┐ ┌───────────┐ │ │
│ │ │ Camera │ │ jsPDF │ │ Service │ │ Upload │ │ │
│ │ │ API │──▶│ PDF Gen │──▶│ Worker │──▶│ Queue │ │ │
│ │ └───────────┘ └───────────┘ └───────────┘ └───────────┘ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │ │
│ │ HTTPS │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ ASP.NET CORE API │ │
│ │ ┌────────────────┐ ┌────────────────┐ ┌────────────────┐ │ │
│ │ │ MobileAuth │ │ MobileUpload │ │ AIInsights │ │ │
│ │ │ Controller │──▶│ Controller │──▶│ Service │ │ │
│ │ └────────────────┘ └────────────────┘ └────────────────┘ │ │
│ │ │ │ │
│ │ ▼ │ │
│ │ ┌────────────────────────────┐ │ │
│ │ │ AWS Bedrock Nova Lite │ │ │
│ │ │ (Vendor Name Extraction) │ │ │
│ │ └────────────────────────────┘ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │ │
│ │ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ AWS S3 │ │
│ │ s3://erpcrystal-docs/MFGReports/Docs/{dbname}/Bill/{YearBillNo}.pdf│ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘Step-by-Step Flow
Step 1: Initiate Scan (Desktop)
User clicks [Scan Vendor's Bill]
│
▼
┌────────────────────────────────────────────┐
│ GeneratePasscode() │
│ Payload: {dbname}_{billId}_{timestamp}} │
│ Expiry: 5 minutes │
│ Display: "Enter code: X7K2M" │
└────────────────────────────────────────────┘API Call:
POST /api/mobileauth/generate
Body: { dbname, billId }
Response:
{
passcode: "X7K2M",
expiresAt: "2024-02-15T14:35:00Z",
billId: 123,
dbname: "erpcrystal_mfg",
partyName: "Crystal Metals Pvt Ltd",
partyShortName: "CRM"
}Step 2: Mobile Authentication
┌─────────────────────────────────────────────────────────────────────┐
│ User enters passcode: X7K2M │
│ │
│ POST /api/mobileauth/validate │
│ Body: { passcode: "X7K2M" } │
│ │
│ Response: │
│ { │
│ authenticated: true, │
│ sessionToken: "jwt_token", │
│ expiresIn: 300 // 5 minutes │
│ } │
└─────────────────────────────────────────────────────────────────────┘Step 3: Scan PDF (Mobile PWA)
Camera Access:
navigator.mediaDevices.getUserMedia({
video: { facingMode: 'environment' } // Rear camera
})PDF Generation (using jsPDF):
import jsPDF from 'jspdf'
const doc = new jsPDF()
doc.addImage(imageData, 'JPEG', x, y, width, height)
doc.save('scan.pdf')PWA Libraries:
| Library | Purpose |
|---|---|
| jsPDF | Create PDF from images, text |
| pdf-lib | Modify/create PDFs in browser |
| html2canvas | Capture DOM as PDF |
| pdfjs-dist | Read/analyze PDFs |
Step 4: Upload PDF (Mobile → Server)
POST /api/mobile/upload
Headers: { Authorization: "Bearer {sessionToken}" }
Content-Type: multipart/form-data
Body:
------Boundary123
Content-Disposition: form-data; name="pdf"; filename="scan.pdf"
Content-Type: application/pdf
[PDF BINARY DATA]
------Boundary123--Step 5: Server Processing
API RECEIVES PDF
│
▼
┌──────────────────────────────────────────────────────────────────┐
│ 1. Read PDF file │
│ 2. Convert to Base64 │
│ 3. Call AIInsightsService with new prompt │
└──────────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────┐
│ AIInsightsService.ExtractVendorName(pdfBase64) │
│ │
│ AWS Bedrock Nova Lite: │
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ PROMPT: │ │
│ │ "Extract vendor/party name from this invoice PDF. │ │
│ │ Return ONLY the vendor name as plain text." │ │
│ └────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────┐
│ Fuzzy Match against PartyName from BillMain │
│ │
│ extracted: "Crystal Metals Pvt Ltd" │
│ expected: "Crystal Metals Pvt Ltd" │
│ similarity: 100% → MATCH │
│ │
│ OR: │
│ │
│ extracted: "Crystal Industries" │
│ expected: "Crystal Metals Pvt Ltd" │
│ similarity: 72% → NO MATCH │
└──────────────────────────────────────────────────────────────────┘Step 6a: Match Success
POST /api/bill/uploadpresignedurl
→ Get S3 pre-signed URL
Mobile uploads PDF to S3 directly
Return to mobile:
{
status: "success",
extractedVendor: "Crystal Metals Pvt Ltd",
matchedVendor: "Crystal Metals Pvt Ltd",
confidence: 1.0,
s3Url: "https://.../2024-25BL00123.pdf"
}Step 6b: Match Failure → Regenerate Passcode
Return to mobile:
{
status: "match_failed",
extractedVendor: "Crystal Industries", // What AI found
expectedVendor: "Crystal Metals Pvt Ltd", // What DB expects
similarity: 0.72,
newPasscode: "K9M4N", // NEW CODE
expiresAt: "2024-02-15T14:40:00Z", // +5 MINUTES
message: "Vendor name doesn't match. Scan again?"
}Mobile UI on Failure:
┌─────────────────────────────────────────────────────────────────┐
│ │
│ ❌ Vendor name doesn't match │
│ │
│ AI found: "Crystal Industries" │
│ Bill expects: "Crystal Metals Pvt Ltd" │
│ │
│ ┌───────────┐ │
│ │ NEW CODE │ K9M4N │
│ │ GENERATED│ (5 min expiry) │
│ └───────────┘ │
│ │
│ ┌───────────┐ │
│ │ SCAN │ Try again │
│ │ AGAIN │ │
│ └───────────┘ │
│ │
└─────────────────────────────────────────────────────────────────┘API Endpoints
POST /api/mobileauth/generate
| Parameter | Type | Description |
|---|---|---|
| dbname | string | Database name |
| billId | int | Bill ID |
Response:
{
"passcode": "X7K2M",
"expiresAt": "2024-02-15T14:35:00Z",
"billId": 123,
"dbname": "erpcrystal_mfg",
"partyName": "Crystal Metals Pvt Ltd",
"partyShortName": "CRM"
}POST /api/mobileauth/validate
| Parameter | Type | Description |
|---|---|---|
| passcode | string | 6-character code |
Response:
{
"authenticated": true,
"sessionToken": "jwt_token",
"expiresIn": 300
}POST /api/mobile/upload
| Header | Value |
|---|---|
| Authorization | Bearer {sessionToken} |
| Content-Type | multipart/form-data |
Body: PDF file as multipart/form-data
Success Response:
{
"status": "success",
"extractedVendor": "Crystal Metals Pvt Ltd",
"matchedVendor": "Crystal Metals Pvt Ltd",
"confidence": 1.0,
"s3Url": "https://..."
}Failure Response:
{
"status": "match_failed",
"extractedVendor": "Crystal Industries",
"expectedVendor": "Crystal Metals Pvt Ltd",
"similarity": 0.72,
"newPasscode": "K9M4N",
"expiresAt": "2024-02-15T14:40:00Z",
"message": "Vendor name doesn't match. Scan again?"
}PUT {preSignedUrl}
Upload PDF binary directly to S3.
Design Decisions
| Decision | Choice | Rationale |
|---|---|---|
| Mobile Platform | Next.js PWA | No app store submission, installable, cross-platform |
| AI Processing | Server-side (Bedrock Nova Lite) | Cost control, simpler mobile app |
| Vendor Matching | Fuzzy match + User confirmation | Balances automation with safety |
| Passcode Flow | Server generates, mobile validates | Lightweight, no QR complexity |
| Bill Context | Encoded in passcode | Keeps mobile app simple |
| S3 Path | Same as desktop uploads | Maintains consistency |
Error Handling
| Scenario | Handling |
|---|---|
| Camera permission denied | Fallback to file picker |
| PDF generation fails | Show error, allow retry |
| Upload timeout | Queue in Service Worker |
| Network offline | Background Sync API |
| AI extraction fails | Return error, prompt retry |
| Session expires during upload | Return 401, prompt new code |
| Multiple consecutive failures | Block after 3 attempts |
PWA Features
- Service Worker: Offline queue for uploads
- Web App Manifest: Install to homescreen
- Push Notifications: Upload complete alerts
- Background Sync: Retry failed uploads
- File System Access API: Direct file handling
Related Files
| Component | File Path |
|---|---|
| Bill Upload UI | ErpCrystal_MFG.Web/Pages/BillUpload.razor |
| Bill Details UI | ErpCrystal_MFG.Web/Pages/BillDetails.razor |
| Bill Service | ErpCrystal_MFG.Web/Services/BillService.cs |
| Bill API Controller | ErpCrystal_MFG.Api/Controllers/BillController.cs |
| Bill Model | ErpCrystal_MFG.Models/Bill.cs |
| Party/Vendor Model | ErpCrystal_MFG.Models/PartyMst.cs |
| S3 Configuration | ErpCrystal_MFG.Web/Components/S3Credentials.cs |
| PDF Extractor | ErpCrystal_MFG.Web/Services/PdfExtractorService.cs |
| API Auth Attribute | ErpCrystal_MFG.Api/CustomAttributes/ApiKeyAuthAttribute.cs |
| AI Insights Service | ErpCrystal_MFG.Web/Services/AIInsightsService.cs |
To Be Created
/api/mobileauth/generateendpoint/api/mobileauth/validateendpoint/api/mobile/uploadendpoint- Next.js PWA mobile application
- Integration with AIInsightsService for vendor extraction
- Fuzzy matching logic
- Passcode generation and validation
- SignalR notification to desktop (optional)
Questions for Further Clarification
Question: When match fails, should the new passcode go back to desktop to generate, or should the server auto-generate and send it to mobile?
Answer: Server alone will create the passcode which should expire in 5 minutes. If first attempt of scanning resulted in error, a second passcode is needed. The server auto-generates and sends the new passcode directly to the mobile client.
Document generated: 2024-02-08 Source: OpenCode exploration session