Video KYC Agent

The Video KYC Agent provides automated identity verification by processing video files containing Aadhaar card information. It extracts and validates identity details from both audio (spoken information) and visual (Aadhaar card image) components of the video, performing cross-validation to generate a verification score.

Base URL

/api/agents/kyc_agent

Authentication

All endpoints require authentication. Sign up to the https://nextneural.superteams.ai to get your API key.

How It Works

The Video KYC Agent performs comprehensive identity verification:

Audio Processing: Extracts audio from video, transcribes speech, and extracts Aadhaar details
Visual Processing: Detects Aadhaar card in video frames and extracts text using OCR
Cross-Validation: Compares audio and visual data to calculate a verification score (0-100)
Quality Assessment: Analyzes multiple frames to find the best quality image for processing

Endpoints

1. Health Check

Check if the Video KYC agent service is running.

Endpoint: GET /health

Authentication: None required

Response:

{
  "status": "healthy",
  "service": "VIDEO KYC Agent"
}

2. Process KYC Video

Process a video file to extract and verify Aadhaar card information from both audio and visual components.

Endpoint: POST /kyc_process

Authentication: Required

Request Body:

{
  "file_name": "kyc_video_12345.mp4",
  "document_id": 456,
  "force_reprocess": false
}

Parameters:

file_name (required): Filename of the video in the media directory
document_id (required): Reference to the knowledge base document ID for ownership verification
force_reprocess (optional, default: false): Re-process video even if already analyzed

Request Example:

curl -X POST "https://nextneural-api.superteams.ai/api/agents/kyc_agent/kyc_process" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "file_name": "customer_kyc.mp4",
    "document_id": 456,
    "force_reprocess": false
  }'

Response (Successful Processing):

{
  "id": 789,
  "kb_document_id": 456,
  "user_id": 1,
  "audio_data": {
    "name": "Rajesh Kumar Singh",
    "dob": "15/08/1990",
    "aadhaar_number": "1234 5678 9012"
  },
  "audio_transcript": "My name is Rajesh Kumar Singh. My date of birth is 15th August 1990. My Aadhaar number is 1234 5678 9012.",
  "visual_data": {
    "name": "RAJESH KUMAR SINGH",
    "dob": "15/08/1990",
    "gender": "Male",
    "aadhar_no": "1234 5678 9012",
    "confidence": 0.92,
    "frame_name": "customer_kyc_frame000045_20250114T103000Z.jpg"
  },
  "verification_score": 95.5,
  "warning_message": null,
  "created_at": "2025-01-14T10:30:00Z",
  "updated_at": "2025-01-14T10:30:00Z",
  "is_reevaluation": false,
  "was_cached": false
}

Response (Cached Result):

{
  "id": 789,
  "kb_document_id": 456,
  "user_id": 1,
  "audio_data": { ... },
  "visual_data": { ... },
  "verification_score": 95.5,
  "warning_message": null,
  "created_at": "2025-01-14T10:30:00Z",
  "updated_at": "2025-01-14T10:30:00Z",
  "is_reevaluation": false,
  "was_cached": true
}

Response (With Warnings):

{
  "id": 790,
  "kb_document_id": 457,
  "verification_score": 45.0,
  "warning_message": "Incomplete data extracted from video. Please upload a clearer video",
  "visual_data": {
    "name": "RAJESH KUMAR",
    "dob": null,
    "aadhar_no": null,
    "confidence": 0.65
  },
  ...
}

Notes:

The video file must exist in the configured media directory
Document ownership is verified before processing
If force_reprocess=true, existing KYC data will be deleted and re-processed
The system automatically selects the best frame from the video for OCR processing
Verification score is calculated by comparing audio and visual data:
- Name match: 40 points (exact) or 20 points (partial)
- DOB match: 20 points
- Aadhaar number match: 40 points
Maximum verification score is 100

3. Get My KYC Records

Retrieve all KYC records for the authenticated user.

Endpoint: GET /my_kyc_records

Authentication: Required

Query Parameters:

limit (optional, default: 100): Maximum number of records to return

Request Example:

curl -X GET "https://nextneural-api.superteams.ai/api/agents/kyc_agent/my_kyc_records?limit=50" \
  -H "Authorization: Bearer YOUR_TOKEN"

Response:

{
  "success": true,
  "userId": 1,
  "count": 3,
  "records": [
    {
      "id": 789,
      "analyzedAt": "2025-01-14T10:30:00Z",
      "verificationScore": 95.5,
      "name": "Rajesh Kumar Singh"
    },
    {
      "id": 788,
      "analyzedAt": "2025-01-13T15:20:00Z",
      "verificationScore": 88.0,
      "name": "Priya Sharma"
    },
    {
      "id": 787,
      "analyzedAt": "2025-01-12T09:45:00Z",
      "verificationScore": 72.5,
      "name": "Unknown"
    }
  ]
}

Notes:

Returns records in reverse chronological order (newest first)
Only returns records belonging to the authenticated user
name field shows the name from audio data, falling back to visual data or "Unknown"

4. Get Specific KYC Record

Retrieve detailed information for a specific KYC record.

Endpoint: GET /kyc_record/{record_id}

Authentication: Required

Path Parameters:

record_id (required): The ID of the KYC record to retrieve

Request Example:

curl -X GET "https://nextneural-api.superteams.ai/api/agents/kyc_agent/kyc_record/789" \
  -H "Authorization: Bearer YOUR_TOKEN"

Response:

{
  "success": true,
  "record": {
    "id": 789,
    "kb_document_id": 456,
    "user_id": 1,
    "audio_data": {
      "name": "Rajesh Kumar Singh",
      "dob": "15/08/1990",
      "aadhaar_number": "1234 5678 9012"
    },
    "audio_transcript": "My name is Rajesh Kumar Singh. My date of birth is 15th August 1990. My Aadhaar number is 1234 5678 9012.",
    "visual_data": {
      "name": "RAJESH KUMAR SINGH",
      "dob": "15/08/1990",
      "gender": "Male",
      "aadhar_no": "1234 5678 9012",
      "confidence": 0.92,
      "frame_name": "customer_kyc_frame000045_20250114T103000Z.jpg"
    },
    "verification_score": 95.5,
    "warning_message": null,
    "created_at": "2025-01-14T10:30:00Z",
    "updated_at": "2025-01-14T10:30:00Z",
    "is_reevaluation": false,
    "was_cached": false
  }
}

Notes:

Only the record owner can access their KYC records
Returns 404 if record doesn't exist or user doesn't have access

Data Models

Audio Data Structure

{
  "name": "Full Name",
  "dob": "DD/MM/YYYY",
  "aadhaar_number": "XXXX XXXX XXXX"
}

Visual Data Structure

{
  "name": "FULL NAME",
  "dob": "DD/MM/YYYY",
  "gender": "Male/Female",
  "aadhar_no": "XXXX XXXX XXXX",
  "confidence": 0.92,
  "frame_name": "filename_frameXXXXXX_timestamp.jpg"
}

Field Descriptions:

Audio Data:

name: Name extracted from speech transcription
dob: Date of birth in DD/MM/YYYY format
aadhaar_number: 12-digit Aadhaar number with spaces

Visual Data:

name: Name extracted from Aadhaar card OCR (usually uppercase)
dob: Date of birth extracted from card
gender: Gender information from card
aadhar_no: Aadhaar number from card OCR
confidence: Detection confidence score (0-1)
frame_name: Filename of the extracted frame used for OCR

Verification Score Calculation

The verification score (0-100) is calculated by comparing audio and visual data:

Comparison	Points	Condition
Name - Exact Match	40	Audio name exactly matches visual name
Name - Partial Match	20	Some name parts match between audio and visual
DOB - Exact Match	20	Date of birth matches exactly
Aadhaar - Exact Match	40	Aadhaar number matches exactly

Score Interpretation:

90-100: Excellent match, high confidence verification
70-89: Good match, acceptable verification
50-69: Moderate match, manual review recommended
0-49: Poor match, verification failed

Error Responses

All endpoints may return the following error responses:

400 Bad Request:

{
  "detail": "document_id is required for security validation"
}

403 Forbidden:

{
  "detail": "Document does not belong to this user"
}

404 Not Found:

{
  "detail": "File not found in media directory."
}

404 Not Found (Record):

{
  "detail": "KYC record not found or you don't have access to it"
}

500 Internal Server Error:

{
  "detail": "Error processing video: [error message]"
}

Warning Messages

The system may return warning messages in certain scenarios:

"No Aadhaar card detected in video": Vision model failed to detect any Aadhaar card in the analyzed frames
"Incomplete data extracted from video. Please upload a clearer video": Some required fields (name, DOB, or Aadhaar number) are missing from the extracted data

Best Practices

Video Quality Requirements

Resolution: Minimum 720p recommended
Duration: 10-30 seconds optimal
Lighting: Good, even lighting on the Aadhaar card
Focus: Card should be clearly visible and in focus
Audio: Clear speech, minimal background noise
Content: User should speak name, DOB, and Aadhaar number clearly

Recording Guidelines

Hold the Aadhaar card steady in frame for at least 3-5 seconds
Ensure card is flat and not tilted
Avoid glare or reflections on the card
Speak clearly and at moderate pace
Pronounce Aadhaar number digit by digit
Use proper date format (day, month, year)

Integration Workflow

# 1. Upload video to media directory (via your file upload system)
# Video file: customer_kyc_20250114.mp4

# 2. Process the video for KYC verification
curl -X POST "https://nextneural-api.superteams.ai/api/agents/kyc_agent/kyc_process" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "file_name": "customer_kyc_20250114.mp4",
    "document_id": 456
  }'

# 3. Check verification score and validate
# If score >= 70: Approved
# If score < 70: Manual review or re-capture

# 4. Retrieve record later if needed
curl -X GET "https://nextneural-api.superteams.ai/api/agents/kyc_agent/kyc_record/789" \
  -H "Authorization: Bearer YOUR_TOKEN"

# 5. List all KYC records for auditing
curl -X GET "https://nextneural-api.superteams.ai/api/agents/kyc_agent/my_kyc_records" \
  -H "Authorization: Bearer YOUR_TOKEN"

Performance Considerations

Processing Time: Typically 15-30 seconds per video
Frame Analysis: Default analyzes up to 30 frames (configurable)
Caching: Results are cached; re-processing requires force_reprocess=true
Concurrent Processing: Multiple videos can be processed simultaneously
Storage: Cropped Aadhaar frames are saved for audit purposes

Security Features

Document Ownership Verification: Ensures users can only process their own documents
User Isolation: KYC records are strictly isolated by user ID
Authentication: All endpoints require valid API tokens with appropriate scopes
Frame Storage: Extracted frames stored with unique timestamps for audit trails
Data Validation: Cross-validation between audio and visual data prevents fraud

Troubleshooting

Low Verification Scores

Possible Causes:

Poor video quality or lighting
Unclear audio or background noise
Card not fully visible in frame
Incorrect pronunciation of details
Translation errors for non-English speech

Solutions:

Re-record video with better quality
Improve lighting conditions
Speak more clearly and slowly
Hold card steady in frame
Use force_reprocess=true to retry

No Aadhaar Card Detected

Possible Causes:

Card not visible in any frame
Card too small or far from camera
Poor lighting or focus
Card obstructed or partially hidden

Solutions:

Ensure card is clearly visible
Move card closer to camera
Improve lighting
Hold card steady for longer duration

Missing Data Fields

Possible Causes:

OCR failed to extract text
Text on card is blurry or unclear
Partial card visibility
Damaged or worn card

Solutions:

Use clearer video with better resolution
Ensure entire card is visible
Clean card surface before recording
Adjust camera angle to avoid glare

Base URL​

Authentication​

How It Works​

Endpoints​

1. Health Check​

2. Process KYC Video​

3. Get My KYC Records​

4. Get Specific KYC Record​

Data Models​

Audio Data Structure​

Visual Data Structure​

Verification Score Calculation​

Error Responses​

Warning Messages​

Best Practices​

Video Quality Requirements​

Recording Guidelines​

Integration Workflow​

Performance Considerations​

Security Features​

Troubleshooting​

Low Verification Scores​

No Aadhaar Card Detected​

Missing Data Fields​

Base URL

Authentication

How It Works

Endpoints

1. Health Check

2. Process KYC Video

3. Get My KYC Records

4. Get Specific KYC Record

Data Models

Audio Data Structure

Visual Data Structure

Verification Score Calculation

Error Responses

Warning Messages

Best Practices

Video Quality Requirements

Recording Guidelines

Integration Workflow

Performance Considerations

Security Features

Troubleshooting

Low Verification Scores

No Aadhaar Card Detected

Missing Data Fields