Camera-Follows-Aim Calibration Guide

🎯 New Targeting System Overview

With the camera mounted to move with the aim mechanism, targeting is now much simpler and more accurate:

Goal: Center the target mouth in the camera view

Red crosshair = aiming point
Blue circle = target deadzone (30 pixel radius)
When mouth is in deadzone = ready to fire

🔧 Calibration Parameters

Edit these values in octv2_server_v2.py to match your hardware:

1. Targeting Sensitivity

# In MouthDetector.__init__():
self.target_deadzone_pixels = 30      # Larger = more tolerant centering
self.max_adjustment_degrees = 10      # Smaller = more precise movements

# In calculate_centering_adjustment():
pixels_per_degree_rotation = 15       # Adjust for your setup
pixels_per_degree_pitch = 12      # Adjust for your setup

2. Distance Estimation

# Camera parameters (measure your actual camera)
self.camera_focal_length_mm = 3.04    # Pi Camera focal length
self.sensor_width_mm = 3.68           # Pi Camera sensor width
self.average_face_width_cm = 16.0     # Average human face width

3. Aiming Offsets

# Mechanical compensation
self.rotation_offset_degrees = 0.0    # If camera/launcher not aligned
self.pitch_offset_degrees = 0.0   # For gravity/drop compensation
self.distance_pitch_factor = 0.5  # Higher pitch for closer targets

🎮 Calibration Process

Step 1: Basic Functionality Test

# Run the server
python3 octv2_server_v2.py

# Test manual aiming first:
# 1. Use iOS app manual controls
# 2. Check that ESP32 responds to commands
# 3. Verify camera moves with motors

Step 2: Centering Calibration

Position test subject at known distance (e.g., 1 meter)
Switch to AUTO mode in iOS app

Open mouth wide and observe behavior:

Expected sequence:
1. Detects WIDE_OPEN mouth
2. Calculates centering adjustment
3. Moves motors to center the mouth
4. Fires when mouth is in target zone

Step 3: Pixel-to-Degree Tuning

If the system over/under-corrects, adjust these values:

# Too much movement (overshoots):
pixels_per_degree_rotation = 20       # Increase value
pixels_per_degree_pitch = 16      # Increase value

# Too little movement (doesn't reach target):
pixels_per_degree_rotation = 10       # Decrease value
pixels_per_degree_pitch = 8       # Decrease value

Step 4: Distance Accuracy Check

Test distance estimation accuracy:

Measure actual distance to test subject
Check displayed distance on video overlay

Adjust parameters if needed:

# If distance reads too high:
self.average_face_width_cm = 15.0     # Decrease face width

# If distance reads too low:
self.average_face_width_cm = 17.0     # Increase face width

Step 5: Elevation Compensation

For accurate trajectory at different distances:

# If shooting too low at close range:
self.distance_pitch_factor = 0.7  # Increase compensation

# If shooting too high at close range:
self.distance_pitch_factor = 0.3  # Decrease compensation

📊 Understanding the Algorithm

Distance Estimation Formula:

distance = (face_width_real * focal_length * image_width) / (face_width_pixels * sensor_width)

Centering Logic:

# 1. Calculate offset from center
dx = mouth_x - center_x
dy = mouth_y - center_y

# 2. Convert to angle adjustments
rotation_adj = dx / pixels_per_degree_rotation
pitch_adj = -dy / pixels_per_degree_pitch

# 3. Apply distance scaling (closer = smaller adjustments)
distance_factor = 100.0 / estimated_distance
adjusted_pixels_per_degree *= distance_factor

# 4. Add compensation offsets
rotation_adj += rotation_offset_degrees
pitch_adj += pitch_offset_degrees + distance_compensation

🎯 Visual Feedback Elements

On Video Stream:

🟢 Green thick border + "🎯 TARGET!" = Will fire at this mouth
🟠 Orange border "SPEAKING" = Ignored
🟡 Cyan border "SMILING" = Ignored
⚪ Gray border "CLOSED" = Ignored
Red crosshair = Current aim point
Blue circle = Target zone (deadzone)
Distance display = "~150cm" for WIDE_OPEN mouths

In Logs:

🎯 AUTO TARGET: Mouth detected (confidence 0.85, distance ~120cm)
🎯 CENTERING: Adjusting R:+2.3° E:-1.5° -> R:15.3° E:28.5°
🔥 AUTO FIRE: Launching Oreo at centered target! (offset: 12px)

🔧 Common Issues & Solutions

System doesn't fire:

Check mouth is truly WIDE_OPEN (not just speaking)
Verify mouth is detected as green target
Ensure mouth gets centered in blue circle

Overshooting targets:

Decrease pixels_per_degree_rotation/pitch values
Increase max_adjustment_degrees for smaller steps

Undershooting targets:

Increase pixels_per_degree_rotation/pitch values
Check motor gear ratios match ESP32 firmware

Wrong distance estimates:

Measure actual face width of test subject
Adjust average_face_width_cm accordingly
Verify camera focal length specification

Systematic aiming errors:

Use offset parameters to compensate:

self.rotation_offset_degrees = -2.0   # Aim 2° left
self.pitch_offset_degrees = 1.5   # Aim 1.5° higher

🎪 Testing Tips

Start with stationary targets - easier to tune
Use consistent lighting - improves detection
Test at multiple distances - 0.5m, 1m, 2m, 3m
Mark successful positions - note what worked
Incremental adjustments - change one parameter at a time

🍪 Advanced Features

Distance-Based Trajectory:

The system automatically adjusts pitch based on distance:

Closer targets (50-100cm) = higher pitch
Farther targets (200cm+) = lower pitch

Iterative Centering:

If target not centered on first try:

System makes smaller adjustments each iteration
Fires when target enters deadzone
Max 10° adjustment per iteration prevents overshooting

Your OCTv2 now has precision camera-follows-aim targeting with distance compensation! 🎯📷

camera_aim_calibration.md 6.1 KB History Raw