OCTv2/raspberry_pi_server

OCTv2 (Oreo Cookie Thrower v2) - Advanced System

Enhanced system with ESP32 motor control and automatic mouth detection.

🚀 System Architecture

iOS App ←→ Raspberry Pi ←→ ESP32 ←→ Stepper Motors
    ↓            ↓              ↓
Video Stream  AI Vision    Hardware Control

🏗️ Hardware Components

Raspberry Pi

• Main controller running Python server
• Camera for video streaming and mouth detection
• Serial connection to ESP32 (USB/UART)

ESP32 Motor Controller

• Two stepper motors with A4988 drivers
• Rotation motor: -90° to +90° (horizontal aiming)
• Elevation motor: 0° to 60° (vertical aiming)
• Fire mechanism: Servo or solenoid
• Limit switches for homing

Motors & Mechanics

• Stepper motors: 200 steps/rev with 16x microstepping
• Gear ratios: Configurable (5:1 rotation, 3:1 pitch)
• Precision: ~0.1° accuracy with proper gearing

🚀 Quick Setup

1. Raspberry Pi Setup

# Copy files to Pi
scp -r raspberry_pi_server/ pi@your-pi-ip:~/octv2/

# SSH into Pi
ssh pi@your-pi-ip
cd ~/octv2

# Install dependencies
sudo apt update
sudo apt install python3-picamera2 python3-opencv python3-serial
pip3 install -r requirements.txt

2. ESP32 Setup

Hardware Connections:

ESP32 Pin → Component
  2,3,4   → Rotation Stepper (Step, Dir, Enable)
  6,7,8   → Elevation Stepper (Step, Dir, Enable)
  5,9     → Limit Switches (Rotation, Elevation)
  10      → Fire Servo
  11      → Fire Solenoid (alternative)
  13      → Status LED

Programming:

1. Open esp32_firmware/octv2_motor_controller.ino in Arduino IDE
2. Install libraries: AccelStepper, ESP32Servo
3. Configure your hardware pins/ratios if needed
4. Upload to ESP32

3. Run the System

# Connect ESP32 to Pi via USB
# Check USB device (usually /dev/ttyUSB0 or /dev/ttyACM0)
ls /dev/tty*

# Update port in Python code if needed
# Edit octv2_server.py line: ESP32Controller(port='/dev/ttyUSB0')

# Run the enhanced server (will wait for ESP32 connection)
python3 octv2_server.py
# Note: Server will retry ESP32 connection indefinitely until successful

4. iOS App Connection

Same as before - connect to Pi's IP address on port 8080.

🎯 New Features

🤖 Automatic Mode

• Face detection using OpenCV Haar cascades
• Mouth detection within detected faces
• Auto-aiming calculates angles from pixel coordinates
• Auto-firing with 2-second cooldown between shots
• Visual feedback with detection overlay on video stream

🎮 Enhanced Manual Control

• Precise positioning with stepper motors
• Real-time position feedback from ESP32
• Smooth acceleration and speed control
• Limit protection and homing capability

📡 Serial Protocol

Commands sent to ESP32:

HOME                    → Home both motors
MOVE 45.0 30.0         → Move to absolute position (rot, elev)
REL -5.0 2.5           → Move relative to current position
FIRE                   → Trigger fire mechanism
POS                    → Get current position
STATUS                 → Get detailed status
STOP                   → Emergency stop

Responses from ESP32:

OK                     → Command successful
ERROR: message         → Command failed
45.0 30.0             → Position response (rotation pitch)
HOMED:1 ROT:45.0 ...  → Status response

🔧 Configuration

Camera Calibration

Edit in octv2_server.py:

# Camera FOV for targeting calculations
self.camera_fov_h = 62.2  # Horizontal FOV degrees
self.camera_fov_v = 48.8  # Vertical FOV degrees

Motor Configuration

Edit in octv2_motor_controller.ino:

#define ROTATION_GEAR_RATIO   5.0   // Your gear ratio
#define ELEVATION_GEAR_RATIO  3.0   // Your gear ratio
#define STEPS_PER_REVOLUTION  200   // Your stepper motor
#define MICROSTEPS           16     // Your driver setting

Detection Sensitivity

Edit in octv2_server.py:

# Auto mode timing
self.target_cooldown = 2.0        # Seconds between shots
self.auto_fire_enabled = True     # Enable/disable auto firing

# Detection parameters
faces = self.face_cascade.detectMultiScale(gray, 1.3, 5)  # Adjust sensitivity
mouths = self.mouth_cascade.detectMultiScale(face_roi, 1.3, 5)

🐛 Troubleshooting

ESP32 Connection Issues

# Check USB connection
lsusb | grep -i esp
dmesg | tail

# Check permissions
sudo usermod -a -G dialout $USER
# Logout and login

# Test serial connection
screen /dev/ttyUSB0 115200
# Type "STATUS" and press Enter

Motor Not Moving

1. Check power supply - Steppers need adequate current
2. Verify wiring - Step/Dir/Enable pins
3. Test motors individually - Use Arduino examples
4. Check driver settings - Microstepping, current limit

Mouth Detection Not Working

# Test camera
libcamera-hello --preview

# Check OpenCV installation
python3 -c "import cv2; print(cv2.__version__)"

# Test detection manually
python3 -c "
import cv2
face_cascade = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_frontalface_default.xml')
print('Cascade loaded:', face_cascade.empty() == False)
"

Poor Auto-Aiming Accuracy

1. Calibrate camera FOV - Measure actual field of view
2. Adjust gear ratios - Match your mechanical setup
3. Fine-tune detection - Modify cascade parameters
4. Add manual offset - Compensate for systematic errors

🎮 iOS App Changes

The iOS app works unchanged! All new features are handled on the Pi side:

• Auto mode toggle activates mouth detection
• Video stream now shows detection overlays
• Manual controls have improved precision
• Status updates include ESP32 connection info

🍪 Usage Tips

Manual Mode

• Use Aim Left/Right for quick adjustments
• Tap camera area for photo capture during streaming
• Home button recalibrates motor positions

Auto Mode

• Toggle to Auto and step back
• System will automatically detect and target open mouths
• Green rectangles = high confidence detection
• Yellow rectangles = lower confidence
• Red crosshair = aiming center

Optimal Setup

• Good lighting improves detection accuracy
• Clear background reduces false positives
• Eye-level mounting works best for face detection
• 2-3 meter range for optimal targeting

🔒 Safety Notes

• Emergency stop always available via STOP command
• Limit switches prevent mechanical damage
• Soft limits in software prevent over-travel
• Auto-fire cooldown prevents rapid-fire accidents

🍪 Happy Advanced Oreo Launching!

Your OCTv2 now has AI-powered targeting and precision stepper control!