# Email-Based Bounce Handling Setup

This document explains the email-based bounce handling system implemented as an alternative to SNS webhooks for environments without SES production access.

## Overview

The system processes email bounces directly within the Postfix container by:
1. Rewriting return paths to direct bounces to a processing address
2. Processing bounce emails via Python script
3. Updating member bounce statistics in MySQL database
4. Automatically disabling members with excessive bounces
2. Setting up an alias that pipes bounce emails to a Python processing script
3. The script parses bounce emails, extracts bounced addresses, and updates the database
4. Members with hard bounces are automatically deactivated
5. Bounce history is tracked and displayed in the UI (same as SNS method)

## Advantages

- **Works with SES Sandbox**: No production SES access required
- **No External Dependencies**: Doesn't require SNS, webhooks, or HTTPS domains
- **Self-Contained**: All processing happens within the existing containers
- **Real-time Processing**: Bounces are processed as soon as emails arrive
- **Compatible**: Uses the same database schema and UI as SNS bounce handling

## Configuration

### 1. Enable Email Bounce Processing

In your `.env` file:

```bash
# Enable email-based bounce processing
ENABLE_EMAIL_BOUNCE_PROCESSING=true

# This will automatically enable bounce handling features
ENABLE_BOUNCE_HANDLING=true  # Automatically set to true when email processing is enabled
```

### 2. Restart the System

```bash
sudo docker-compose down
sudo docker-compose up --build -d
```

### 3. Verify Configuration

Check that email bounce processing is enabled:

```bash
curl -s http://localhost:8000/config | jq .
```

Expected output:
```json
{
  "bounce_handling_enabled": true,
  "sns_webhooks_enabled": false,
  "email_bounce_processing_enabled": true
}
```

## How It Works

### Postfix Configuration

The system configures Postfix with:

- `bounce_notice_recipient = bounces@lists.sasalliance.org`
- `2bounce_notice_recipient = bounces@lists.sasalliance.org`
- `error_notice_recipient = bounces@lists.sasalliance.org`

### Aliases Configuration

The `bounces` address is configured to pipe emails to the processing script:

```
bounces: "|/usr/local/bin/process-bounce.py"
```

### Bounce Processing Script

The Python script (`/usr/local/bin/process-bounce.py`):

1. **Reads bounce emails** from stdin (via pipe)
2. **Parses email content** using multiple regex patterns to extract bounced addresses
3. **Analyzes bounce type** based on SMTP error codes:
   - 5xx codes = Permanent bounces
   - 4xx codes = Transient bounces
   - Unknown = Undetermined
4. **Updates database** using the same schema as SNS bounce handling:
   - Logs bounce in `bounce_logs` table
   - Updates member `bounce_count`, `bounce_status`, and `last_bounce_at`
   - Deactivates members with permanent bounces
   - Marks members with soft bounce status after 3 transient bounces

## Testing

### Test the Processing Script

You can test the bounce processing script in test mode:

```bash
# Test with sample bounce email
sudo docker-compose exec postfix /usr/local/bin/process-bounce.py --test
```

Expected output:
```
2025-10-14 15:49:16,041 - bounce-processor - INFO - Starting bounce processing
2025-10-14 15:49:16,041 - bounce-processor - INFO - Running in test mode with sample bounce email
2025-10-14 15:49:16,050 - bounce-processor - INFO - Extracted addresses: ['testuser@example.com']
2025-10-14 15:49:16,050 - bounce-processor - INFO - Test mode - would process 1 bounce(s):
2025-10-14 15:49:16,050 - bounce-processor - INFO -   {'email': 'testuser@example.com', 'bounce_type': 'Permanent', 'bounce_subtype': 'General', 'diagnostic_code': '', 'timestamp': '2025-10-14 15:49:16'}
```

### Test with Real Bounce Email

To test with a real bounce:

1. Send an email to a non-existent address via your mailing list
2. Wait for the bounce to be processed
3. Check the database for bounce logs:

```bash
# Check bounce logs
sudo docker-compose exec mysql mysql -u maillist -pmaillist maillist -e "SELECT * FROM bounce_logs ORDER BY created_at DESC LIMIT 5;"

# Check member bounce status
sudo docker-compose exec mysql mysql -u maillist -pmaillist maillist -e "SELECT member_id, email, bounce_count, bounce_status, last_bounce_at FROM members WHERE bounce_count > 0;"
```

### View in Web Interface

1. Open http://localhost:3000
2. Navigate to the Members tab
3. Look for bounce badges and bounce counts next to member names
4. Click the "Bounces" button next to a member to view bounce history

## Supported Bounce Email Formats

The processing script recognizes these bounce patterns:

### SMTP Error Codes
- 550, 551, 553, 552, 554 (permanent failures)
- 450, 451, 452 (temporary failures)

### Delivery Status Notification (DSN)
- `Final-Recipient:` headers
- `Original-Recipient:` headers

### Common Bounce Messages
- "user unknown"
- "does not exist"
- "not found"
- "mailbox unavailable"
- "recipient rejected"

## Monitoring

### View Processing Logs

```bash
# View bounce processing logs
sudo docker-compose logs postfix | grep bounce-processor

# Follow logs in real-time
sudo docker-compose logs -f postfix | grep bounce-processor
```

### Check Aliases Configuration

```bash
# Verify aliases are configured correctly
sudo docker-compose exec postfix cat /etc/aliases

# Check alias database
sudo docker-compose exec postfix postmap -q bounces /etc/aliases
```

## Troubleshooting

### Bounces Not Being Processed

1. **Check aliases configuration:**
   ```bash
   sudo docker-compose exec postfix cat /etc/aliases
   ```

2. **Verify script permissions:**
   ```bash
   sudo docker-compose exec postfix ls -la /usr/local/bin/process-bounce.py
   ```

3. **Test script manually:**
   ```bash
   sudo docker-compose exec postfix /usr/local/bin/process-bounce.py --test
   ```

4. **Check Postfix logs:**
   ```bash
   sudo docker-compose logs postfix | grep -i bounce
   ```

### Database Connection Issues

1. **Check environment variables:**
   ```bash
   sudo docker-compose exec postfix env | grep MYSQL
   ```

2. **Test database connection:**
   ```bash
   sudo docker-compose exec postfix python3 -c "import pymysql; print('PyMySQL available')"
   ```

### Script Errors

View detailed error logs:
```bash
sudo docker-compose logs postfix | grep -A 10 -B 10 "bounce-processor.*ERROR"
```

## Comparison with SNS Webhooks

| Feature | Email-Based | SNS Webhooks |
|---------|-------------|--------------|
| **SES Requirement** | Sandbox OK | Production access required |
| **External Dependencies** | None | SNS, HTTPS domain |
| **Processing Speed** | Real-time | Real-time |
| **Setup Complexity** | Low | High |
| **Reliability** | High | High |
| **Bounce Detection** | Regex-based | AWS-provided |
| **Cost** | Free | SNS charges apply |

## Next Steps

1. **Monitor bounce processing** to ensure it's working correctly
2. **Review bounce patterns** in the logs to improve detection if needed
3. **Set up bounce notification alerts** (optional)
4. **Consider upgrading to SNS webhooks** when SES production access is available

Email-based bounce handling provides a robust alternative that works immediately with any SES configuration while providing the same bounce management features as the SNS webhook method.