sasa-maillist/BOUNCE_HANDLING_SETUP.md

SES SNS Bounce Handling Setup (Optional)

⚠️ NOTICE: Bounce handling is optional and disabled by default.

This document describes how to configure AWS SES and SNS to handle email bounces automatically in the Mail List Manager.

Prerequisites:

• SES production access (not available in sandbox mode)
• Valid HTTPS domain for webhook endpoint
• Bounce handling must be enabled in configuration

Overview

The system can optionally use AWS Simple Notification Service (SNS) to receive real-time bounce notifications from AWS Simple Email Service (SES). When bounce handling is enabled and an email bounces:

1. SES sends a notification to an SNS topic
2. SNS forwards the notification to your webhook endpoint
3. The API processes the notification and updates the database
4. Members with hard bounces are automatically deactivated
5. Bounce history is tracked and displayed in the UI

Bounce Status Types

• Clean: No bounces recorded
• Soft Bounce: Temporary delivery issues (e.g., mailbox full, temporary server issues)
  • After 3 soft bounces, the member is marked with soft bounce status
• Hard Bounce: Permanent delivery failure (e.g., invalid email address, domain doesn't exist)
  • Member is automatically deactivated and cannot receive emails

Setup Instructions

1. Enable Bounce Handling

First, enable bounce handling in your .env file:

# Enable SNS webhook bounce handling
ENABLE_SNS_WEBHOOKS=true
ENABLE_BOUNCE_HANDLING=true

Restart the API container after making this change:

sudo docker-compose restart api

2. Prerequisites

• AWS account with SES configured and verified
• Your Mail List Manager deployed and accessible via HTTPS (required for SNS webhook)
• Domain or subdomain for webhook (e.g., https://lists.yourdomain.com)

3. Create SNS Topic

1. Log in to AWS Console and navigate to SNS
2. Click "Create topic"
3. Choose "Standard" topic type
4. Name: ses-bounce-notifications (or your preferred name)
5. Display name: SES Bounce Notifications
6. Click "Create topic"
7. Save the Topic ARN (you'll need it in step 4) arn:aws:sns:eu-west-2:827164363113:ses-bounces

4. Subscribe Your Webhook to SNS Topic

1. In the SNS topic details, click "Create subscription"
2. Protocol: HTTPS
3. Endpoint: https://yourdomain.com:8000/webhooks/sns
   • Replace yourdomain.com with your actual domain
   • The API must be accessible via HTTPS (SNS doesn't support HTTP)
4. Enable raw message delivery: Unchecked
5. Click "Create subscription"
6. The subscription will be in "PendingConfirmation" status

5. Confirm SNS Subscription

When you create the subscription, SNS will send a SubscriptionConfirmation request to your webhook endpoint. The Mail List Manager API automatically confirms this subscription.

1. Check your API logs:

   sudo docker-compose logs -f api
   

2. You should see a log entry indicating the subscription was confirmed
3. In the AWS SNS console, refresh the subscriptions list
4. The status should change from "PendingConfirmation" to "Confirmed"

6. Configure SES to Send Bounce Notifications

1. Navigate to AWS SES console
2. Go to "Configuration Sets" (or "Verified identities" > select your domain > "Notifications")
3. For configuration sets:
   • Create a new configuration set or select existing
   • Add "Event destination"
   • Event types: Select "Bounce" (and optionally "Complaint")
   • Destination: SNS topic
   • Select your SNS topic created in step 2
4. For verified identities:
   • Select your sending domain/email
   • Click "Edit" in the "Notifications" section
   • Bounce feedback: Select your SNS topic
   • Include original headers: Enabled (optional)
   • Click "Save changes"

7. Verify Setup

Test with a Bounce Simulator

AWS SES provides bounce simulator addresses:

# From inside Postfix container
docker-compose exec postfix bash
echo "Test bounce" | mail -s "Test" bounce@simulator.amazonses.com

Or send to your mailing list with a test recipient:

1. Add bounce@simulator.amazonses.com as a member
2. Subscribe to a test list
3. Send an email to the list

Check the Results

1. Wait a few minutes for SES to process and send the notification
2. Check API logs:

   sudo docker-compose logs api | grep -i bounce
   

3. Log in to the web UI
4. Go to Members tab
5. Find the test member and click the "Bounces" button
6. You should see the bounce event recorded

8. Security Considerations

SNS Signature Verification

The webhook endpoint automatically verifies SNS message signatures to ensure notifications are genuine AWS messages. This prevents unauthorized parties from sending fake bounce notifications.

HTTPS Requirement

SNS requires HTTPS for webhooks. You'll need:

• Valid SSL/TLS certificate for your domain
• Reverse proxy (e.g., Nginx, Apache) in front of the API container
• Or use AWS API Gateway as a proxy

Example Nginx Configuration

server {
    listen 443 ssl http2;
    server_name lists.yourdomain.com;
    
    ssl_certificate /path/to/cert.pem;
    ssl_certificate_key /path/to/key.pem;
    
    # Webhook endpoint
    location /webhooks/sns {
        proxy_pass http://localhost:8000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
    
    # Optional: proxy API for web UI
    location /api {
        proxy_pass http://localhost:8000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

9. Managing Bounces in the UI

View Bounce Status

In the Members tab, bounced emails are indicated with:

• Warning badge showing bounce count
• Color-coded status (yellow for soft bounce, red for hard bounce)
• Last bounce timestamp

View Bounce History

1. Click the "Bounces" button next to a member
2. View detailed bounce history including:
   • Bounce type (Permanent, Transient, Undetermined)
   • Bounce subtype
   • Diagnostic code from the receiving mail server
   • Timestamp of each bounce

Reset Bounce Status

If a member's email has been corrected or verified:

1. Open the bounce history modal
2. Click "Reset Bounce Status"
3. Confirm the action
4. The member's bounce count is cleared and they can receive emails again

Note: Only users with write access (administrators and operators) can reset bounce status.

10. Monitoring and Maintenance

Check Bounce Logs

# View all bounces in database
sudo docker-compose exec mysql mysql -u maillist -p maillist -e "SELECT * FROM bounce_logs ORDER BY timestamp DESC LIMIT 20;"

# Count bounces by type
sudo docker-compose exec mysql mysql -u maillist -p maillist -e "SELECT bounce_type, COUNT(*) as count FROM bounce_logs GROUP BY bounce_type;"

# Find members with bounces
sudo docker-compose exec mysql mysql -u maillist -p maillist -e "SELECT name, email, bounce_count, bounce_status FROM members WHERE bounce_count > 0;"

API Health Check

# Check if webhook is accessible
curl -X POST https://yourdomain.com:8000/webhooks/sns \
  -H "Content-Type: application/json" \
  -d '{"Type":"test"}'

Clean Up Old Bounce Records

Periodically review and clean up old bounce records:

-- Delete bounce logs older than 90 days
DELETE FROM bounce_logs WHERE created_at < DATE_SUB(NOW(), INTERVAL 90 DAY);

Troubleshooting

SNS Subscription Not Confirming

• Ensure the API container is running and accessible via HTTPS
• Check API logs for errors
• Verify firewall rules allow HTTPS traffic to port 8000
• Test the endpoint manually: curl https://yourdomain.com:8000/health

Bounces Not Being Recorded

1. Verify SNS topic is receiving messages:
   • Check SNS topic metrics in AWS Console
2. Verify subscription is active:
   • Check subscription status in SNS console
3. Check API logs for webhook errors:

   sudo docker-compose logs api | grep -i "sns\|bounce"
   

4. Test signature verification:
   • Temporarily add debug logging to the webhook endpoint

Members Not Being Deactivated

• Check if bounce type is "Permanent"
• Review member's bounce_status in database:

  sudo docker-compose exec mysql mysql -u maillist -p maillist -e "SELECT * FROM members WHERE email='problem@example.com';"
  

• Verify bounce processing logic in API logs

SSL Certificate Issues

If using self-signed certificates, SNS will reject the webhook. You must use:

• Valid certificate from a trusted CA (Let's Encrypt, etc.)
• Or use AWS Certificate Manager with API Gateway

Additional Resources

• AWS SES Bounce Handling
• AWS SNS HTTPS Subscriptions
• SES Bounce Types