Fixed bounce handling

2025-10-13 16:53:22 +00:00
parent 72f3297a80
commit d37027ee5a
5 changed files with 660 additions and 41 deletions
--- a/SES_BOUNCE_TESTING_GUIDE.md
+++ b/SES_BOUNCE_TESTING_GUIDE.md
@@ -0,0 +1,398 @@
+# Testing Bounces from SES Sandbox
+
+AWS SES provides **built-in bounce simulator addresses** that work even in sandbox mode. This guide shows you how to use them to test your bounce handling.
+
+## Quick Answer
+
+Send email to these special AWS addresses to simulate different bounce types:
+
+### Hard Bounce (Permanent - Invalid Address)
+```bash
+bounce@simulator.amazonses.com
+```
+
+### Soft Bounce (Temporary - Mailbox Full)
+```bash
+ooto@simulator.amazonses.com
+```
+
+### Complaint (Spam Report)
+```bash
+complaint@simulator.amazonses.com
+```
+
+### Successful Delivery (No Bounce)
+```bash
+success@simulator.amazonses.com
+```
+
+## Step-by-Step Testing Guide
+
+### Option 1: Using Your Mailing Lists (Recommended)
+
+This tests the complete flow: Postfix → SES → SNS → API
+
+1. **Add the simulator address as a member:**
+   ```bash
+   # Using the API
+   curl -X POST http://localhost:8000/members \
+     -H "Authorization: Bearer $API_TOKEN" \
+     -H "Content-Type: application/json" \
+     -d '{
+       "name": "Bounce Test",
+       "email": "bounce@simulator.amazonses.com",
+       "active": true
+     }'
+   ```
+
+   Or use the Web UI:
+   - Go to http://localhost:3000
+   - Members tab → Add Member
+   - Name: `Bounce Test`
+   - Email: `bounce@simulator.amazonses.com`
+
+2. **Subscribe to a test list:**
+   - Click "Subscriptions" button for the test member
+   - Toggle on one of your mailing lists
+
+3. **Send email to the list:**
+   ```bash
+   # From inside Postfix container
+   sudo docker compose exec postfix bash
+   echo "This will bounce" | mail -s "Test Bounce" community@lists.sasalliance.org
+   exit
+   ```
+
+   Replace `community@lists.sasalliance.org` with your actual list email.
+
+4. **Wait 30-60 seconds** for:
+   - Email to be sent via SES
+   - SES to process the bounce
+   - SNS to send notification to your webhook
+
+5. **Check the results:**
+
+   **Watch API logs in real-time:**
+   ```bash
+   sudo docker compose logs api -f
+   ```
+   
+   You should see:
+   ```
+   ============================================================
+   SNS Webhook Request Received
+   ============================================================
+   Content-Type: text/plain; charset=UTF-8
+   User-Agent: Amazon Simple Notification Service Agent
+   
+   ✓ Notification Received
+     Notification Type: Bounce
+   
+   ✓ Processing Bounce
+     Bounce Type: Permanent
+     Recipients: ['bounce@simulator.amazonses.com']
+     ✓ Bounce processed successfully
+   ============================================================
+   ```
+
+   **Check the database:**
+   ```bash
+   # View bounce logs
+   sudo docker compose exec mysql mysql -u maillist -pmaillist maillist \
+     -e "SELECT * FROM bounce_logs ORDER BY created_at DESC LIMIT 5;"
+   
+   # Check member status
+   sudo docker compose exec mysql mysql -u maillist -pmaillist maillist \
+     -e "SELECT email, active, bounce_count, bounce_status FROM members WHERE email='bounce@simulator.amazonses.com';"
+   ```
+
+   **View in Web UI:**
+   - Open http://localhost:3000
+   - Go to Members tab
+   - Find "Bounce Test" member
+   - Should show: ❌ Inactive (red), bounce badge, last bounce timestamp
+   - Click "Bounces" button to see detailed history
+
+### Option 2: Direct Email via Postfix (Simpler)
+
+Send directly to the simulator without going through your list:
+
+```bash
+# Enter Postfix container
+sudo docker compose exec postfix bash
+
+# Send test email
+echo "Testing hard bounce" | mail -s "Hard Bounce Test" bounce@simulator.amazonses.com
+
+# Or soft bounce
+echo "Testing soft bounce" | mail -s "Soft Bounce Test" ooto@simulator.amazonses.com
+
+# Exit container
+exit
+```
+
+### Option 3: Using AWS SES Console (For Non-Sandbox Testing)
+
+If you have SES production access:
+
+1. Go to AWS SES Console
+2. Click "Send test email"
+3. To: `bounce@simulator.amazonses.com`
+4. From: Your verified email/domain
+5. Subject: "Bounce test"
+6. Body: "Testing bounce handling"
+7. Click "Send test email"
+
+## Testing Different Bounce Types
+
+### 1. Hard Bounce (Permanent Failure)
+```bash
+echo "Test" | mail -s "Test" bounce@simulator.amazonses.com
+```
+
+**Expected Result:**
+- Member marked as `hard_bounce`
+- Member deactivated (`active = 0`)
+- `bounce_count` incremented
+- Entry in `bounce_logs` table
+
+### 2. Soft Bounce (Transient Failure)
+```bash
+echo "Test" | mail -s "Test" ooto@simulator.amazonses.com
+```
+
+**Expected Result:**
+- Member marked as `clean` (first time)
+- After 3 soft bounces → `soft_bounce` status
+- `bounce_count` incremented
+- Member stays active
+
+### 3. Complaint (Spam Report)
+```bash
+echo "Test" | mail -s "Test" complaint@simulator.amazonses.com
+```
+
+**Expected Result:**
+- API receives complaint notification
+- Currently logged but not processed (you can extend the handler)
+
+## Monitoring the Test
+
+### Real-Time Monitoring (Recommended)
+
+Open 3 terminal windows:
+
+**Terminal 1 - API Logs:**
+```bash
+sudo docker compose logs api -f
+```
+
+**Terminal 2 - Postfix Logs:**
+```bash
+sudo docker compose logs postfix -f
+```
+
+**Terminal 3 - Send Test Email:**
+```bash
+sudo docker compose exec postfix bash
+echo "Test" | mail -s "Bounce Test" bounce@simulator.amazonses.com
+```
+
+### Timeline
+
+Here's what happens and when:
+
+- **T+0s**: Email sent to Postfix
+- **T+1-3s**: Postfix relays to SES
+- **T+5-10s**: SES processes and generates bounce
+- **T+10-30s**: SNS sends notification to your webhook
+- **T+30-60s**: API processes bounce and updates database
+
+## Verifying the Complete Flow
+
+### 1. Check Postfix Logs
+```bash
+sudo docker compose logs postfix | grep bounce@simulator
+```
+
+Should show:
+```
+postfix/smtp[xxx]: ... to=<bounce@simulator.amazonses.com>, relay=email-smtp.eu-west-2.amazonaws.com[...], ... status=sent
+```
+
+### 2. Check SNS Subscription Status
+- Go to AWS SNS Console
+- Find your topic
+- Check "Subscriptions" tab
+- Status should be "Confirmed"
+- Messages delivered should be > 0
+
+### 3. Check API Logs
+```bash
+sudo docker compose logs api | grep -A 20 "SNS Webhook"
+```
+
+Should show successful processing.
+
+### 4. Check Database
+```bash
+sudo docker compose exec mysql mysql -u maillist -pmaillist maillist <<EOF
+-- Show recent bounces
+SELECT 
+    b.bounce_id,
+    m.email,
+    b.bounce_type,
+    b.diagnostic_code,
+    b.timestamp
+FROM bounce_logs b
+JOIN members m ON b.member_id = m.member_id
+ORDER BY b.timestamp DESC
+LIMIT 5;
+
+-- Show members with bounces
+SELECT 
+    email,
+    active,
+    bounce_count,
+    bounce_status,
+    last_bounce_at
+FROM members
+WHERE bounce_count > 0;
+EOF
+```
+
+## Troubleshooting
+
+### "Email not sent" or "Relay access denied"
+
+**Problem**: Postfix not configured to relay via SES
+
+**Check**:
+```bash
+sudo docker compose exec postfix postconf relayhost
+sudo docker compose exec postfix postconf smtp_sasl_auth_enable
+```
+
+Should show:
+```
+relayhost = [email-smtp.eu-west-2.amazonaws.com]:587
+smtp_sasl_auth_enable = yes
+```
+
+### "No bounce received after 5 minutes"
+
+**Possible causes**:
+
+1. **SNS subscription not confirmed**
+   - Check AWS SNS console
+   - Status should be "Confirmed", not "Pending"
+
+2. **SNS topic not configured in SES**
+   - Check SES → Configuration Sets or Verified Identities → Notifications
+   - Bounce notifications should point to your SNS topic
+
+3. **Webhook endpoint not accessible**
+   - SNS requires HTTPS
+   - Test: `curl https://your-domain.com:8000/health`
+
+4. **API container not running**
+   ```bash
+   sudo docker compose ps api
+   ```
+
+### "Bounce received but not in database"
+
+**Check API logs for errors**:
+```bash
+sudo docker compose logs api | grep -i error
+```
+
+**Check database tables exist**:
+```bash
+sudo docker compose exec mysql mysql -u maillist -pmaillist maillist -e "SHOW TABLES;"
+```
+
+Should include: `bounce_logs`, `members`
+
+## Testing Multiple Bounces
+
+To test the "3 soft bounces = soft_bounce status" logic:
+
+```bash
+sudo docker compose exec postfix bash
+
+# Send 3 emails to soft bounce simulator
+for i in {1..3}; do
+    echo "Soft bounce test $i" | mail -s "Test $i" ooto@simulator.amazonses.com
+    sleep 70  # Wait between sends for SNS processing
+done
+```
+
+After the 3rd bounce:
+- Member's `bounce_status` should change from `clean` to `soft_bounce`
+- `bounce_count` should be 3
+
+## Cleanup After Testing
+
+Remove test bounce data:
+
+```bash
+sudo docker compose exec mysql mysql -u maillist -pmaillist maillist <<EOF
+-- Delete test bounce logs
+DELETE FROM bounce_logs WHERE email IN ('bounce@simulator.amazonses.com', 'ooto@simulator.amazonses.com');
+
+-- Reset test member
+UPDATE members 
+SET bounce_count = 0, 
+    bounce_status = 'clean', 
+    last_bounce_at = NULL,
+    active = 1
+WHERE email IN ('bounce@simulator.amazonses.com', 'ooto@simulator.amazonses.com');
+EOF
+```
+
+Or use the Web UI:
+- Go to Members tab
+- Find the test member
+- Click "Bounces" button
+- Click "Reset Bounce Status"
+
+## Alternative: Simulate Bounces Without SES
+
+If SNS isn't set up yet, use the included script to simulate bounces directly in the database:
+
+```bash
+./simulate_bounce.sh
+```
+
+This is useful for:
+- Testing the UI without AWS
+- Development environments
+- Demonstrating bounce handling to stakeholders
+
+## Next Steps
+
+Once bounce handling is working:
+
+1. **Remove simulator addresses** from your member list
+2. **Monitor real bounces** in production
+3. **Set up alerts** for high bounce rates
+4. **Review bounced members** regularly and update/remove invalid addresses
+5. **Consider complaint handling** (similar to bounces, for spam reports)
+
+## Summary Commands
+
+**Quick test sequence**:
+```bash
+# 1. Watch logs
+sudo docker compose logs api -f &
+
+# 2. Send test bounce
+echo "Test" | sudo docker compose exec -T postfix mail -s "Bounce Test" bounce@simulator.amazonses.com
+
+# 3. Wait 60 seconds, then check database
+sleep 60
+sudo docker compose exec mysql mysql -u maillist -pmaillist maillist -e "SELECT * FROM bounce_logs ORDER BY created_at DESC LIMIT 1;"
+```
+
+That's it! The bounce simulator is the easiest way to test your bounce handling without needing real bounced emails.