Fixed bounce handling

This commit is contained in:
James Pattinson
2025-10-13 16:53:22 +00:00
parent 72f3297a80
commit d37027ee5a
5 changed files with 660 additions and 41 deletions

398
SES_BOUNCE_TESTING_GUIDE.md Normal file
View File

@@ -0,0 +1,398 @@
# Testing Bounces from SES Sandbox
AWS SES provides **built-in bounce simulator addresses** that work even in sandbox mode. This guide shows you how to use them to test your bounce handling.
## Quick Answer
Send email to these special AWS addresses to simulate different bounce types:
### Hard Bounce (Permanent - Invalid Address)
```bash
bounce@simulator.amazonses.com
```
### Soft Bounce (Temporary - Mailbox Full)
```bash
ooto@simulator.amazonses.com
```
### Complaint (Spam Report)
```bash
complaint@simulator.amazonses.com
```
### Successful Delivery (No Bounce)
```bash
success@simulator.amazonses.com
```
## Step-by-Step Testing Guide
### Option 1: Using Your Mailing Lists (Recommended)
This tests the complete flow: Postfix → SES → SNS → API
1. **Add the simulator address as a member:**
```bash
# Using the API
curl -X POST http://localhost:8000/members \
-H "Authorization: Bearer $API_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"name": "Bounce Test",
"email": "bounce@simulator.amazonses.com",
"active": true
}'
```
Or use the Web UI:
- Go to http://localhost:3000
- Members tab → Add Member
- Name: `Bounce Test`
- Email: `bounce@simulator.amazonses.com`
2. **Subscribe to a test list:**
- Click "Subscriptions" button for the test member
- Toggle on one of your mailing lists
3. **Send email to the list:**
```bash
# From inside Postfix container
sudo docker compose exec postfix bash
echo "This will bounce" | mail -s "Test Bounce" community@lists.sasalliance.org
exit
```
Replace `community@lists.sasalliance.org` with your actual list email.
4. **Wait 30-60 seconds** for:
- Email to be sent via SES
- SES to process the bounce
- SNS to send notification to your webhook
5. **Check the results:**
**Watch API logs in real-time:**
```bash
sudo docker compose logs api -f
```
You should see:
```
============================================================
SNS Webhook Request Received
============================================================
Content-Type: text/plain; charset=UTF-8
User-Agent: Amazon Simple Notification Service Agent
✓ Notification Received
Notification Type: Bounce
✓ Processing Bounce
Bounce Type: Permanent
Recipients: ['bounce@simulator.amazonses.com']
✓ Bounce processed successfully
============================================================
```
**Check the database:**
```bash
# View bounce logs
sudo docker compose exec mysql mysql -u maillist -pmaillist maillist \
-e "SELECT * FROM bounce_logs ORDER BY created_at DESC LIMIT 5;"
# Check member status
sudo docker compose exec mysql mysql -u maillist -pmaillist maillist \
-e "SELECT email, active, bounce_count, bounce_status FROM members WHERE email='bounce@simulator.amazonses.com';"
```
**View in Web UI:**
- Open http://localhost:3000
- Go to Members tab
- Find "Bounce Test" member
- Should show: ❌ Inactive (red), bounce badge, last bounce timestamp
- Click "Bounces" button to see detailed history
### Option 2: Direct Email via Postfix (Simpler)
Send directly to the simulator without going through your list:
```bash
# Enter Postfix container
sudo docker compose exec postfix bash
# Send test email
echo "Testing hard bounce" | mail -s "Hard Bounce Test" bounce@simulator.amazonses.com
# Or soft bounce
echo "Testing soft bounce" | mail -s "Soft Bounce Test" ooto@simulator.amazonses.com
# Exit container
exit
```
### Option 3: Using AWS SES Console (For Non-Sandbox Testing)
If you have SES production access:
1. Go to AWS SES Console
2. Click "Send test email"
3. To: `bounce@simulator.amazonses.com`
4. From: Your verified email/domain
5. Subject: "Bounce test"
6. Body: "Testing bounce handling"
7. Click "Send test email"
## Testing Different Bounce Types
### 1. Hard Bounce (Permanent Failure)
```bash
echo "Test" | mail -s "Test" bounce@simulator.amazonses.com
```
**Expected Result:**
- Member marked as `hard_bounce`
- Member deactivated (`active = 0`)
- `bounce_count` incremented
- Entry in `bounce_logs` table
### 2. Soft Bounce (Transient Failure)
```bash
echo "Test" | mail -s "Test" ooto@simulator.amazonses.com
```
**Expected Result:**
- Member marked as `clean` (first time)
- After 3 soft bounces → `soft_bounce` status
- `bounce_count` incremented
- Member stays active
### 3. Complaint (Spam Report)
```bash
echo "Test" | mail -s "Test" complaint@simulator.amazonses.com
```
**Expected Result:**
- API receives complaint notification
- Currently logged but not processed (you can extend the handler)
## Monitoring the Test
### Real-Time Monitoring (Recommended)
Open 3 terminal windows:
**Terminal 1 - API Logs:**
```bash
sudo docker compose logs api -f
```
**Terminal 2 - Postfix Logs:**
```bash
sudo docker compose logs postfix -f
```
**Terminal 3 - Send Test Email:**
```bash
sudo docker compose exec postfix bash
echo "Test" | mail -s "Bounce Test" bounce@simulator.amazonses.com
```
### Timeline
Here's what happens and when:
- **T+0s**: Email sent to Postfix
- **T+1-3s**: Postfix relays to SES
- **T+5-10s**: SES processes and generates bounce
- **T+10-30s**: SNS sends notification to your webhook
- **T+30-60s**: API processes bounce and updates database
## Verifying the Complete Flow
### 1. Check Postfix Logs
```bash
sudo docker compose logs postfix | grep bounce@simulator
```
Should show:
```
postfix/smtp[xxx]: ... to=<bounce@simulator.amazonses.com>, relay=email-smtp.eu-west-2.amazonaws.com[...], ... status=sent
```
### 2. Check SNS Subscription Status
- Go to AWS SNS Console
- Find your topic
- Check "Subscriptions" tab
- Status should be "Confirmed"
- Messages delivered should be > 0
### 3. Check API Logs
```bash
sudo docker compose logs api | grep -A 20 "SNS Webhook"
```
Should show successful processing.
### 4. Check Database
```bash
sudo docker compose exec mysql mysql -u maillist -pmaillist maillist <<EOF
-- Show recent bounces
SELECT
b.bounce_id,
m.email,
b.bounce_type,
b.diagnostic_code,
b.timestamp
FROM bounce_logs b
JOIN members m ON b.member_id = m.member_id
ORDER BY b.timestamp DESC
LIMIT 5;
-- Show members with bounces
SELECT
email,
active,
bounce_count,
bounce_status,
last_bounce_at
FROM members
WHERE bounce_count > 0;
EOF
```
## Troubleshooting
### "Email not sent" or "Relay access denied"
**Problem**: Postfix not configured to relay via SES
**Check**:
```bash
sudo docker compose exec postfix postconf relayhost
sudo docker compose exec postfix postconf smtp_sasl_auth_enable
```
Should show:
```
relayhost = [email-smtp.eu-west-2.amazonaws.com]:587
smtp_sasl_auth_enable = yes
```
### "No bounce received after 5 minutes"
**Possible causes**:
1. **SNS subscription not confirmed**
- Check AWS SNS console
- Status should be "Confirmed", not "Pending"
2. **SNS topic not configured in SES**
- Check SES → Configuration Sets or Verified Identities → Notifications
- Bounce notifications should point to your SNS topic
3. **Webhook endpoint not accessible**
- SNS requires HTTPS
- Test: `curl https://your-domain.com:8000/health`
4. **API container not running**
```bash
sudo docker compose ps api
```
### "Bounce received but not in database"
**Check API logs for errors**:
```bash
sudo docker compose logs api | grep -i error
```
**Check database tables exist**:
```bash
sudo docker compose exec mysql mysql -u maillist -pmaillist maillist -e "SHOW TABLES;"
```
Should include: `bounce_logs`, `members`
## Testing Multiple Bounces
To test the "3 soft bounces = soft_bounce status" logic:
```bash
sudo docker compose exec postfix bash
# Send 3 emails to soft bounce simulator
for i in {1..3}; do
echo "Soft bounce test $i" | mail -s "Test $i" ooto@simulator.amazonses.com
sleep 70 # Wait between sends for SNS processing
done
```
After the 3rd bounce:
- Member's `bounce_status` should change from `clean` to `soft_bounce`
- `bounce_count` should be 3
## Cleanup After Testing
Remove test bounce data:
```bash
sudo docker compose exec mysql mysql -u maillist -pmaillist maillist <<EOF
-- Delete test bounce logs
DELETE FROM bounce_logs WHERE email IN ('bounce@simulator.amazonses.com', 'ooto@simulator.amazonses.com');
-- Reset test member
UPDATE members
SET bounce_count = 0,
bounce_status = 'clean',
last_bounce_at = NULL,
active = 1
WHERE email IN ('bounce@simulator.amazonses.com', 'ooto@simulator.amazonses.com');
EOF
```
Or use the Web UI:
- Go to Members tab
- Find the test member
- Click "Bounces" button
- Click "Reset Bounce Status"
## Alternative: Simulate Bounces Without SES
If SNS isn't set up yet, use the included script to simulate bounces directly in the database:
```bash
./simulate_bounce.sh
```
This is useful for:
- Testing the UI without AWS
- Development environments
- Demonstrating bounce handling to stakeholders
## Next Steps
Once bounce handling is working:
1. **Remove simulator addresses** from your member list
2. **Monitor real bounces** in production
3. **Set up alerts** for high bounce rates
4. **Review bounced members** regularly and update/remove invalid addresses
5. **Consider complaint handling** (similar to bounces, for spam reports)
## Summary Commands
**Quick test sequence**:
```bash
# 1. Watch logs
sudo docker compose logs api -f &
# 2. Send test bounce
echo "Test" | sudo docker compose exec -T postfix mail -s "Bounce Test" bounce@simulator.amazonses.com
# 3. Wait 60 seconds, then check database
sleep 60
sudo docker compose exec mysql mysql -u maillist -pmaillist maillist -e "SELECT * FROM bounce_logs ORDER BY created_at DESC LIMIT 1;"
```
That's it! The bounce simulator is the easiest way to test your bounce handling without needing real bounced emails.

132
SNS_DEBUG_GUIDE.md Normal file
View File

@@ -0,0 +1,132 @@
# SNS Webhook Debugging Guide
## Current Status
**Detailed logging is now active!** The API will log all incoming SNS requests with:
- Full headers (including SNS-specific headers)
- Request body content
- Parsed message structure
- Processing steps and results
## How to View Logs in Real-Time
```bash
# Watch API logs as SNS sends notifications
sudo docker compose logs api -f
```
## Testing with Real AWS SNS
### Step 1: Trigger a Test from SNS Console
1. Go to AWS SNS Console → Your Topic
2. Click "Publish message"
3. Send a test notification
### Step 2: Check the Logs
The logs will show you exactly what SNS sent:
```
============================================================
SNS Webhook Request Received
============================================================
Content-Type: text/plain; charset=UTF-8
User-Agent: Amazon Simple Notification Service Agent
X-Amz-SNS-Message-Type: Notification
X-Amz-SNS-Topic-Arn: arn:aws:sns:...
Message Type: Notification
Message Keys: ['Type', 'MessageId', 'TopicArn', 'Message', ...]
✓ Notification Received
Message (first 200 chars): {"notificationType":"Bounce",...
Notification Type: Bounce
✓ Processing Bounce
Bounce Type: Permanent
Recipients: ['bounce@example.com']
✓ Bounce processed successfully
============================================================
```
## What the Logs Tell You
### If you see this:
- **"SNS webhook received body: b''"** → SNS sent empty body (check SNS configuration)
- **"Missing SigningCertURL"** → Message missing required SNS fields
- **"Invalid certificate URL"** → Certificate URL not from amazonaws.com
- **"Invalid signature"** → Signature verification failed (message tampered or wrong cert)
### Successful Flow:
1. Headers logged → Shows SNS user agent and message type
2. Body logged → Shows the raw JSON
3. Message parsed → Shows the Type field
4. Notification processed → Shows bounce/complaint details
5. Database updated → Confirmation of processing
## Understanding Previous Errors
The error you saw earlier:
```
SNS webhook error: Expecting value: line 1 column 1 (char 0)
```
This means SNS was sending an **empty body** or the body was already consumed. Possible causes:
1. SNS subscription confirmation URL was opened in a browser (GET request, not POST)
2. Network issue causing body to be lost
3. SNS configuration issue
## Testing Bounce Handling
### Option 1: AWS SES Bounce Simulator
Send email to: `bounce@simulator.amazonses.com`
### Option 2: Verify Database Updates
After a bounce is processed:
```bash
# Check bounce logs
sudo docker compose exec mysql mysql -u maillist -pmaillist maillist -e "SELECT * FROM bounce_logs ORDER BY created_at DESC LIMIT 5;"
# Check member bounce status
sudo docker compose exec mysql mysql -u maillist -pmaillist maillist -e "SELECT member_id, email, bounce_count, bounce_status, last_bounce_at FROM members WHERE bounce_count > 0;"
```
## Common SNS Message Types
### 1. SubscriptionConfirmation
First message when you create the subscription. API auto-confirms by calling SubscribeURL.
### 2. Notification (Bounce)
```json
{
"Type": "Notification",
"Message": "{\"notificationType\":\"Bounce\",\"bounce\":{...}}"
}
```
### 3. Notification (Complaint)
```json
{
"Type": "Notification",
"Message": "{\"notificationType\":\"Complaint\",\"complaint\":{...}}"
}
```
## Next Steps
1. **Monitor logs while SNS sends**: `sudo docker compose logs api -f`
2. **Trigger a real bounce**: Send to `bounce@simulator.amazonses.com`
3. **Check the detailed logs** to see exactly what SNS is sending
4. **Verify database updates** to confirm bounces are being recorded
## If You Still See Errors
The enhanced logging will now show you:
- What headers SNS is sending
- What body content is arriving (or if it's empty)
- Where in the processing pipeline the error occurs
- Full stack traces for any exceptions
Copy the relevant log section and we can diagnose the exact issue!

View File

@@ -826,7 +826,16 @@ async def verify_sns_signature(request: Request) -> dict:
"""Verify SNS message signature""" """Verify SNS message signature"""
try: try:
body = await request.body() body = await request.body()
print(f"SNS webhook received body: {body}")
print(f"SNS webhook body length: {len(body)}")
print(f"SNS webhook headers: {dict(request.headers)}")
if not body:
print("ERROR: Empty body received")
raise HTTPException(status_code=400, detail="Empty request body")
message = json.loads(body.decode('utf-8')) message = json.loads(body.decode('utf-8'))
print(f"SNS webhook parsed message type: {message.get('Type')}")
# For SubscriptionConfirmation and UnsubscribeConfirmation, we don't validate signature # For SubscriptionConfirmation and UnsubscribeConfirmation, we don't validate signature
# AWS will send a URL to confirm # AWS will send a URL to confirm
@@ -901,9 +910,15 @@ async def process_bounce(bounce_data: dict):
try: try:
bounce_type = bounce_data.get('bounceType') # Permanent, Transient, Undetermined bounce_type = bounce_data.get('bounceType') # Permanent, Transient, Undetermined
bounce_subtype = bounce_data.get('bounceSubType', '') bounce_subtype = bounce_data.get('bounceSubType', '')
timestamp = bounce_data.get('timestamp') timestamp_str = bounce_data.get('timestamp')
feedback_id = bounce_data.get('feedbackId', '') feedback_id = bounce_data.get('feedbackId', '')
# Convert ISO 8601 timestamp to MySQL datetime format
# SES sends: '2025-10-13T16:22:40.359Z'
# MySQL needs: '2025-10-13 16:22:40'
from datetime import datetime as dt
timestamp = dt.fromisoformat(timestamp_str.replace('Z', '+00:00')).strftime('%Y-%m-%d %H:%M:%S')
bounced_recipients = bounce_data.get('bouncedRecipients', []) bounced_recipients = bounce_data.get('bouncedRecipients', [])
with get_db() as conn: with get_db() as conn:
@@ -972,48 +987,75 @@ async def process_bounce(bounce_data: dict):
cursor.close() cursor.close()
except Exception as e: except Exception as e:
print(f"Error processing bounce: {str(e)}") print(f"Error processing bounce: {str(e)}")
print(f"Error type: {type(e).__name__}")
print(f"Bounce data: {bounce_data}")
import traceback
traceback.print_exc()
raise raise
@app.post("/webhooks/sns", response_class=PlainTextResponse) @app.post("/webhooks/sns", response_class=PlainTextResponse)
async def sns_webhook(request: Request): async def sns_webhook(request: Request):
"""Handle SNS notifications for bounces and complaints""" """Handle SNS notifications for bounces and complaints"""
try: try:
print(f"=== SNS Webhook Request ===")
print(f"Headers: {dict(request.headers)}")
print(f"Content-Type: {request.headers.get('content-type')}")
print(f"User-Agent: {request.headers.get('user-agent')}")
# Verify SNS signature # Verify SNS signature
message = await verify_sns_signature(request) message = await verify_sns_signature(request)
print(f"Message Type: {message.get('Type')}")
print(f"Message Keys: {list(message.keys())}")
message_type = message.get('Type') message_type = message.get('Type')
# Handle subscription confirmation # Handle subscription confirmation
if message_type == 'SubscriptionConfirmation': if message_type == 'SubscriptionConfirmation':
subscribe_url = message.get('SubscribeURL') subscribe_url = message.get('SubscribeURL')
print(f"Subscription confirmation received, URL: {subscribe_url}")
if subscribe_url: if subscribe_url:
# Confirm subscription # Confirm subscription
async with httpx.AsyncClient() as client: async with httpx.AsyncClient() as client:
await client.get(subscribe_url) response = await client.get(subscribe_url)
print(f"Subscription confirmation response: {response.status_code}")
return "Subscription confirmed" return "Subscription confirmed"
# Handle notification # Handle notification
elif message_type == 'Notification': elif message_type == 'Notification':
# Parse the message # Parse the message
notification = json.loads(message.get('Message', '{}')) inner_message = message.get('Message', '{}')
notification_type = notification.get('notificationType') print(f"Inner message (first 500 chars): {inner_message[:500]}")
notification = json.loads(inner_message)
# SES can send either 'notificationType' or 'eventType' depending on configuration
notification_type = notification.get('notificationType') or notification.get('eventType')
print(f"Notification type: {notification_type}")
if notification_type == 'Bounce': if notification_type == 'Bounce':
bounce = notification.get('bounce', {}) bounce = notification.get('bounce', {})
print(f"\n✓ Processing Bounce")
print(f" Bounce Type: {bounce.get('bounceType')}")
print(f" Recipients: {[r.get('emailAddress') for r in bounce.get('bouncedRecipients', [])]}")
await process_bounce(bounce) await process_bounce(bounce)
print(f" ✓ Bounce processed successfully")
return "Bounce processed" return "Bounce processed"
elif notification_type == 'Complaint': elif notification_type == 'Complaint':
# We could also track complaints similarly to bounces # We could also track complaints similarly to bounces
print(f"\n✓ Complaint received")
return "Complaint received" return "Complaint received"
print(f"=== End SNS Webhook Request ===")
return "OK" return "OK"
except HTTPException: except HTTPException:
raise raise
except Exception as e: except Exception as e:
print(f"SNS webhook error: {str(e)}") print(f"SNS webhook error: {str(e)}")
import traceback
traceback.print_exc()
raise HTTPException(status_code=500, detail=str(e)) raise HTTPException(status_code=500, detail=str(e))
# Bounce management endpoints # Bounce management endpoints

45
test_bounce_quick.sh Normal file
View File

@@ -0,0 +1,45 @@
#!/bin/bash
# Quick SES Bounce Test - One-liner to test bounce handling
echo "🚀 Testing SES Bounce Handling"
echo "================================"
echo ""
# Check if API and Postfix are running
if ! sudo docker compose ps | grep -q "maillist-api.*Up"; then
echo "❌ API container not running. Start with: sudo docker compose up -d"
exit 1
fi
if ! sudo docker compose ps | grep -q "maillist-postfix.*Up"; then
echo "❌ Postfix container not running. Start with: sudo docker compose up -d"
exit 1
fi
echo "✅ Containers are running"
echo ""
# Start watching logs in background
echo "📋 Opening log viewer (press Ctrl+C to stop)..."
echo ""
sleep 2
# Show the command they can run
echo "Run this command to send a test bounce:"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "echo 'Test bounce' | sudo docker compose exec -T postfix mail -s 'Bounce Test' bounce@simulator.amazonses.com"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo ""
echo "Then wait 30-60 seconds and check:"
echo ""
echo "1. API logs (in this window)"
echo "2. Database: sudo docker compose exec mysql mysql -u maillist -pmaillist maillist -e 'SELECT * FROM bounce_logs ORDER BY created_at DESC LIMIT 5;'"
echo "3. Web UI: http://localhost:3000 → Members tab"
echo ""
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "Watching API logs now..."
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo ""
# Follow logs
sudo docker compose logs api -f --tail 20

View File

@@ -25,8 +25,8 @@ class APIClient {
return `${protocol}//${hostname}:8000`; return `${protocol}//${hostname}:8000`;
} }
// If running in production, assume API is on port 8000 // If running in production behind a reverse proxy, use /api path
return `${protocol}//${hostname}:8000`; return `${protocol}//${hostname}/api`;
} }
/** /**
@@ -104,16 +104,18 @@ class APIClient {
return this.request('/'); return this.request('/');
} }
// Authentication API
async login(username, password) { async login(username, password) {
// Don't include Authorization header for login
const tempHeaders = { ...this.headers };
delete tempHeaders['Authorization'];
const response = await this.request('/auth/login', { const response = await this.request('/auth/login', {
method: 'POST', method: 'POST',
body: JSON.stringify({ headers: tempHeaders,
username: username, body: JSON.stringify({ username, password })
password: password
})
}); });
// Set the token from the response
if (response.access_token) { if (response.access_token) {
this.setToken(response.access_token); this.setToken(response.access_token);
} }
@@ -126,41 +128,16 @@ class APIClient {
await this.request('/auth/logout', { await this.request('/auth/logout', {
method: 'POST' method: 'POST'
}); });
} catch (error) { } finally {
// Ignore logout errors, we'll clear the token anyway // Clear token even if logout fails
}
this.clearToken(); this.clearToken();
} }
}
async getCurrentUser() { async getCurrentUser() {
return this.request('/auth/me'); return this.request('/auth/me');
} }
// User management API
async getUsers() {
return this.request('/users');
}
async createUser(userData) {
return this.request('/users', {
method: 'POST',
body: JSON.stringify(userData)
});
}
async updateUser(userId, userData) {
return this.request(`/users/${userId}`, {
method: 'PATCH',
body: JSON.stringify(userData)
});
}
async deleteUser(userId) {
return this.request(`/users/${userId}`, {
method: 'DELETE'
});
}
// Mailing Lists API // Mailing Lists API
async getLists() { async getLists() {
return this.request('/lists'); return this.request('/lists');
@@ -255,7 +232,32 @@ class APIClient {
}); });
} }
// Bounce management API // User Management API
async getUsers() {
return this.request('/users');
}
async createUser(userData) {
return this.request('/users', {
method: 'POST',
body: JSON.stringify(userData)
});
}
async updateUser(userId, userData) {
return this.request(`/users/${userId}`, {
method: 'PATCH',
body: JSON.stringify(userData)
});
}
async deleteUser(userId) {
return this.request(`/users/${userId}`, {
method: 'DELETE'
});
}
// Bounce Management API
async getMemberBounces(memberId) { async getMemberBounces(memberId) {
return this.request(`/members/${memberId}/bounces`); return this.request(`/members/${memberId}/bounces`);
} }