sasa-maillist/SES_BOUNCE_TESTING_GUIDE.md

Testing Bounces from SES Sandbox

AWS SES provides built-in bounce simulator addresses that work even in sandbox mode. This guide shows you how to use them to test your bounce handling.

Quick Answer

Send email to these special AWS addresses to simulate different bounce types:

Hard Bounce (Permanent - Invalid Address)

bounce@simulator.amazonses.com

Soft Bounce (Temporary - Mailbox Full)

ooto@simulator.amazonses.com

Complaint (Spam Report)

complaint@simulator.amazonses.com

Successful Delivery (No Bounce)

success@simulator.amazonses.com

Step-by-Step Testing Guide

Option 1: Using Your Mailing Lists (Recommended)

This tests the complete flow: Postfix → SES → SNS → API

1. Add the simulator address as a member:

   # Using the API
   curl -X POST http://localhost:8000/members \
     -H "Authorization: Bearer $API_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
       "name": "Bounce Test",
       "email": "bounce@simulator.amazonses.com",
       "active": true
     }'
   

   Or use the Web UI:

   • Go to http://localhost:3000
   • Members tab → Add Member
   • Name: Bounce Test
   • Email: bounce@simulator.amazonses.com

2. Subscribe to a test list:

   • Click "Subscriptions" button for the test member
   • Toggle on one of your mailing lists

3. Send email to the list:

   # From inside Postfix container
   sudo docker compose exec postfix bash
   echo "This will bounce" | mail -s "Test Bounce" community@lists.sasalliance.org
   exit
   

   Replace community@lists.sasalliance.org with your actual list email.

4. Wait 30-60 seconds for:

   • Email to be sent via SES
   • SES to process the bounce
   • SNS to send notification to your webhook

5. Check the results:

   Watch API logs in real-time:

   sudo docker compose logs api -f
   

   You should see:

   ============================================================
   SNS Webhook Request Received
   ============================================================
   Content-Type: text/plain; charset=UTF-8
   User-Agent: Amazon Simple Notification Service Agent
   
   ✓ Notification Received
     Notification Type: Bounce
   
   ✓ Processing Bounce
     Bounce Type: Permanent
     Recipients: ['bounce@simulator.amazonses.com']
     ✓ Bounce processed successfully
   ============================================================
   

   Check the database:

   # View bounce logs
   sudo docker compose exec mysql mysql -u maillist -pmaillist maillist \
     -e "SELECT * FROM bounce_logs ORDER BY created_at DESC LIMIT 5;"
   
   # Check member status
   sudo docker compose exec mysql mysql -u maillist -pmaillist maillist \
     -e "SELECT email, active, bounce_count, bounce_status FROM members WHERE email='bounce@simulator.amazonses.com';"
   

   View in Web UI:

   • Open http://localhost:3000
   • Go to Members tab
   • Find "Bounce Test" member
   • Should show: ❌ Inactive (red), bounce badge, last bounce timestamp
   • Click "Bounces" button to see detailed history

Option 2: Direct Email via Postfix (Simpler)

Send directly to the simulator without going through your list:

# Enter Postfix container
sudo docker compose exec postfix bash

# Send test email
echo "Testing hard bounce" | mail -s "Hard Bounce Test" bounce@simulator.amazonses.com

# Or soft bounce
echo "Testing soft bounce" | mail -s "Soft Bounce Test" ooto@simulator.amazonses.com

# Exit container
exit

Option 3: Using AWS SES Console (For Non-Sandbox Testing)

If you have SES production access:

1. Go to AWS SES Console
2. Click "Send test email"
3. To: bounce@simulator.amazonses.com
4. From: Your verified email/domain
5. Subject: "Bounce test"
6. Body: "Testing bounce handling"
7. Click "Send test email"

Testing Different Bounce Types

1. Hard Bounce (Permanent Failure)

echo "Test" | mail -s "Test" bounce@simulator.amazonses.com

Expected Result:

• Member marked as hard_bounce
• Member deactivated (active = 0)
• bounce_count incremented
• Entry in bounce_logs table

2. Soft Bounce (Transient Failure)

echo "Test" | mail -s "Test" ooto@simulator.amazonses.com

Expected Result:

• Member marked as clean (first time)
• After 3 soft bounces → soft_bounce status
• bounce_count incremented
• Member stays active

3. Complaint (Spam Report)

echo "Test" | mail -s "Test" complaint@simulator.amazonses.com

Expected Result:

• API receives complaint notification
• Currently logged but not processed (you can extend the handler)

Monitoring the Test

Real-Time Monitoring (Recommended)

Open 3 terminal windows:

Terminal 1 - API Logs:

sudo docker compose logs api -f

Terminal 2 - Postfix Logs:

sudo docker compose logs postfix -f

Terminal 3 - Send Test Email:

sudo docker compose exec postfix bash
echo "Test" | mail -s "Bounce Test" bounce@simulator.amazonses.com

Timeline

Here's what happens and when:

• T+0s: Email sent to Postfix
• T+1-3s: Postfix relays to SES
• T+5-10s: SES processes and generates bounce
• T+10-30s: SNS sends notification to your webhook
• T+30-60s: API processes bounce and updates database

Verifying the Complete Flow

1. Check Postfix Logs

sudo docker compose logs postfix | grep bounce@simulator

Should show:

postfix/smtp[xxx]: ... to=<bounce@simulator.amazonses.com>, relay=email-smtp.eu-west-2.amazonaws.com[...], ... status=sent

2. Check SNS Subscription Status

• Go to AWS SNS Console
• Find your topic
• Check "Subscriptions" tab
• Status should be "Confirmed"
• Messages delivered should be > 0

3. Check API Logs

sudo docker compose logs api | grep -A 20 "SNS Webhook"

Should show successful processing.

4. Check Database

sudo docker compose exec mysql mysql -u maillist -pmaillist maillist <<EOF
-- Show recent bounces
SELECT 
    b.bounce_id,
    m.email,
    b.bounce_type,
    b.diagnostic_code,
    b.timestamp
FROM bounce_logs b
JOIN members m ON b.member_id = m.member_id
ORDER BY b.timestamp DESC
LIMIT 5;

-- Show members with bounces
SELECT 
    email,
    active,
    bounce_count,
    bounce_status,
    last_bounce_at
FROM members
WHERE bounce_count > 0;
EOF

Troubleshooting

"Email not sent" or "Relay access denied"

Problem: Postfix not configured to relay via SES

Check:

sudo docker compose exec postfix postconf relayhost
sudo docker compose exec postfix postconf smtp_sasl_auth_enable

Should show:

relayhost = [email-smtp.eu-west-2.amazonaws.com]:587
smtp_sasl_auth_enable = yes

"No bounce received after 5 minutes"

Possible causes:

1. SNS subscription not confirmed

   • Check AWS SNS console
   • Status should be "Confirmed", not "Pending"

2. SNS topic not configured in SES

   • Check SES → Configuration Sets or Verified Identities → Notifications
   • Bounce notifications should point to your SNS topic

3. Webhook endpoint not accessible

   • SNS requires HTTPS
   • Test: curl https://your-domain.com:8000/health

4. API container not running

   sudo docker compose ps api
   

"Bounce received but not in database"

Check API logs for errors:

sudo docker compose logs api | grep -i error

Check database tables exist:

sudo docker compose exec mysql mysql -u maillist -pmaillist maillist -e "SHOW TABLES;"

Should include: bounce_logs, members

Testing Multiple Bounces

To test the "3 soft bounces = soft_bounce status" logic:

sudo docker compose exec postfix bash

# Send 3 emails to soft bounce simulator
for i in {1..3}; do
    echo "Soft bounce test $i" | mail -s "Test $i" ooto@simulator.amazonses.com
    sleep 70  # Wait between sends for SNS processing
done

After the 3rd bounce:

• Member's bounce_status should change from clean to soft_bounce
• bounce_count should be 3

Cleanup After Testing

Remove test bounce data:

sudo docker compose exec mysql mysql -u maillist -pmaillist maillist <<EOF
-- Delete test bounce logs
DELETE FROM bounce_logs WHERE email IN ('bounce@simulator.amazonses.com', 'ooto@simulator.amazonses.com');

-- Reset test member
UPDATE members 
SET bounce_count = 0, 
    bounce_status = 'clean', 
    last_bounce_at = NULL,
    active = 1
WHERE email IN ('bounce@simulator.amazonses.com', 'ooto@simulator.amazonses.com');
EOF

Or use the Web UI:

• Go to Members tab
• Find the test member
• Click "Bounces" button
• Click "Reset Bounce Status"

Alternative: Simulate Bounces Without SES

If SNS isn't set up yet, use the included script to simulate bounces directly in the database:

./simulate_bounce.sh

This is useful for:

• Testing the UI without AWS
• Development environments
• Demonstrating bounce handling to stakeholders

Next Steps

Once bounce handling is working:

1. Remove simulator addresses from your member list
2. Monitor real bounces in production
3. Set up alerts for high bounce rates
4. Review bounced members regularly and update/remove invalid addresses
5. Consider complaint handling (similar to bounces, for spam reports)

Summary Commands

Quick test sequence:

# 1. Watch logs
sudo docker compose logs api -f &

# 2. Send test bounce
echo "Test" | sudo docker compose exec -T postfix mail -s "Bounce Test" bounce@simulator.amazonses.com

# 3. Wait 60 seconds, then check database
sleep 60
sudo docker compose exec mysql mysql -u maillist -pmaillist maillist -e "SELECT * FROM bounce_logs ORDER BY created_at DESC LIMIT 1;"

That's it! The bounce simulator is the easiest way to test your bounce handling without needing real bounced emails.