Ability to disable SNS bounce handling
This commit is contained in:
12
.env.example
12
.env.example
@@ -20,3 +20,15 @@ MYSQL_ROOT_PASSWORD=change_this_root_password
|
||||
|
||||
# API Configuration
|
||||
API_TOKEN=change_this_to_a_secure_random_token
|
||||
|
||||
# Bounce Handling Configuration (Optional)
|
||||
# Set to 'true' to enable SNS webhook bounce handling
|
||||
# Set to 'false' to disable and rely on email-based bounce handling
|
||||
ENABLE_SNS_WEBHOOKS=false
|
||||
ENABLE_BOUNCE_HANDLING=false
|
||||
|
||||
# If ENABLE_SNS_WEBHOOKS=true, you need:
|
||||
# 1. AWS SNS topic configured
|
||||
# 2. SES configured to send notifications to SNS topic
|
||||
# 3. Valid HTTPS domain for webhook endpoint
|
||||
# 4. SNS subscription confirmed to your webhook endpoint
|
||||
@@ -1,10 +1,17 @@
|
||||
# SES SNS Bounce Handling Setup
|
||||
# SES SNS Bounce Handling Setup (Optional)
|
||||
|
||||
**⚠️ NOTICE: Bounce handling is optional and disabled by default.**
|
||||
|
||||
This document describes how to configure AWS SES and SNS to handle email bounces automatically in the Mail List Manager.
|
||||
|
||||
**Prerequisites:**
|
||||
- SES production access (not available in sandbox mode)
|
||||
- Valid HTTPS domain for webhook endpoint
|
||||
- Bounce handling must be enabled in configuration
|
||||
|
||||
## Overview
|
||||
|
||||
The system uses AWS Simple Notification Service (SNS) to receive real-time bounce notifications from AWS Simple Email Service (SES). When an email bounces:
|
||||
The system can optionally use AWS Simple Notification Service (SNS) to receive real-time bounce notifications from AWS Simple Email Service (SES). When bounce handling is enabled and an email bounces:
|
||||
|
||||
1. SES sends a notification to an SNS topic
|
||||
2. SNS forwards the notification to your webhook endpoint
|
||||
@@ -22,13 +29,28 @@ The system uses AWS Simple Notification Service (SNS) to receive real-time bounc
|
||||
|
||||
## Setup Instructions
|
||||
|
||||
### 1. Prerequisites
|
||||
### 1. Enable Bounce Handling
|
||||
|
||||
First, enable bounce handling in your `.env` file:
|
||||
|
||||
```bash
|
||||
# Enable SNS webhook bounce handling
|
||||
ENABLE_SNS_WEBHOOKS=true
|
||||
ENABLE_BOUNCE_HANDLING=true
|
||||
```
|
||||
|
||||
Restart the API container after making this change:
|
||||
```bash
|
||||
sudo docker-compose restart api
|
||||
```
|
||||
|
||||
### 2. Prerequisites
|
||||
|
||||
- AWS account with SES configured and verified
|
||||
- Your Mail List Manager deployed and accessible via HTTPS (required for SNS webhook)
|
||||
- Domain or subdomain for webhook (e.g., `https://lists.yourdomain.com`)
|
||||
|
||||
### 2. Create SNS Topic
|
||||
### 3. Create SNS Topic
|
||||
|
||||
1. Log in to AWS Console and navigate to SNS
|
||||
2. Click "Create topic"
|
||||
@@ -38,7 +60,7 @@ The system uses AWS Simple Notification Service (SNS) to receive real-time bounc
|
||||
6. Click "Create topic"
|
||||
7. **Save the Topic ARN** (you'll need it in step 4) arn:aws:sns:eu-west-2:827164363113:ses-bounces
|
||||
|
||||
### 3. Subscribe Your Webhook to SNS Topic
|
||||
### 4. Subscribe Your Webhook to SNS Topic
|
||||
|
||||
1. In the SNS topic details, click "Create subscription"
|
||||
2. Protocol: `HTTPS`
|
||||
@@ -49,7 +71,7 @@ The system uses AWS Simple Notification Service (SNS) to receive real-time bounc
|
||||
5. Click "Create subscription"
|
||||
6. The subscription will be in "PendingConfirmation" status
|
||||
|
||||
### 4. Confirm SNS Subscription
|
||||
### 5. Confirm SNS Subscription
|
||||
|
||||
When you create the subscription, SNS will send a `SubscriptionConfirmation` request to your webhook endpoint. The Mail List Manager API automatically confirms this subscription.
|
||||
|
||||
@@ -61,7 +83,7 @@ When you create the subscription, SNS will send a `SubscriptionConfirmation` req
|
||||
3. In the AWS SNS console, refresh the subscriptions list
|
||||
4. The status should change from "PendingConfirmation" to "Confirmed"
|
||||
|
||||
### 5. Configure SES to Send Bounce Notifications
|
||||
### 6. Configure SES to Send Bounce Notifications
|
||||
|
||||
1. Navigate to AWS SES console
|
||||
2. Go to "Configuration Sets" (or "Verified identities" > select your domain > "Notifications")
|
||||
@@ -78,7 +100,7 @@ When you create the subscription, SNS will send a `SubscriptionConfirmation` req
|
||||
- Include original headers: Enabled (optional)
|
||||
- Click "Save changes"
|
||||
|
||||
### 6. Verify Setup
|
||||
### 7. Verify Setup
|
||||
|
||||
#### Test with a Bounce Simulator
|
||||
|
||||
@@ -108,7 +130,7 @@ Or send to your mailing list with a test recipient:
|
||||
5. Find the test member and click the "Bounces" button
|
||||
6. You should see the bounce event recorded
|
||||
|
||||
### 7. Security Considerations
|
||||
### 8. Security Considerations
|
||||
|
||||
#### SNS Signature Verification
|
||||
|
||||
@@ -149,7 +171,7 @@ server {
|
||||
}
|
||||
```
|
||||
|
||||
### 8. Managing Bounces in the UI
|
||||
### 9. Managing Bounces in the UI
|
||||
|
||||
#### View Bounce Status
|
||||
|
||||
@@ -178,7 +200,7 @@ If a member's email has been corrected or verified:
|
||||
|
||||
**Note**: Only users with write access (administrators and operators) can reset bounce status.
|
||||
|
||||
### 9. Monitoring and Maintenance
|
||||
### 10. Monitoring and Maintenance
|
||||
|
||||
#### Check Bounce Logs
|
||||
|
||||
|
||||
35
README.md
35
README.md
@@ -172,6 +172,41 @@ docker-compose exec postfix postmap -q "community@lists.sasalliance.org" \
|
||||
mysql:/etc/postfix/mysql_virtual_alias_maps.cf
|
||||
```
|
||||
|
||||
### Bounce Handling (Optional)
|
||||
|
||||
**Email bounce handling is optional and disabled by default.**
|
||||
|
||||
**Two Configuration Options:**
|
||||
|
||||
1. **SNS Webhooks** (Requires SES Production Access):
|
||||
```bash
|
||||
# In .env file
|
||||
ENABLE_SNS_WEBHOOKS=true
|
||||
ENABLE_BOUNCE_HANDLING=true
|
||||
```
|
||||
- Real-time bounce notifications via AWS SNS
|
||||
- Automatic member deactivation for hard bounces
|
||||
- Bounce history tracking and management
|
||||
- Requires valid HTTPS domain and SES production access
|
||||
- See `BOUNCE_HANDLING_SETUP.md` for complete setup
|
||||
|
||||
2. **Email-Based Handling** (Default for SES Sandbox):
|
||||
```bash
|
||||
# In .env file (or leave these commented out)
|
||||
ENABLE_SNS_WEBHOOKS=false
|
||||
ENABLE_BOUNCE_HANDLING=false
|
||||
```
|
||||
- Manual bounce management via email notifications
|
||||
- No automatic processing - requires manual member cleanup
|
||||
- Works with SES sandbox accounts
|
||||
- Bounce-related UI elements are hidden
|
||||
|
||||
**When bounce handling is disabled:**
|
||||
- `/webhooks/sns` endpoint is not registered
|
||||
- Bounce history endpoints return empty results
|
||||
- Web UI hides bounce badges and bounce management buttons
|
||||
- No automatic member deactivation occurs
|
||||
|
||||
## Security
|
||||
|
||||
- **Environment Variables**: All credentials stored in `.env` (git-ignored)
|
||||
|
||||
562
api/main.py
562
api/main.py
@@ -43,6 +43,10 @@ MYSQL_DATABASE = os.getenv('MYSQL_DATABASE', 'maillist')
|
||||
MYSQL_USER = os.getenv('MYSQL_USER', 'maillist')
|
||||
MYSQL_PASSWORD = os.getenv('MYSQL_PASSWORD', '')
|
||||
|
||||
# Bounce handling configuration
|
||||
ENABLE_SNS_WEBHOOKS = os.getenv('ENABLE_SNS_WEBHOOKS', 'false').lower() == 'true'
|
||||
ENABLE_BOUNCE_HANDLING = os.getenv('ENABLE_BOUNCE_HANDLING', 'false').lower() == 'true'
|
||||
|
||||
# Password hashing
|
||||
pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto")
|
||||
|
||||
@@ -463,6 +467,14 @@ async def health():
|
||||
except Exception as e:
|
||||
raise HTTPException(status_code=503, detail=f"Unhealthy: {str(e)}")
|
||||
|
||||
@app.get("/config")
|
||||
async def get_config():
|
||||
"""Get public configuration settings"""
|
||||
return {
|
||||
"bounce_handling_enabled": ENABLE_BOUNCE_HANDLING,
|
||||
"sns_webhooks_enabled": ENABLE_SNS_WEBHOOKS
|
||||
}
|
||||
|
||||
# Mailing Lists endpoints
|
||||
@app.get("/lists", response_model=List[MailingList])
|
||||
async def get_lists(current_user: CurrentUser = require_read_access()):
|
||||
@@ -824,298 +836,316 @@ async def bulk_import_members(bulk_request: BulkImportRequest, current_user: Cur
|
||||
cursor.close()
|
||||
raise HTTPException(status_code=500, detail=f"Bulk import failed: {str(e)}")
|
||||
|
||||
# SNS Webhook for Bounce Handling
|
||||
async def verify_sns_signature(request: Request) -> dict:
|
||||
"""Verify SNS message signature"""
|
||||
try:
|
||||
body = await request.body()
|
||||
print(f"SNS webhook received body: {body}")
|
||||
print(f"SNS webhook body length: {len(body)}")
|
||||
print(f"SNS webhook headers: {dict(request.headers)}")
|
||||
# SNS Webhook for Bounce Handling (conditionally enabled)
|
||||
if ENABLE_SNS_WEBHOOKS:
|
||||
async def verify_sns_signature(request: Request) -> dict:
|
||||
"""Verify SNS message signature"""
|
||||
try:
|
||||
body = await request.body()
|
||||
print(f"SNS webhook received body: {body}")
|
||||
print(f"SNS webhook body length: {len(body)}")
|
||||
print(f"SNS webhook headers: {dict(request.headers)}")
|
||||
|
||||
if not body:
|
||||
print("ERROR: Empty body received")
|
||||
raise HTTPException(status_code=400, detail="Empty request body")
|
||||
if not body:
|
||||
print("ERROR: Empty body received")
|
||||
raise HTTPException(status_code=400, detail="Empty request body")
|
||||
|
||||
message = json.loads(body.decode('utf-8'))
|
||||
print(f"SNS webhook parsed message type: {message.get('Type')}")
|
||||
message = json.loads(body.decode('utf-8'))
|
||||
print(f"SNS webhook parsed message type: {message.get('Type')}")
|
||||
|
||||
# For SubscriptionConfirmation and UnsubscribeConfirmation, we don't validate signature
|
||||
# AWS will send a URL to confirm
|
||||
if message.get('Type') in ['SubscriptionConfirmation', 'UnsubscribeConfirmation']:
|
||||
return message
|
||||
|
||||
# Get certificate URL and download certificate
|
||||
cert_url = message.get('SigningCertURL')
|
||||
if not cert_url:
|
||||
raise HTTPException(status_code=400, detail="Missing SigningCertURL")
|
||||
|
||||
# Verify certificate URL is from AWS
|
||||
parsed_url = urlparse(cert_url)
|
||||
if not parsed_url.hostname.endswith('.amazonaws.com'):
|
||||
raise HTTPException(status_code=400, detail="Invalid certificate URL")
|
||||
|
||||
# Download certificate
|
||||
async with httpx.AsyncClient() as client:
|
||||
cert_response = await client.get(cert_url)
|
||||
cert_response.raise_for_status()
|
||||
cert_pem = cert_response.content
|
||||
|
||||
# Load certificate and extract public key
|
||||
cert = x509.load_pem_x509_certificate(cert_pem, default_backend())
|
||||
public_key = cert.public_key()
|
||||
|
||||
# Build string to sign
|
||||
if message.get('Type') == 'Notification':
|
||||
string_to_sign = (
|
||||
f"Message\n{message['Message']}\n"
|
||||
f"MessageId\n{message['MessageId']}\n"
|
||||
)
|
||||
if 'Subject' in message:
|
||||
string_to_sign += f"Subject\n{message['Subject']}\n"
|
||||
string_to_sign += (
|
||||
f"Timestamp\n{message['Timestamp']}\n"
|
||||
f"TopicArn\n{message['TopicArn']}\n"
|
||||
f"Type\n{message['Type']}\n"
|
||||
)
|
||||
else:
|
||||
string_to_sign = (
|
||||
f"Message\n{message['Message']}\n"
|
||||
f"MessageId\n{message['MessageId']}\n"
|
||||
f"SubscribeURL\n{message['SubscribeURL']}\n"
|
||||
f"Timestamp\n{message['Timestamp']}\n"
|
||||
f"Token\n{message['Token']}\n"
|
||||
f"TopicArn\n{message['TopicArn']}\n"
|
||||
f"Type\n{message['Type']}\n"
|
||||
)
|
||||
|
||||
# Verify signature
|
||||
signature = base64.b64decode(message['Signature'])
|
||||
try:
|
||||
public_key.verify(
|
||||
signature,
|
||||
string_to_sign.encode('utf-8'),
|
||||
padding.PKCS1v15(),
|
||||
hashes.SHA1()
|
||||
)
|
||||
except Exception as e:
|
||||
raise HTTPException(status_code=400, detail=f"Invalid signature: {str(e)}")
|
||||
|
||||
# For SubscriptionConfirmation and UnsubscribeConfirmation, we don't validate signature
|
||||
# AWS will send a URL to confirm
|
||||
if message.get('Type') in ['SubscriptionConfirmation', 'UnsubscribeConfirmation']:
|
||||
return message
|
||||
|
||||
# Get certificate URL and download certificate
|
||||
cert_url = message.get('SigningCertURL')
|
||||
if not cert_url:
|
||||
raise HTTPException(status_code=400, detail="Missing SigningCertURL")
|
||||
|
||||
# Verify certificate URL is from AWS
|
||||
parsed_url = urlparse(cert_url)
|
||||
if not parsed_url.hostname.endswith('.amazonaws.com'):
|
||||
raise HTTPException(status_code=400, detail="Invalid certificate URL")
|
||||
|
||||
# Download certificate
|
||||
async with httpx.AsyncClient() as client:
|
||||
cert_response = await client.get(cert_url)
|
||||
cert_response.raise_for_status()
|
||||
cert_pem = cert_response.content
|
||||
|
||||
# Load certificate and extract public key
|
||||
cert = x509.load_pem_x509_certificate(cert_pem, default_backend())
|
||||
public_key = cert.public_key()
|
||||
|
||||
# Build string to sign
|
||||
if message.get('Type') == 'Notification':
|
||||
string_to_sign = (
|
||||
f"Message\n{message['Message']}\n"
|
||||
f"MessageId\n{message['MessageId']}\n"
|
||||
)
|
||||
if 'Subject' in message:
|
||||
string_to_sign += f"Subject\n{message['Subject']}\n"
|
||||
string_to_sign += (
|
||||
f"Timestamp\n{message['Timestamp']}\n"
|
||||
f"TopicArn\n{message['TopicArn']}\n"
|
||||
f"Type\n{message['Type']}\n"
|
||||
)
|
||||
else:
|
||||
string_to_sign = (
|
||||
f"Message\n{message['Message']}\n"
|
||||
f"MessageId\n{message['MessageId']}\n"
|
||||
f"SubscribeURL\n{message['SubscribeURL']}\n"
|
||||
f"Timestamp\n{message['Timestamp']}\n"
|
||||
f"Token\n{message['Token']}\n"
|
||||
f"TopicArn\n{message['TopicArn']}\n"
|
||||
f"Type\n{message['Type']}\n"
|
||||
)
|
||||
|
||||
# Verify signature
|
||||
signature = base64.b64decode(message['Signature'])
|
||||
try:
|
||||
public_key.verify(
|
||||
signature,
|
||||
string_to_sign.encode('utf-8'),
|
||||
padding.PKCS1v15(),
|
||||
hashes.SHA1()
|
||||
)
|
||||
except json.JSONDecodeError:
|
||||
raise HTTPException(status_code=400, detail="Invalid JSON")
|
||||
except Exception as e:
|
||||
raise HTTPException(status_code=400, detail=f"Invalid signature: {str(e)}")
|
||||
raise HTTPException(status_code=400, detail=f"Signature verification failed: {str(e)}")
|
||||
|
||||
return message
|
||||
async def process_bounce(bounce_data: dict):
|
||||
"""Process bounce notification and update database"""
|
||||
try:
|
||||
bounce_type = bounce_data.get('bounceType') # Permanent, Transient, Undetermined
|
||||
bounce_subtype = bounce_data.get('bounceSubType', '')
|
||||
timestamp_str = bounce_data.get('timestamp')
|
||||
feedback_id = bounce_data.get('feedbackId', '')
|
||||
|
||||
except json.JSONDecodeError:
|
||||
raise HTTPException(status_code=400, detail="Invalid JSON")
|
||||
except Exception as e:
|
||||
raise HTTPException(status_code=400, detail=f"Signature verification failed: {str(e)}")
|
||||
# Convert ISO 8601 timestamp to MySQL datetime format
|
||||
# SES sends: '2025-10-13T16:22:40.359Z'
|
||||
# MySQL needs: '2025-10-13 16:22:40'
|
||||
from datetime import datetime as dt
|
||||
timestamp = dt.fromisoformat(timestamp_str.replace('Z', '+00:00')).strftime('%Y-%m-%d %H:%M:%S')
|
||||
|
||||
async def process_bounce(bounce_data: dict):
|
||||
"""Process bounce notification and update database"""
|
||||
try:
|
||||
bounce_type = bounce_data.get('bounceType') # Permanent, Transient, Undetermined
|
||||
bounce_subtype = bounce_data.get('bounceSubType', '')
|
||||
timestamp_str = bounce_data.get('timestamp')
|
||||
feedback_id = bounce_data.get('feedbackId', '')
|
||||
bounced_recipients = bounce_data.get('bouncedRecipients', [])
|
||||
|
||||
# Convert ISO 8601 timestamp to MySQL datetime format
|
||||
# SES sends: '2025-10-13T16:22:40.359Z'
|
||||
# MySQL needs: '2025-10-13 16:22:40'
|
||||
from datetime import datetime as dt
|
||||
timestamp = dt.fromisoformat(timestamp_str.replace('Z', '+00:00')).strftime('%Y-%m-%d %H:%M:%S')
|
||||
with get_db() as conn:
|
||||
cursor = conn.cursor(dictionary=True)
|
||||
|
||||
bounced_recipients = bounce_data.get('bouncedRecipients', [])
|
||||
for recipient in bounced_recipients:
|
||||
email = recipient.get('emailAddress')
|
||||
diagnostic_code = recipient.get('diagnosticCode', '')
|
||||
|
||||
with get_db() as conn:
|
||||
cursor = conn.cursor(dictionary=True)
|
||||
if not email:
|
||||
continue
|
||||
|
||||
for recipient in bounced_recipients:
|
||||
email = recipient.get('emailAddress')
|
||||
diagnostic_code = recipient.get('diagnosticCode', '')
|
||||
# Find member by email
|
||||
cursor.execute("SELECT member_id FROM members WHERE email = %s", (email,))
|
||||
member = cursor.fetchone()
|
||||
member_id = member['member_id'] if member else None
|
||||
|
||||
if not email:
|
||||
continue
|
||||
|
||||
# Find member by email
|
||||
cursor.execute("SELECT member_id FROM members WHERE email = %s", (email,))
|
||||
member = cursor.fetchone()
|
||||
member_id = member['member_id'] if member else None
|
||||
|
||||
# Log the bounce
|
||||
cursor.execute("""
|
||||
INSERT INTO bounce_logs
|
||||
(member_id, email, bounce_type, bounce_subtype, diagnostic_code, timestamp, feedback_id)
|
||||
VALUES (%s, %s, %s, %s, %s, %s, %s)
|
||||
""", (member_id, email, bounce_type, bounce_subtype, diagnostic_code, timestamp, feedback_id))
|
||||
|
||||
# Update member bounce status
|
||||
if member_id:
|
||||
# Determine bounce status
|
||||
if bounce_type == 'Permanent':
|
||||
new_status = 'hard_bounce'
|
||||
# Deactivate member with hard bounce
|
||||
cursor.execute("""
|
||||
UPDATE members
|
||||
SET bounce_count = bounce_count + 1,
|
||||
last_bounce_at = %s,
|
||||
bounce_status = %s,
|
||||
active = 0
|
||||
WHERE member_id = %s
|
||||
""", (timestamp, new_status, member_id))
|
||||
elif bounce_type == 'Transient':
|
||||
# Check current bounce count
|
||||
cursor.execute("SELECT bounce_count, bounce_status FROM members WHERE member_id = %s", (member_id,))
|
||||
current = cursor.fetchone()
|
||||
|
||||
# If already hard bounced, don't change status
|
||||
if current and current['bounce_status'] != 'hard_bounce':
|
||||
new_count = current['bounce_count'] + 1
|
||||
# After 3 soft bounces, mark as soft_bounce status
|
||||
new_status = 'soft_bounce' if new_count >= 3 else 'clean'
|
||||
# Log the bounce
|
||||
cursor.execute("""
|
||||
INSERT INTO bounce_logs
|
||||
(member_id, email, bounce_type, bounce_subtype, diagnostic_code, timestamp, feedback_id)
|
||||
VALUES (%s, %s, %s, %s, %s, %s, %s)
|
||||
""", (member_id, email, bounce_type, bounce_subtype, diagnostic_code, timestamp, feedback_id))
|
||||
|
||||
# Update member bounce status
|
||||
if member_id:
|
||||
# Determine bounce status
|
||||
if bounce_type == 'Permanent':
|
||||
new_status = 'hard_bounce'
|
||||
# Deactivate member with hard bounce
|
||||
cursor.execute("""
|
||||
UPDATE members
|
||||
SET bounce_count = %s,
|
||||
SET bounce_count = bounce_count + 1,
|
||||
last_bounce_at = %s,
|
||||
bounce_status = %s
|
||||
bounce_status = %s,
|
||||
active = 0
|
||||
WHERE member_id = %s
|
||||
""", (new_count, timestamp, new_status, member_id))
|
||||
else: # Undetermined
|
||||
cursor.execute("""
|
||||
UPDATE members
|
||||
SET bounce_count = bounce_count + 1,
|
||||
last_bounce_at = %s
|
||||
WHERE member_id = %s
|
||||
""", (timestamp, member_id))
|
||||
""", (timestamp, new_status, member_id))
|
||||
elif bounce_type == 'Transient':
|
||||
# Check current bounce count
|
||||
cursor.execute("SELECT bounce_count, bounce_status FROM members WHERE member_id = %s", (member_id,))
|
||||
current = cursor.fetchone()
|
||||
|
||||
conn.commit()
|
||||
# If already hard bounced, don't change status
|
||||
if current and current['bounce_status'] != 'hard_bounce':
|
||||
new_count = current['bounce_count'] + 1
|
||||
# After 3 soft bounces, mark as soft_bounce status
|
||||
new_status = 'soft_bounce' if new_count >= 3 else 'clean'
|
||||
|
||||
cursor.execute("""
|
||||
UPDATE members
|
||||
SET bounce_count = %s,
|
||||
last_bounce_at = %s,
|
||||
bounce_status = %s
|
||||
WHERE member_id = %s
|
||||
""", (new_count, timestamp, new_status, member_id))
|
||||
else: # Undetermined
|
||||
cursor.execute("""
|
||||
UPDATE members
|
||||
SET bounce_count = bounce_count + 1,
|
||||
last_bounce_at = %s
|
||||
WHERE member_id = %s
|
||||
""", (timestamp, member_id))
|
||||
|
||||
conn.commit()
|
||||
cursor.close()
|
||||
|
||||
except Exception as e:
|
||||
print(f"✗ Error processing bounce: {str(e)}")
|
||||
print(f"Error type: {type(e).__name__}")
|
||||
print(f"Bounce data: {bounce_data}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
raise
|
||||
|
||||
@app.post("/webhooks/sns", response_class=PlainTextResponse)
|
||||
async def sns_webhook(request: Request):
|
||||
"""Handle SNS notifications for bounces and complaints"""
|
||||
try:
|
||||
print(f"=== SNS Webhook Request ===")
|
||||
print(f"Headers: {dict(request.headers)}")
|
||||
print(f"Content-Type: {request.headers.get('content-type')}")
|
||||
print(f"User-Agent: {request.headers.get('user-agent')}")
|
||||
|
||||
# Verify SNS signature
|
||||
message = await verify_sns_signature(request)
|
||||
|
||||
print(f"Message Type: {message.get('Type')}")
|
||||
print(f"Message Keys: {list(message.keys())}")
|
||||
|
||||
message_type = message.get('Type')
|
||||
|
||||
# Handle subscription confirmation
|
||||
if message_type == 'SubscriptionConfirmation':
|
||||
subscribe_url = message.get('SubscribeURL')
|
||||
print(f"Subscription confirmation received, URL: {subscribe_url}")
|
||||
if subscribe_url:
|
||||
# Confirm subscription
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.get(subscribe_url)
|
||||
print(f"Subscription confirmation response: {response.status_code}")
|
||||
return "Subscription confirmed"
|
||||
|
||||
# Handle notification
|
||||
elif message_type == 'Notification':
|
||||
# Parse the message
|
||||
inner_message = message.get('Message', '{}')
|
||||
print(f"Inner message (first 500 chars): {inner_message[:500]}")
|
||||
notification = json.loads(inner_message)
|
||||
|
||||
# SES can send either 'notificationType' or 'eventType' depending on configuration
|
||||
notification_type = notification.get('notificationType') or notification.get('eventType')
|
||||
print(f"Notification type: {notification_type}")
|
||||
|
||||
if notification_type == 'Bounce':
|
||||
bounce = notification.get('bounce', {})
|
||||
print(f"\n✓ Processing Bounce")
|
||||
print(f" Bounce Type: {bounce.get('bounceType')}")
|
||||
print(f" Recipients: {[r.get('emailAddress') for r in bounce.get('bouncedRecipients', [])]}")
|
||||
await process_bounce(bounce)
|
||||
print(f" ✓ Bounce processed successfully")
|
||||
return "Bounce processed"
|
||||
|
||||
elif notification_type == 'Complaint':
|
||||
# We could also track complaints similarly to bounces
|
||||
print(f"\n✓ Complaint received")
|
||||
return "Complaint received"
|
||||
|
||||
print(f"=== End SNS Webhook Request ===")
|
||||
return "OK"
|
||||
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as e:
|
||||
print(f"SNS webhook error: {str(e)}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
raise HTTPException(status_code=500, detail=str(e))
|
||||
|
||||
else:
|
||||
# Provide stub functions when SNS webhooks are disabled
|
||||
print("SNS webhooks disabled - bounce handling via email only")
|
||||
|
||||
# Bounce management endpoints (conditionally enabled)
|
||||
if ENABLE_BOUNCE_HANDLING:
|
||||
class BounceLog(BaseModel):
|
||||
bounce_id: int
|
||||
email: str
|
||||
bounce_type: str
|
||||
bounce_subtype: Optional[str] = None
|
||||
diagnostic_code: Optional[str] = None
|
||||
timestamp: datetime
|
||||
feedback_id: Optional[str] = None
|
||||
created_at: datetime
|
||||
|
||||
class MemberWithBounces(BaseModel):
|
||||
member_id: int
|
||||
name: str
|
||||
email: str
|
||||
active: bool
|
||||
bounce_count: int
|
||||
last_bounce_at: Optional[datetime] = None
|
||||
bounce_status: str
|
||||
|
||||
@app.get("/members/{member_id}/bounces", response_model=List[BounceLog])
|
||||
async def get_member_bounces(member_id: int, current_user: CurrentUser = require_read_access()):
|
||||
"""Get bounce history for a member"""
|
||||
with get_db() as conn:
|
||||
cursor = conn.cursor(dictionary=True)
|
||||
cursor.execute("""
|
||||
SELECT bounce_id, email, bounce_type, bounce_subtype, diagnostic_code,
|
||||
timestamp, feedback_id, created_at
|
||||
FROM bounce_logs
|
||||
WHERE member_id = %s
|
||||
ORDER BY timestamp DESC
|
||||
""", (member_id,))
|
||||
bounces = cursor.fetchall()
|
||||
cursor.close()
|
||||
return bounces
|
||||
|
||||
except Exception as e:
|
||||
print(f"✗ Error processing bounce: {str(e)}")
|
||||
print(f"Error type: {type(e).__name__}")
|
||||
print(f"Bounce data: {bounce_data}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
raise
|
||||
@app.patch("/members/{member_id}/bounce-status")
|
||||
async def reset_bounce_status(member_id: int, current_user: CurrentUser = require_write_access()):
|
||||
"""Reset bounce status for a member (e.g., after email address is corrected)"""
|
||||
with get_db() as conn:
|
||||
cursor = conn.cursor()
|
||||
cursor.execute("""
|
||||
UPDATE members
|
||||
SET bounce_count = 0,
|
||||
last_bounce_at = NULL,
|
||||
bounce_status = 'clean'
|
||||
WHERE member_id = %s
|
||||
""", (member_id,))
|
||||
conn.commit()
|
||||
|
||||
@app.post("/webhooks/sns", response_class=PlainTextResponse)
|
||||
async def sns_webhook(request: Request):
|
||||
"""Handle SNS notifications for bounces and complaints"""
|
||||
try:
|
||||
print(f"=== SNS Webhook Request ===")
|
||||
print(f"Headers: {dict(request.headers)}")
|
||||
print(f"Content-Type: {request.headers.get('content-type')}")
|
||||
print(f"User-Agent: {request.headers.get('user-agent')}")
|
||||
if cursor.rowcount == 0:
|
||||
raise HTTPException(status_code=404, detail="Member not found")
|
||||
|
||||
# Verify SNS signature
|
||||
message = await verify_sns_signature(request)
|
||||
cursor.close()
|
||||
return {"message": "Bounce status reset successfully"}
|
||||
|
||||
print(f"Message Type: {message.get('Type')}")
|
||||
print(f"Message Keys: {list(message.keys())}")
|
||||
else:
|
||||
# When bounce handling is disabled, provide stub endpoints that return appropriate responses
|
||||
@app.get("/members/{member_id}/bounces")
|
||||
async def get_member_bounces_disabled(member_id: int, current_user: CurrentUser = require_read_access()):
|
||||
"""Bounce history disabled - returns empty list"""
|
||||
return []
|
||||
|
||||
message_type = message.get('Type')
|
||||
|
||||
# Handle subscription confirmation
|
||||
if message_type == 'SubscriptionConfirmation':
|
||||
subscribe_url = message.get('SubscribeURL')
|
||||
print(f"Subscription confirmation received, URL: {subscribe_url}")
|
||||
if subscribe_url:
|
||||
# Confirm subscription
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.get(subscribe_url)
|
||||
print(f"Subscription confirmation response: {response.status_code}")
|
||||
return "Subscription confirmed"
|
||||
|
||||
# Handle notification
|
||||
elif message_type == 'Notification':
|
||||
# Parse the message
|
||||
inner_message = message.get('Message', '{}')
|
||||
print(f"Inner message (first 500 chars): {inner_message[:500]}")
|
||||
notification = json.loads(inner_message)
|
||||
|
||||
# SES can send either 'notificationType' or 'eventType' depending on configuration
|
||||
notification_type = notification.get('notificationType') or notification.get('eventType')
|
||||
print(f"Notification type: {notification_type}")
|
||||
|
||||
if notification_type == 'Bounce':
|
||||
bounce = notification.get('bounce', {})
|
||||
print(f"\n✓ Processing Bounce")
|
||||
print(f" Bounce Type: {bounce.get('bounceType')}")
|
||||
print(f" Recipients: {[r.get('emailAddress') for r in bounce.get('bouncedRecipients', [])]}")
|
||||
await process_bounce(bounce)
|
||||
print(f" ✓ Bounce processed successfully")
|
||||
return "Bounce processed"
|
||||
|
||||
elif notification_type == 'Complaint':
|
||||
# We could also track complaints similarly to bounces
|
||||
print(f"\n✓ Complaint received")
|
||||
return "Complaint received"
|
||||
|
||||
print(f"=== End SNS Webhook Request ===")
|
||||
return "OK"
|
||||
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as e:
|
||||
print(f"SNS webhook error: {str(e)}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
raise HTTPException(status_code=500, detail=str(e))
|
||||
|
||||
# Bounce management endpoints
|
||||
class BounceLog(BaseModel):
|
||||
bounce_id: int
|
||||
email: str
|
||||
bounce_type: str
|
||||
bounce_subtype: Optional[str] = None
|
||||
diagnostic_code: Optional[str] = None
|
||||
timestamp: datetime
|
||||
feedback_id: Optional[str] = None
|
||||
created_at: datetime
|
||||
|
||||
class MemberWithBounces(BaseModel):
|
||||
member_id: int
|
||||
name: str
|
||||
email: str
|
||||
active: bool
|
||||
bounce_count: int
|
||||
last_bounce_at: Optional[datetime] = None
|
||||
bounce_status: str
|
||||
|
||||
@app.get("/members/{member_id}/bounces", response_model=List[BounceLog])
|
||||
async def get_member_bounces(member_id: int, current_user: CurrentUser = require_read_access()):
|
||||
"""Get bounce history for a member"""
|
||||
with get_db() as conn:
|
||||
cursor = conn.cursor(dictionary=True)
|
||||
cursor.execute("""
|
||||
SELECT bounce_id, email, bounce_type, bounce_subtype, diagnostic_code,
|
||||
timestamp, feedback_id, created_at
|
||||
FROM bounce_logs
|
||||
WHERE member_id = %s
|
||||
ORDER BY timestamp DESC
|
||||
""", (member_id,))
|
||||
bounces = cursor.fetchall()
|
||||
cursor.close()
|
||||
return bounces
|
||||
|
||||
@app.patch("/members/{member_id}/bounce-status")
|
||||
async def reset_bounce_status(member_id: int, current_user: CurrentUser = require_write_access()):
|
||||
"""Reset bounce status for a member (e.g., after email address is corrected)"""
|
||||
with get_db() as conn:
|
||||
cursor = conn.cursor()
|
||||
cursor.execute("""
|
||||
UPDATE members
|
||||
SET bounce_count = 0,
|
||||
last_bounce_at = NULL,
|
||||
bounce_status = 'clean'
|
||||
WHERE member_id = %s
|
||||
""", (member_id,))
|
||||
conn.commit()
|
||||
|
||||
if cursor.rowcount == 0:
|
||||
raise HTTPException(status_code=404, detail="Member not found")
|
||||
|
||||
cursor.close()
|
||||
return {"message": "Bounce status reset successfully"}
|
||||
@app.patch("/members/{member_id}/bounce-status")
|
||||
async def reset_bounce_status_disabled(member_id: int, current_user: CurrentUser = require_write_access()):
|
||||
"""Bounce status reset disabled"""
|
||||
raise HTTPException(status_code=501, detail="Bounce handling is disabled")
|
||||
|
||||
if __name__ == "__main__":
|
||||
import uvicorn
|
||||
|
||||
@@ -105,6 +105,10 @@ class APIClient {
|
||||
return this.request('/health');
|
||||
}
|
||||
|
||||
async getConfig() {
|
||||
return this.request('/config');
|
||||
}
|
||||
|
||||
async testAuth() {
|
||||
return this.request('/');
|
||||
}
|
||||
|
||||
@@ -352,12 +352,15 @@ class MailingListApp {
|
||||
try {
|
||||
uiManager.setLoading(true);
|
||||
|
||||
// Load lists and members in parallel
|
||||
const [lists, members] = await Promise.all([
|
||||
// Load configuration, lists and members in parallel
|
||||
const [config, lists, members] = await Promise.all([
|
||||
apiClient.getConfig(),
|
||||
apiClient.getLists(),
|
||||
apiClient.getMembers()
|
||||
]);
|
||||
|
||||
this.config = config;
|
||||
|
||||
this.lists = lists;
|
||||
this.members = members;
|
||||
|
||||
@@ -531,7 +534,7 @@ class MailingListApp {
|
||||
row.innerHTML = `
|
||||
<td>
|
||||
<div class="font-medium">${uiManager.escapeHtml(member.name)}</div>
|
||||
${member.bounce_count > 0 ? `<div class="text-xs text-muted" style="margin-top: 2px;"></div>` : ''}
|
||||
${this.config?.bounce_handling_enabled && member.bounce_count > 0 ? `<div class="text-xs text-muted" style="margin-top: 2px;"></div>` : ''}
|
||||
</td>
|
||||
<td>
|
||||
<a href="mailto:${member.email}" style="color: var(--primary-color)">
|
||||
@@ -552,8 +555,8 @@ class MailingListApp {
|
||||
</td>
|
||||
`;
|
||||
|
||||
// Add bounce badge if member has bounces
|
||||
if (member.bounce_count > 0) {
|
||||
// Add bounce badge if member has bounces (only if bounce handling is enabled)
|
||||
if (this.config?.bounce_handling_enabled && member.bounce_count > 0) {
|
||||
const bounceInfoDiv = row.cells[0].querySelector('.text-xs');
|
||||
const bounceBadge = uiManager.createBounceStatusBadge(member.bounce_status, member.bounce_count);
|
||||
if (bounceBadge) {
|
||||
@@ -577,8 +580,8 @@ class MailingListApp {
|
||||
uiManager.showMemberSubscriptionsModal(member);
|
||||
});
|
||||
|
||||
// Create Bounces button (show if member has any bounces or for admins/operators)
|
||||
if (member.bounce_count > 0 || hasWriteAccess) {
|
||||
// Create Bounces button (show if bounce handling is enabled and member has bounces or for admins/operators)
|
||||
if (this.config?.bounce_handling_enabled && (member.bounce_count > 0 || hasWriteAccess)) {
|
||||
const bouncesBtn = document.createElement('button');
|
||||
bouncesBtn.className = `btn btn-sm ${member.bounce_count > 0 ? 'btn-warning' : 'btn-secondary'}`;
|
||||
bouncesBtn.innerHTML = `<i class="fas fa-exclamation-triangle"></i> Bounces${member.bounce_count > 0 ? ` (${member.bounce_count})` : ''}`;
|
||||
@@ -852,8 +855,8 @@ class MailingListApp {
|
||||
statusBadge.className = `status-badge ${member.active ? 'active' : 'inactive'}`;
|
||||
statusBadge.innerHTML = `<i class="fas fa-${member.active ? 'check' : 'times'}"></i> ${member.active ? 'Active' : 'Inactive'}`;
|
||||
|
||||
// Add bounce status if exists
|
||||
if (member.bounce_status && member.bounce_status !== 'clean') {
|
||||
// Add bounce status if exists and bounce handling is enabled
|
||||
if (this.config?.bounce_handling_enabled && member.bounce_status && member.bounce_status !== 'clean') {
|
||||
const bounceIndicator = document.createElement('span');
|
||||
bounceIndicator.className = `bounce-badge bounce-${member.bounce_status === 'hard_bounce' ? 'hard' : 'soft'}`;
|
||||
bounceIndicator.innerHTML = `<i class="fas fa-exclamation-triangle"></i> Bounces`;
|
||||
@@ -868,8 +871,8 @@ class MailingListApp {
|
||||
const actionsCell = row.insertCell();
|
||||
actionsCell.className = 'action-buttons';
|
||||
|
||||
// Bounces button (if member has bounce data)
|
||||
if (member.bounce_count > 0) {
|
||||
// Bounces button (if bounce handling is enabled and member has bounce data)
|
||||
if (this.config?.bounce_handling_enabled && member.bounce_count > 0) {
|
||||
const bouncesBtn = uiManager.createActionButton('Bounces', 'exclamation-triangle', 'btn-warning', () => {
|
||||
uiManager.showBounceHistory(member);
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user