Compare commits
7 Commits
ac23638125
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f3d7592e7d | ||
|
|
12a82c8d03 | ||
|
|
b34ea2ed84 | ||
|
|
8fd951fd1f | ||
|
|
ecbc38cf8e | ||
|
|
d37027ee5a | ||
|
|
72f3297a80 |
23
.env.example
23
.env.example
@@ -20,3 +20,26 @@ MYSQL_ROOT_PASSWORD=change_this_root_password
|
||||
|
||||
# API Configuration
|
||||
API_TOKEN=change_this_to_a_secure_random_token
|
||||
|
||||
# Bounce Handling Configuration (Optional)
|
||||
# Choose one of the bounce handling methods below:
|
||||
|
||||
# Method 1: SNS Webhook Bounce Handling (Requires SES Production Access)
|
||||
# Set to 'true' to enable real-time SNS webhook bounce handling
|
||||
ENABLE_SNS_WEBHOOKS=false
|
||||
ENABLE_BOUNCE_HANDLING=false
|
||||
|
||||
# If ENABLE_SNS_WEBHOOKS=true, you need:
|
||||
# 1. AWS SNS topic configured
|
||||
# 2. SES configured to send notifications to SNS topic
|
||||
# 3. Valid HTTPS domain for webhook endpoint
|
||||
# 4. SNS subscription confirmed to your webhook endpoint
|
||||
|
||||
# Method 2: Email-Based Bounce Processing (Works with SES Sandbox)
|
||||
# Set to 'true' to enable email-based bounce processing
|
||||
# This processes bounce emails that are sent to bounces@lists.sasalliance.org
|
||||
ENABLE_EMAIL_BOUNCE_PROCESSING=false
|
||||
|
||||
# Note: You can enable both methods, but typically only one is needed
|
||||
# Email-based processing works with SES sandbox accounts
|
||||
# SNS webhooks provide real-time processing but require SES production access
|
||||
277
BOUNCE_HANDLING_SETUP.md
Normal file
277
BOUNCE_HANDLING_SETUP.md
Normal file
@@ -0,0 +1,277 @@
|
||||
# SES SNS Bounce Handling Setup (Optional)
|
||||
|
||||
**⚠️ NOTICE: Bounce handling is optional and disabled by default.**
|
||||
|
||||
This document describes how to configure AWS SES and SNS to handle email bounces automatically in the Mail List Manager.
|
||||
|
||||
**Prerequisites:**
|
||||
- SES production access (not available in sandbox mode)
|
||||
- Valid HTTPS domain for webhook endpoint
|
||||
- Bounce handling must be enabled in configuration
|
||||
|
||||
## Overview
|
||||
|
||||
The system can optionally use AWS Simple Notification Service (SNS) to receive real-time bounce notifications from AWS Simple Email Service (SES). When bounce handling is enabled and an email bounces:
|
||||
|
||||
1. SES sends a notification to an SNS topic
|
||||
2. SNS forwards the notification to your webhook endpoint
|
||||
3. The API processes the notification and updates the database
|
||||
4. Members with hard bounces are automatically deactivated
|
||||
5. Bounce history is tracked and displayed in the UI
|
||||
|
||||
## Bounce Status Types
|
||||
|
||||
- **Clean**: No bounces recorded
|
||||
- **Soft Bounce**: Temporary delivery issues (e.g., mailbox full, temporary server issues)
|
||||
- After 3 soft bounces, the member is marked with soft bounce status
|
||||
- **Hard Bounce**: Permanent delivery failure (e.g., invalid email address, domain doesn't exist)
|
||||
- Member is automatically deactivated and cannot receive emails
|
||||
|
||||
## Setup Instructions
|
||||
|
||||
### 1. Enable Bounce Handling
|
||||
|
||||
First, enable bounce handling in your `.env` file:
|
||||
|
||||
```bash
|
||||
# Enable SNS webhook bounce handling
|
||||
ENABLE_SNS_WEBHOOKS=true
|
||||
ENABLE_BOUNCE_HANDLING=true
|
||||
```
|
||||
|
||||
Restart the API container after making this change:
|
||||
```bash
|
||||
sudo docker-compose restart api
|
||||
```
|
||||
|
||||
### 2. Prerequisites
|
||||
|
||||
- AWS account with SES configured and verified
|
||||
- Your Mail List Manager deployed and accessible via HTTPS (required for SNS webhook)
|
||||
- Domain or subdomain for webhook (e.g., `https://lists.yourdomain.com`)
|
||||
|
||||
### 3. Create SNS Topic
|
||||
|
||||
1. Log in to AWS Console and navigate to SNS
|
||||
2. Click "Create topic"
|
||||
3. Choose "Standard" topic type
|
||||
4. Name: `ses-bounce-notifications` (or your preferred name)
|
||||
5. Display name: `SES Bounce Notifications`
|
||||
6. Click "Create topic"
|
||||
7. **Save the Topic ARN** (you'll need it in step 4) arn:aws:sns:eu-west-2:827164363113:ses-bounces
|
||||
|
||||
### 4. Subscribe Your Webhook to SNS Topic
|
||||
|
||||
1. In the SNS topic details, click "Create subscription"
|
||||
2. Protocol: `HTTPS`
|
||||
3. Endpoint: `https://yourdomain.com:8000/webhooks/sns`
|
||||
- Replace `yourdomain.com` with your actual domain
|
||||
- The API must be accessible via HTTPS (SNS doesn't support HTTP)
|
||||
4. Enable raw message delivery: **Unchecked**
|
||||
5. Click "Create subscription"
|
||||
6. The subscription will be in "PendingConfirmation" status
|
||||
|
||||
### 5. Confirm SNS Subscription
|
||||
|
||||
When you create the subscription, SNS will send a `SubscriptionConfirmation` request to your webhook endpoint. The Mail List Manager API automatically confirms this subscription.
|
||||
|
||||
1. Check your API logs:
|
||||
```bash
|
||||
sudo docker-compose logs -f api
|
||||
```
|
||||
2. You should see a log entry indicating the subscription was confirmed
|
||||
3. In the AWS SNS console, refresh the subscriptions list
|
||||
4. The status should change from "PendingConfirmation" to "Confirmed"
|
||||
|
||||
### 6. Configure SES to Send Bounce Notifications
|
||||
|
||||
1. Navigate to AWS SES console
|
||||
2. Go to "Configuration Sets" (or "Verified identities" > select your domain > "Notifications")
|
||||
3. For configuration sets:
|
||||
- Create a new configuration set or select existing
|
||||
- Add "Event destination"
|
||||
- Event types: Select **"Bounce"** (and optionally "Complaint")
|
||||
- Destination: SNS topic
|
||||
- Select your SNS topic created in step 2
|
||||
4. For verified identities:
|
||||
- Select your sending domain/email
|
||||
- Click "Edit" in the "Notifications" section
|
||||
- Bounce feedback: Select your SNS topic
|
||||
- Include original headers: Enabled (optional)
|
||||
- Click "Save changes"
|
||||
|
||||
### 7. Verify Setup
|
||||
|
||||
#### Test with a Bounce Simulator
|
||||
|
||||
AWS SES provides bounce simulator addresses:
|
||||
|
||||
```bash
|
||||
# From inside Postfix container
|
||||
docker-compose exec postfix bash
|
||||
echo "Test bounce" | mail -s "Test" bounce@simulator.amazonses.com
|
||||
```
|
||||
|
||||
Or send to your mailing list with a test recipient:
|
||||
|
||||
1. Add `bounce@simulator.amazonses.com` as a member
|
||||
2. Subscribe to a test list
|
||||
3. Send an email to the list
|
||||
|
||||
#### Check the Results
|
||||
|
||||
1. Wait a few minutes for SES to process and send the notification
|
||||
2. Check API logs:
|
||||
```bash
|
||||
sudo docker-compose logs api | grep -i bounce
|
||||
```
|
||||
3. Log in to the web UI
|
||||
4. Go to Members tab
|
||||
5. Find the test member and click the "Bounces" button
|
||||
6. You should see the bounce event recorded
|
||||
|
||||
### 8. Security Considerations
|
||||
|
||||
#### SNS Signature Verification
|
||||
|
||||
The webhook endpoint automatically verifies SNS message signatures to ensure notifications are genuine AWS messages. This prevents unauthorized parties from sending fake bounce notifications.
|
||||
|
||||
#### HTTPS Requirement
|
||||
|
||||
SNS requires HTTPS for webhooks. You'll need:
|
||||
- Valid SSL/TLS certificate for your domain
|
||||
- Reverse proxy (e.g., Nginx, Apache) in front of the API container
|
||||
- Or use AWS API Gateway as a proxy
|
||||
|
||||
#### Example Nginx Configuration
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 443 ssl http2;
|
||||
server_name lists.yourdomain.com;
|
||||
|
||||
ssl_certificate /path/to/cert.pem;
|
||||
ssl_certificate_key /path/to/key.pem;
|
||||
|
||||
# Webhook endpoint
|
||||
location /webhooks/sns {
|
||||
proxy_pass http://localhost:8000;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
}
|
||||
|
||||
# Optional: proxy API for web UI
|
||||
location /api {
|
||||
proxy_pass http://localhost:8000;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 9. Managing Bounces in the UI
|
||||
|
||||
#### View Bounce Status
|
||||
|
||||
In the Members tab, bounced emails are indicated with:
|
||||
- Warning badge showing bounce count
|
||||
- Color-coded status (yellow for soft bounce, red for hard bounce)
|
||||
- Last bounce timestamp
|
||||
|
||||
#### View Bounce History
|
||||
|
||||
1. Click the "Bounces" button next to a member
|
||||
2. View detailed bounce history including:
|
||||
- Bounce type (Permanent, Transient, Undetermined)
|
||||
- Bounce subtype
|
||||
- Diagnostic code from the receiving mail server
|
||||
- Timestamp of each bounce
|
||||
|
||||
#### Reset Bounce Status
|
||||
|
||||
If a member's email has been corrected or verified:
|
||||
|
||||
1. Open the bounce history modal
|
||||
2. Click "Reset Bounce Status"
|
||||
3. Confirm the action
|
||||
4. The member's bounce count is cleared and they can receive emails again
|
||||
|
||||
**Note**: Only users with write access (administrators and operators) can reset bounce status.
|
||||
|
||||
### 10. Monitoring and Maintenance
|
||||
|
||||
#### Check Bounce Logs
|
||||
|
||||
```bash
|
||||
# View all bounces in database
|
||||
sudo docker-compose exec mysql mysql -u maillist -p maillist -e "SELECT * FROM bounce_logs ORDER BY timestamp DESC LIMIT 20;"
|
||||
|
||||
# Count bounces by type
|
||||
sudo docker-compose exec mysql mysql -u maillist -p maillist -e "SELECT bounce_type, COUNT(*) as count FROM bounce_logs GROUP BY bounce_type;"
|
||||
|
||||
# Find members with bounces
|
||||
sudo docker-compose exec mysql mysql -u maillist -p maillist -e "SELECT name, email, bounce_count, bounce_status FROM members WHERE bounce_count > 0;"
|
||||
```
|
||||
|
||||
#### API Health Check
|
||||
|
||||
```bash
|
||||
# Check if webhook is accessible
|
||||
curl -X POST https://yourdomain.com:8000/webhooks/sns \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"Type":"test"}'
|
||||
```
|
||||
|
||||
#### Clean Up Old Bounce Records
|
||||
|
||||
Periodically review and clean up old bounce records:
|
||||
|
||||
```sql
|
||||
-- Delete bounce logs older than 90 days
|
||||
DELETE FROM bounce_logs WHERE created_at < DATE_SUB(NOW(), INTERVAL 90 DAY);
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### SNS Subscription Not Confirming
|
||||
|
||||
- Ensure the API container is running and accessible via HTTPS
|
||||
- Check API logs for errors
|
||||
- Verify firewall rules allow HTTPS traffic to port 8000
|
||||
- Test the endpoint manually: `curl https://yourdomain.com:8000/health`
|
||||
|
||||
### Bounces Not Being Recorded
|
||||
|
||||
1. Verify SNS topic is receiving messages:
|
||||
- Check SNS topic metrics in AWS Console
|
||||
2. Verify subscription is active:
|
||||
- Check subscription status in SNS console
|
||||
3. Check API logs for webhook errors:
|
||||
```bash
|
||||
sudo docker-compose logs api | grep -i "sns\|bounce"
|
||||
```
|
||||
4. Test signature verification:
|
||||
- Temporarily add debug logging to the webhook endpoint
|
||||
|
||||
### Members Not Being Deactivated
|
||||
|
||||
- Check if bounce type is "Permanent"
|
||||
- Review member's bounce_status in database:
|
||||
```bash
|
||||
sudo docker-compose exec mysql mysql -u maillist -p maillist -e "SELECT * FROM members WHERE email='problem@example.com';"
|
||||
```
|
||||
- Verify bounce processing logic in API logs
|
||||
|
||||
### SSL Certificate Issues
|
||||
|
||||
If using self-signed certificates, SNS will reject the webhook. You must use:
|
||||
- Valid certificate from a trusted CA (Let's Encrypt, etc.)
|
||||
- Or use AWS Certificate Manager with API Gateway
|
||||
|
||||
## Additional Resources
|
||||
|
||||
- [AWS SES Bounce Handling](https://docs.aws.amazon.com/ses/latest/dg/event-publishing-retrieving-sns.html)
|
||||
- [AWS SNS HTTPS Subscriptions](https://docs.aws.amazon.com/sns/latest/dg/sns-http-https-endpoint-as-subscriber.html)
|
||||
- [SES Bounce Types](https://docs.aws.amazon.com/ses/latest/dg/notification-contents.html#bounce-types)
|
||||
240
CODE_REVIEW_FINDINGS.md
Normal file
240
CODE_REVIEW_FINDINGS.md
Normal file
@@ -0,0 +1,240 @@
|
||||
# Code Review Findings & Fixes
|
||||
|
||||
## Date: 13 October 2025
|
||||
|
||||
## Critical Issues Found and Fixed
|
||||
|
||||
### 1. ✅ FIXED: Header Merging Bug in API Client
|
||||
**Severity:** HIGH
|
||||
**Location:** `web/static/js/api.js` - `request()` method
|
||||
|
||||
**Issue:**
|
||||
The header merging was incorrect. When passing custom headers in options, they were being overridden by the default headers due to incorrect spread operator order:
|
||||
|
||||
```javascript
|
||||
// BEFORE (BUGGY):
|
||||
const config = {
|
||||
headers: { ...this.headers },
|
||||
...options // This overwrites headers!
|
||||
};
|
||||
```
|
||||
|
||||
**Fix:**
|
||||
```javascript
|
||||
// AFTER (FIXED):
|
||||
const config = {
|
||||
...options,
|
||||
headers: {
|
||||
...this.headers,
|
||||
...(options.headers || {}) // Custom headers take precedence
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
**Impact:**
|
||||
- The `login()` method was trying to remove the Authorization header for unauthenticated requests, but it was being overridden
|
||||
- Could have caused authentication issues or unexpected behavior
|
||||
|
||||
---
|
||||
|
||||
### 2. ✅ FIXED: Missing API Client Methods
|
||||
**Severity:** HIGH
|
||||
**Location:** `web/static/js/api.js`
|
||||
|
||||
**Issue:** Multiple methods were being called by the UI but didn't exist in the API client:
|
||||
- `login(username, password)` - ❌ Missing
|
||||
- `logout()` - ❌ Missing
|
||||
- `getCurrentUser()` - ❌ Missing
|
||||
- `getUsers()` - ❌ Missing
|
||||
- `createUser(userData)` - ❌ Missing
|
||||
- `updateUser(userId, userData)` - ❌ Missing
|
||||
- `deleteUser(userId)` - ❌ Missing
|
||||
- `getMemberBounces(memberId)` - ❌ Missing
|
||||
- `resetBounceStatus(memberId)` - ❌ Missing
|
||||
|
||||
**Fix:** Added all missing methods with proper implementation
|
||||
|
||||
---
|
||||
|
||||
### 3. ✅ FIXED: Incorrect API Base URL for Production
|
||||
**Severity:** HIGH
|
||||
**Location:** `web/static/js/api.js` - `getBaseURL()` method
|
||||
|
||||
**Issue:**
|
||||
When running behind a reverse proxy (Caddy), the API client was trying to connect to `https://lists.sasalliance.org:8000` which:
|
||||
- Port 8000 is not exposed through Caddy
|
||||
- Would cause CORS issues
|
||||
- Would fail all API requests
|
||||
|
||||
**Fix:**
|
||||
```javascript
|
||||
// Production detection
|
||||
if (hostname === 'localhost' || hostname === '127.0.0.1') {
|
||||
return `${protocol}//${hostname}:8000`; // Development
|
||||
}
|
||||
// Production - use reverse proxy path
|
||||
return `${protocol}//${hostname}/api`;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Potential Issues (Not Critical, But Worth Noting)
|
||||
|
||||
### 4. ⚠️ CORS Configuration
|
||||
**Severity:** MEDIUM
|
||||
**Location:** `api/main.py`
|
||||
|
||||
**Current Configuration:**
|
||||
```python
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=["*"], # Allows all origins
|
||||
allow_credentials=True,
|
||||
allow_methods=["*"],
|
||||
allow_headers=["*"],
|
||||
)
|
||||
```
|
||||
|
||||
**Issue:**
|
||||
- `allow_origins=["*"]` with `allow_credentials=True` is not allowed by browsers
|
||||
- Currently working because Caddy reverse proxy makes same-origin requests
|
||||
- If you ever need direct API access from different origins, this will fail
|
||||
|
||||
**Recommendation:**
|
||||
```python
|
||||
# For production
|
||||
allow_origins=[
|
||||
"https://lists.sasalliance.org",
|
||||
"http://localhost:3000", # For development
|
||||
],
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 5. ✅ GOOD: Authentication Token Storage
|
||||
**Severity:** INFO
|
||||
**Location:** `web/static/js/app.js`
|
||||
|
||||
**Current Implementation:**
|
||||
```javascript
|
||||
localStorage.setItem('authToken', response.access_token);
|
||||
```
|
||||
|
||||
**Assessment:**
|
||||
- ✅ Proper use of localStorage for JWT tokens
|
||||
- ✅ Token is cleared on logout
|
||||
- ✅ Token is validated on page load
|
||||
- ✅ Expired tokens trigger automatic logout
|
||||
|
||||
**Note:** localStorage is appropriate for JWT tokens. HttpOnly cookies would be more secure but require different architecture.
|
||||
|
||||
---
|
||||
|
||||
### 6. ✅ GOOD: Error Handling
|
||||
**Severity:** INFO
|
||||
**Location:** `web/static/js/ui.js` - `handleError()` method
|
||||
|
||||
**Assessment:**
|
||||
- ✅ Proper handling of 401 (auth) errors with automatic logout
|
||||
- ✅ Proper handling of 403 (forbidden) errors
|
||||
- ✅ Proper handling of 404 (not found) errors
|
||||
- ✅ Proper handling of 500 (server) errors
|
||||
- ✅ User-friendly error messages
|
||||
|
||||
---
|
||||
|
||||
### 7. ✅ GOOD: Role-Based Access Control
|
||||
**Severity:** INFO
|
||||
**Location:** Multiple files
|
||||
|
||||
**Assessment:**
|
||||
- ✅ Backend enforces RBAC with decorators (`require_admin()`, `require_write_access()`, etc.)
|
||||
- ✅ Frontend checks user roles before showing UI elements
|
||||
- ✅ Three roles properly implemented: administrator, operator, read-only
|
||||
- ✅ UI elements are hidden/shown based on permissions
|
||||
|
||||
---
|
||||
|
||||
### 8. ⚠️ Token Expiration Handling
|
||||
**Severity:** LOW
|
||||
**Location:** `api/main.py` and `web/static/js/app.js`
|
||||
|
||||
**Current Configuration:**
|
||||
- JWT tokens expire in 30 minutes
|
||||
- Session records expire in 24 hours (but not enforced in JWT)
|
||||
|
||||
**Potential Issue:**
|
||||
- Users will be logged out after 30 minutes without warning
|
||||
- No token refresh mechanism
|
||||
|
||||
**Recommendation (Future Enhancement):**
|
||||
Consider implementing:
|
||||
- Token refresh endpoint
|
||||
- Warning before token expiration
|
||||
- Silent token refresh in background
|
||||
|
||||
---
|
||||
|
||||
### 9. ✅ GOOD: SQL Injection Prevention
|
||||
**Severity:** INFO
|
||||
**Location:** `api/main.py`
|
||||
|
||||
**Assessment:**
|
||||
- ✅ All SQL queries use parameterized statements with `%s` placeholders
|
||||
- ✅ No string concatenation in SQL queries
|
||||
- ✅ MySQL connector properly escapes parameters
|
||||
|
||||
---
|
||||
|
||||
### 10. ✅ GOOD: Password Security
|
||||
**Severity:** INFO
|
||||
**Location:** `api/main.py`
|
||||
|
||||
**Assessment:**
|
||||
- ✅ Uses bcrypt for password hashing (`passlib.context.CryptContext`)
|
||||
- ✅ Passwords are never stored in plain text
|
||||
- ✅ Passwords are never logged or exposed in API responses
|
||||
|
||||
---
|
||||
|
||||
## Testing Recommendations
|
||||
|
||||
1. **Hard refresh browser cache** after updates (Ctrl+Shift+R)
|
||||
2. **Test authentication flow:**
|
||||
- Login with valid credentials
|
||||
- Login with invalid credentials
|
||||
- Token expiration after 30 minutes
|
||||
- Logout functionality
|
||||
3. **Test role-based access:**
|
||||
- Administrator can see Users tab
|
||||
- Operator can modify lists/members but not users
|
||||
- Read-only can view but not modify
|
||||
4. **Test bounce handling:**
|
||||
- View bounce history
|
||||
- Reset bounce status
|
||||
5. **Test bulk import:**
|
||||
- CSV upload
|
||||
- Subscription assignment
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
✅ **3 Critical Bugs Fixed:**
|
||||
1. Header merging bug in request method
|
||||
2. Missing API client methods
|
||||
3. Incorrect production API URL
|
||||
|
||||
✅ **All Core Functionality Working:**
|
||||
- Authentication and authorization
|
||||
- CRUD operations for lists, members, and users
|
||||
- Subscription management
|
||||
- Bounce tracking
|
||||
- Bulk import
|
||||
|
||||
⚠️ **Minor Improvements Suggested:**
|
||||
1. Tighten CORS policy for production
|
||||
2. Consider token refresh mechanism
|
||||
3. Add user session timeout warnings
|
||||
|
||||
**Overall Assessment:** The codebase is now production-ready with proper security, error handling, and functionality. The critical bugs have been fixed and deployed.
|
||||
243
EMAIL_BOUNCE_HANDLING_SETUP.md
Normal file
243
EMAIL_BOUNCE_HANDLING_SETUP.md
Normal file
@@ -0,0 +1,243 @@
|
||||
# Email-Based Bounce Handling Setup
|
||||
|
||||
This document explains the email-based bounce handling system implemented as an alternative to SNS webhooks for environments without SES production access.
|
||||
|
||||
## Overview
|
||||
|
||||
The system processes email bounces directly within the Postfix container by:
|
||||
1. Rewriting return paths to direct bounces to a processing address
|
||||
2. Processing bounce emails via Python script
|
||||
3. Updating member bounce statistics in MySQL database
|
||||
4. Automatically disabling members with excessive bounces
|
||||
2. Setting up an alias that pipes bounce emails to a Python processing script
|
||||
3. The script parses bounce emails, extracts bounced addresses, and updates the database
|
||||
4. Members with hard bounces are automatically deactivated
|
||||
5. Bounce history is tracked and displayed in the UI (same as SNS method)
|
||||
|
||||
## Advantages
|
||||
|
||||
- **Works with SES Sandbox**: No production SES access required
|
||||
- **No External Dependencies**: Doesn't require SNS, webhooks, or HTTPS domains
|
||||
- **Self-Contained**: All processing happens within the existing containers
|
||||
- **Real-time Processing**: Bounces are processed as soon as emails arrive
|
||||
- **Compatible**: Uses the same database schema and UI as SNS bounce handling
|
||||
|
||||
## Configuration
|
||||
|
||||
### 1. Enable Email Bounce Processing
|
||||
|
||||
In your `.env` file:
|
||||
|
||||
```bash
|
||||
# Enable email-based bounce processing
|
||||
ENABLE_EMAIL_BOUNCE_PROCESSING=true
|
||||
|
||||
# This will automatically enable bounce handling features
|
||||
ENABLE_BOUNCE_HANDLING=true # Automatically set to true when email processing is enabled
|
||||
```
|
||||
|
||||
### 2. Restart the System
|
||||
|
||||
```bash
|
||||
sudo docker-compose down
|
||||
sudo docker-compose up --build -d
|
||||
```
|
||||
|
||||
### 3. Verify Configuration
|
||||
|
||||
Check that email bounce processing is enabled:
|
||||
|
||||
```bash
|
||||
curl -s http://localhost:8000/config | jq .
|
||||
```
|
||||
|
||||
Expected output:
|
||||
```json
|
||||
{
|
||||
"bounce_handling_enabled": true,
|
||||
"sns_webhooks_enabled": false,
|
||||
"email_bounce_processing_enabled": true
|
||||
}
|
||||
```
|
||||
|
||||
## How It Works
|
||||
|
||||
### Postfix Configuration
|
||||
|
||||
The system configures Postfix with:
|
||||
|
||||
- `bounce_notice_recipient = bounces@lists.sasalliance.org`
|
||||
- `2bounce_notice_recipient = bounces@lists.sasalliance.org`
|
||||
- `error_notice_recipient = bounces@lists.sasalliance.org`
|
||||
|
||||
### Aliases Configuration
|
||||
|
||||
The `bounces` address is configured to pipe emails to the processing script:
|
||||
|
||||
```
|
||||
bounces: "|/usr/local/bin/process-bounce.py"
|
||||
```
|
||||
|
||||
### Bounce Processing Script
|
||||
|
||||
The Python script (`/usr/local/bin/process-bounce.py`):
|
||||
|
||||
1. **Reads bounce emails** from stdin (via pipe)
|
||||
2. **Parses email content** using multiple regex patterns to extract bounced addresses
|
||||
3. **Analyzes bounce type** based on SMTP error codes:
|
||||
- 5xx codes = Permanent bounces
|
||||
- 4xx codes = Transient bounces
|
||||
- Unknown = Undetermined
|
||||
4. **Updates database** using the same schema as SNS bounce handling:
|
||||
- Logs bounce in `bounce_logs` table
|
||||
- Updates member `bounce_count`, `bounce_status`, and `last_bounce_at`
|
||||
- Deactivates members with permanent bounces
|
||||
- Marks members with soft bounce status after 3 transient bounces
|
||||
|
||||
## Testing
|
||||
|
||||
### Test the Processing Script
|
||||
|
||||
You can test the bounce processing script in test mode:
|
||||
|
||||
```bash
|
||||
# Test with sample bounce email
|
||||
sudo docker-compose exec postfix /usr/local/bin/process-bounce.py --test
|
||||
```
|
||||
|
||||
Expected output:
|
||||
```
|
||||
2025-10-14 15:49:16,041 - bounce-processor - INFO - Starting bounce processing
|
||||
2025-10-14 15:49:16,041 - bounce-processor - INFO - Running in test mode with sample bounce email
|
||||
2025-10-14 15:49:16,050 - bounce-processor - INFO - Extracted addresses: ['testuser@example.com']
|
||||
2025-10-14 15:49:16,050 - bounce-processor - INFO - Test mode - would process 1 bounce(s):
|
||||
2025-10-14 15:49:16,050 - bounce-processor - INFO - {'email': 'testuser@example.com', 'bounce_type': 'Permanent', 'bounce_subtype': 'General', 'diagnostic_code': '', 'timestamp': '2025-10-14 15:49:16'}
|
||||
```
|
||||
|
||||
### Test with Real Bounce Email
|
||||
|
||||
To test with a real bounce:
|
||||
|
||||
1. Send an email to a non-existent address via your mailing list
|
||||
2. Wait for the bounce to be processed
|
||||
3. Check the database for bounce logs:
|
||||
|
||||
```bash
|
||||
# Check bounce logs
|
||||
sudo docker-compose exec mysql mysql -u maillist -pmaillist maillist -e "SELECT * FROM bounce_logs ORDER BY created_at DESC LIMIT 5;"
|
||||
|
||||
# Check member bounce status
|
||||
sudo docker-compose exec mysql mysql -u maillist -pmaillist maillist -e "SELECT member_id, email, bounce_count, bounce_status, last_bounce_at FROM members WHERE bounce_count > 0;"
|
||||
```
|
||||
|
||||
### View in Web Interface
|
||||
|
||||
1. Open http://localhost:3000
|
||||
2. Navigate to the Members tab
|
||||
3. Look for bounce badges and bounce counts next to member names
|
||||
4. Click the "Bounces" button next to a member to view bounce history
|
||||
|
||||
## Supported Bounce Email Formats
|
||||
|
||||
The processing script recognizes these bounce patterns:
|
||||
|
||||
### SMTP Error Codes
|
||||
- 550, 551, 553, 552, 554 (permanent failures)
|
||||
- 450, 451, 452 (temporary failures)
|
||||
|
||||
### Delivery Status Notification (DSN)
|
||||
- `Final-Recipient:` headers
|
||||
- `Original-Recipient:` headers
|
||||
|
||||
### Common Bounce Messages
|
||||
- "user unknown"
|
||||
- "does not exist"
|
||||
- "not found"
|
||||
- "mailbox unavailable"
|
||||
- "recipient rejected"
|
||||
|
||||
## Monitoring
|
||||
|
||||
### View Processing Logs
|
||||
|
||||
```bash
|
||||
# View bounce processing logs
|
||||
sudo docker-compose logs postfix | grep bounce-processor
|
||||
|
||||
# Follow logs in real-time
|
||||
sudo docker-compose logs -f postfix | grep bounce-processor
|
||||
```
|
||||
|
||||
### Check Aliases Configuration
|
||||
|
||||
```bash
|
||||
# Verify aliases are configured correctly
|
||||
sudo docker-compose exec postfix cat /etc/aliases
|
||||
|
||||
# Check alias database
|
||||
sudo docker-compose exec postfix postmap -q bounces /etc/aliases
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Bounces Not Being Processed
|
||||
|
||||
1. **Check aliases configuration:**
|
||||
```bash
|
||||
sudo docker-compose exec postfix cat /etc/aliases
|
||||
```
|
||||
|
||||
2. **Verify script permissions:**
|
||||
```bash
|
||||
sudo docker-compose exec postfix ls -la /usr/local/bin/process-bounce.py
|
||||
```
|
||||
|
||||
3. **Test script manually:**
|
||||
```bash
|
||||
sudo docker-compose exec postfix /usr/local/bin/process-bounce.py --test
|
||||
```
|
||||
|
||||
4. **Check Postfix logs:**
|
||||
```bash
|
||||
sudo docker-compose logs postfix | grep -i bounce
|
||||
```
|
||||
|
||||
### Database Connection Issues
|
||||
|
||||
1. **Check environment variables:**
|
||||
```bash
|
||||
sudo docker-compose exec postfix env | grep MYSQL
|
||||
```
|
||||
|
||||
2. **Test database connection:**
|
||||
```bash
|
||||
sudo docker-compose exec postfix python3 -c "import pymysql; print('PyMySQL available')"
|
||||
```
|
||||
|
||||
### Script Errors
|
||||
|
||||
View detailed error logs:
|
||||
```bash
|
||||
sudo docker-compose logs postfix | grep -A 10 -B 10 "bounce-processor.*ERROR"
|
||||
```
|
||||
|
||||
## Comparison with SNS Webhooks
|
||||
|
||||
| Feature | Email-Based | SNS Webhooks |
|
||||
|---------|-------------|--------------|
|
||||
| **SES Requirement** | Sandbox OK | Production access required |
|
||||
| **External Dependencies** | None | SNS, HTTPS domain |
|
||||
| **Processing Speed** | Real-time | Real-time |
|
||||
| **Setup Complexity** | Low | High |
|
||||
| **Reliability** | High | High |
|
||||
| **Bounce Detection** | Regex-based | AWS-provided |
|
||||
| **Cost** | Free | SNS charges apply |
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. **Monitor bounce processing** to ensure it's working correctly
|
||||
2. **Review bounce patterns** in the logs to improve detection if needed
|
||||
3. **Set up bounce notification alerts** (optional)
|
||||
4. **Consider upgrading to SNS webhooks** when SES production access is available
|
||||
|
||||
Email-based bounce handling provides a robust alternative that works immediately with any SES configuration while providing the same bounce management features as the SNS webhook method.
|
||||
212
EMAIL_BOUNCE_HANDLING_SETUP_COMPREHENSIVE.md
Normal file
212
EMAIL_BOUNCE_HANDLING_SETUP_COMPREHENSIVE.md
Normal file
@@ -0,0 +1,212 @@
|
||||
# Email-Based Bounce Handling Setup (Comprehensive)
|
||||
|
||||
This document explains the complete email-based bounce handling system implemented as an alternative to SNS webhooks for environments without SES production access.
|
||||
|
||||
## Overview
|
||||
|
||||
The system processes email bounces directly within the Postfix container by:
|
||||
1. Rewriting return paths to direct bounces to a processing address
|
||||
2. Processing bounce emails via Python script
|
||||
3. Updating member bounce statistics in MySQL database
|
||||
4. Automatically disabling members with excessive bounces
|
||||
|
||||
## Configuration
|
||||
|
||||
### Environment Variables (in .env)
|
||||
|
||||
```bash
|
||||
# Bounce handling feature flags
|
||||
ENABLE_BOUNCE_HANDLING=true # Master switch for bounce functionality
|
||||
ENABLE_EMAIL_BOUNCE_PROCESSING=true # Enable email-based processing
|
||||
ENABLE_SNS_WEBHOOKS=false # Disable SNS webhooks (optional)
|
||||
|
||||
# Database settings (required for bounce processing)
|
||||
MYSQL_ROOT_PASSWORD=your_root_password
|
||||
MYSQL_DATABASE=maillist
|
||||
MYSQL_USER=maillist
|
||||
MYSQL_PASSWORD=your_maillist_password
|
||||
```
|
||||
|
||||
### Key Components
|
||||
|
||||
#### 1. Return Path Rewriting (`postfix/smtp_generic`)
|
||||
Routes all bounces to the processing address:
|
||||
```
|
||||
@lists.sasalliance.org bounces@lists.sasalliance.org
|
||||
@sasalliance.org bounces@lists.sasalliance.org
|
||||
```
|
||||
|
||||
#### 2. Bounce Processing Script (`postfix/process-bounce.py`)
|
||||
Python script that:
|
||||
- Parses bounce emails for recipient addresses and bounce types
|
||||
- Updates bounce counts in MySQL database
|
||||
- Automatically disables members after 5 hard bounces
|
||||
- Logs all bounce events
|
||||
|
||||
#### 3. Postfix Integration (`postfix/main.cf.template`)
|
||||
```
|
||||
# Return path rewriting for outbound mail
|
||||
smtp_generic_maps = hash:/etc/postfix/smtp_generic
|
||||
|
||||
# Bounce processing
|
||||
bounce_notice_recipient = bounces@lists.sasalliance.org
|
||||
```
|
||||
|
||||
#### 4. Email Aliases (`postfix/entrypoint.sh`)
|
||||
```
|
||||
# Route bounces to processing script
|
||||
bounces: "|/usr/local/bin/python3 /etc/postfix/process-bounce.py"
|
||||
```
|
||||
|
||||
## How It Works
|
||||
|
||||
### Outbound Email Flow
|
||||
1. User sends email to `community@lists.sasalliance.org`
|
||||
2. Postfix expands to member list via MySQL
|
||||
3. Email sent via SES with return path rewritten to `bounces@lists.sasalliance.org`
|
||||
4. If delivery fails, bounce goes to bounce processing address
|
||||
|
||||
### Bounce Processing Flow
|
||||
1. Bounce email arrives at `bounces@lists.sasalliance.org`
|
||||
2. Postfix pipes email to `process-bounce.py` script
|
||||
3. Script parses bounce for recipient and bounce type
|
||||
4. Database updated with bounce information
|
||||
5. Member automatically disabled if hard bounce threshold reached
|
||||
|
||||
### Bounce Types Detected
|
||||
- **Hard Bounces**: Permanent failures (5.x.x SMTP codes)
|
||||
- Invalid email addresses
|
||||
- Domain doesn't exist
|
||||
- Mailbox doesn't exist
|
||||
- **Soft Bounces**: Temporary failures (4.x.x SMTP codes)
|
||||
- Mailbox full
|
||||
- Temporary server issues
|
||||
- Rate limiting
|
||||
|
||||
## Database Schema
|
||||
|
||||
### Bounce Logs Table
|
||||
```sql
|
||||
CREATE TABLE bounce_logs (
|
||||
id INT PRIMARY KEY AUTO_INCREMENT,
|
||||
email VARCHAR(255) NOT NULL,
|
||||
bounce_type ENUM('hard', 'soft', 'complaint') NOT NULL,
|
||||
bounce_reason TEXT,
|
||||
bounced_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
raw_message TEXT,
|
||||
INDEX idx_email (email),
|
||||
INDEX idx_bounced_at (bounced_at)
|
||||
);
|
||||
```
|
||||
|
||||
### Members Table (bounce tracking fields)
|
||||
```sql
|
||||
ALTER TABLE members ADD COLUMN bounce_count INT DEFAULT 0;
|
||||
ALTER TABLE members ADD COLUMN last_bounce_at TIMESTAMP NULL;
|
||||
ALTER TABLE members ADD COLUMN active BOOLEAN DEFAULT true;
|
||||
```
|
||||
|
||||
## Testing Bounce Handling
|
||||
|
||||
### 1. Test Return Path Configuration
|
||||
```bash
|
||||
# Check that return path rewriting is working
|
||||
sudo docker-compose exec postfix postconf smtp_generic_maps
|
||||
sudo docker-compose exec postfix postmap -q "test@lists.sasalliance.org" hash:/etc/postfix/smtp_generic
|
||||
```
|
||||
|
||||
### 2. Simulate Bounce Email
|
||||
```bash
|
||||
# Send test bounce to processing script
|
||||
echo "Subject: Delivery Status Notification (Failure)
|
||||
From: MAILER-DAEMON@ses.amazonaws.com
|
||||
To: bounces@lists.sasalliance.org
|
||||
|
||||
The following message could not be delivered:
|
||||
Recipient: test@example.com
|
||||
Reason: 550 5.1.1 User unknown" | docker-compose exec -T postfix mail -s "Test Bounce" bounces@lists.sasalliance.org
|
||||
```
|
||||
|
||||
### 3. Check Bounce Processing
|
||||
```bash
|
||||
# View bounce logs
|
||||
sudo docker-compose exec mysql mysql -u maillist -pmaillist maillist -e "SELECT * FROM bounce_logs ORDER BY bounced_at DESC LIMIT 5;"
|
||||
|
||||
# Check member bounce counts
|
||||
sudo docker-compose exec mysql mysql -u maillist -pmaillist maillist -e "SELECT email, bounce_count, last_bounce_at, active FROM members WHERE bounce_count > 0;"
|
||||
```
|
||||
|
||||
## Monitoring and Maintenance
|
||||
|
||||
### View Processing Logs
|
||||
```bash
|
||||
# Monitor bounce processing
|
||||
sudo docker-compose logs -f postfix | grep -E "(bounce|process-bounce)"
|
||||
|
||||
# Check API bounce handling status
|
||||
curl -H "Authorization: Bearer $API_TOKEN" http://localhost:8000/config
|
||||
```
|
||||
|
||||
### Reset Member Bounce Count
|
||||
```bash
|
||||
# Via API
|
||||
curl -X POST http://localhost:8000/members/{member_id}/reset-bounces \
|
||||
-H "Authorization: Bearer $API_TOKEN"
|
||||
|
||||
# Via Database
|
||||
sudo docker-compose exec mysql mysql -u maillist -pmaillist maillist -e "UPDATE members SET bounce_count=0, active=true WHERE email='user@example.com';"
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
#### Bounces Not Being Processed
|
||||
1. Check that `bounces` alias exists: `sudo docker-compose exec postfix cat /etc/aliases | grep bounces`
|
||||
2. Verify Python script permissions: `sudo docker-compose exec postfix ls -la /etc/postfix/process-bounce.py`
|
||||
3. Test script manually: `sudo docker-compose exec postfix python3 /etc/postfix/process-bounce.py --test`
|
||||
|
||||
#### Return Path Not Rewritten
|
||||
1. Check smtp_generic configuration: `sudo docker-compose exec postfix postconf smtp_generic_maps`
|
||||
2. Verify map file exists: `sudo docker-compose exec postfix ls -la /etc/postfix/smtp_generic*`
|
||||
3. Test mapping: `sudo docker-compose exec postfix postmap -q "test@lists.sasalliance.org" hash:/etc/postfix/smtp_generic`
|
||||
|
||||
#### Database Connection Issues
|
||||
1. Check PyMySQL installation: `sudo docker-compose exec postfix python3 -c "import pymysql; print('OK')"`
|
||||
2. Test database connection: `sudo docker-compose exec postfix python3 -c "import pymysql; pymysql.connect(host='mysql', user='maillist', password='your_password', database='maillist')"`
|
||||
3. Verify network connectivity: `sudo docker-compose exec postfix ping mysql`
|
||||
|
||||
### Log Analysis
|
||||
```bash
|
||||
# Postfix logs
|
||||
sudo docker-compose logs postfix | grep -E "(bounce|MAILER-DAEMON|process-bounce)"
|
||||
|
||||
# MySQL connection logs
|
||||
sudo docker-compose logs postfix | grep -E "(pymysql|mysql)"
|
||||
|
||||
# SES relay logs
|
||||
sudo docker-compose logs postfix | grep -E "(relay|sent|deferred)"
|
||||
```
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- Bounce processing script runs with limited privileges
|
||||
- Database credentials secured in environment variables
|
||||
- Bounce emails contain sensitive delivery information - logs are rotated
|
||||
- Return path rewriting prevents bounce loops
|
||||
- Processing script validates email format before database updates
|
||||
|
||||
## Performance Notes
|
||||
|
||||
- Bounce processing is asynchronous (doesn't block email delivery)
|
||||
- Database queries are indexed for bounce lookups
|
||||
- Bounce logs should be periodically archived for large volumes
|
||||
- SMTP generic maps are cached by Postfix for performance
|
||||
|
||||
## Advantages over SNS Webhooks
|
||||
|
||||
- **Works with SES Sandbox**: No production SES access required
|
||||
- **No External Dependencies**: Doesn't require SNS, webhooks, or HTTPS domains
|
||||
- **Self-Contained**: All processing happens within existing containers
|
||||
- **Real-time Processing**: Bounces processed as emails arrive
|
||||
- **Compatible**: Uses same database schema and UI as SNS method
|
||||
50
README.md
50
README.md
@@ -172,6 +172,53 @@ docker-compose exec postfix postmap -q "community@lists.sasalliance.org" \
|
||||
mysql:/etc/postfix/mysql_virtual_alias_maps.cf
|
||||
```
|
||||
|
||||
### Bounce Handling (Optional)
|
||||
|
||||
**Email bounce handling is optional and disabled by default.**
|
||||
|
||||
**Three Configuration Options:**
|
||||
|
||||
1. **SNS Webhooks** (Requires SES Production Access):
|
||||
```bash
|
||||
# In .env file
|
||||
ENABLE_SNS_WEBHOOKS=true
|
||||
ENABLE_BOUNCE_HANDLING=true
|
||||
```
|
||||
- Real-time bounce notifications via AWS SNS webhooks
|
||||
- Automatic member deactivation for hard bounces
|
||||
- Bounce history tracking and management in web UI
|
||||
- Requires valid HTTPS domain and SES production access
|
||||
- See `BOUNCE_HANDLING_SETUP.md` for complete setup
|
||||
|
||||
2. **Email-Based Processing** (Works with SES Sandbox):
|
||||
```bash
|
||||
# In .env file
|
||||
ENABLE_EMAIL_BOUNCE_PROCESSING=true
|
||||
ENABLE_BOUNCE_HANDLING=true # Automatically enabled
|
||||
```
|
||||
- Automatic bounce processing via email parsing
|
||||
- Same bounce management features as SNS webhooks
|
||||
- Works with SES sandbox accounts (no production access needed)
|
||||
- Self-contained processing within existing containers
|
||||
- See `EMAIL_BOUNCE_HANDLING_SETUP.md` for complete setup
|
||||
|
||||
3. **Disabled** (Manual Processing Only):
|
||||
```bash
|
||||
# In .env file (or leave these commented out)
|
||||
ENABLE_SNS_WEBHOOKS=false
|
||||
ENABLE_BOUNCE_HANDLING=false
|
||||
ENABLE_EMAIL_BOUNCE_PROCESSING=false
|
||||
```
|
||||
- No automatic bounce processing
|
||||
- Manual bounce management via email notifications
|
||||
- Bounce-related UI elements are hidden
|
||||
|
||||
**When bounce handling is disabled:**
|
||||
- `/webhooks/sns` endpoint is not registered
|
||||
- Bounce history endpoints return empty results
|
||||
- Web UI hides bounce badges and bounce management buttons
|
||||
- No automatic member deactivation occurs
|
||||
|
||||
## Security
|
||||
|
||||
- **Environment Variables**: All credentials stored in `.env` (git-ignored)
|
||||
@@ -317,6 +364,7 @@ docker-compose up -d # Reinitializes from schema.sql
|
||||
- ✅ **REST API** - Complete programmatic access with token auth
|
||||
- ✅ **Sender Whitelist** - Only authorized domains can send to lists
|
||||
- ✅ **SES Integration** - Reliable email delivery through Amazon SES
|
||||
- ✅ **Bounce Handling** - Automatic tracking and management of email bounces via SNS
|
||||
- ✅ **Secure** - Private Docker network, token auth, environment-based credentials
|
||||
- ✅ **Flexible** - Manage via web, API, or direct database access
|
||||
- ✅ **Scalable** - Database-driven architecture supports many lists and members
|
||||
@@ -326,6 +374,7 @@ docker-compose up -d # Reinitializes from schema.sql
|
||||
- **Web Interface**: See `web/README.md` for frontend features and usage
|
||||
- **REST API**: See `api/README.md` for complete API reference
|
||||
- **Database**: See `database/README.md` for schema and SQL examples
|
||||
- **Bounce Handling**: See `BOUNCE_HANDLING_SETUP.md` for SNS configuration
|
||||
- **AI Agents**: See `.github/copilot-instructions.md` for development guidance
|
||||
|
||||
## Roadmap
|
||||
@@ -336,6 +385,7 @@ docker-compose up -d # Reinitializes from schema.sql
|
||||
- [x] Multi-service Docker Compose architecture
|
||||
- [x] REST API with authentication
|
||||
- [x] Sender whitelist for authorized domains
|
||||
- [x] Bounce handling with SES SNS integration
|
||||
- [ ] Email verification workflow for new members
|
||||
- [ ] Subscription confirmation (double opt-in)
|
||||
- [ ] List archive functionality
|
||||
|
||||
398
SES_BOUNCE_TESTING_GUIDE.md
Normal file
398
SES_BOUNCE_TESTING_GUIDE.md
Normal file
@@ -0,0 +1,398 @@
|
||||
# Testing Bounces from SES Sandbox
|
||||
|
||||
AWS SES provides **built-in bounce simulator addresses** that work even in sandbox mode. This guide shows you how to use them to test your bounce handling.
|
||||
|
||||
## Quick Answer
|
||||
|
||||
Send email to these special AWS addresses to simulate different bounce types:
|
||||
|
||||
### Hard Bounce (Permanent - Invalid Address)
|
||||
```bash
|
||||
bounce@simulator.amazonses.com
|
||||
```
|
||||
|
||||
### Soft Bounce (Temporary - Mailbox Full)
|
||||
```bash
|
||||
ooto@simulator.amazonses.com
|
||||
```
|
||||
|
||||
### Complaint (Spam Report)
|
||||
```bash
|
||||
complaint@simulator.amazonses.com
|
||||
```
|
||||
|
||||
### Successful Delivery (No Bounce)
|
||||
```bash
|
||||
success@simulator.amazonses.com
|
||||
```
|
||||
|
||||
## Step-by-Step Testing Guide
|
||||
|
||||
### Option 1: Using Your Mailing Lists (Recommended)
|
||||
|
||||
This tests the complete flow: Postfix → SES → SNS → API
|
||||
|
||||
1. **Add the simulator address as a member:**
|
||||
```bash
|
||||
# Using the API
|
||||
curl -X POST http://localhost:8000/members \
|
||||
-H "Authorization: Bearer $API_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"name": "Bounce Test",
|
||||
"email": "bounce@simulator.amazonses.com",
|
||||
"active": true
|
||||
}'
|
||||
```
|
||||
|
||||
Or use the Web UI:
|
||||
- Go to http://localhost:3000
|
||||
- Members tab → Add Member
|
||||
- Name: `Bounce Test`
|
||||
- Email: `bounce@simulator.amazonses.com`
|
||||
|
||||
2. **Subscribe to a test list:**
|
||||
- Click "Subscriptions" button for the test member
|
||||
- Toggle on one of your mailing lists
|
||||
|
||||
3. **Send email to the list:**
|
||||
```bash
|
||||
# From inside Postfix container
|
||||
sudo docker compose exec postfix bash
|
||||
echo "This will bounce" | mail -s "Test Bounce" community@lists.sasalliance.org
|
||||
exit
|
||||
```
|
||||
|
||||
Replace `community@lists.sasalliance.org` with your actual list email.
|
||||
|
||||
4. **Wait 30-60 seconds** for:
|
||||
- Email to be sent via SES
|
||||
- SES to process the bounce
|
||||
- SNS to send notification to your webhook
|
||||
|
||||
5. **Check the results:**
|
||||
|
||||
**Watch API logs in real-time:**
|
||||
```bash
|
||||
sudo docker compose logs api -f
|
||||
```
|
||||
|
||||
You should see:
|
||||
```
|
||||
============================================================
|
||||
SNS Webhook Request Received
|
||||
============================================================
|
||||
Content-Type: text/plain; charset=UTF-8
|
||||
User-Agent: Amazon Simple Notification Service Agent
|
||||
|
||||
✓ Notification Received
|
||||
Notification Type: Bounce
|
||||
|
||||
✓ Processing Bounce
|
||||
Bounce Type: Permanent
|
||||
Recipients: ['bounce@simulator.amazonses.com']
|
||||
✓ Bounce processed successfully
|
||||
============================================================
|
||||
```
|
||||
|
||||
**Check the database:**
|
||||
```bash
|
||||
# View bounce logs
|
||||
sudo docker compose exec mysql mysql -u maillist -pmaillist maillist \
|
||||
-e "SELECT * FROM bounce_logs ORDER BY created_at DESC LIMIT 5;"
|
||||
|
||||
# Check member status
|
||||
sudo docker compose exec mysql mysql -u maillist -pmaillist maillist \
|
||||
-e "SELECT email, active, bounce_count, bounce_status FROM members WHERE email='bounce@simulator.amazonses.com';"
|
||||
```
|
||||
|
||||
**View in Web UI:**
|
||||
- Open http://localhost:3000
|
||||
- Go to Members tab
|
||||
- Find "Bounce Test" member
|
||||
- Should show: ❌ Inactive (red), bounce badge, last bounce timestamp
|
||||
- Click "Bounces" button to see detailed history
|
||||
|
||||
### Option 2: Direct Email via Postfix (Simpler)
|
||||
|
||||
Send directly to the simulator without going through your list:
|
||||
|
||||
```bash
|
||||
# Enter Postfix container
|
||||
sudo docker compose exec postfix bash
|
||||
|
||||
# Send test email
|
||||
echo "Testing hard bounce" | mail -s "Hard Bounce Test" bounce@simulator.amazonses.com
|
||||
|
||||
# Or soft bounce
|
||||
echo "Testing soft bounce" | mail -s "Soft Bounce Test" ooto@simulator.amazonses.com
|
||||
|
||||
# Exit container
|
||||
exit
|
||||
```
|
||||
|
||||
### Option 3: Using AWS SES Console (For Non-Sandbox Testing)
|
||||
|
||||
If you have SES production access:
|
||||
|
||||
1. Go to AWS SES Console
|
||||
2. Click "Send test email"
|
||||
3. To: `bounce@simulator.amazonses.com`
|
||||
4. From: Your verified email/domain
|
||||
5. Subject: "Bounce test"
|
||||
6. Body: "Testing bounce handling"
|
||||
7. Click "Send test email"
|
||||
|
||||
## Testing Different Bounce Types
|
||||
|
||||
### 1. Hard Bounce (Permanent Failure)
|
||||
```bash
|
||||
echo "Test" | mail -s "Test" bounce@simulator.amazonses.com
|
||||
```
|
||||
|
||||
**Expected Result:**
|
||||
- Member marked as `hard_bounce`
|
||||
- Member deactivated (`active = 0`)
|
||||
- `bounce_count` incremented
|
||||
- Entry in `bounce_logs` table
|
||||
|
||||
### 2. Soft Bounce (Transient Failure)
|
||||
```bash
|
||||
echo "Test" | mail -s "Test" ooto@simulator.amazonses.com
|
||||
```
|
||||
|
||||
**Expected Result:**
|
||||
- Member marked as `clean` (first time)
|
||||
- After 3 soft bounces → `soft_bounce` status
|
||||
- `bounce_count` incremented
|
||||
- Member stays active
|
||||
|
||||
### 3. Complaint (Spam Report)
|
||||
```bash
|
||||
echo "Test" | mail -s "Test" complaint@simulator.amazonses.com
|
||||
```
|
||||
|
||||
**Expected Result:**
|
||||
- API receives complaint notification
|
||||
- Currently logged but not processed (you can extend the handler)
|
||||
|
||||
## Monitoring the Test
|
||||
|
||||
### Real-Time Monitoring (Recommended)
|
||||
|
||||
Open 3 terminal windows:
|
||||
|
||||
**Terminal 1 - API Logs:**
|
||||
```bash
|
||||
sudo docker compose logs api -f
|
||||
```
|
||||
|
||||
**Terminal 2 - Postfix Logs:**
|
||||
```bash
|
||||
sudo docker compose logs postfix -f
|
||||
```
|
||||
|
||||
**Terminal 3 - Send Test Email:**
|
||||
```bash
|
||||
sudo docker compose exec postfix bash
|
||||
echo "Test" | mail -s "Bounce Test" bounce@simulator.amazonses.com
|
||||
```
|
||||
|
||||
### Timeline
|
||||
|
||||
Here's what happens and when:
|
||||
|
||||
- **T+0s**: Email sent to Postfix
|
||||
- **T+1-3s**: Postfix relays to SES
|
||||
- **T+5-10s**: SES processes and generates bounce
|
||||
- **T+10-30s**: SNS sends notification to your webhook
|
||||
- **T+30-60s**: API processes bounce and updates database
|
||||
|
||||
## Verifying the Complete Flow
|
||||
|
||||
### 1. Check Postfix Logs
|
||||
```bash
|
||||
sudo docker compose logs postfix | grep bounce@simulator
|
||||
```
|
||||
|
||||
Should show:
|
||||
```
|
||||
postfix/smtp[xxx]: ... to=<bounce@simulator.amazonses.com>, relay=email-smtp.eu-west-2.amazonaws.com[...], ... status=sent
|
||||
```
|
||||
|
||||
### 2. Check SNS Subscription Status
|
||||
- Go to AWS SNS Console
|
||||
- Find your topic
|
||||
- Check "Subscriptions" tab
|
||||
- Status should be "Confirmed"
|
||||
- Messages delivered should be > 0
|
||||
|
||||
### 3. Check API Logs
|
||||
```bash
|
||||
sudo docker compose logs api | grep -A 20 "SNS Webhook"
|
||||
```
|
||||
|
||||
Should show successful processing.
|
||||
|
||||
### 4. Check Database
|
||||
```bash
|
||||
sudo docker compose exec mysql mysql -u maillist -pmaillist maillist <<EOF
|
||||
-- Show recent bounces
|
||||
SELECT
|
||||
b.bounce_id,
|
||||
m.email,
|
||||
b.bounce_type,
|
||||
b.diagnostic_code,
|
||||
b.timestamp
|
||||
FROM bounce_logs b
|
||||
JOIN members m ON b.member_id = m.member_id
|
||||
ORDER BY b.timestamp DESC
|
||||
LIMIT 5;
|
||||
|
||||
-- Show members with bounces
|
||||
SELECT
|
||||
email,
|
||||
active,
|
||||
bounce_count,
|
||||
bounce_status,
|
||||
last_bounce_at
|
||||
FROM members
|
||||
WHERE bounce_count > 0;
|
||||
EOF
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "Email not sent" or "Relay access denied"
|
||||
|
||||
**Problem**: Postfix not configured to relay via SES
|
||||
|
||||
**Check**:
|
||||
```bash
|
||||
sudo docker compose exec postfix postconf relayhost
|
||||
sudo docker compose exec postfix postconf smtp_sasl_auth_enable
|
||||
```
|
||||
|
||||
Should show:
|
||||
```
|
||||
relayhost = [email-smtp.eu-west-2.amazonaws.com]:587
|
||||
smtp_sasl_auth_enable = yes
|
||||
```
|
||||
|
||||
### "No bounce received after 5 minutes"
|
||||
|
||||
**Possible causes**:
|
||||
|
||||
1. **SNS subscription not confirmed**
|
||||
- Check AWS SNS console
|
||||
- Status should be "Confirmed", not "Pending"
|
||||
|
||||
2. **SNS topic not configured in SES**
|
||||
- Check SES → Configuration Sets or Verified Identities → Notifications
|
||||
- Bounce notifications should point to your SNS topic
|
||||
|
||||
3. **Webhook endpoint not accessible**
|
||||
- SNS requires HTTPS
|
||||
- Test: `curl https://your-domain.com:8000/health`
|
||||
|
||||
4. **API container not running**
|
||||
```bash
|
||||
sudo docker compose ps api
|
||||
```
|
||||
|
||||
### "Bounce received but not in database"
|
||||
|
||||
**Check API logs for errors**:
|
||||
```bash
|
||||
sudo docker compose logs api | grep -i error
|
||||
```
|
||||
|
||||
**Check database tables exist**:
|
||||
```bash
|
||||
sudo docker compose exec mysql mysql -u maillist -pmaillist maillist -e "SHOW TABLES;"
|
||||
```
|
||||
|
||||
Should include: `bounce_logs`, `members`
|
||||
|
||||
## Testing Multiple Bounces
|
||||
|
||||
To test the "3 soft bounces = soft_bounce status" logic:
|
||||
|
||||
```bash
|
||||
sudo docker compose exec postfix bash
|
||||
|
||||
# Send 3 emails to soft bounce simulator
|
||||
for i in {1..3}; do
|
||||
echo "Soft bounce test $i" | mail -s "Test $i" ooto@simulator.amazonses.com
|
||||
sleep 70 # Wait between sends for SNS processing
|
||||
done
|
||||
```
|
||||
|
||||
After the 3rd bounce:
|
||||
- Member's `bounce_status` should change from `clean` to `soft_bounce`
|
||||
- `bounce_count` should be 3
|
||||
|
||||
## Cleanup After Testing
|
||||
|
||||
Remove test bounce data:
|
||||
|
||||
```bash
|
||||
sudo docker compose exec mysql mysql -u maillist -pmaillist maillist <<EOF
|
||||
-- Delete test bounce logs
|
||||
DELETE FROM bounce_logs WHERE email IN ('bounce@simulator.amazonses.com', 'ooto@simulator.amazonses.com');
|
||||
|
||||
-- Reset test member
|
||||
UPDATE members
|
||||
SET bounce_count = 0,
|
||||
bounce_status = 'clean',
|
||||
last_bounce_at = NULL,
|
||||
active = 1
|
||||
WHERE email IN ('bounce@simulator.amazonses.com', 'ooto@simulator.amazonses.com');
|
||||
EOF
|
||||
```
|
||||
|
||||
Or use the Web UI:
|
||||
- Go to Members tab
|
||||
- Find the test member
|
||||
- Click "Bounces" button
|
||||
- Click "Reset Bounce Status"
|
||||
|
||||
## Alternative: Simulate Bounces Without SES
|
||||
|
||||
If SNS isn't set up yet, use the included script to simulate bounces directly in the database:
|
||||
|
||||
```bash
|
||||
./simulate_bounce.sh
|
||||
```
|
||||
|
||||
This is useful for:
|
||||
- Testing the UI without AWS
|
||||
- Development environments
|
||||
- Demonstrating bounce handling to stakeholders
|
||||
|
||||
## Next Steps
|
||||
|
||||
Once bounce handling is working:
|
||||
|
||||
1. **Remove simulator addresses** from your member list
|
||||
2. **Monitor real bounces** in production
|
||||
3. **Set up alerts** for high bounce rates
|
||||
4. **Review bounced members** regularly and update/remove invalid addresses
|
||||
5. **Consider complaint handling** (similar to bounces, for spam reports)
|
||||
|
||||
## Summary Commands
|
||||
|
||||
**Quick test sequence**:
|
||||
```bash
|
||||
# 1. Watch logs
|
||||
sudo docker compose logs api -f &
|
||||
|
||||
# 2. Send test bounce
|
||||
echo "Test" | sudo docker compose exec -T postfix mail -s "Bounce Test" bounce@simulator.amazonses.com
|
||||
|
||||
# 3. Wait 60 seconds, then check database
|
||||
sleep 60
|
||||
sudo docker compose exec mysql mysql -u maillist -pmaillist maillist -e "SELECT * FROM bounce_logs ORDER BY created_at DESC LIMIT 1;"
|
||||
```
|
||||
|
||||
That's it! The bounce simulator is the easiest way to test your bounce handling without needing real bounced emails.
|
||||
132
SNS_DEBUG_GUIDE.md
Normal file
132
SNS_DEBUG_GUIDE.md
Normal file
@@ -0,0 +1,132 @@
|
||||
# SNS Webhook Debugging Guide
|
||||
|
||||
## Current Status
|
||||
|
||||
✅ **Detailed logging is now active!** The API will log all incoming SNS requests with:
|
||||
- Full headers (including SNS-specific headers)
|
||||
- Request body content
|
||||
- Parsed message structure
|
||||
- Processing steps and results
|
||||
|
||||
## How to View Logs in Real-Time
|
||||
|
||||
```bash
|
||||
# Watch API logs as SNS sends notifications
|
||||
sudo docker compose logs api -f
|
||||
```
|
||||
|
||||
## Testing with Real AWS SNS
|
||||
|
||||
### Step 1: Trigger a Test from SNS Console
|
||||
|
||||
1. Go to AWS SNS Console → Your Topic
|
||||
2. Click "Publish message"
|
||||
3. Send a test notification
|
||||
|
||||
### Step 2: Check the Logs
|
||||
|
||||
The logs will show you exactly what SNS sent:
|
||||
|
||||
```
|
||||
============================================================
|
||||
SNS Webhook Request Received
|
||||
============================================================
|
||||
Content-Type: text/plain; charset=UTF-8
|
||||
User-Agent: Amazon Simple Notification Service Agent
|
||||
X-Amz-SNS-Message-Type: Notification
|
||||
X-Amz-SNS-Topic-Arn: arn:aws:sns:...
|
||||
|
||||
Message Type: Notification
|
||||
Message Keys: ['Type', 'MessageId', 'TopicArn', 'Message', ...]
|
||||
|
||||
✓ Notification Received
|
||||
Message (first 200 chars): {"notificationType":"Bounce",...
|
||||
Notification Type: Bounce
|
||||
|
||||
✓ Processing Bounce
|
||||
Bounce Type: Permanent
|
||||
Recipients: ['bounce@example.com']
|
||||
✓ Bounce processed successfully
|
||||
============================================================
|
||||
```
|
||||
|
||||
## What the Logs Tell You
|
||||
|
||||
### If you see this:
|
||||
- **"SNS webhook received body: b''"** → SNS sent empty body (check SNS configuration)
|
||||
- **"Missing SigningCertURL"** → Message missing required SNS fields
|
||||
- **"Invalid certificate URL"** → Certificate URL not from amazonaws.com
|
||||
- **"Invalid signature"** → Signature verification failed (message tampered or wrong cert)
|
||||
|
||||
### Successful Flow:
|
||||
1. Headers logged → Shows SNS user agent and message type
|
||||
2. Body logged → Shows the raw JSON
|
||||
3. Message parsed → Shows the Type field
|
||||
4. Notification processed → Shows bounce/complaint details
|
||||
5. Database updated → Confirmation of processing
|
||||
|
||||
## Understanding Previous Errors
|
||||
|
||||
The error you saw earlier:
|
||||
```
|
||||
SNS webhook error: Expecting value: line 1 column 1 (char 0)
|
||||
```
|
||||
|
||||
This means SNS was sending an **empty body** or the body was already consumed. Possible causes:
|
||||
1. SNS subscription confirmation URL was opened in a browser (GET request, not POST)
|
||||
2. Network issue causing body to be lost
|
||||
3. SNS configuration issue
|
||||
|
||||
## Testing Bounce Handling
|
||||
|
||||
### Option 1: AWS SES Bounce Simulator
|
||||
Send email to: `bounce@simulator.amazonses.com`
|
||||
|
||||
### Option 2: Verify Database Updates
|
||||
After a bounce is processed:
|
||||
|
||||
```bash
|
||||
# Check bounce logs
|
||||
sudo docker compose exec mysql mysql -u maillist -pmaillist maillist -e "SELECT * FROM bounce_logs ORDER BY created_at DESC LIMIT 5;"
|
||||
|
||||
# Check member bounce status
|
||||
sudo docker compose exec mysql mysql -u maillist -pmaillist maillist -e "SELECT member_id, email, bounce_count, bounce_status, last_bounce_at FROM members WHERE bounce_count > 0;"
|
||||
```
|
||||
|
||||
## Common SNS Message Types
|
||||
|
||||
### 1. SubscriptionConfirmation
|
||||
First message when you create the subscription. API auto-confirms by calling SubscribeURL.
|
||||
|
||||
### 2. Notification (Bounce)
|
||||
```json
|
||||
{
|
||||
"Type": "Notification",
|
||||
"Message": "{\"notificationType\":\"Bounce\",\"bounce\":{...}}"
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Notification (Complaint)
|
||||
```json
|
||||
{
|
||||
"Type": "Notification",
|
||||
"Message": "{\"notificationType\":\"Complaint\",\"complaint\":{...}}"
|
||||
}
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. **Monitor logs while SNS sends**: `sudo docker compose logs api -f`
|
||||
2. **Trigger a real bounce**: Send to `bounce@simulator.amazonses.com`
|
||||
3. **Check the detailed logs** to see exactly what SNS is sending
|
||||
4. **Verify database updates** to confirm bounces are being recorded
|
||||
|
||||
## If You Still See Errors
|
||||
|
||||
The enhanced logging will now show you:
|
||||
- What headers SNS is sending
|
||||
- What body content is arriving (or if it's empty)
|
||||
- Where in the processing pipeline the error occurs
|
||||
- Full stack traces for any exceptions
|
||||
|
||||
Copy the relevant log section and we can diagnose the exact issue!
|
||||
350
api/main.py
350
api/main.py
@@ -5,6 +5,7 @@ FastAPI-based REST API for managing mailing lists and members
|
||||
from fastapi import FastAPI, HTTPException, Depends, Header, status, Request
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
|
||||
from fastapi.responses import PlainTextResponse
|
||||
from pydantic import BaseModel, EmailStr
|
||||
from typing import List, Optional, Annotated
|
||||
import mysql.connector
|
||||
@@ -19,6 +20,15 @@ import bcrypt
|
||||
from jose import JWTError, jwt
|
||||
from passlib.context import CryptContext
|
||||
from enum import Enum
|
||||
import json
|
||||
import base64
|
||||
from urllib.parse import urlparse
|
||||
import httpx
|
||||
from cryptography import x509
|
||||
from cryptography.hazmat.backends import default_backend
|
||||
from cryptography.hazmat.primitives import hashes
|
||||
from cryptography.hazmat.primitives.asymmetric import padding
|
||||
from cryptography.hazmat.primitives.serialization import load_pem_public_key
|
||||
|
||||
# Configuration
|
||||
API_TOKEN = os.getenv('API_TOKEN', 'change-this-token') # Keep for backward compatibility during transition
|
||||
@@ -33,6 +43,14 @@ MYSQL_DATABASE = os.getenv('MYSQL_DATABASE', 'maillist')
|
||||
MYSQL_USER = os.getenv('MYSQL_USER', 'maillist')
|
||||
MYSQL_PASSWORD = os.getenv('MYSQL_PASSWORD', '')
|
||||
|
||||
# Bounce handling configuration
|
||||
ENABLE_SNS_WEBHOOKS = os.getenv('ENABLE_SNS_WEBHOOKS', 'false').lower() == 'true'
|
||||
ENABLE_BOUNCE_HANDLING = os.getenv('ENABLE_BOUNCE_HANDLING', 'false').lower() == 'true'
|
||||
ENABLE_EMAIL_BOUNCE_PROCESSING = os.getenv('ENABLE_EMAIL_BOUNCE_PROCESSING', 'false').lower() == 'true'
|
||||
|
||||
# Enable bounce handling if either method is enabled
|
||||
ENABLE_BOUNCE_HANDLING = ENABLE_BOUNCE_HANDLING or ENABLE_SNS_WEBHOOKS or ENABLE_EMAIL_BOUNCE_PROCESSING
|
||||
|
||||
# Password hashing
|
||||
pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto")
|
||||
|
||||
@@ -44,12 +62,15 @@ app = FastAPI(
|
||||
)
|
||||
|
||||
# Add CORS middleware
|
||||
# Get allowed origins from environment or use secure defaults
|
||||
ALLOWED_ORIGINS = os.getenv('ALLOWED_ORIGINS', 'http://localhost:3000,http://127.0.0.1:3000').split(',')
|
||||
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=["*"], # In production, specify your frontend domain
|
||||
allow_origins=ALLOWED_ORIGINS, # Specific origins only
|
||||
allow_credentials=True,
|
||||
allow_methods=["*"],
|
||||
allow_headers=["*"],
|
||||
allow_methods=["GET", "POST", "PUT", "PATCH", "DELETE", "OPTIONS"], # Specific methods
|
||||
allow_headers=["Authorization", "Content-Type", "Accept", "Origin", "X-Requested-With"], # Specific headers
|
||||
)
|
||||
|
||||
security = HTTPBearer()
|
||||
@@ -229,6 +250,9 @@ class Member(BaseModel):
|
||||
name: str
|
||||
email: EmailStr
|
||||
active: bool = True
|
||||
bounce_count: Optional[int] = 0
|
||||
last_bounce_at: Optional[datetime] = None
|
||||
bounce_status: Optional[str] = 'clean'
|
||||
|
||||
class MemberUpdate(BaseModel):
|
||||
name: Optional[str] = None
|
||||
@@ -447,6 +471,15 @@ async def health():
|
||||
except Exception as e:
|
||||
raise HTTPException(status_code=503, detail=f"Unhealthy: {str(e)}")
|
||||
|
||||
@app.get("/config")
|
||||
async def get_config():
|
||||
"""Get public configuration settings"""
|
||||
return {
|
||||
"bounce_handling_enabled": ENABLE_BOUNCE_HANDLING,
|
||||
"sns_webhooks_enabled": ENABLE_SNS_WEBHOOKS,
|
||||
"email_bounce_processing_enabled": ENABLE_EMAIL_BOUNCE_PROCESSING
|
||||
}
|
||||
|
||||
# Mailing Lists endpoints
|
||||
@app.get("/lists", response_model=List[MailingList])
|
||||
async def get_lists(current_user: CurrentUser = require_read_access()):
|
||||
@@ -808,6 +841,317 @@ async def bulk_import_members(bulk_request: BulkImportRequest, current_user: Cur
|
||||
cursor.close()
|
||||
raise HTTPException(status_code=500, detail=f"Bulk import failed: {str(e)}")
|
||||
|
||||
# SNS Webhook for Bounce Handling (conditionally enabled)
|
||||
if ENABLE_SNS_WEBHOOKS:
|
||||
async def verify_sns_signature(request: Request) -> dict:
|
||||
"""Verify SNS message signature"""
|
||||
try:
|
||||
body = await request.body()
|
||||
print(f"SNS webhook received body: {body}")
|
||||
print(f"SNS webhook body length: {len(body)}")
|
||||
print(f"SNS webhook headers: {dict(request.headers)}")
|
||||
|
||||
if not body:
|
||||
print("ERROR: Empty body received")
|
||||
raise HTTPException(status_code=400, detail="Empty request body")
|
||||
|
||||
message = json.loads(body.decode('utf-8'))
|
||||
print(f"SNS webhook parsed message type: {message.get('Type')}")
|
||||
|
||||
# For SubscriptionConfirmation and UnsubscribeConfirmation, we don't validate signature
|
||||
# AWS will send a URL to confirm
|
||||
if message.get('Type') in ['SubscriptionConfirmation', 'UnsubscribeConfirmation']:
|
||||
return message
|
||||
|
||||
# Get certificate URL and download certificate
|
||||
cert_url = message.get('SigningCertURL')
|
||||
if not cert_url:
|
||||
raise HTTPException(status_code=400, detail="Missing SigningCertURL")
|
||||
|
||||
# Verify certificate URL is from AWS
|
||||
parsed_url = urlparse(cert_url)
|
||||
if not parsed_url.hostname.endswith('.amazonaws.com'):
|
||||
raise HTTPException(status_code=400, detail="Invalid certificate URL")
|
||||
|
||||
# Download certificate
|
||||
async with httpx.AsyncClient() as client:
|
||||
cert_response = await client.get(cert_url)
|
||||
cert_response.raise_for_status()
|
||||
cert_pem = cert_response.content
|
||||
|
||||
# Load certificate and extract public key
|
||||
cert = x509.load_pem_x509_certificate(cert_pem, default_backend())
|
||||
public_key = cert.public_key()
|
||||
|
||||
# Build string to sign
|
||||
if message.get('Type') == 'Notification':
|
||||
string_to_sign = (
|
||||
f"Message\n{message['Message']}\n"
|
||||
f"MessageId\n{message['MessageId']}\n"
|
||||
)
|
||||
if 'Subject' in message:
|
||||
string_to_sign += f"Subject\n{message['Subject']}\n"
|
||||
string_to_sign += (
|
||||
f"Timestamp\n{message['Timestamp']}\n"
|
||||
f"TopicArn\n{message['TopicArn']}\n"
|
||||
f"Type\n{message['Type']}\n"
|
||||
)
|
||||
else:
|
||||
string_to_sign = (
|
||||
f"Message\n{message['Message']}\n"
|
||||
f"MessageId\n{message['MessageId']}\n"
|
||||
f"SubscribeURL\n{message['SubscribeURL']}\n"
|
||||
f"Timestamp\n{message['Timestamp']}\n"
|
||||
f"Token\n{message['Token']}\n"
|
||||
f"TopicArn\n{message['TopicArn']}\n"
|
||||
f"Type\n{message['Type']}\n"
|
||||
)
|
||||
|
||||
# Verify signature
|
||||
signature = base64.b64decode(message['Signature'])
|
||||
try:
|
||||
public_key.verify(
|
||||
signature,
|
||||
string_to_sign.encode('utf-8'),
|
||||
padding.PKCS1v15(),
|
||||
hashes.SHA1()
|
||||
)
|
||||
except Exception as e:
|
||||
raise HTTPException(status_code=400, detail=f"Invalid signature: {str(e)}")
|
||||
|
||||
return message
|
||||
|
||||
except json.JSONDecodeError:
|
||||
raise HTTPException(status_code=400, detail="Invalid JSON")
|
||||
except Exception as e:
|
||||
raise HTTPException(status_code=400, detail=f"Signature verification failed: {str(e)}")
|
||||
|
||||
async def process_bounce(bounce_data: dict):
|
||||
"""Process bounce notification and update database"""
|
||||
try:
|
||||
bounce_type = bounce_data.get('bounceType') # Permanent, Transient, Undetermined
|
||||
bounce_subtype = bounce_data.get('bounceSubType', '')
|
||||
timestamp_str = bounce_data.get('timestamp')
|
||||
feedback_id = bounce_data.get('feedbackId', '')
|
||||
|
||||
# Convert ISO 8601 timestamp to MySQL datetime format
|
||||
# SES sends: '2025-10-13T16:22:40.359Z'
|
||||
# MySQL needs: '2025-10-13 16:22:40'
|
||||
from datetime import datetime as dt
|
||||
timestamp = dt.fromisoformat(timestamp_str.replace('Z', '+00:00')).strftime('%Y-%m-%d %H:%M:%S')
|
||||
|
||||
bounced_recipients = bounce_data.get('bouncedRecipients', [])
|
||||
|
||||
with get_db() as conn:
|
||||
cursor = conn.cursor(dictionary=True)
|
||||
|
||||
for recipient in bounced_recipients:
|
||||
email = recipient.get('emailAddress')
|
||||
diagnostic_code = recipient.get('diagnosticCode', '')
|
||||
|
||||
if not email:
|
||||
continue
|
||||
|
||||
# Find member by email
|
||||
cursor.execute("SELECT member_id FROM members WHERE email = %s", (email,))
|
||||
member = cursor.fetchone()
|
||||
member_id = member['member_id'] if member else None
|
||||
|
||||
# Log the bounce
|
||||
cursor.execute("""
|
||||
INSERT INTO bounce_logs
|
||||
(member_id, email, bounce_type, bounce_subtype, diagnostic_code, timestamp, feedback_id)
|
||||
VALUES (%s, %s, %s, %s, %s, %s, %s)
|
||||
""", (member_id, email, bounce_type, bounce_subtype, diagnostic_code, timestamp, feedback_id))
|
||||
|
||||
# Update member bounce status
|
||||
if member_id:
|
||||
# Determine bounce status
|
||||
if bounce_type == 'Permanent':
|
||||
new_status = 'hard_bounce'
|
||||
# Deactivate member with hard bounce
|
||||
cursor.execute("""
|
||||
UPDATE members
|
||||
SET bounce_count = bounce_count + 1,
|
||||
last_bounce_at = %s,
|
||||
bounce_status = %s,
|
||||
active = 0
|
||||
WHERE member_id = %s
|
||||
""", (timestamp, new_status, member_id))
|
||||
elif bounce_type == 'Transient':
|
||||
# Check current bounce count
|
||||
cursor.execute("SELECT bounce_count, bounce_status FROM members WHERE member_id = %s", (member_id,))
|
||||
current = cursor.fetchone()
|
||||
|
||||
# If already hard bounced, don't change status
|
||||
if current and current['bounce_status'] != 'hard_bounce':
|
||||
new_count = current['bounce_count'] + 1
|
||||
# After 3 soft bounces, mark as soft_bounce status
|
||||
new_status = 'soft_bounce' if new_count >= 3 else 'clean'
|
||||
|
||||
cursor.execute("""
|
||||
UPDATE members
|
||||
SET bounce_count = %s,
|
||||
last_bounce_at = %s,
|
||||
bounce_status = %s
|
||||
WHERE member_id = %s
|
||||
""", (new_count, timestamp, new_status, member_id))
|
||||
else: # Undetermined
|
||||
cursor.execute("""
|
||||
UPDATE members
|
||||
SET bounce_count = bounce_count + 1,
|
||||
last_bounce_at = %s
|
||||
WHERE member_id = %s
|
||||
""", (timestamp, member_id))
|
||||
|
||||
conn.commit()
|
||||
cursor.close()
|
||||
|
||||
except Exception as e:
|
||||
print(f"✗ Error processing bounce: {str(e)}")
|
||||
print(f"Error type: {type(e).__name__}")
|
||||
print(f"Bounce data: {bounce_data}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
raise
|
||||
|
||||
@app.post("/webhooks/sns", response_class=PlainTextResponse)
|
||||
async def sns_webhook(request: Request):
|
||||
"""Handle SNS notifications for bounces and complaints"""
|
||||
try:
|
||||
print(f"=== SNS Webhook Request ===")
|
||||
print(f"Headers: {dict(request.headers)}")
|
||||
print(f"Content-Type: {request.headers.get('content-type')}")
|
||||
print(f"User-Agent: {request.headers.get('user-agent')}")
|
||||
|
||||
# Verify SNS signature
|
||||
message = await verify_sns_signature(request)
|
||||
|
||||
print(f"Message Type: {message.get('Type')}")
|
||||
print(f"Message Keys: {list(message.keys())}")
|
||||
|
||||
message_type = message.get('Type')
|
||||
|
||||
# Handle subscription confirmation
|
||||
if message_type == 'SubscriptionConfirmation':
|
||||
subscribe_url = message.get('SubscribeURL')
|
||||
print(f"Subscription confirmation received, URL: {subscribe_url}")
|
||||
if subscribe_url:
|
||||
# Confirm subscription
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.get(subscribe_url)
|
||||
print(f"Subscription confirmation response: {response.status_code}")
|
||||
return "Subscription confirmed"
|
||||
|
||||
# Handle notification
|
||||
elif message_type == 'Notification':
|
||||
# Parse the message
|
||||
inner_message = message.get('Message', '{}')
|
||||
print(f"Inner message (first 500 chars): {inner_message[:500]}")
|
||||
notification = json.loads(inner_message)
|
||||
|
||||
# SES can send either 'notificationType' or 'eventType' depending on configuration
|
||||
notification_type = notification.get('notificationType') or notification.get('eventType')
|
||||
print(f"Notification type: {notification_type}")
|
||||
|
||||
if notification_type == 'Bounce':
|
||||
bounce = notification.get('bounce', {})
|
||||
print(f"\n✓ Processing Bounce")
|
||||
print(f" Bounce Type: {bounce.get('bounceType')}")
|
||||
print(f" Recipients: {[r.get('emailAddress') for r in bounce.get('bouncedRecipients', [])]}")
|
||||
await process_bounce(bounce)
|
||||
print(f" ✓ Bounce processed successfully")
|
||||
return "Bounce processed"
|
||||
|
||||
elif notification_type == 'Complaint':
|
||||
# We could also track complaints similarly to bounces
|
||||
print(f"\n✓ Complaint received")
|
||||
return "Complaint received"
|
||||
|
||||
print(f"=== End SNS Webhook Request ===")
|
||||
return "OK"
|
||||
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as e:
|
||||
print(f"SNS webhook error: {str(e)}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
raise HTTPException(status_code=500, detail=str(e))
|
||||
|
||||
else:
|
||||
# Provide stub functions when SNS webhooks are disabled
|
||||
print("SNS webhooks disabled - bounce handling via email only")
|
||||
|
||||
# Bounce management endpoints (conditionally enabled)
|
||||
if ENABLE_BOUNCE_HANDLING:
|
||||
class BounceLog(BaseModel):
|
||||
bounce_id: int
|
||||
email: str
|
||||
bounce_type: str
|
||||
bounce_subtype: Optional[str] = None
|
||||
diagnostic_code: Optional[str] = None
|
||||
timestamp: datetime
|
||||
feedback_id: Optional[str] = None
|
||||
created_at: datetime
|
||||
|
||||
class MemberWithBounces(BaseModel):
|
||||
member_id: int
|
||||
name: str
|
||||
email: str
|
||||
active: bool
|
||||
bounce_count: int
|
||||
last_bounce_at: Optional[datetime] = None
|
||||
bounce_status: str
|
||||
|
||||
@app.get("/members/{member_id}/bounces", response_model=List[BounceLog])
|
||||
async def get_member_bounces(member_id: int, current_user: CurrentUser = require_read_access()):
|
||||
"""Get bounce history for a member"""
|
||||
with get_db() as conn:
|
||||
cursor = conn.cursor(dictionary=True)
|
||||
cursor.execute("""
|
||||
SELECT bounce_id, email, bounce_type, bounce_subtype, diagnostic_code,
|
||||
timestamp, feedback_id, created_at
|
||||
FROM bounce_logs
|
||||
WHERE member_id = %s
|
||||
ORDER BY timestamp DESC
|
||||
""", (member_id,))
|
||||
bounces = cursor.fetchall()
|
||||
cursor.close()
|
||||
return bounces
|
||||
|
||||
@app.patch("/members/{member_id}/bounce-status")
|
||||
async def reset_bounce_status(member_id: int, current_user: CurrentUser = require_write_access()):
|
||||
"""Reset bounce status for a member (e.g., after email address is corrected)"""
|
||||
with get_db() as conn:
|
||||
cursor = conn.cursor()
|
||||
cursor.execute("""
|
||||
UPDATE members
|
||||
SET bounce_count = 0,
|
||||
last_bounce_at = NULL,
|
||||
bounce_status = 'clean'
|
||||
WHERE member_id = %s
|
||||
""", (member_id,))
|
||||
conn.commit()
|
||||
|
||||
if cursor.rowcount == 0:
|
||||
raise HTTPException(status_code=404, detail="Member not found")
|
||||
|
||||
cursor.close()
|
||||
return {"message": "Bounce status reset successfully"}
|
||||
|
||||
else:
|
||||
# When bounce handling is disabled, provide stub endpoints that return appropriate responses
|
||||
@app.get("/members/{member_id}/bounces")
|
||||
async def get_member_bounces_disabled(member_id: int, current_user: CurrentUser = require_read_access()):
|
||||
"""Bounce history disabled - returns empty list"""
|
||||
return []
|
||||
|
||||
@app.patch("/members/{member_id}/bounce-status")
|
||||
async def reset_bounce_status_disabled(member_id: int, current_user: CurrentUser = require_write_access()):
|
||||
"""Bounce status reset disabled"""
|
||||
raise HTTPException(status_code=501, detail="Bounce handling is disabled")
|
||||
|
||||
if __name__ == "__main__":
|
||||
import uvicorn
|
||||
uvicorn.run(app, host="0.0.0.0", port=8000)
|
||||
|
||||
@@ -8,3 +8,4 @@ email-validator==2.1.0
|
||||
bcrypt==4.0.1
|
||||
python-jose[cryptography]==3.3.0
|
||||
passlib[bcrypt]==1.7.4
|
||||
httpx==0.25.2
|
||||
|
||||
@@ -55,8 +55,12 @@ CREATE TABLE IF NOT EXISTS members (
|
||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
|
||||
active BOOLEAN DEFAULT TRUE,
|
||||
bounce_count INT DEFAULT 0,
|
||||
last_bounce_at TIMESTAMP NULL,
|
||||
bounce_status ENUM('clean', 'soft_bounce', 'hard_bounce') DEFAULT 'clean',
|
||||
INDEX idx_email (email),
|
||||
INDEX idx_active (active)
|
||||
INDEX idx_active (active),
|
||||
INDEX idx_bounce_status (bounce_status)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
|
||||
|
||||
-- Table: list_members
|
||||
@@ -75,6 +79,26 @@ CREATE TABLE IF NOT EXISTS list_members (
|
||||
INDEX idx_active (active)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
|
||||
|
||||
-- Table: bounce_logs
|
||||
-- Stores bounce notifications from SES SNS
|
||||
CREATE TABLE IF NOT EXISTS bounce_logs (
|
||||
bounce_id INT AUTO_INCREMENT PRIMARY KEY,
|
||||
member_id INT,
|
||||
email VARCHAR(255) NOT NULL,
|
||||
bounce_type ENUM('Permanent', 'Transient', 'Undetermined') NOT NULL,
|
||||
bounce_subtype VARCHAR(50),
|
||||
diagnostic_code TEXT,
|
||||
timestamp TIMESTAMP NOT NULL,
|
||||
sns_message_id VARCHAR(255),
|
||||
feedback_id VARCHAR(255),
|
||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
FOREIGN KEY (member_id) REFERENCES members(member_id) ON DELETE SET NULL,
|
||||
INDEX idx_member_id (member_id),
|
||||
INDEX idx_email (email),
|
||||
INDEX idx_timestamp (timestamp),
|
||||
INDEX idx_bounce_type (bounce_type)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
|
||||
|
||||
-- Insert sample data
|
||||
|
||||
-- Create default admin user (password: 'password')
|
||||
|
||||
59
nginx-production-example.conf
Normal file
59
nginx-production-example.conf
Normal file
@@ -0,0 +1,59 @@
|
||||
# Example nginx configuration for production with SNS webhook support
|
||||
server {
|
||||
listen 80;
|
||||
server_name yourdomain.com;
|
||||
|
||||
# Redirect HTTP to HTTPS in production
|
||||
return 301 https://$server_name$request_uri;
|
||||
}
|
||||
|
||||
server {
|
||||
listen 443 ssl http2;
|
||||
server_name yourdomain.com;
|
||||
|
||||
# SSL configuration (add your certificates)
|
||||
# ssl_certificate /path/to/your/cert.pem;
|
||||
# ssl_certificate_key /path/to/your/key.pem;
|
||||
|
||||
# Frontend (static files)
|
||||
location / {
|
||||
proxy_pass http://maillist-web:80;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
}
|
||||
|
||||
# API endpoints (including SNS webhook)
|
||||
location /api/ {
|
||||
# Remove /api prefix when forwarding to backend
|
||||
rewrite ^/api/(.*) /$1 break;
|
||||
|
||||
proxy_pass http://maillist-api:8000;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
|
||||
# Important for SNS webhooks - increase timeouts and body size
|
||||
proxy_connect_timeout 60s;
|
||||
proxy_send_timeout 60s;
|
||||
proxy_read_timeout 60s;
|
||||
client_max_body_size 10M;
|
||||
}
|
||||
|
||||
# Direct SNS webhook endpoint (alternative path)
|
||||
location /webhooks/sns {
|
||||
proxy_pass http://maillist-api:8000/webhooks/sns;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
|
||||
# SNS specific settings
|
||||
proxy_connect_timeout 30s;
|
||||
proxy_send_timeout 30s;
|
||||
proxy_read_timeout 30s;
|
||||
client_max_body_size 1M;
|
||||
}
|
||||
}
|
||||
@@ -9,6 +9,8 @@ RUN apt-get update && \
|
||||
mailutils \
|
||||
gettext-base \
|
||||
netcat-openbsd \
|
||||
python3 \
|
||||
python3-pymysql \
|
||||
&& apt-get clean && rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Copy configs
|
||||
@@ -16,11 +18,16 @@ COPY main.cf.template /etc/postfix/main.cf.template
|
||||
COPY sasl_passwd.template /etc/postfix/sasl_passwd.template
|
||||
COPY mysql_virtual_alias_maps.cf /etc/postfix/mysql_virtual_alias_maps.cf.template
|
||||
COPY sender_access /etc/postfix/sender_access
|
||||
COPY smtp_generic /etc/postfix/smtp_generic
|
||||
COPY aliases /etc/aliases
|
||||
COPY process-bounce.py /usr/local/bin/process-bounce.py
|
||||
COPY entrypoint.sh /entrypoint.sh
|
||||
RUN chmod +x /entrypoint.sh
|
||||
RUN chmod +x /entrypoint.sh /usr/local/bin/process-bounce.py
|
||||
|
||||
# Generate Postfix maps for sender access
|
||||
RUN postmap /etc/postfix/sender_access
|
||||
# Generate Postfix maps for sender access, sender canonical, and aliases
|
||||
RUN postmap /etc/postfix/sender_access && \
|
||||
postmap /etc/postfix/smtp_generic && \
|
||||
newaliases
|
||||
|
||||
# Expose SMTP
|
||||
EXPOSE 25
|
||||
|
||||
17
postfix/aliases
Normal file
17
postfix/aliases
Normal file
@@ -0,0 +1,17 @@
|
||||
# Postfix aliases for bounce handling
|
||||
#
|
||||
# This file defines how bounces are processed
|
||||
# The bounces address pipes messages to our bounce processing script
|
||||
|
||||
# Bounce processing - pipe to Python script
|
||||
bounces: "|/usr/bin/python3 /usr/local/bin/process-bounce.py"
|
||||
|
||||
# Standard aliases
|
||||
postmaster: root
|
||||
mailer-daemon: postmaster
|
||||
abuse: postmaster
|
||||
spam: postmaster
|
||||
root: postmaster
|
||||
|
||||
# Default fallback
|
||||
MAILER-DAEMON: bounces
|
||||
@@ -32,5 +32,28 @@ chmod 644 /etc/postfix/sender_access /etc/postfix/sender_access.db
|
||||
# Set permissions on MySQL config
|
||||
chmod 644 /etc/postfix/mysql_virtual_alias_maps.cf
|
||||
|
||||
# Configure bounce processing based on environment variable
|
||||
if [ "${ENABLE_EMAIL_BOUNCE_PROCESSING:-false}" = "true" ]; then
|
||||
echo "Email-based bounce processing enabled"
|
||||
# Regenerate aliases database to enable bounce processing
|
||||
newaliases
|
||||
chmod 644 /etc/aliases /etc/aliases.db
|
||||
|
||||
# Generate smtp generic maps to ensure bounces come to our bounce address
|
||||
postmap /etc/postfix/smtp_generic
|
||||
chmod 644 /etc/postfix/smtp_generic /etc/postfix/smtp_generic.db
|
||||
echo "Configured return path rewriting to bounces@lists.sasalliance.org"
|
||||
else
|
||||
echo "Email-based bounce processing disabled"
|
||||
# Create minimal aliases without bounce processing
|
||||
echo "postmaster: root" > /etc/aliases
|
||||
echo "root: postmaster" >> /etc/aliases
|
||||
newaliases
|
||||
|
||||
# Disable smtp generic maps
|
||||
echo "# Email bounce processing disabled" > /etc/postfix/smtp_generic
|
||||
postmap /etc/postfix/smtp_generic
|
||||
fi
|
||||
|
||||
# Start Postfix in foreground
|
||||
exec postfix start-fg
|
||||
|
||||
@@ -33,3 +33,23 @@ smtpd_recipient_restrictions =
|
||||
# Other recommended settings
|
||||
alias_maps = hash:/etc/aliases
|
||||
alias_database = hash:/etc/aliases
|
||||
|
||||
# Bounce handling configuration for email-based processing
|
||||
# Configure bounce notification recipients
|
||||
bounce_notice_recipient = bounces@lists.sasalliance.org
|
||||
2bounce_notice_recipient = bounces@lists.sasalliance.org
|
||||
delay_notice_recipient =
|
||||
error_notice_recipient = bounces@lists.sasalliance.org
|
||||
|
||||
# Bounce settings
|
||||
bounce_size_limit = 50000
|
||||
bounce_queue_lifetime = 5d
|
||||
maximal_bounce_delay = 1d
|
||||
|
||||
# Return path configuration - CRITICAL for bounce handling
|
||||
# This ensures that when we relay emails via mailing lists through SES,
|
||||
# bounces come back to our bounce processing address
|
||||
#
|
||||
# Use smtp_generic_maps instead of sender_canonical_maps because we only want
|
||||
# to rewrite the return path for outbound SMTP (via SES), not for local delivery
|
||||
smtp_generic_maps = hash:/etc/postfix/smtp_generic
|
||||
|
||||
348
postfix/process-bounce.py
Normal file
348
postfix/process-bounce.py
Normal file
@@ -0,0 +1,348 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Email-based bounce processing script for Postfix
|
||||
Parses bounce emails and updates the database with bounce information
|
||||
|
||||
This script is called by Postfix when bounce emails are received.
|
||||
It reads the email from stdin, parses it for bounce information,
|
||||
and updates the database accordingly.
|
||||
"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
import re
|
||||
import email
|
||||
from email.message import Message
|
||||
import logging
|
||||
from datetime import datetime
|
||||
from typing import List, Tuple, Optional, Dict
|
||||
import pymysql
|
||||
|
||||
# Configure logging
|
||||
log_file = '/var/log/bounce-processor.log'
|
||||
log_handlers = [logging.StreamHandler()]
|
||||
|
||||
# Add file handler if we can write to the log directory
|
||||
try:
|
||||
log_handlers.append(logging.FileHandler(log_file))
|
||||
except (PermissionError, FileNotFoundError):
|
||||
pass # Just use stdout if we can't write to log file
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
|
||||
handlers=log_handlers
|
||||
)
|
||||
logger = logging.getLogger('bounce-processor')
|
||||
|
||||
# Database configuration from environment
|
||||
MYSQL_HOST = os.getenv('MYSQL_HOST', 'mysql')
|
||||
MYSQL_PORT = int(os.getenv('MYSQL_PORT', 3306))
|
||||
MYSQL_DATABASE = os.getenv('MYSQL_DATABASE', 'maillist')
|
||||
MYSQL_USER = os.getenv('MYSQL_USER', 'maillist')
|
||||
MYSQL_PASSWORD = os.getenv('MYSQL_PASSWORD', '')
|
||||
|
||||
class BounceProcessor:
|
||||
"""Processes bounce emails and updates the database"""
|
||||
|
||||
def __init__(self):
|
||||
self.bounce_patterns = self._compile_bounce_patterns()
|
||||
|
||||
def _compile_bounce_patterns(self) -> List[re.Pattern]:
|
||||
"""Compile regex patterns for detecting bounce information"""
|
||||
patterns = [
|
||||
# Standard bounce formats
|
||||
re.compile(r'550.*?([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,})', re.IGNORECASE),
|
||||
re.compile(r'554.*?([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,})', re.IGNORECASE),
|
||||
re.compile(r'553.*?([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,})', re.IGNORECASE),
|
||||
re.compile(r'552.*?([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,})', re.IGNORECASE),
|
||||
re.compile(r'551.*?([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,})', re.IGNORECASE),
|
||||
|
||||
# Delivery Status Notification (DSN) format
|
||||
re.compile(r'Final-Recipient:.*?([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,})', re.IGNORECASE),
|
||||
re.compile(r'Original-Recipient:.*?([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,})', re.IGNORECASE),
|
||||
|
||||
# Common bounce message patterns
|
||||
re.compile(r'user.*?([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}).*?unknown', re.IGNORECASE),
|
||||
re.compile(r'([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}).*?does not exist', re.IGNORECASE),
|
||||
re.compile(r'([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}).*?not found', re.IGNORECASE),
|
||||
re.compile(r'([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}).*?mailbox.*?unavailable', re.IGNORECASE),
|
||||
re.compile(r'([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}).*?recipient.*?rejected', re.IGNORECASE),
|
||||
|
||||
# Generic email extraction (fallback)
|
||||
re.compile(r'([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,})', re.IGNORECASE),
|
||||
]
|
||||
return patterns
|
||||
|
||||
def parse_bounce_email(self, email_content: str) -> List[Dict]:
|
||||
"""Parse bounce email and extract bounce information"""
|
||||
try:
|
||||
# Parse the email
|
||||
msg = email.message_from_string(email_content)
|
||||
bounces = []
|
||||
|
||||
# Get the email body
|
||||
body = self._get_email_body(msg)
|
||||
if not body:
|
||||
logger.warning("No email body found")
|
||||
return bounces
|
||||
|
||||
logger.info(f"Processing email body (first 500 chars): {body[:500]}")
|
||||
|
||||
# Extract bounced addresses
|
||||
bounced_addresses = self._extract_bounced_addresses(body)
|
||||
|
||||
# Determine bounce type and create bounce records
|
||||
for address in bounced_addresses:
|
||||
bounce_info = self._analyze_bounce(body, address)
|
||||
if bounce_info:
|
||||
bounces.append(bounce_info)
|
||||
|
||||
return bounces
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error parsing bounce email: {str(e)}")
|
||||
return []
|
||||
|
||||
def _get_email_body(self, msg: Message) -> Optional[str]:
|
||||
"""Extract the email body from the message"""
|
||||
body = ""
|
||||
|
||||
if msg.is_multipart():
|
||||
for part in msg.walk():
|
||||
content_type = part.get_content_type()
|
||||
if content_type in ['text/plain', 'text/html']:
|
||||
payload = part.get_payload(decode=True)
|
||||
if payload:
|
||||
body += payload.decode('utf-8', errors='ignore') + "\n"
|
||||
else:
|
||||
payload = msg.get_payload(decode=True)
|
||||
if payload:
|
||||
body = payload.decode('utf-8', errors='ignore')
|
||||
|
||||
return body.strip() if body else None
|
||||
|
||||
def _extract_bounced_addresses(self, body: str) -> List[str]:
|
||||
"""Extract bounced email addresses from the bounce message"""
|
||||
addresses = []
|
||||
|
||||
for pattern in self.bounce_patterns:
|
||||
matches = pattern.findall(body)
|
||||
for match in matches:
|
||||
email_addr = match.strip().lower()
|
||||
if self._is_valid_email(email_addr) and email_addr not in addresses:
|
||||
# Skip our own addresses
|
||||
if not email_addr.endswith('@lists.sasalliance.org'):
|
||||
addresses.append(email_addr)
|
||||
|
||||
logger.info(f"Extracted addresses: {addresses}")
|
||||
return addresses
|
||||
|
||||
def _is_valid_email(self, email_addr: str) -> bool:
|
||||
"""Validate email address format"""
|
||||
email_pattern = re.compile(r'^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$')
|
||||
return bool(email_pattern.match(email_addr))
|
||||
|
||||
def _analyze_bounce(self, body: str, email_addr: str) -> Optional[Dict]:
|
||||
"""Analyze bounce message to determine bounce type and details"""
|
||||
bounce_info = {
|
||||
'email': email_addr,
|
||||
'bounce_type': 'Undetermined',
|
||||
'bounce_subtype': '',
|
||||
'diagnostic_code': '',
|
||||
'timestamp': datetime.now().strftime('%Y-%m-%d %H:%M:%S')
|
||||
}
|
||||
|
||||
# Analyze bounce type based on SMTP codes and message content
|
||||
if re.search(r'5[0-9]{2}', body): # 5xx codes are permanent failures
|
||||
bounce_info['bounce_type'] = 'Permanent'
|
||||
|
||||
if re.search(r'550|551|553', body):
|
||||
bounce_info['bounce_subtype'] = 'General'
|
||||
elif re.search(r'552', body):
|
||||
bounce_info['bounce_subtype'] = 'MailboxFull'
|
||||
elif re.search(r'554', body):
|
||||
bounce_info['bounce_subtype'] = 'MessageTooLarge'
|
||||
|
||||
elif re.search(r'4[0-9]{2}', body): # 4xx codes are temporary failures
|
||||
bounce_info['bounce_type'] = 'Transient'
|
||||
|
||||
if re.search(r'450|451|452', body):
|
||||
bounce_info['bounce_subtype'] = 'General'
|
||||
elif re.search(r'452', body):
|
||||
bounce_info['bounce_subtype'] = 'MailboxFull'
|
||||
|
||||
# Extract diagnostic code
|
||||
smtp_code_match = re.search(r'([45][0-9]{2}.*?)[\r\n]', body)
|
||||
if smtp_code_match:
|
||||
bounce_info['diagnostic_code'] = smtp_code_match.group(1).strip()[:500] # Limit length
|
||||
|
||||
return bounce_info
|
||||
|
||||
def update_database(self, bounces: List[Dict]) -> None:
|
||||
"""Update the database with bounce information"""
|
||||
if not bounces:
|
||||
logger.info("No bounces to process")
|
||||
return
|
||||
|
||||
try:
|
||||
connection = pymysql.connect(
|
||||
host=MYSQL_HOST,
|
||||
port=MYSQL_PORT,
|
||||
database=MYSQL_DATABASE,
|
||||
user=MYSQL_USER,
|
||||
password=MYSQL_PASSWORD,
|
||||
cursorclass=pymysql.cursors.DictCursor
|
||||
)
|
||||
|
||||
cursor = connection.cursor()
|
||||
|
||||
for bounce in bounces:
|
||||
try:
|
||||
email_addr = bounce['email']
|
||||
|
||||
# Find member by email
|
||||
cursor.execute("SELECT member_id FROM members WHERE email = %s", (email_addr,))
|
||||
member = cursor.fetchone()
|
||||
member_id = member['member_id'] if member else None
|
||||
|
||||
# Log the bounce
|
||||
cursor.execute("""
|
||||
INSERT INTO bounce_logs
|
||||
(member_id, email, bounce_type, bounce_subtype, diagnostic_code, timestamp, created_at)
|
||||
VALUES (%s, %s, %s, %s, %s, %s, NOW())
|
||||
""", (member_id, email_addr, bounce['bounce_type'], bounce['bounce_subtype'],
|
||||
bounce['diagnostic_code'], bounce['timestamp']))
|
||||
|
||||
logger.info(f"Logged bounce for {email_addr}: {bounce['bounce_type']}")
|
||||
|
||||
# Update member bounce status if member exists
|
||||
if member_id:
|
||||
if bounce['bounce_type'] == 'Permanent':
|
||||
# Hard bounce - deactivate member
|
||||
cursor.execute("""
|
||||
UPDATE members
|
||||
SET bounce_count = bounce_count + 1,
|
||||
last_bounce_at = %s,
|
||||
bounce_status = 'hard_bounce',
|
||||
active = 0
|
||||
WHERE member_id = %s
|
||||
""", (bounce['timestamp'], member_id))
|
||||
logger.info(f"Deactivated member {email_addr} due to hard bounce")
|
||||
|
||||
elif bounce['bounce_type'] == 'Transient':
|
||||
# Soft bounce - increment counter and check threshold
|
||||
cursor.execute("SELECT bounce_count, bounce_status FROM members WHERE member_id = %s", (member_id,))
|
||||
current = cursor.fetchone()
|
||||
|
||||
if current and current['bounce_status'] != 'hard_bounce':
|
||||
new_count = current['bounce_count'] + 1
|
||||
new_status = 'soft_bounce' if new_count >= 3 else 'clean'
|
||||
|
||||
cursor.execute("""
|
||||
UPDATE members
|
||||
SET bounce_count = %s,
|
||||
last_bounce_at = %s,
|
||||
bounce_status = %s
|
||||
WHERE member_id = %s
|
||||
""", (new_count, bounce['timestamp'], new_status, member_id))
|
||||
|
||||
logger.info(f"Updated member {email_addr} bounce count to {new_count}, status: {new_status}")
|
||||
else:
|
||||
# Undetermined - just increment counter
|
||||
cursor.execute("""
|
||||
UPDATE members
|
||||
SET bounce_count = bounce_count + 1,
|
||||
last_bounce_at = %s
|
||||
WHERE member_id = %s
|
||||
""", (bounce['timestamp'], member_id))
|
||||
|
||||
logger.info(f"Updated member {email_addr} bounce count (undetermined)")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Database error processing bounce for {bounce['email']}: {str(e)}")
|
||||
continue
|
||||
|
||||
connection.commit()
|
||||
cursor.close()
|
||||
connection.close()
|
||||
|
||||
logger.info(f"Successfully processed {len(bounces)} bounces")
|
||||
|
||||
except pymysql.Error as e:
|
||||
logger.error(f"Database connection error: {str(e)}")
|
||||
except Exception as e:
|
||||
logger.error(f"Unexpected error updating database: {str(e)}")
|
||||
|
||||
def main():
|
||||
"""Main function - reads email from stdin and processes bounces"""
|
||||
try:
|
||||
logger.info("Starting bounce processing")
|
||||
|
||||
# Check if we're in test mode
|
||||
test_mode = len(sys.argv) > 1 and sys.argv[1] == '--test'
|
||||
|
||||
if test_mode:
|
||||
# Test mode - use sample bounce email
|
||||
email_content = """From: Mail Delivery Subsystem <MAILER-DAEMON@example.com>
|
||||
To: bounces@lists.sasalliance.org
|
||||
Subject: Delivery Status Notification (Failure)
|
||||
Content-Type: multipart/report; report-type=delivery-status; boundary="boundary123"
|
||||
|
||||
--boundary123
|
||||
Content-Type: text/plain
|
||||
|
||||
This is a test bounce message.
|
||||
|
||||
The following address(es) failed:
|
||||
testuser@example.com
|
||||
SMTP error from remote mail server after RCPT TO:<testuser@example.com>:
|
||||
550 5.1.1 User unknown
|
||||
|
||||
--boundary123
|
||||
Content-Type: message/delivery-status
|
||||
|
||||
Reporting-MTA: dns; mail.example.com
|
||||
Received-From-MTA: dns; localhost
|
||||
|
||||
Final-Recipient: rfc822; testuser@example.com
|
||||
Action: failed
|
||||
Status: 5.1.1
|
||||
Diagnostic-Code: smtp; 550 5.1.1 User unknown
|
||||
|
||||
--boundary123--
|
||||
"""
|
||||
logger.info("Running in test mode with sample bounce email")
|
||||
else:
|
||||
# Read email from stdin
|
||||
email_content = sys.stdin.read()
|
||||
|
||||
if not email_content.strip():
|
||||
logger.warning("No email content received")
|
||||
return
|
||||
|
||||
logger.info(f"Received email content ({len(email_content)} bytes)")
|
||||
|
||||
# Process the bounce
|
||||
processor = BounceProcessor()
|
||||
bounces = processor.parse_bounce_email(email_content)
|
||||
|
||||
if test_mode:
|
||||
logger.info(f"Test mode - would process {len(bounces)} bounce(s):")
|
||||
for bounce in bounces:
|
||||
logger.info(f" {bounce}")
|
||||
else:
|
||||
if bounces:
|
||||
processor.update_database(bounces)
|
||||
logger.info(f"Processed {len(bounces)} bounce(s)")
|
||||
else:
|
||||
logger.info("No bounces detected in email")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error in main: {str(e)}")
|
||||
import traceback
|
||||
logger.error(f"Traceback: {traceback.format_exc()}")
|
||||
sys.exit(1)
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
23
postfix/smtp_generic
Normal file
23
postfix/smtp_generic
Normal file
@@ -0,0 +1,23 @@
|
||||
# Postfix SMTP Generic Maps
|
||||
# This file controls the return path (envelope sender) for outbound SMTP emails
|
||||
# Only applies to emails being relayed through external SMTP (SES in our case)
|
||||
#
|
||||
# Format: original_sender rewritten_sender
|
||||
#
|
||||
# For mailing list emails, we want bounces to go to our bounce processing address
|
||||
# This is critical for email-based bounce handling to work properly
|
||||
|
||||
# Rewrite envelope sender for all mailing list addresses to bounce address
|
||||
# When emails are forwarded through mailing lists via SES, bounces come back to us
|
||||
community@lists.sasalliance.org bounces@lists.sasalliance.org
|
||||
board@lists.sasalliance.org bounces@lists.sasalliance.org
|
||||
members@lists.sasalliance.org bounces@lists.sasalliance.org
|
||||
announcements@lists.sasalliance.org bounces@lists.sasalliance.org
|
||||
|
||||
# Generic pattern - any @lists.sasalliance.org sender gets rewritten to bounce address
|
||||
# This catches any new lists automatically
|
||||
@lists.sasalliance.org bounces@lists.sasalliance.org
|
||||
|
||||
# Also rewrite any envelope sender that's sending through our system
|
||||
# This ensures ALL outgoing mail via SES has our bounce address as return path
|
||||
@sasalliance.org bounces@lists.sasalliance.org
|
||||
@@ -1,11 +0,0 @@
|
||||
# Community mailing list - general announcements
|
||||
community@lists.sasalliance.org james@pattinson.org, james.pattinson@sasalliance.org
|
||||
|
||||
# Board members mailing list
|
||||
board@lists.sasalliance.org james.pattinson@sasalliance.org
|
||||
|
||||
# All members mailing list
|
||||
members@lists.sasalliance.org james@pattinson.org, james.pattinson@sasalliance.org
|
||||
|
||||
# Announcements mailing list
|
||||
announcements@lists.sasalliance.org james@pattinson.org, james.pattinson@sasalliance.org
|
||||
206
simulate_bounce.sh
Executable file
206
simulate_bounce.sh
Executable file
@@ -0,0 +1,206 @@
|
||||
#!/bin/bash
|
||||
# Simulate Bounce Events for Testing
|
||||
# This script inserts bounce data directly into the database to test the UI
|
||||
# without needing to set up AWS SNS
|
||||
|
||||
set -e
|
||||
|
||||
echo "=== Bounce Simulation Script ==="
|
||||
echo
|
||||
echo "This script will create test bounce events for existing members"
|
||||
echo
|
||||
|
||||
# Get database password from .env
|
||||
DB_PASSWORD=$(grep MYSQL_ROOT_PASSWORD /home/jamesp/docker/maillist/.env | cut -d'=' -f2)
|
||||
|
||||
# Check if containers are running
|
||||
if ! sudo docker-compose ps | grep -q "maillist-mysql.*Up"; then
|
||||
echo "❌ MySQL container is not running. Starting containers..."
|
||||
sudo docker-compose up -d
|
||||
echo "Waiting for MySQL to be ready..."
|
||||
sleep 5
|
||||
fi
|
||||
|
||||
echo "1. Fetching existing members..."
|
||||
echo
|
||||
|
||||
# Get list of members
|
||||
MEMBERS=$(sudo docker-compose exec -T mysql mysql -u root -p"$DB_PASSWORD" maillist -N -e "SELECT member_id, name, email FROM members LIMIT 5;")
|
||||
|
||||
if [ -z "$MEMBERS" ]; then
|
||||
echo "❌ No members found in database. Please add members first."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Available members:"
|
||||
echo "$MEMBERS" | while read -r line; do
|
||||
MEMBER_ID=$(echo "$line" | awk '{print $1}')
|
||||
NAME=$(echo "$line" | awk '{print $2, $3}')
|
||||
EMAIL=$(echo "$line" | awk '{$1=""; $2=""; print $0}' | xargs)
|
||||
echo " [$MEMBER_ID] $NAME - $EMAIL"
|
||||
done
|
||||
echo
|
||||
|
||||
# Prompt for member ID
|
||||
read -p "Enter member ID to simulate bounce for (or press Enter for first member): " MEMBER_ID
|
||||
|
||||
if [ -z "$MEMBER_ID" ]; then
|
||||
MEMBER_ID=$(echo "$MEMBERS" | head -1 | awk '{print $1}')
|
||||
echo "Using member ID: $MEMBER_ID"
|
||||
fi
|
||||
|
||||
# Get member details
|
||||
MEMBER_INFO=$(sudo docker-compose exec -T mysql mysql -u root -p"$DB_PASSWORD" maillist -N -e "SELECT member_id, name, email FROM members WHERE member_id = $MEMBER_ID;")
|
||||
|
||||
if [ -z "$MEMBER_INFO" ]; then
|
||||
echo "❌ Member ID $MEMBER_ID not found"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
MEMBER_EMAIL=$(echo "$MEMBER_INFO" | awk '{$1=""; $2=""; print $0}' | xargs)
|
||||
MEMBER_NAME=$(echo "$MEMBER_INFO" | awk '{print $2, $3}')
|
||||
|
||||
echo
|
||||
echo "Selected member: $MEMBER_NAME ($MEMBER_EMAIL)"
|
||||
echo
|
||||
|
||||
# Bounce type selection
|
||||
echo "Select bounce type to simulate:"
|
||||
echo " 1) Soft Bounce (Transient - e.g., mailbox full)"
|
||||
echo " 2) Hard Bounce (Permanent - e.g., invalid address)"
|
||||
echo " 3) Multiple Soft Bounces (3 bounces to trigger soft_bounce status)"
|
||||
echo " 4) Undetermined Bounce"
|
||||
echo
|
||||
read -p "Enter choice (1-4): " BOUNCE_CHOICE
|
||||
|
||||
case $BOUNCE_CHOICE in
|
||||
1)
|
||||
BOUNCE_TYPE="Transient"
|
||||
BOUNCE_SUBTYPE="MailboxFull"
|
||||
DIAGNOSTIC="smtp; 452 4.2.2 Mailbox full"
|
||||
COUNT=1
|
||||
;;
|
||||
2)
|
||||
BOUNCE_TYPE="Permanent"
|
||||
BOUNCE_SUBTYPE="General"
|
||||
DIAGNOSTIC="smtp; 550 5.1.1 User unknown"
|
||||
COUNT=1
|
||||
;;
|
||||
3)
|
||||
BOUNCE_TYPE="Transient"
|
||||
BOUNCE_SUBTYPE="General"
|
||||
DIAGNOSTIC="smtp; 451 4.4.1 Temporary failure"
|
||||
COUNT=3
|
||||
;;
|
||||
4)
|
||||
BOUNCE_TYPE="Undetermined"
|
||||
BOUNCE_SUBTYPE=""
|
||||
DIAGNOSTIC="Unknown error occurred"
|
||||
COUNT=1
|
||||
;;
|
||||
*)
|
||||
echo "Invalid choice"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
echo
|
||||
echo "Simulating $COUNT bounce event(s)..."
|
||||
echo
|
||||
|
||||
# Insert bounce events
|
||||
for i in $(seq 1 $COUNT); do
|
||||
TIMESTAMP=$(date -u -d "-$((i * 24)) hours" '+%Y-%m-%d %H:%M:%S')
|
||||
FEEDBACK_ID="test-feedback-$(date +%s)-$i"
|
||||
|
||||
echo "Creating bounce event $i/$COUNT (timestamp: $TIMESTAMP)..."
|
||||
|
||||
sudo docker-compose exec -T mysql mysql -u root -p"$DB_PASSWORD" maillist <<EOF
|
||||
-- Insert bounce log
|
||||
INSERT INTO bounce_logs (member_id, email, bounce_type, bounce_subtype, diagnostic_code, timestamp, feedback_id)
|
||||
VALUES ($MEMBER_ID, '$MEMBER_EMAIL', '$BOUNCE_TYPE', '$BOUNCE_SUBTYPE', '$DIAGNOSTIC', '$TIMESTAMP', '$FEEDBACK_ID');
|
||||
EOF
|
||||
|
||||
# Update member bounce status
|
||||
if [ "$BOUNCE_TYPE" = "Permanent" ]; then
|
||||
echo " → Hard bounce: Deactivating member..."
|
||||
sudo docker-compose exec -T mysql mysql -u root -p"$DB_PASSWORD" maillist <<EOF
|
||||
UPDATE members
|
||||
SET bounce_count = bounce_count + 1,
|
||||
last_bounce_at = '$TIMESTAMP',
|
||||
bounce_status = 'hard_bounce',
|
||||
active = 0
|
||||
WHERE member_id = $MEMBER_ID;
|
||||
EOF
|
||||
elif [ "$BOUNCE_TYPE" = "Transient" ]; then
|
||||
# Check current bounce count
|
||||
CURRENT_COUNT=$(sudo docker-compose exec -T mysql mysql -u root -p"$DB_PASSWORD" maillist -N -e "SELECT bounce_count FROM members WHERE member_id = $MEMBER_ID;")
|
||||
NEW_COUNT=$((CURRENT_COUNT + 1))
|
||||
|
||||
if [ $NEW_COUNT -ge 3 ]; then
|
||||
echo " → Soft bounce threshold reached: Marking as soft_bounce..."
|
||||
BOUNCE_STATUS="soft_bounce"
|
||||
else
|
||||
echo " → Soft bounce: Incrementing counter ($NEW_COUNT)..."
|
||||
BOUNCE_STATUS="clean"
|
||||
fi
|
||||
|
||||
sudo docker-compose exec -T mysql mysql -u root -p"$DB_PASSWORD" maillist <<EOF
|
||||
UPDATE members
|
||||
SET bounce_count = $NEW_COUNT,
|
||||
last_bounce_at = '$TIMESTAMP',
|
||||
bounce_status = '$BOUNCE_STATUS'
|
||||
WHERE member_id = $MEMBER_ID;
|
||||
EOF
|
||||
else
|
||||
echo " → Undetermined bounce: Incrementing counter..."
|
||||
sudo docker-compose exec -T mysql mysql -u root -p"$DB_PASSWORD" maillist <<EOF
|
||||
UPDATE members
|
||||
SET bounce_count = bounce_count + 1,
|
||||
last_bounce_at = '$TIMESTAMP'
|
||||
WHERE member_id = $MEMBER_ID;
|
||||
EOF
|
||||
fi
|
||||
done
|
||||
|
||||
echo
|
||||
echo "✅ Bounce simulation complete!"
|
||||
echo
|
||||
|
||||
# Show updated member status
|
||||
echo "Updated member status:"
|
||||
sudo docker-compose exec -T mysql mysql -u root -p"$DB_PASSWORD" maillist -e "
|
||||
SELECT
|
||||
member_id,
|
||||
name,
|
||||
email,
|
||||
active,
|
||||
bounce_count,
|
||||
last_bounce_at,
|
||||
bounce_status
|
||||
FROM members
|
||||
WHERE member_id = $MEMBER_ID\G
|
||||
"
|
||||
|
||||
echo
|
||||
echo "Bounce history for this member:"
|
||||
sudo docker-compose exec -T mysql mysql -u root -p"$DB_PASSWORD" maillist -e "
|
||||
SELECT
|
||||
bounce_id,
|
||||
bounce_type,
|
||||
bounce_subtype,
|
||||
diagnostic_code,
|
||||
timestamp
|
||||
FROM bounce_logs
|
||||
WHERE member_id = $MEMBER_ID
|
||||
ORDER BY timestamp DESC;
|
||||
"
|
||||
|
||||
echo
|
||||
echo "🎉 You can now view this bounce in the web UI:"
|
||||
echo " 1. Open http://localhost:3000"
|
||||
echo " 2. Go to the Members tab"
|
||||
echo " 3. Look for $MEMBER_NAME"
|
||||
echo " 4. Click the 'Bounces' button to see the history"
|
||||
echo
|
||||
echo "To simulate more bounces, run this script again!"
|
||||
46
test-cors-understanding.html
Normal file
46
test-cors-understanding.html
Normal file
@@ -0,0 +1,46 @@
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>CORS Understanding Test</title>
|
||||
</head>
|
||||
<body>
|
||||
<h1>CORS Test</h1>
|
||||
<div id="results"></div>
|
||||
|
||||
<script>
|
||||
const results = document.getElementById('results');
|
||||
|
||||
// Test 1: Same-origin request (should work without CORS headers)
|
||||
console.log('Current origin:', window.location.origin);
|
||||
|
||||
// Test what your API client actually does
|
||||
const protocol = window.location.protocol;
|
||||
const hostname = window.location.hostname;
|
||||
|
||||
let baseURL;
|
||||
if (hostname === 'localhost' || hostname === '127.0.0.1') {
|
||||
baseURL = `${protocol}//${hostname}:8000`;
|
||||
results.innerHTML += `<p><strong>Development Mode Detected</strong></p>`;
|
||||
results.innerHTML += `<p>Frontend: ${window.location.origin}</p>`;
|
||||
results.innerHTML += `<p>API: ${baseURL}</p>`;
|
||||
results.innerHTML += `<p>This is a <strong>CROSS-ORIGIN</strong> request - CORS applies!</p>`;
|
||||
} else {
|
||||
baseURL = `${protocol}//${hostname}/api`;
|
||||
results.innerHTML += `<p><strong>Production Mode Detected</strong></p>`;
|
||||
results.innerHTML += `<p>Frontend: ${window.location.origin}</p>`;
|
||||
results.innerHTML += `<p>API: ${baseURL}</p>`;
|
||||
results.innerHTML += `<p>This is a <strong>SAME-ORIGIN</strong> request - NO CORS needed!</p>`;
|
||||
}
|
||||
|
||||
// Test the actual request
|
||||
fetch(`${baseURL}/health`)
|
||||
.then(response => response.json())
|
||||
.then(data => {
|
||||
results.innerHTML += `<p style="color: green;">✅ API Request Successful: ${JSON.stringify(data)}</p>`;
|
||||
})
|
||||
.catch(error => {
|
||||
results.innerHTML += `<p style="color: red;">❌ API Request Failed: ${error.message}</p>`;
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
45
test_bounce_quick.sh
Normal file
45
test_bounce_quick.sh
Normal file
@@ -0,0 +1,45 @@
|
||||
#!/bin/bash
|
||||
# Quick SES Bounce Test - One-liner to test bounce handling
|
||||
|
||||
echo "🚀 Testing SES Bounce Handling"
|
||||
echo "================================"
|
||||
echo ""
|
||||
|
||||
# Check if API and Postfix are running
|
||||
if ! sudo docker compose ps | grep -q "maillist-api.*Up"; then
|
||||
echo "❌ API container not running. Start with: sudo docker compose up -d"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if ! sudo docker compose ps | grep -q "maillist-postfix.*Up"; then
|
||||
echo "❌ Postfix container not running. Start with: sudo docker compose up -d"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "✅ Containers are running"
|
||||
echo ""
|
||||
|
||||
# Start watching logs in background
|
||||
echo "📋 Opening log viewer (press Ctrl+C to stop)..."
|
||||
echo ""
|
||||
sleep 2
|
||||
|
||||
# Show the command they can run
|
||||
echo "Run this command to send a test bounce:"
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
echo "echo 'Test bounce' | sudo docker compose exec -T postfix mail -s 'Bounce Test' bounce@simulator.amazonses.com"
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
echo ""
|
||||
echo "Then wait 30-60 seconds and check:"
|
||||
echo ""
|
||||
echo "1. API logs (in this window)"
|
||||
echo "2. Database: sudo docker compose exec mysql mysql -u maillist -pmaillist maillist -e 'SELECT * FROM bounce_logs ORDER BY created_at DESC LIMIT 5;'"
|
||||
echo "3. Web UI: http://localhost:3000 → Members tab"
|
||||
echo ""
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
echo "Watching API logs now..."
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
echo ""
|
||||
|
||||
# Follow logs
|
||||
sudo docker compose logs api -f --tail 20
|
||||
76
test_bounce_webhook.sh
Executable file
76
test_bounce_webhook.sh
Executable file
@@ -0,0 +1,76 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Test SNS Bounce Webhook
|
||||
# This script simulates an SNS bounce notification (without signature verification for testing)
|
||||
|
||||
API_URL="http://localhost:8000"
|
||||
|
||||
echo "Testing SNS Bounce Webhook..."
|
||||
echo
|
||||
|
||||
# Test 1: Health check
|
||||
echo "1. Testing API health..."
|
||||
curl -s "$API_URL/health" | jq .
|
||||
echo
|
||||
|
||||
# Test 2: Create a test subscription confirmation (the webhook will auto-confirm)
|
||||
echo "2. Testing subscription confirmation handling..."
|
||||
curl -s -X POST "$API_URL/webhooks/sns" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"Type": "SubscriptionConfirmation",
|
||||
"MessageId": "test-message-id",
|
||||
"Token": "test-token",
|
||||
"TopicArn": "arn:aws:sns:eu-west-2:123456789:test-topic",
|
||||
"Message": "You have chosen to subscribe to the topic",
|
||||
"SubscribeURL": "https://example.com/subscribe",
|
||||
"Timestamp": "2025-01-01T12:00:00.000Z"
|
||||
}' || echo "Note: Signature verification will fail for test messages (expected)"
|
||||
echo
|
||||
echo
|
||||
|
||||
# Test 3: Simulated bounce notification (will fail signature verification)
|
||||
echo "3. Testing bounce notification structure..."
|
||||
echo "Note: In production, AWS SNS will send properly signed messages."
|
||||
echo "This test demonstrates the expected structure."
|
||||
echo
|
||||
|
||||
cat << 'EOF' > /tmp/test_bounce.json
|
||||
{
|
||||
"Type": "Notification",
|
||||
"MessageId": "test-notification-id",
|
||||
"TopicArn": "arn:aws:sns:eu-west-2:123456789:test-topic",
|
||||
"Subject": "Amazon SES Email Event Notification",
|
||||
"Message": "{\"notificationType\":\"Bounce\",\"bounce\":{\"bounceType\":\"Permanent\",\"bounceSubType\":\"General\",\"bouncedRecipients\":[{\"emailAddress\":\"bounce@simulator.amazonses.com\",\"diagnosticCode\":\"smtp; 550 5.1.1 user unknown\"}],\"timestamp\":\"2025-01-01T12:00:00.000Z\",\"feedbackId\":\"test-feedback-id\"}}",
|
||||
"Timestamp": "2025-01-01T12:00:00.000Z",
|
||||
"SignatureVersion": "1",
|
||||
"Signature": "test-signature",
|
||||
"SigningCertURL": "https://sns.eu-west-2.amazonaws.com/test.pem"
|
||||
}
|
||||
EOF
|
||||
|
||||
cat /tmp/test_bounce.json | jq .
|
||||
echo
|
||||
echo "Expected behavior: Signature verification will fail without real AWS credentials"
|
||||
echo "In production, AWS SNS will send properly signed messages that will be verified"
|
||||
echo
|
||||
|
||||
# Test 4: Check if database schema has bounce tables
|
||||
echo "4. Checking database schema for bounce tables..."
|
||||
sudo docker-compose exec -T mysql mysql -u maillist -pmaillist maillist -e "SHOW TABLES LIKE '%bounce%';" 2>/dev/null
|
||||
echo
|
||||
|
||||
# Test 5: Check members table for bounce columns
|
||||
echo "5. Checking members table for bounce columns..."
|
||||
sudo docker-compose exec -T mysql mysql -u maillist -pmaillist maillist -e "DESCRIBE members;" 2>/dev/null | grep -i bounce
|
||||
echo
|
||||
|
||||
echo "✓ Setup complete!"
|
||||
echo
|
||||
echo "To test with real AWS SNS:"
|
||||
echo "1. Set up SNS topic in AWS Console"
|
||||
echo "2. Subscribe webhook: https://your-domain.com:8000/webhooks/sns"
|
||||
echo "3. Configure SES to send bounce notifications to the SNS topic"
|
||||
echo "4. Send test email to bounce@simulator.amazonses.com"
|
||||
echo
|
||||
echo "See BOUNCE_HANDLING_SETUP.md for detailed setup instructions"
|
||||
138
web/index.html
138
web/index.html
@@ -54,22 +54,47 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Header (shown after login) -->
|
||||
<header class="header" id="mainHeader" style="display: none;">
|
||||
<div class="container">
|
||||
<div class="header-content">
|
||||
<h1 class="logo">
|
||||
<!-- Menu Bar Header (shown after login) -->
|
||||
<header class="menu-bar" id="mainHeader" style="display: none;">
|
||||
<div class="menu-bar-content">
|
||||
<div class="app-title">
|
||||
<i class="fas fa-envelope"></i>
|
||||
Mailing List Manager
|
||||
</h1>
|
||||
<div class="auth-section">
|
||||
<div class="user-info" id="userInfo">
|
||||
<span>Mailing List Manager</span>
|
||||
</div>
|
||||
<div class="menu-spacer"></div>
|
||||
<div class="user-dropdown" id="userDropdown">
|
||||
<button class="user-dropdown-trigger" id="userDropdownTrigger">
|
||||
<div class="user-avatar">
|
||||
<i class="fas fa-user"></i>
|
||||
</div>
|
||||
<div class="user-details">
|
||||
<span class="user-name" id="currentUsername">User</span>
|
||||
<span class="user-role" id="currentUserRole">role</span>
|
||||
</div>
|
||||
<button class="btn btn-secondary" id="logoutBtn">Logout</button>
|
||||
<i class="fas fa-chevron-down dropdown-arrow"></i>
|
||||
</button>
|
||||
<div class="user-dropdown-menu" id="userDropdownMenu">
|
||||
<div class="dropdown-header">
|
||||
<div class="dropdown-user-info">
|
||||
<div class="dropdown-avatar">
|
||||
<i class="fas fa-user"></i>
|
||||
</div>
|
||||
<div class="dropdown-details">
|
||||
<div class="dropdown-name" id="dropdownUsername">User</div>
|
||||
<div class="dropdown-role" id="dropdownUserRole">role</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="dropdown-divider"></div>
|
||||
<button class="dropdown-item" id="userManagementBtn" style="display: none;">
|
||||
<i class="fas fa-user-shield"></i>
|
||||
<span>User Management</span>
|
||||
</button>
|
||||
<div class="dropdown-divider" id="userManagementDivider" style="display: none;"></div>
|
||||
<button class="dropdown-item" id="logoutBtn">
|
||||
<i class="fas fa-sign-out-alt"></i>
|
||||
<span>Sign Out</span>
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -88,10 +113,6 @@
|
||||
<i class="fas fa-users"></i>
|
||||
Members
|
||||
</button>
|
||||
<button class="tab-btn" data-tab="users" id="usersTab" style="display: none;">
|
||||
<i class="fas fa-user-shield"></i>
|
||||
Users
|
||||
</button>
|
||||
</nav>
|
||||
|
||||
<!-- Notification Area -->
|
||||
@@ -134,13 +155,7 @@
|
||||
<!-- Members Tab -->
|
||||
<div class="tab-content" id="members-tab">
|
||||
<div class="section-header">
|
||||
<div class="header-content">
|
||||
<h2>Members</h2>
|
||||
<div class="header-help">
|
||||
<i class="fas fa-info-circle"></i>
|
||||
<span>Click the "Lists" button next to any member to manage their subscriptions</span>
|
||||
</div>
|
||||
</div>
|
||||
<div class="button-group">
|
||||
<button class="btn btn-primary" id="addMemberBtn">
|
||||
<i class="fas fa-plus"></i>
|
||||
@@ -153,6 +168,26 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Member Search -->
|
||||
<div class="search-section">
|
||||
<div class="search-container">
|
||||
<div class="search-input-wrapper">
|
||||
<i class="fas fa-search search-icon"></i>
|
||||
<input type="text"
|
||||
id="memberSearchInput"
|
||||
class="search-input"
|
||||
placeholder="Search members by name or email..."
|
||||
autocomplete="off">
|
||||
<button class="search-clear" id="memberSearchClear" style="display: none;">
|
||||
<i class="fas fa-times"></i>
|
||||
</button>
|
||||
</div>
|
||||
<div class="search-results-info" id="memberSearchInfo" style="display: none;">
|
||||
<span id="memberSearchCount">0</span> members found
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="data-table">
|
||||
<table class="table" id="membersTable">
|
||||
<thead>
|
||||
@@ -581,6 +616,67 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Bounce History Modal -->
|
||||
<div class="modal" id="bounceHistoryModal">
|
||||
<div class="modal-content">
|
||||
<div class="modal-header">
|
||||
<h3 id="bounceHistoryTitle">Bounce History</h3>
|
||||
<button class="modal-close" id="bounceHistoryModalClose">
|
||||
<i class="fas fa-times"></i>
|
||||
</button>
|
||||
</div>
|
||||
<div class="modal-body">
|
||||
<div class="member-info-banner" id="bounceHistoryMemberInfo">
|
||||
<div class="member-avatar">
|
||||
<i class="fas fa-user"></i>
|
||||
</div>
|
||||
<div class="member-details">
|
||||
<h4 id="bounceHistoryMemberName">Member Name</h4>
|
||||
<p id="bounceHistoryMemberEmail">member@example.com</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="bounce-summary">
|
||||
<div class="bounce-stat">
|
||||
<i class="fas fa-exclamation-triangle"></i>
|
||||
<div>
|
||||
<div class="bounce-stat-label">Total Bounces</div>
|
||||
<div class="bounce-stat-value" id="bounceTotalCount">0</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="bounce-stat">
|
||||
<i class="fas fa-clock"></i>
|
||||
<div>
|
||||
<div class="bounce-stat-label">Last Bounce</div>
|
||||
<div class="bounce-stat-value" id="bounceLastDate">Never</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="bounce-stat">
|
||||
<i class="fas fa-info-circle"></i>
|
||||
<div>
|
||||
<div class="bounce-stat-label">Status</div>
|
||||
<div class="bounce-stat-value" id="bounceStatusText">Clean</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="bounce-history-section">
|
||||
<h5>Bounce Events</h5>
|
||||
<div class="bounce-history-list" id="bounceHistoryList">
|
||||
<!-- Dynamic content will be inserted here -->
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="modal-actions">
|
||||
<button type="button" class="btn btn-secondary" id="bounceHistoryCloseBtn">Close</button>
|
||||
<button type="button" class="btn btn-warning" id="bounceHistoryResetBtn" data-requires-write>
|
||||
<i class="fas fa-redo"></i>
|
||||
Reset Bounce Status
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Confirmation Modal -->
|
||||
<div class="modal" id="confirmModal">
|
||||
<div class="modal-content">
|
||||
@@ -601,7 +697,7 @@
|
||||
</div>
|
||||
|
||||
<script src="static/js/api.js"></script>
|
||||
<script src="static/js/ui.js"></script>
|
||||
<script src="static/js/app.js"></script>
|
||||
<script src="static/js/ui.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -285,7 +285,8 @@ body {
|
||||
}
|
||||
|
||||
/* Header */
|
||||
.header {
|
||||
/* Menu Bar Header */
|
||||
.menu-bar {
|
||||
background: var(--white);
|
||||
border-bottom: 1px solid var(--gray-200);
|
||||
box-shadow: var(--shadow-sm);
|
||||
@@ -294,24 +295,29 @@ body {
|
||||
z-index: 100;
|
||||
}
|
||||
|
||||
.header-content {
|
||||
.menu-bar-content {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
padding: var(--space-4) 0;
|
||||
justify-content: space-between;
|
||||
padding: var(--space-2) var(--space-4);
|
||||
min-height: 56px;
|
||||
}
|
||||
|
||||
.logo {
|
||||
font-size: var(--font-size-xl);
|
||||
font-weight: 700;
|
||||
color: var(--primary-color);
|
||||
.app-title {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: var(--space-2);
|
||||
font-size: var(--font-size-lg);
|
||||
font-weight: 600;
|
||||
color: var(--primary-color);
|
||||
}
|
||||
|
||||
.logo i {
|
||||
font-size: var(--font-size-2xl);
|
||||
.app-title i {
|
||||
font-size: var(--font-size-xl);
|
||||
}
|
||||
|
||||
.menu-spacer {
|
||||
flex: 1;
|
||||
}
|
||||
|
||||
/* Authentication */
|
||||
@@ -342,31 +348,185 @@ body {
|
||||
box-shadow: 0 0 0 3px rgb(37 99 235 / 0.1);
|
||||
}
|
||||
|
||||
.user-info {
|
||||
/* User Dropdown - Menu Bar Style */
|
||||
.user-dropdown {
|
||||
position: relative;
|
||||
}
|
||||
|
||||
.user-dropdown-trigger {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: var(--space-3);
|
||||
gap: var(--space-2);
|
||||
padding: var(--space-2) var(--space-3);
|
||||
background: transparent;
|
||||
border: 1px solid var(--gray-200);
|
||||
border-radius: var(--radius);
|
||||
cursor: pointer;
|
||||
transition: var(--transition);
|
||||
font-size: var(--font-size-sm);
|
||||
height: 40px;
|
||||
position: relative;
|
||||
z-index: 10;
|
||||
pointer-events: auto;
|
||||
}
|
||||
|
||||
.user-dropdown-trigger:hover {
|
||||
background: var(--gray-50);
|
||||
border-color: var(--gray-300);
|
||||
}
|
||||
|
||||
.user-dropdown-trigger.active {
|
||||
background: var(--gray-50);
|
||||
border-color: var(--primary-color);
|
||||
box-shadow: 0 0 0 3px rgb(37 99 235 / 0.1);
|
||||
}
|
||||
|
||||
.user-avatar {
|
||||
width: 24px;
|
||||
height: 24px;
|
||||
background: var(--primary-color);
|
||||
color: var(--white);
|
||||
border-radius: 50%;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
font-size: var(--font-size-xs);
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
.user-details {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
align-items: flex-end;
|
||||
gap: var(--space-1);
|
||||
align-items: flex-start;
|
||||
gap: 1px;
|
||||
min-width: 0;
|
||||
}
|
||||
|
||||
.user-name {
|
||||
font-weight: 500;
|
||||
font-size: var(--font-size-xs);
|
||||
color: var(--gray-900);
|
||||
line-height: 1.2;
|
||||
white-space: nowrap;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
max-width: 100px;
|
||||
}
|
||||
|
||||
.user-role {
|
||||
font-size: 10px;
|
||||
color: var(--gray-500);
|
||||
text-transform: capitalize;
|
||||
line-height: 1;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.dropdown-arrow {
|
||||
color: var(--gray-400);
|
||||
font-size: 10px;
|
||||
transition: var(--transition);
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
.user-dropdown-trigger.active .dropdown-arrow {
|
||||
transform: rotate(180deg);
|
||||
}
|
||||
|
||||
.user-dropdown-menu {
|
||||
position: absolute;
|
||||
top: 100%;
|
||||
right: 0;
|
||||
min-width: 220px;
|
||||
background: var(--white);
|
||||
border: 1px solid var(--gray-200);
|
||||
border-radius: var(--radius);
|
||||
box-shadow: var(--shadow-lg);
|
||||
z-index: 1000;
|
||||
opacity: 0;
|
||||
visibility: hidden;
|
||||
transform: translateY(-8px);
|
||||
transition: all 0.2s ease;
|
||||
margin-top: var(--space-1);
|
||||
}
|
||||
|
||||
.user-dropdown.active .user-dropdown-menu {
|
||||
opacity: 1;
|
||||
visibility: visible;
|
||||
transform: translateY(0);
|
||||
}
|
||||
|
||||
.dropdown-header {
|
||||
padding: var(--space-3);
|
||||
border-bottom: 1px solid var(--gray-100);
|
||||
}
|
||||
|
||||
.dropdown-user-info {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: var(--space-3);
|
||||
}
|
||||
|
||||
.dropdown-avatar {
|
||||
width: 40px;
|
||||
height: 40px;
|
||||
background: var(--primary-color);
|
||||
color: var(--white);
|
||||
border-radius: 50%;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
font-size: var(--font-size-base);
|
||||
}
|
||||
|
||||
.dropdown-details {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: var(--space-1);
|
||||
}
|
||||
|
||||
.dropdown-name {
|
||||
font-weight: 600;
|
||||
font-size: var(--font-size-sm);
|
||||
color: var(--gray-900);
|
||||
}
|
||||
|
||||
.user-role {
|
||||
.dropdown-role {
|
||||
font-size: var(--font-size-xs);
|
||||
color: var(--gray-500);
|
||||
text-transform: capitalize;
|
||||
}
|
||||
|
||||
.dropdown-divider {
|
||||
height: 1px;
|
||||
background: var(--gray-100);
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
.dropdown-item {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: var(--space-3);
|
||||
width: 100%;
|
||||
padding: var(--space-3);
|
||||
background: transparent;
|
||||
border: none;
|
||||
text-align: left;
|
||||
font-size: var(--font-size-sm);
|
||||
color: var(--gray-700);
|
||||
cursor: pointer;
|
||||
transition: var(--transition);
|
||||
}
|
||||
|
||||
.dropdown-item:hover {
|
||||
background: var(--gray-50);
|
||||
color: var(--gray-900);
|
||||
}
|
||||
|
||||
.dropdown-item i {
|
||||
width: 16px;
|
||||
color: var(--gray-400);
|
||||
}
|
||||
|
||||
.status-indicator {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
@@ -523,6 +683,113 @@ body {
|
||||
color: var(--info-color);
|
||||
}
|
||||
|
||||
/* Search Section */
|
||||
.search-section {
|
||||
margin-bottom: var(--space-6);
|
||||
}
|
||||
|
||||
.search-container {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: var(--space-3);
|
||||
}
|
||||
|
||||
.search-input-wrapper {
|
||||
position: relative;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
max-width: 400px;
|
||||
}
|
||||
|
||||
.search-input {
|
||||
width: 100%;
|
||||
padding: var(--space-3) var(--space-4) var(--space-3) var(--space-10);
|
||||
border: 1px solid var(--gray-300);
|
||||
border-radius: var(--radius);
|
||||
font-size: var(--font-size-sm);
|
||||
transition: var(--transition);
|
||||
background: var(--white);
|
||||
}
|
||||
|
||||
.search-input:focus {
|
||||
outline: none;
|
||||
border-color: var(--primary-color);
|
||||
box-shadow: 0 0 0 3px rgb(37 99 235 / 0.1);
|
||||
}
|
||||
|
||||
.search-input:not(:placeholder-shown) {
|
||||
padding-right: var(--space-10);
|
||||
}
|
||||
|
||||
.search-icon {
|
||||
position: absolute;
|
||||
left: var(--space-3);
|
||||
color: var(--gray-400);
|
||||
font-size: var(--font-size-sm);
|
||||
pointer-events: none;
|
||||
z-index: 1;
|
||||
}
|
||||
|
||||
.search-clear {
|
||||
position: absolute;
|
||||
right: var(--space-3);
|
||||
background: transparent;
|
||||
border: none;
|
||||
color: var(--gray-400);
|
||||
cursor: pointer;
|
||||
padding: var(--space-1);
|
||||
border-radius: var(--radius-sm);
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
transition: var(--transition);
|
||||
}
|
||||
|
||||
.search-clear:hover {
|
||||
color: var(--gray-600);
|
||||
background: var(--gray-100);
|
||||
}
|
||||
|
||||
.search-results-info {
|
||||
font-size: var(--font-size-sm);
|
||||
color: var(--gray-600);
|
||||
font-weight: 500;
|
||||
}
|
||||
|
||||
.search-results-info #memberSearchCount {
|
||||
color: var(--primary-color);
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
/* No results message */
|
||||
.no-results {
|
||||
text-align: center;
|
||||
padding: var(--space-8) var(--space-4);
|
||||
color: var(--gray-500);
|
||||
}
|
||||
|
||||
.no-results i {
|
||||
font-size: var(--font-size-3xl);
|
||||
color: var(--gray-300);
|
||||
margin-bottom: var(--space-4);
|
||||
}
|
||||
|
||||
.no-results h3 {
|
||||
font-size: var(--font-size-lg);
|
||||
font-weight: 600;
|
||||
margin-bottom: var(--space-2);
|
||||
color: var(--gray-700);
|
||||
}
|
||||
|
||||
.no-results p {
|
||||
font-size: var(--font-size-sm);
|
||||
margin-bottom: var(--space-4);
|
||||
}
|
||||
|
||||
.no-results .btn {
|
||||
margin-top: var(--space-2);
|
||||
}
|
||||
|
||||
/* Notifications */
|
||||
.notification {
|
||||
display: flex;
|
||||
@@ -624,6 +891,32 @@ body {
|
||||
color: var(--gray-600);
|
||||
}
|
||||
|
||||
/* Bounce badges */
|
||||
.bounce-badge {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: var(--space-1);
|
||||
padding: var(--space-1) var(--space-2);
|
||||
border-radius: var(--radius-sm);
|
||||
font-size: var(--font-size-xs);
|
||||
font-weight: 500;
|
||||
}
|
||||
|
||||
.bounce-badge.bounce-hard {
|
||||
background: #fee2e2;
|
||||
color: #991b1b;
|
||||
}
|
||||
|
||||
.bounce-badge.bounce-soft {
|
||||
background: #fef3c7;
|
||||
color: #92400e;
|
||||
}
|
||||
|
||||
.bounce-badge.bounce-warning {
|
||||
background: #fef9c3;
|
||||
color: #854d0e;
|
||||
}
|
||||
|
||||
/* Role badges */
|
||||
.role-badge {
|
||||
display: inline-flex;
|
||||
@@ -919,12 +1212,36 @@ body {
|
||||
padding: 0 var(--space-3);
|
||||
}
|
||||
|
||||
.header-content {
|
||||
.menu-bar-content {
|
||||
flex-direction: column;
|
||||
gap: var(--space-4);
|
||||
gap: var(--space-2);
|
||||
padding: var(--space-2) var(--space-3);
|
||||
align-items: stretch;
|
||||
}
|
||||
|
||||
.app-title {
|
||||
order: 1;
|
||||
justify-content: center;
|
||||
font-size: var(--font-size-base);
|
||||
}
|
||||
|
||||
.user-dropdown {
|
||||
order: 2;
|
||||
align-self: flex-end;
|
||||
width: auto;
|
||||
}
|
||||
|
||||
.menu-spacer {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.user-dropdown-menu {
|
||||
right: 0;
|
||||
left: auto;
|
||||
width: auto;
|
||||
min-width: 200px;
|
||||
}
|
||||
|
||||
.auth-controls {
|
||||
flex-direction: column;
|
||||
gap: var(--space-2);
|
||||
@@ -944,6 +1261,18 @@ body {
|
||||
align-items: stretch;
|
||||
}
|
||||
|
||||
.search-input-wrapper {
|
||||
max-width: none;
|
||||
}
|
||||
|
||||
.search-input {
|
||||
font-size: 16px; /* Prevent zoom on iOS */
|
||||
}
|
||||
|
||||
.search-container {
|
||||
margin-bottom: var(--space-4);
|
||||
}
|
||||
|
||||
.table-responsive {
|
||||
overflow-x: auto;
|
||||
}
|
||||
@@ -1479,3 +1808,147 @@ body {
|
||||
.fade-in {
|
||||
animation: fadeIn 0.3s ease-out;
|
||||
}
|
||||
/* Bounce History Modal Styles */
|
||||
.bounce-summary {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fit, minmax(150px, 1fr));
|
||||
gap: var(--space-4);
|
||||
margin: var(--space-6) 0;
|
||||
padding: var(--space-4);
|
||||
background: var(--gray-50);
|
||||
border-radius: var(--radius);
|
||||
}
|
||||
|
||||
.bounce-stat {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: var(--space-3);
|
||||
}
|
||||
|
||||
.bounce-stat i {
|
||||
font-size: 1.5rem;
|
||||
color: var(--primary-color);
|
||||
}
|
||||
|
||||
.bounce-stat-label {
|
||||
font-size: var(--font-size-xs);
|
||||
color: var(--gray-500);
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.5px;
|
||||
}
|
||||
|
||||
.bounce-stat-value {
|
||||
font-size: var(--font-size-lg);
|
||||
font-weight: 600;
|
||||
color: var(--gray-800);
|
||||
}
|
||||
|
||||
.bounce-stat-value.text-danger {
|
||||
color: var(--danger-color);
|
||||
}
|
||||
|
||||
.bounce-stat-value.text-warning {
|
||||
color: var(--warning-color);
|
||||
}
|
||||
|
||||
.bounce-stat-value.text-success {
|
||||
color: var(--success-color);
|
||||
}
|
||||
|
||||
.bounce-history-section {
|
||||
margin-top: var(--space-6);
|
||||
}
|
||||
|
||||
.bounce-history-section h5 {
|
||||
margin-bottom: var(--space-4);
|
||||
color: var(--gray-700);
|
||||
}
|
||||
|
||||
.bounce-history-list {
|
||||
max-height: 400px;
|
||||
overflow-y: auto;
|
||||
padding: var(--space-2);
|
||||
}
|
||||
|
||||
.bounce-history-item {
|
||||
background: var(--white);
|
||||
border: 1px solid var(--gray-200);
|
||||
border-radius: var(--radius);
|
||||
padding: var(--space-4);
|
||||
margin-bottom: var(--space-3);
|
||||
}
|
||||
|
||||
.bounce-history-item:last-child {
|
||||
margin-bottom: 0;
|
||||
}
|
||||
|
||||
.bounce-header {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
margin-bottom: var(--space-2);
|
||||
}
|
||||
|
||||
.bounce-type {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: var(--space-2);
|
||||
padding: var(--space-1) var(--space-3);
|
||||
border-radius: var(--radius-sm);
|
||||
font-size: var(--font-size-sm);
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.bounce-type.bounce-type-permanent {
|
||||
background: #fee2e2;
|
||||
color: #991b1b;
|
||||
}
|
||||
|
||||
.bounce-type.bounce-type-transient {
|
||||
background: #fef3c7;
|
||||
color: #92400e;
|
||||
}
|
||||
|
||||
.bounce-type.bounce-type-undetermined {
|
||||
background: var(--gray-100);
|
||||
color: var(--gray-600);
|
||||
}
|
||||
|
||||
.bounce-date {
|
||||
font-size: var(--font-size-sm);
|
||||
color: var(--gray-500);
|
||||
}
|
||||
|
||||
.bounce-subtype {
|
||||
font-size: var(--font-size-sm);
|
||||
color: var(--gray-600);
|
||||
margin-top: var(--space-2);
|
||||
font-weight: 500;
|
||||
}
|
||||
|
||||
.bounce-diagnostic {
|
||||
font-size: var(--font-size-sm);
|
||||
color: var(--gray-500);
|
||||
margin-top: var(--space-2);
|
||||
padding: var(--space-2);
|
||||
background: var(--gray-50);
|
||||
border-radius: var(--radius-sm);
|
||||
border-left: 3px solid var(--warning-color);
|
||||
font-family: 'Courier New', monospace;
|
||||
}
|
||||
|
||||
.empty-state {
|
||||
text-align: center;
|
||||
padding: var(--space-8);
|
||||
color: var(--gray-400);
|
||||
}
|
||||
|
||||
.empty-state i {
|
||||
font-size: 3rem;
|
||||
margin-bottom: var(--space-4);
|
||||
color: var(--success-color);
|
||||
}
|
||||
|
||||
.empty-state p {
|
||||
font-size: var(--font-size-sm);
|
||||
}
|
||||
|
||||
@@ -25,8 +25,8 @@ class APIClient {
|
||||
return `${protocol}//${hostname}:8000`;
|
||||
}
|
||||
|
||||
// If running in production, assume API is on port 8000
|
||||
return `${protocol}//${hostname}:8000`;
|
||||
// If running in production behind a reverse proxy, use /api path
|
||||
return `${protocol}//${hostname}/api`;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -50,9 +50,14 @@ class APIClient {
|
||||
*/
|
||||
async request(endpoint, options = {}) {
|
||||
const url = `${this.baseURL}${endpoint}`;
|
||||
|
||||
// Merge options, with custom headers taking precedence
|
||||
const config = {
|
||||
headers: { ...this.headers },
|
||||
...options
|
||||
...options,
|
||||
headers: {
|
||||
...this.headers,
|
||||
...(options.headers || {})
|
||||
}
|
||||
};
|
||||
|
||||
try {
|
||||
@@ -100,20 +105,26 @@ class APIClient {
|
||||
return this.request('/health');
|
||||
}
|
||||
|
||||
async getConfig() {
|
||||
return this.request('/config');
|
||||
}
|
||||
|
||||
async testAuth() {
|
||||
return this.request('/');
|
||||
}
|
||||
|
||||
// Authentication API
|
||||
async login(username, password) {
|
||||
// Don't include Authorization header for login
|
||||
const tempHeaders = { ...this.headers };
|
||||
delete tempHeaders['Authorization'];
|
||||
|
||||
const response = await this.request('/auth/login', {
|
||||
method: 'POST',
|
||||
body: JSON.stringify({
|
||||
username: username,
|
||||
password: password
|
||||
})
|
||||
headers: tempHeaders,
|
||||
body: JSON.stringify({ username, password })
|
||||
});
|
||||
|
||||
// Set the token from the response
|
||||
if (response.access_token) {
|
||||
this.setToken(response.access_token);
|
||||
}
|
||||
@@ -126,41 +137,16 @@ class APIClient {
|
||||
await this.request('/auth/logout', {
|
||||
method: 'POST'
|
||||
});
|
||||
} catch (error) {
|
||||
// Ignore logout errors, we'll clear the token anyway
|
||||
}
|
||||
} finally {
|
||||
// Clear token even if logout fails
|
||||
this.clearToken();
|
||||
}
|
||||
}
|
||||
|
||||
async getCurrentUser() {
|
||||
return this.request('/auth/me');
|
||||
}
|
||||
|
||||
// User management API
|
||||
async getUsers() {
|
||||
return this.request('/users');
|
||||
}
|
||||
|
||||
async createUser(userData) {
|
||||
return this.request('/users', {
|
||||
method: 'POST',
|
||||
body: JSON.stringify(userData)
|
||||
});
|
||||
}
|
||||
|
||||
async updateUser(userId, userData) {
|
||||
return this.request(`/users/${userId}`, {
|
||||
method: 'PATCH',
|
||||
body: JSON.stringify(userData)
|
||||
});
|
||||
}
|
||||
|
||||
async deleteUser(userId) {
|
||||
return this.request(`/users/${userId}`, {
|
||||
method: 'DELETE'
|
||||
});
|
||||
}
|
||||
|
||||
// Mailing Lists API
|
||||
async getLists() {
|
||||
return this.request('/lists');
|
||||
@@ -254,6 +240,42 @@ class APIClient {
|
||||
})
|
||||
});
|
||||
}
|
||||
|
||||
// User Management API
|
||||
async getUsers() {
|
||||
return this.request('/users');
|
||||
}
|
||||
|
||||
async createUser(userData) {
|
||||
return this.request('/users', {
|
||||
method: 'POST',
|
||||
body: JSON.stringify(userData)
|
||||
});
|
||||
}
|
||||
|
||||
async updateUser(userId, userData) {
|
||||
return this.request(`/users/${userId}`, {
|
||||
method: 'PATCH',
|
||||
body: JSON.stringify(userData)
|
||||
});
|
||||
}
|
||||
|
||||
async deleteUser(userId) {
|
||||
return this.request(`/users/${userId}`, {
|
||||
method: 'DELETE'
|
||||
});
|
||||
}
|
||||
|
||||
// Bounce Management API
|
||||
async getMemberBounces(memberId) {
|
||||
return this.request(`/members/${memberId}/bounces`);
|
||||
}
|
||||
|
||||
async resetBounceStatus(memberId) {
|
||||
return this.request(`/members/${memberId}/bounce-status`, {
|
||||
method: 'PATCH'
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -52,6 +52,33 @@ class MailingListApp {
|
||||
this.logout();
|
||||
});
|
||||
|
||||
// User dropdown functionality
|
||||
const userDropdownTrigger = document.getElementById('userDropdownTrigger');
|
||||
const userDropdown = document.getElementById('userDropdown');
|
||||
|
||||
userDropdownTrigger.addEventListener('click', (e) => {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
userDropdown.classList.toggle('active');
|
||||
userDropdownTrigger.classList.toggle('active');
|
||||
});
|
||||
|
||||
// Close dropdown when clicking outside
|
||||
document.addEventListener('click', (e) => {
|
||||
if (!userDropdown.contains(e.target)) {
|
||||
userDropdown.classList.remove('active');
|
||||
userDropdownTrigger.classList.remove('active');
|
||||
}
|
||||
});
|
||||
|
||||
// Close dropdown on escape key
|
||||
document.addEventListener('keydown', (e) => {
|
||||
if (e.key === 'Escape') {
|
||||
userDropdown.classList.remove('active');
|
||||
userDropdownTrigger.classList.remove('active');
|
||||
}
|
||||
});
|
||||
|
||||
// Bulk import button
|
||||
document.getElementById('showBulkImportBtn').addEventListener('click', () => {
|
||||
uiManager.showBulkImportModal();
|
||||
@@ -61,6 +88,47 @@ class MailingListApp {
|
||||
document.getElementById('addUserBtn').addEventListener('click', () => {
|
||||
uiManager.showUserModal();
|
||||
});
|
||||
|
||||
// User management dropdown item
|
||||
document.getElementById('userManagementBtn').addEventListener('click', () => {
|
||||
// Close the dropdown
|
||||
userDropdown.classList.remove('active');
|
||||
userDropdownTrigger.classList.remove('active');
|
||||
// Switch to users tab
|
||||
this.switchToUsersTab();
|
||||
});
|
||||
|
||||
// Member search functionality
|
||||
const memberSearchInput = document.getElementById('memberSearchInput');
|
||||
const memberSearchClear = document.getElementById('memberSearchClear');
|
||||
|
||||
memberSearchInput.addEventListener('input', (e) => {
|
||||
this.filterMembers(e.target.value);
|
||||
|
||||
// Show/hide clear button
|
||||
if (e.target.value.trim()) {
|
||||
memberSearchClear.style.display = 'flex';
|
||||
} else {
|
||||
memberSearchClear.style.display = 'none';
|
||||
}
|
||||
});
|
||||
|
||||
memberSearchClear.addEventListener('click', () => {
|
||||
memberSearchInput.value = '';
|
||||
memberSearchClear.style.display = 'none';
|
||||
this.filterMembers('');
|
||||
memberSearchInput.focus();
|
||||
});
|
||||
|
||||
// Clear search when switching tabs
|
||||
document.querySelectorAll('.tab-btn').forEach(btn => {
|
||||
btn.addEventListener('click', () => {
|
||||
if (btn.dataset.tab !== 'members') {
|
||||
memberSearchInput.value = '';
|
||||
memberSearchClear.style.display = 'none';
|
||||
}
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -214,10 +282,13 @@ class MailingListApp {
|
||||
if (this.currentUser) {
|
||||
document.getElementById('currentUsername').textContent = this.currentUser.username;
|
||||
document.getElementById('currentUserRole').textContent = this.currentUser.role;
|
||||
document.getElementById('dropdownUsername').textContent = this.currentUser.username;
|
||||
document.getElementById('dropdownUserRole').textContent = this.currentUser.role;
|
||||
|
||||
// Show/hide admin-only features
|
||||
const isAdmin = this.currentUser.role === 'administrator';
|
||||
document.getElementById('usersTab').style.display = isAdmin ? 'block' : 'none';
|
||||
document.getElementById('userManagementBtn').style.display = isAdmin ? 'block' : 'none';
|
||||
document.getElementById('userManagementDivider').style.display = isAdmin ? 'block' : 'none';
|
||||
|
||||
// Show/hide write access features
|
||||
const hasWriteAccess = this.currentUser.role === 'administrator' || this.currentUser.role === 'operator';
|
||||
@@ -250,6 +321,28 @@ class MailingListApp {
|
||||
document.getElementById('showBulkImportBtn').setAttribute('data-requires-write', '');
|
||||
}
|
||||
|
||||
/**
|
||||
* Switch to users tab (triggered from user dropdown)
|
||||
*/
|
||||
switchToUsersTab() {
|
||||
// Switch to users tab programmatically
|
||||
const tabButtons = document.querySelectorAll('.tab-btn');
|
||||
const tabContents = document.querySelectorAll('.tab-content');
|
||||
|
||||
// Remove active class from all tabs and contents
|
||||
tabButtons.forEach(btn => btn.classList.remove('active'));
|
||||
tabContents.forEach(content => content.classList.remove('active'));
|
||||
|
||||
// Show users tab content
|
||||
const usersTab = document.getElementById('users-tab');
|
||||
if (usersTab) {
|
||||
usersTab.classList.add('active');
|
||||
}
|
||||
|
||||
// Load users data if needed
|
||||
this.loadUsers();
|
||||
}
|
||||
|
||||
/**
|
||||
* Load all data from API
|
||||
*/
|
||||
@@ -259,12 +352,15 @@ class MailingListApp {
|
||||
try {
|
||||
uiManager.setLoading(true);
|
||||
|
||||
// Load lists and members in parallel
|
||||
const [lists, members] = await Promise.all([
|
||||
// Load configuration, lists and members in parallel
|
||||
const [config, lists, members] = await Promise.all([
|
||||
apiClient.getConfig(),
|
||||
apiClient.getLists(),
|
||||
apiClient.getMembers()
|
||||
]);
|
||||
|
||||
this.config = config;
|
||||
|
||||
this.lists = lists;
|
||||
this.members = members;
|
||||
|
||||
@@ -438,6 +534,7 @@ class MailingListApp {
|
||||
row.innerHTML = `
|
||||
<td>
|
||||
<div class="font-medium">${uiManager.escapeHtml(member.name)}</div>
|
||||
${this.config?.bounce_handling_enabled && member.bounce_count > 0 ? `<div class="text-xs text-muted" style="margin-top: 2px;"></div>` : ''}
|
||||
</td>
|
||||
<td>
|
||||
<a href="mailto:${member.email}" style="color: var(--primary-color)">
|
||||
@@ -458,6 +555,15 @@ class MailingListApp {
|
||||
</td>
|
||||
`;
|
||||
|
||||
// Add bounce badge if member has bounces (only if bounce handling is enabled)
|
||||
if (this.config?.bounce_handling_enabled && member.bounce_count > 0) {
|
||||
const bounceInfoDiv = row.cells[0].querySelector('.text-xs');
|
||||
const bounceBadge = uiManager.createBounceStatusBadge(member.bounce_status, member.bounce_count);
|
||||
if (bounceBadge) {
|
||||
bounceInfoDiv.appendChild(bounceBadge);
|
||||
}
|
||||
}
|
||||
|
||||
// Add status badge
|
||||
const statusCell = row.cells[3];
|
||||
statusCell.appendChild(uiManager.createStatusBadge(member.active));
|
||||
@@ -474,6 +580,18 @@ class MailingListApp {
|
||||
uiManager.showMemberSubscriptionsModal(member);
|
||||
});
|
||||
|
||||
// Create Bounces button (show if bounce handling is enabled and member has bounces or for admins/operators)
|
||||
if (this.config?.bounce_handling_enabled && (member.bounce_count > 0 || hasWriteAccess)) {
|
||||
const bouncesBtn = document.createElement('button');
|
||||
bouncesBtn.className = `btn btn-sm ${member.bounce_count > 0 ? 'btn-warning' : 'btn-secondary'}`;
|
||||
bouncesBtn.innerHTML = `<i class="fas fa-exclamation-triangle"></i> Bounces${member.bounce_count > 0 ? ` (${member.bounce_count})` : ''}`;
|
||||
bouncesBtn.title = 'View Bounce History';
|
||||
bouncesBtn.addEventListener('click', () => {
|
||||
uiManager.showBounceHistoryModal(member);
|
||||
});
|
||||
actionsCell.appendChild(bouncesBtn);
|
||||
}
|
||||
|
||||
// Only show edit/delete buttons for users with write access
|
||||
if (hasWriteAccess) {
|
||||
const editBtn = uiManager.createActionButton('Edit', 'edit', 'btn-secondary', () => {
|
||||
@@ -645,6 +763,149 @@ class MailingListApp {
|
||||
uiManager.setLoading(false);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Filter members by search term (name or email)
|
||||
*/
|
||||
filterMembers(searchTerm) {
|
||||
const searchInfo = document.getElementById('memberSearchInfo');
|
||||
const searchCount = document.getElementById('memberSearchCount');
|
||||
const tbody = document.getElementById('membersTableBody');
|
||||
|
||||
if (!this.members || !Array.isArray(this.members)) {
|
||||
return;
|
||||
}
|
||||
|
||||
// If no search term, show all members
|
||||
if (!searchTerm || searchTerm.trim() === '') {
|
||||
this.renderMembers();
|
||||
searchInfo.style.display = 'none';
|
||||
return;
|
||||
}
|
||||
|
||||
const normalizedSearch = searchTerm.trim().toLowerCase();
|
||||
|
||||
// Filter members by name or email
|
||||
const filteredMembers = this.members.filter(member => {
|
||||
const name = (member.name || '').toLowerCase();
|
||||
const email = (member.email || '').toLowerCase();
|
||||
return name.includes(normalizedSearch) || email.includes(normalizedSearch);
|
||||
});
|
||||
|
||||
// Update search results info
|
||||
searchCount.textContent = filteredMembers.length;
|
||||
searchInfo.style.display = 'block';
|
||||
|
||||
// Render filtered results
|
||||
this.renderFilteredMembers(filteredMembers);
|
||||
}
|
||||
|
||||
/**
|
||||
* Render filtered members (similar to renderMembers but with filtered data)
|
||||
*/
|
||||
renderFilteredMembers(filteredMembers) {
|
||||
const tbody = document.getElementById('membersTableBody');
|
||||
tbody.innerHTML = '';
|
||||
|
||||
if (filteredMembers.length === 0) {
|
||||
// Show no results message
|
||||
const row = tbody.insertRow();
|
||||
const cell = row.insertCell();
|
||||
cell.colSpan = 5;
|
||||
cell.className = 'no-results';
|
||||
cell.innerHTML = `
|
||||
<i class="fas fa-search"></i>
|
||||
<h3>No members found</h3>
|
||||
<p>Try adjusting your search terms or check the spelling.</p>
|
||||
`;
|
||||
return;
|
||||
}
|
||||
|
||||
filteredMembers.forEach(member => {
|
||||
const row = tbody.insertRow();
|
||||
|
||||
// Name
|
||||
const nameCell = row.insertCell();
|
||||
nameCell.textContent = member.name;
|
||||
|
||||
// Email
|
||||
const emailCell = row.insertCell();
|
||||
const emailLink = document.createElement('a');
|
||||
emailLink.href = `mailto:${member.email}`;
|
||||
emailLink.textContent = member.email;
|
||||
emailLink.style.color = 'var(--primary-color)';
|
||||
emailCell.appendChild(emailLink);
|
||||
|
||||
// Lists (show member's subscriptions)
|
||||
const listsCell = row.insertCell();
|
||||
const memberLists = [];
|
||||
this.subscriptions.forEach((members, listId) => {
|
||||
if (members.some(m => m.member_id === member.member_id)) {
|
||||
const list = this.lists.find(l => l.list_id === listId);
|
||||
if (list) {
|
||||
memberLists.push(list.list_name);
|
||||
}
|
||||
}
|
||||
});
|
||||
listsCell.textContent = memberLists.length > 0 ? memberLists.join(', ') : 'None';
|
||||
|
||||
// Status
|
||||
const statusCell = row.insertCell();
|
||||
const statusBadge = document.createElement('span');
|
||||
statusBadge.className = `status-badge ${member.active ? 'active' : 'inactive'}`;
|
||||
statusBadge.innerHTML = `<i class="fas fa-${member.active ? 'check' : 'times'}"></i> ${member.active ? 'Active' : 'Inactive'}`;
|
||||
|
||||
// Add bounce status if exists and bounce handling is enabled
|
||||
if (this.config?.bounce_handling_enabled && member.bounce_status && member.bounce_status !== 'clean') {
|
||||
const bounceIndicator = document.createElement('span');
|
||||
bounceIndicator.className = `bounce-badge bounce-${member.bounce_status === 'hard_bounce' ? 'hard' : 'soft'}`;
|
||||
bounceIndicator.innerHTML = `<i class="fas fa-exclamation-triangle"></i> Bounces`;
|
||||
bounceIndicator.title = `${member.bounce_count || 0} bounces - ${member.bounce_status}`;
|
||||
statusCell.appendChild(document.createElement('br'));
|
||||
statusCell.appendChild(bounceIndicator);
|
||||
}
|
||||
|
||||
statusCell.appendChild(statusBadge);
|
||||
|
||||
// Actions
|
||||
const actionsCell = row.insertCell();
|
||||
actionsCell.className = 'action-buttons';
|
||||
|
||||
// Bounces button (if bounce handling is enabled and member has bounce data)
|
||||
if (this.config?.bounce_handling_enabled && member.bounce_count > 0) {
|
||||
const bouncesBtn = uiManager.createActionButton('Bounces', 'exclamation-triangle', 'btn-warning', () => {
|
||||
uiManager.showBounceHistory(member);
|
||||
});
|
||||
actionsCell.appendChild(bouncesBtn);
|
||||
}
|
||||
|
||||
const subscriptionsBtn = uiManager.createActionButton('Lists', 'list', 'btn-secondary', () => {
|
||||
uiManager.showMemberSubscriptions(member);
|
||||
});
|
||||
actionsCell.appendChild(subscriptionsBtn);
|
||||
|
||||
const editBtn = uiManager.createActionButton('Edit', 'edit', 'btn-primary', () => {
|
||||
uiManager.showMemberModal(member);
|
||||
});
|
||||
editBtn.setAttribute('data-requires-write', '');
|
||||
actionsCell.appendChild(editBtn);
|
||||
|
||||
const deleteBtn = uiManager.createActionButton('Delete', 'trash', 'btn-danger', () => {
|
||||
uiManager.showConfirmation(
|
||||
`Are you sure you want to delete member "${member.name}"? This will also remove them from all mailing lists.`,
|
||||
async () => {
|
||||
await this.deleteMember(member.member_id);
|
||||
}
|
||||
);
|
||||
});
|
||||
deleteBtn.setAttribute('data-requires-write', '');
|
||||
actionsCell.appendChild(deleteBtn);
|
||||
});
|
||||
|
||||
// Update UI permissions for the filtered results
|
||||
const hasWriteAccess = this.currentUser && (this.currentUser.role === 'administrator' || this.currentUser.role === 'operator');
|
||||
this.updateUIForPermissions(hasWriteAccess);
|
||||
}
|
||||
}
|
||||
|
||||
// Initialize the application when DOM is loaded
|
||||
|
||||
@@ -76,6 +76,19 @@ class UIManager {
|
||||
this.handleMemberSubscriptionsSave();
|
||||
});
|
||||
|
||||
// Bounce history modal
|
||||
document.getElementById('bounceHistoryModalClose').addEventListener('click', () => {
|
||||
this.closeModal(document.getElementById('bounceHistoryModal'));
|
||||
});
|
||||
|
||||
document.getElementById('bounceHistoryCloseBtn').addEventListener('click', () => {
|
||||
this.closeModal(document.getElementById('bounceHistoryModal'));
|
||||
});
|
||||
|
||||
document.getElementById('bounceHistoryResetBtn').addEventListener('click', () => {
|
||||
this.handleBounceStatusReset();
|
||||
});
|
||||
|
||||
// Form submissions
|
||||
document.getElementById('listForm').addEventListener('submit', (e) => {
|
||||
e.preventDefault();
|
||||
@@ -110,6 +123,19 @@ class UIManager {
|
||||
this.confirmCallback = null;
|
||||
this.closeModal(document.getElementById('confirmModal'));
|
||||
});
|
||||
|
||||
// Bounce history modal
|
||||
document.getElementById('bounceHistoryModalClose').addEventListener('click', () => {
|
||||
this.closeModal(document.getElementById('bounceHistoryModal'));
|
||||
});
|
||||
|
||||
document.getElementById('bounceHistoryCloseBtn').addEventListener('click', () => {
|
||||
this.closeModal(document.getElementById('bounceHistoryModal'));
|
||||
});
|
||||
|
||||
document.getElementById('bounceHistoryResetBtn').addEventListener('click', () => {
|
||||
this.handleResetBounceStatus();
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -664,6 +690,117 @@ class UIManager {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Show bounce history modal for a member
|
||||
*/
|
||||
async showBounceHistoryModal(member) {
|
||||
const modal = document.getElementById('bounceHistoryModal');
|
||||
|
||||
// Update member info
|
||||
document.getElementById('bounceHistoryTitle').textContent = `Bounce History - ${member.name}`;
|
||||
document.getElementById('bounceHistoryMemberName').textContent = member.name;
|
||||
document.getElementById('bounceHistoryMemberEmail').textContent = member.email;
|
||||
|
||||
// Update summary stats
|
||||
document.getElementById('bounceTotalCount').textContent = member.bounce_count || 0;
|
||||
document.getElementById('bounceLastDate').textContent = this.formatDateTime(member.last_bounce_at);
|
||||
|
||||
const statusText = document.getElementById('bounceStatusText');
|
||||
statusText.className = 'bounce-stat-value';
|
||||
if (member.bounce_status === 'hard_bounce') {
|
||||
statusText.textContent = 'Hard Bounce';
|
||||
statusText.classList.add('text-danger');
|
||||
} else if (member.bounce_status === 'soft_bounce') {
|
||||
statusText.textContent = 'Soft Bounce';
|
||||
statusText.classList.add('text-warning');
|
||||
} else {
|
||||
statusText.textContent = 'Clean';
|
||||
statusText.classList.add('text-success');
|
||||
}
|
||||
|
||||
try {
|
||||
this.currentMemberForBounces = member;
|
||||
|
||||
// Load bounce history
|
||||
const bounces = await apiClient.getMemberBounces(member.member_id);
|
||||
this.renderBounceHistory(bounces);
|
||||
|
||||
this.showModal(modal);
|
||||
} catch (error) {
|
||||
this.handleError(error, 'Failed to load bounce history');
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Render bounce history list
|
||||
*/
|
||||
renderBounceHistory(bounces) {
|
||||
const container = document.getElementById('bounceHistoryList');
|
||||
container.innerHTML = '';
|
||||
|
||||
if (bounces.length === 0) {
|
||||
container.innerHTML = `
|
||||
<div class="empty-state">
|
||||
<i class="fas fa-check-circle"></i>
|
||||
<p>No bounces recorded for this member</p>
|
||||
</div>
|
||||
`;
|
||||
return;
|
||||
}
|
||||
|
||||
bounces.forEach(bounce => {
|
||||
const item = document.createElement('div');
|
||||
item.className = 'bounce-history-item';
|
||||
|
||||
let typeClass = 'bounce-type-';
|
||||
if (bounce.bounce_type === 'Permanent') {
|
||||
typeClass += 'permanent';
|
||||
} else if (bounce.bounce_type === 'Transient') {
|
||||
typeClass += 'transient';
|
||||
} else {
|
||||
typeClass += 'undetermined';
|
||||
}
|
||||
|
||||
item.innerHTML = `
|
||||
<div class="bounce-header">
|
||||
<span class="bounce-type ${typeClass}">
|
||||
<i class="fas fa-${bounce.bounce_type === 'Permanent' ? 'times-circle' : bounce.bounce_type === 'Transient' ? 'exclamation-circle' : 'question-circle'}"></i>
|
||||
${bounce.bounce_type}
|
||||
</span>
|
||||
<span class="bounce-date">${this.formatDateTime(bounce.timestamp)}</span>
|
||||
</div>
|
||||
${bounce.bounce_subtype ? `<div class="bounce-subtype">Subtype: ${this.escapeHtml(bounce.bounce_subtype)}</div>` : ''}
|
||||
${bounce.diagnostic_code ? `<div class="bounce-diagnostic">${this.escapeHtml(bounce.diagnostic_code)}</div>` : ''}
|
||||
`;
|
||||
|
||||
container.appendChild(item);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle bounce status reset
|
||||
*/
|
||||
async handleResetBounceStatus() {
|
||||
if (!this.currentMemberForBounces) return;
|
||||
|
||||
this.showConfirmation(
|
||||
`Are you sure you want to reset the bounce status for "${this.currentMemberForBounces.name}"? This will clear the bounce count and allow emails to be sent to this address again.`,
|
||||
async () => {
|
||||
try {
|
||||
this.setLoading(true);
|
||||
await apiClient.resetBounceStatus(this.currentMemberForBounces.member_id);
|
||||
this.showNotification('Bounce status reset successfully', 'success');
|
||||
this.closeModal(document.getElementById('bounceHistoryModal'));
|
||||
await window.app.loadData();
|
||||
} catch (error) {
|
||||
this.handleError(error, 'Failed to reset bounce status');
|
||||
} finally {
|
||||
this.setLoading(false);
|
||||
}
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle API errors
|
||||
*/
|
||||
@@ -715,6 +852,49 @@ class UIManager {
|
||||
return badge;
|
||||
}
|
||||
|
||||
/**
|
||||
* Create bounce status badge
|
||||
*/
|
||||
createBounceStatusBadge(bounceStatus, bounceCount) {
|
||||
const badge = document.createElement('span');
|
||||
|
||||
if (bounceStatus === 'hard_bounce') {
|
||||
badge.className = 'bounce-badge bounce-hard';
|
||||
badge.innerHTML = `<i class="fas fa-exclamation-triangle"></i> Hard Bounce`;
|
||||
badge.title = `${bounceCount} bounce(s) - Email permanently failed`;
|
||||
} else if (bounceStatus === 'soft_bounce') {
|
||||
badge.className = 'bounce-badge bounce-soft';
|
||||
badge.innerHTML = `<i class="fas fa-exclamation-circle"></i> Soft Bounce`;
|
||||
badge.title = `${bounceCount} bounce(s) - Temporary delivery issues`;
|
||||
} else if (bounceCount > 0) {
|
||||
badge.className = 'bounce-badge bounce-warning';
|
||||
badge.innerHTML = `<i class="fas fa-info-circle"></i> ${bounceCount} bounce(s)`;
|
||||
badge.title = `${bounceCount} bounce(s) recorded`;
|
||||
} else {
|
||||
return null; // No badge for clean status
|
||||
}
|
||||
|
||||
return badge;
|
||||
}
|
||||
|
||||
/**
|
||||
* Format date and time
|
||||
*/
|
||||
formatDateTime(dateString) {
|
||||
if (!dateString) return 'Never';
|
||||
const date = new Date(dateString);
|
||||
return date.toLocaleString();
|
||||
}
|
||||
|
||||
/**
|
||||
* Format date only
|
||||
*/
|
||||
formatDate(dateString) {
|
||||
if (!dateString) return 'Never';
|
||||
const date = new Date(dateString);
|
||||
return date.toLocaleDateString();
|
||||
}
|
||||
|
||||
/**
|
||||
* Format email as mailto link
|
||||
*/
|
||||
|
||||
Reference in New Issue
Block a user