Compare commits

...

12 Commits

Author SHA1 Message Date
James Pattinson
f3d7592e7d Email bounce handling with postfix 2025-10-14 16:16:44 +00:00
James Pattinson
12a82c8d03 Ability to disable SNS bounce handling 2025-10-14 15:39:33 +00:00
James Pattinson
b34ea2ed84 Dynamic search 2025-10-13 20:05:08 +00:00
James Pattinson
8fd951fd1f FIx menu bar 2025-10-13 19:58:46 +00:00
James Pattinson
ecbc38cf8e Cleanuop and CORS fixing 2025-10-13 19:30:19 +00:00
James Pattinson
d37027ee5a Fixed bounce handling 2025-10-13 16:53:22 +00:00
James Pattinson
72f3297a80 SES SNS Bounce Handling 2025-10-13 15:05:42 +00:00
James Pattinson
ac23638125 Failed login feedback 2025-10-13 14:23:17 +00:00
James Pattinson
4fc1ed96dd Failed login feedback 2025-10-13 14:23:02 +00:00
James Pattinson
459e16b26f Schema correction 2025-10-13 14:12:33 +00:00
James Pattinson
d4b88e0952 RBAC and Doc updates 2025-10-13 14:05:01 +00:00
James Pattinson
f721be7280 CSV Import 2025-10-13 13:28:12 +00:00
32 changed files with 6844 additions and 281 deletions

View File

@@ -19,4 +19,27 @@ MYSQL_PASSWORD=change_this_password
MYSQL_ROOT_PASSWORD=change_this_root_password MYSQL_ROOT_PASSWORD=change_this_root_password
# API Configuration # API Configuration
API_TOKEN=change_this_to_a_secure_random_token API_TOKEN=change_this_to_a_secure_random_token
# Bounce Handling Configuration (Optional)
# Choose one of the bounce handling methods below:
# Method 1: SNS Webhook Bounce Handling (Requires SES Production Access)
# Set to 'true' to enable real-time SNS webhook bounce handling
ENABLE_SNS_WEBHOOKS=false
ENABLE_BOUNCE_HANDLING=false
# If ENABLE_SNS_WEBHOOKS=true, you need:
# 1. AWS SNS topic configured
# 2. SES configured to send notifications to SNS topic
# 3. Valid HTTPS domain for webhook endpoint
# 4. SNS subscription confirmed to your webhook endpoint
# Method 2: Email-Based Bounce Processing (Works with SES Sandbox)
# Set to 'true' to enable email-based bounce processing
# This processes bounce emails that are sent to bounces@lists.sasalliance.org
ENABLE_EMAIL_BOUNCE_PROCESSING=false
# Note: You can enable both methods, but typically only one is needed
# Email-based processing works with SES sandbox accounts
# SNS webhooks provide real-time processing but require SES production access

View File

@@ -1,97 +1,169 @@
# Copilot Instructions: Mail List Manager (Postfix + SES + Web Interface) # Copilot Instructions: Mail List Manager (Postfix + SES + MySQL + API + Web)
## Architecture Overview ## Architecture Overview
This is a containerized mailing list management system built around Postfix as an SMTP relay through Amazon SES. **Currently in Phase 1** with static configuration; **planned expansion** includes web frontend and SQL database for dynamic member management. This is a complete containerized mailing list management system with four services:
**Current Components (Phase 1):** 1. **Web Frontend** (Port 3000) - Nginx serving modern HTML/CSS/JS interface
- `docker-compose.yaml`: Single-service orchestration with SES credentials 2. **REST API** (Port 8000) - FastAPI backend with token authentication
- `postfix/`: Complete Postfix container configuration 3. **MySQL Database** (Internal) - Stores lists, members, and subscriptions
- Static virtual aliases system for mailing list distribution 4. **Postfix Mail Server** (Port 25) - SMTP server with native MySQL integration
**Planned Architecture (Phase 2+):** **Key Architecture Principles:**
- Web frontend for list management (view/add/remove members) - Real-time operation: Changes take effect immediately without Postfix reload
- SQL database backend for member storage - Native integration: Postfix queries MySQL directly (no Python scripts)
- Dynamic Postfix configuration generation - Security-first: Private Docker network, token auth, sender whitelist
- Multi-service Docker Compose setup - User-friendly: Web interface for easy management by non-technical users
## Configuration Pattern ## Configuration Pattern
**Current (Static):** Environment variable substitution for secure credential management: **Dynamic Database-Driven Architecture:**
1. **Credentials Flow**: SES credentials passed via environment → `sasl_passwd.template` → runtime generation in `entrypoint.sh` 1. **Data Flow**: Web UI → REST API → MySQL ← Postfix (real-time queries)
2. **Config Files**: Static configs (`main.cf`, `virtual_aliases.cf`) + dynamic SASL auth file 2. **Credentials**: All sensitive data in `.env` file (SES, MySQL, API_TOKEN)
3. **Postfix Maps**: Hash databases generated at build time (virtual aliases) and runtime (SASL) 3. **Mail Processing**:
- Email arrives for `list@lists.sasalliance.org`
- Postfix queries MySQL via `mysql_virtual_alias_maps.cf`
- MySQL returns comma-separated member emails
- Postfix expands alias and delivers via SES
4. **Security**: Private Docker network (maillist-internal), sender whitelist (@sasalliance.org)
**Future (Dynamic):** Database-driven configuration: **No Static Configuration:**
- Member lists stored in SQL database, with members able to join multiple lists - Virtual aliases are stored in MySQL database, not config files
- Web interface for CRUD operations on members - Changes take effect immediately without container restarts
- `virtual_aliases.cf` generated from database at runtime - Postfix caches query results for performance
- Postfix reload triggered by configuration changes
## Key Files and Their Roles ## Key Files and Their Roles
- `main.cf`: Core Postfix config - relay through SES, domain settings, security ### Docker & Environment
- `sasl_passwd.template`: Template for SES authentication (uses `${SES_USER}:${SES_PASS}`) - `docker-compose.yaml`: Multi-service orchestration (mysql, postfix, api, web)
- `virtual_aliases.cf`: Static email forwarding rules (one mailing list currently) - `.env`: All credentials and configuration (SES, MySQL, API_TOKEN)
- `entrypoint.sh`: Runtime credential processing and Postfix startup
### Web Frontend (`web/`)
- `index.html`: Single-page application interface
- `static/js/api.js`: API client with authentication
- `static/js/ui.js`: UI helpers and modal management
- `static/js/app.js`: Main application controller
- `static/css/style.css`: Complete styling system
- `nginx.conf`: Nginx configuration for static file serving
### REST API (`api/`)
- `main.py`: FastAPI application with all CRUD endpoints
- `requirements.txt`: Python dependencies (fastapi, mysql-connector-python, etc.)
- `Dockerfile`: Python 3.11 container build
### Database (`database/`)
- `schema.sql`: Database initialization with tables and sample data
- Tables: `lists`, `members`, `list_members` (many-to-many junction)
### Postfix (`postfix/`)
- `main.cf.template`: Core Postfix config with MySQL virtual alias maps
- `mysql_virtual_alias_maps.cf`: Native Postfix MySQL query configuration
- `sender_access`: Sender whitelist (sasalliance.org domain allowed)
- `sasl_passwd.template`: SES authentication template
- `entrypoint.sh`: Container startup with MySQL health check and config generation
## Development Workflows ## Development Workflows
**Building and Running:** **Building and Running:**
Always use sudo for Docker commands. Don't bother opening the simple browser UI for Docker as it is not very useful.
```bash ```bash
docker-compose up --build # Build and start mail server sudo docker-compose up --build # Build and start all services
docker-compose logs -f # Monitor mail delivery logs sudo docker-compose logs -f # Monitor all service logs
sudo docker-compose logs -f api # Monitor specific service
``` ```
**Adding Mailing Lists (Current):** **Managing Lists and Members:**
1. Edit `virtual_aliases.cf` with new list → recipients mapping
2. Rebuild container (postmap runs at build time)
3. No restart needed for existing virtual alias changes
**Future Web Interface Workflow:** **Via Web Interface (Recommended):**
1. Access web frontend at configured port 1. Open http://localhost:3000
2. Use CRUD interface to manage mailing lists and members 2. Enter API_TOKEN from .env file
3. Changes automatically update database and regenerate Postfix configs 3. Use intuitive interface to manage lists, members, and subscriptions
4. Click "Subscriptions" button on any member for toggle-based subscription management
**Via REST API:**
```bash
# Get all lists
curl -H "Authorization: Bearer $API_TOKEN" http://localhost:8000/lists
# Create new list
curl -X POST http://localhost:8000/lists \
-H "Authorization: Bearer $API_TOKEN" \
-H "Content-Type: application/json" \
-d '{"list_name":"Test","list_email":"test@lists.sasalliance.org","active":true}'
# Subscribe member to list
curl -X POST http://localhost:8000/subscriptions \
-H "Authorization: Bearer $API_TOKEN" \
-H "Content-Type: application/json" \
-d '{"list_email":"test@lists.sasalliance.org","member_email":"user@example.com"}'
```
**Via Direct Database Access:**
```bash
docker-compose exec mysql mysql -u maillist -p maillist
# See database/README.md for SQL examples
```
**Testing Mail Delivery:** **Testing Mail Delivery:**
```bash ```bash
# From inside container # From inside container
docker-compose exec postfix bash
echo "Test message" | mail -s "Subject" community@lists.sasalliance.org echo "Test message" | mail -s "Subject" community@lists.sasalliance.org
# Check logs for SES relay status # Check logs for SES relay status
docker-compose logs postfix | grep -E "(sent|bounced|deferred)" docker-compose logs postfix | grep -E "(sent|bounced|deferred)"
# Verify MySQL query works
docker-compose exec postfix postmap -q "community@lists.sasalliance.org" \
mysql:/etc/postfix/mysql_virtual_alias_maps.cf
``` ```
## Security Considerations ## Security Considerations
- SES credentials exposed in docker-compose.yaml (consider using Docker secrets) - All credentials stored in `.env` file (git-ignored)
- MySQL on private Docker network (no host port mapping)
- API requires Bearer token authentication
- Sender whitelist restricts who can send to lists (@sasalliance.org only)
- SASL password file permissions set to 600 in entrypoint - SASL password file permissions set to 600 in entrypoint
- TLS encryption enforced for SES relay (`smtp_tls_security_level = encrypt`) - TLS encryption enforced for SES relay (`smtp_tls_security_level = encrypt`)
- Only localhost and configured hostname accepted for local delivery - Web interface requires token for all operations
## Configuration Conventions ## Configuration Conventions
- **Hostname Pattern**: `lists.sasalliance.org` for mailing lists, origin domain `sasalliance.org` - **Hostname Pattern**: `lists.sasalliance.org` for mailing lists, origin domain `sasalliance.org`
- **Virtual Aliases**: Simple format `listname@lists.domain → recipient1, recipient2` - **Virtual Aliases**: Dynamically queried from MySQL database in real-time
- **SES Region**: EU West 2 (`email-smtp.eu-west-2.amazonaws.com`) - **SES Region**: EU West 2 (`email-smtp.eu-west-2.amazonaws.com`)
- **Port Mapping**: Standard SMTP port 25 exposed to host - **Port Mapping**: Standard SMTP port 25 exposed to host
- **Database Tables**: `lists`, `members`, `list_members` (many-to-many design)
- **API Port**: 8000 for REST API
- **Web Port**: 3000 for web interface
## Common Modifications ## Common Modifications
**Adding Recipients to Existing List (Current):** **Adding Recipients to a List:**
Edit `virtual_aliases.cf`, add comma-separated emails, rebuild container. Use web interface or API - changes are immediate!
**New Mailing List (Current):** **Creating New Mailing List:**
Add line to `virtual_aliases.cf`: `newlist@lists.sasalliance.org recipients...` 1. Via Web: Click "Add List" button, fill in form
2. Via API: POST to /lists endpoint with list details
3. Via MySQL: INSERT into lists table
**Adding New Member:**
1. Via Web: Click "Add Member" button in Members tab
2. Via API: POST to /members endpoint
3. Via MySQL: INSERT into members table
**Managing Subscriptions:**
1. Via Web: Click "Subscriptions" button on member, toggle lists
2. Via API: POST/DELETE to /subscriptions endpoint
3. Via MySQL: INSERT/DELETE in list_members table
**Credential Updates:** **Credential Updates:**
Update `SES_USER`/`SES_PASS` in docker-compose.yaml, restart container. 1. Edit `.env` file with new credentials
2. Restart affected services:
## Migration Considerations - SES: `docker-compose restart postfix`
- MySQL: `docker-compose restart mysql`
When implementing the web frontend and database backend: - API: `docker-compose restart api`
- Preserve existing `community@lists.sasalliance.org` list during migration
- Consider migration script to import current virtual aliases into database
- Plan for zero-downtime deployment with database-driven config generation
- Web interface should validate email addresses and handle duplicate members

277
BOUNCE_HANDLING_SETUP.md Normal file
View File

@@ -0,0 +1,277 @@
# SES SNS Bounce Handling Setup (Optional)
**⚠️ NOTICE: Bounce handling is optional and disabled by default.**
This document describes how to configure AWS SES and SNS to handle email bounces automatically in the Mail List Manager.
**Prerequisites:**
- SES production access (not available in sandbox mode)
- Valid HTTPS domain for webhook endpoint
- Bounce handling must be enabled in configuration
## Overview
The system can optionally use AWS Simple Notification Service (SNS) to receive real-time bounce notifications from AWS Simple Email Service (SES). When bounce handling is enabled and an email bounces:
1. SES sends a notification to an SNS topic
2. SNS forwards the notification to your webhook endpoint
3. The API processes the notification and updates the database
4. Members with hard bounces are automatically deactivated
5. Bounce history is tracked and displayed in the UI
## Bounce Status Types
- **Clean**: No bounces recorded
- **Soft Bounce**: Temporary delivery issues (e.g., mailbox full, temporary server issues)
- After 3 soft bounces, the member is marked with soft bounce status
- **Hard Bounce**: Permanent delivery failure (e.g., invalid email address, domain doesn't exist)
- Member is automatically deactivated and cannot receive emails
## Setup Instructions
### 1. Enable Bounce Handling
First, enable bounce handling in your `.env` file:
```bash
# Enable SNS webhook bounce handling
ENABLE_SNS_WEBHOOKS=true
ENABLE_BOUNCE_HANDLING=true
```
Restart the API container after making this change:
```bash
sudo docker-compose restart api
```
### 2. Prerequisites
- AWS account with SES configured and verified
- Your Mail List Manager deployed and accessible via HTTPS (required for SNS webhook)
- Domain or subdomain for webhook (e.g., `https://lists.yourdomain.com`)
### 3. Create SNS Topic
1. Log in to AWS Console and navigate to SNS
2. Click "Create topic"
3. Choose "Standard" topic type
4. Name: `ses-bounce-notifications` (or your preferred name)
5. Display name: `SES Bounce Notifications`
6. Click "Create topic"
7. **Save the Topic ARN** (you'll need it in step 4) arn:aws:sns:eu-west-2:827164363113:ses-bounces
### 4. Subscribe Your Webhook to SNS Topic
1. In the SNS topic details, click "Create subscription"
2. Protocol: `HTTPS`
3. Endpoint: `https://yourdomain.com:8000/webhooks/sns`
- Replace `yourdomain.com` with your actual domain
- The API must be accessible via HTTPS (SNS doesn't support HTTP)
4. Enable raw message delivery: **Unchecked**
5. Click "Create subscription"
6. The subscription will be in "PendingConfirmation" status
### 5. Confirm SNS Subscription
When you create the subscription, SNS will send a `SubscriptionConfirmation` request to your webhook endpoint. The Mail List Manager API automatically confirms this subscription.
1. Check your API logs:
```bash
sudo docker-compose logs -f api
```
2. You should see a log entry indicating the subscription was confirmed
3. In the AWS SNS console, refresh the subscriptions list
4. The status should change from "PendingConfirmation" to "Confirmed"
### 6. Configure SES to Send Bounce Notifications
1. Navigate to AWS SES console
2. Go to "Configuration Sets" (or "Verified identities" > select your domain > "Notifications")
3. For configuration sets:
- Create a new configuration set or select existing
- Add "Event destination"
- Event types: Select **"Bounce"** (and optionally "Complaint")
- Destination: SNS topic
- Select your SNS topic created in step 2
4. For verified identities:
- Select your sending domain/email
- Click "Edit" in the "Notifications" section
- Bounce feedback: Select your SNS topic
- Include original headers: Enabled (optional)
- Click "Save changes"
### 7. Verify Setup
#### Test with a Bounce Simulator
AWS SES provides bounce simulator addresses:
```bash
# From inside Postfix container
docker-compose exec postfix bash
echo "Test bounce" | mail -s "Test" bounce@simulator.amazonses.com
```
Or send to your mailing list with a test recipient:
1. Add `bounce@simulator.amazonses.com` as a member
2. Subscribe to a test list
3. Send an email to the list
#### Check the Results
1. Wait a few minutes for SES to process and send the notification
2. Check API logs:
```bash
sudo docker-compose logs api | grep -i bounce
```
3. Log in to the web UI
4. Go to Members tab
5. Find the test member and click the "Bounces" button
6. You should see the bounce event recorded
### 8. Security Considerations
#### SNS Signature Verification
The webhook endpoint automatically verifies SNS message signatures to ensure notifications are genuine AWS messages. This prevents unauthorized parties from sending fake bounce notifications.
#### HTTPS Requirement
SNS requires HTTPS for webhooks. You'll need:
- Valid SSL/TLS certificate for your domain
- Reverse proxy (e.g., Nginx, Apache) in front of the API container
- Or use AWS API Gateway as a proxy
#### Example Nginx Configuration
```nginx
server {
listen 443 ssl http2;
server_name lists.yourdomain.com;
ssl_certificate /path/to/cert.pem;
ssl_certificate_key /path/to/key.pem;
# Webhook endpoint
location /webhooks/sns {
proxy_pass http://localhost:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
# Optional: proxy API for web UI
location /api {
proxy_pass http://localhost:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
}
```
### 9. Managing Bounces in the UI
#### View Bounce Status
In the Members tab, bounced emails are indicated with:
- Warning badge showing bounce count
- Color-coded status (yellow for soft bounce, red for hard bounce)
- Last bounce timestamp
#### View Bounce History
1. Click the "Bounces" button next to a member
2. View detailed bounce history including:
- Bounce type (Permanent, Transient, Undetermined)
- Bounce subtype
- Diagnostic code from the receiving mail server
- Timestamp of each bounce
#### Reset Bounce Status
If a member's email has been corrected or verified:
1. Open the bounce history modal
2. Click "Reset Bounce Status"
3. Confirm the action
4. The member's bounce count is cleared and they can receive emails again
**Note**: Only users with write access (administrators and operators) can reset bounce status.
### 10. Monitoring and Maintenance
#### Check Bounce Logs
```bash
# View all bounces in database
sudo docker-compose exec mysql mysql -u maillist -p maillist -e "SELECT * FROM bounce_logs ORDER BY timestamp DESC LIMIT 20;"
# Count bounces by type
sudo docker-compose exec mysql mysql -u maillist -p maillist -e "SELECT bounce_type, COUNT(*) as count FROM bounce_logs GROUP BY bounce_type;"
# Find members with bounces
sudo docker-compose exec mysql mysql -u maillist -p maillist -e "SELECT name, email, bounce_count, bounce_status FROM members WHERE bounce_count > 0;"
```
#### API Health Check
```bash
# Check if webhook is accessible
curl -X POST https://yourdomain.com:8000/webhooks/sns \
-H "Content-Type: application/json" \
-d '{"Type":"test"}'
```
#### Clean Up Old Bounce Records
Periodically review and clean up old bounce records:
```sql
-- Delete bounce logs older than 90 days
DELETE FROM bounce_logs WHERE created_at < DATE_SUB(NOW(), INTERVAL 90 DAY);
```
## Troubleshooting
### SNS Subscription Not Confirming
- Ensure the API container is running and accessible via HTTPS
- Check API logs for errors
- Verify firewall rules allow HTTPS traffic to port 8000
- Test the endpoint manually: `curl https://yourdomain.com:8000/health`
### Bounces Not Being Recorded
1. Verify SNS topic is receiving messages:
- Check SNS topic metrics in AWS Console
2. Verify subscription is active:
- Check subscription status in SNS console
3. Check API logs for webhook errors:
```bash
sudo docker-compose logs api | grep -i "sns\|bounce"
```
4. Test signature verification:
- Temporarily add debug logging to the webhook endpoint
### Members Not Being Deactivated
- Check if bounce type is "Permanent"
- Review member's bounce_status in database:
```bash
sudo docker-compose exec mysql mysql -u maillist -p maillist -e "SELECT * FROM members WHERE email='problem@example.com';"
```
- Verify bounce processing logic in API logs
### SSL Certificate Issues
If using self-signed certificates, SNS will reject the webhook. You must use:
- Valid certificate from a trusted CA (Let's Encrypt, etc.)
- Or use AWS Certificate Manager with API Gateway
## Additional Resources
- [AWS SES Bounce Handling](https://docs.aws.amazon.com/ses/latest/dg/event-publishing-retrieving-sns.html)
- [AWS SNS HTTPS Subscriptions](https://docs.aws.amazon.com/sns/latest/dg/sns-http-https-endpoint-as-subscriber.html)
- [SES Bounce Types](https://docs.aws.amazon.com/ses/latest/dg/notification-contents.html#bounce-types)

240
CODE_REVIEW_FINDINGS.md Normal file
View File

@@ -0,0 +1,240 @@
# Code Review Findings & Fixes
## Date: 13 October 2025
## Critical Issues Found and Fixed
### 1. ✅ FIXED: Header Merging Bug in API Client
**Severity:** HIGH
**Location:** `web/static/js/api.js` - `request()` method
**Issue:**
The header merging was incorrect. When passing custom headers in options, they were being overridden by the default headers due to incorrect spread operator order:
```javascript
// BEFORE (BUGGY):
const config = {
headers: { ...this.headers },
...options // This overwrites headers!
};
```
**Fix:**
```javascript
// AFTER (FIXED):
const config = {
...options,
headers: {
...this.headers,
...(options.headers || {}) // Custom headers take precedence
}
};
```
**Impact:**
- The `login()` method was trying to remove the Authorization header for unauthenticated requests, but it was being overridden
- Could have caused authentication issues or unexpected behavior
---
### 2. ✅ FIXED: Missing API Client Methods
**Severity:** HIGH
**Location:** `web/static/js/api.js`
**Issue:** Multiple methods were being called by the UI but didn't exist in the API client:
- `login(username, password)` - ❌ Missing
- `logout()` - ❌ Missing
- `getCurrentUser()` - ❌ Missing
- `getUsers()` - ❌ Missing
- `createUser(userData)` - ❌ Missing
- `updateUser(userId, userData)` - ❌ Missing
- `deleteUser(userId)` - ❌ Missing
- `getMemberBounces(memberId)` - ❌ Missing
- `resetBounceStatus(memberId)` - ❌ Missing
**Fix:** Added all missing methods with proper implementation
---
### 3. ✅ FIXED: Incorrect API Base URL for Production
**Severity:** HIGH
**Location:** `web/static/js/api.js` - `getBaseURL()` method
**Issue:**
When running behind a reverse proxy (Caddy), the API client was trying to connect to `https://lists.sasalliance.org:8000` which:
- Port 8000 is not exposed through Caddy
- Would cause CORS issues
- Would fail all API requests
**Fix:**
```javascript
// Production detection
if (hostname === 'localhost' || hostname === '127.0.0.1') {
return `${protocol}//${hostname}:8000`; // Development
}
// Production - use reverse proxy path
return `${protocol}//${hostname}/api`;
```
---
## Potential Issues (Not Critical, But Worth Noting)
### 4. ⚠️ CORS Configuration
**Severity:** MEDIUM
**Location:** `api/main.py`
**Current Configuration:**
```python
app.add_middleware(
CORSMiddleware,
allow_origins=["*"], # Allows all origins
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
```
**Issue:**
- `allow_origins=["*"]` with `allow_credentials=True` is not allowed by browsers
- Currently working because Caddy reverse proxy makes same-origin requests
- If you ever need direct API access from different origins, this will fail
**Recommendation:**
```python
# For production
allow_origins=[
"https://lists.sasalliance.org",
"http://localhost:3000", # For development
],
```
---
### 5. ✅ GOOD: Authentication Token Storage
**Severity:** INFO
**Location:** `web/static/js/app.js`
**Current Implementation:**
```javascript
localStorage.setItem('authToken', response.access_token);
```
**Assessment:**
- ✅ Proper use of localStorage for JWT tokens
- ✅ Token is cleared on logout
- ✅ Token is validated on page load
- ✅ Expired tokens trigger automatic logout
**Note:** localStorage is appropriate for JWT tokens. HttpOnly cookies would be more secure but require different architecture.
---
### 6. ✅ GOOD: Error Handling
**Severity:** INFO
**Location:** `web/static/js/ui.js` - `handleError()` method
**Assessment:**
- ✅ Proper handling of 401 (auth) errors with automatic logout
- ✅ Proper handling of 403 (forbidden) errors
- ✅ Proper handling of 404 (not found) errors
- ✅ Proper handling of 500 (server) errors
- ✅ User-friendly error messages
---
### 7. ✅ GOOD: Role-Based Access Control
**Severity:** INFO
**Location:** Multiple files
**Assessment:**
- ✅ Backend enforces RBAC with decorators (`require_admin()`, `require_write_access()`, etc.)
- ✅ Frontend checks user roles before showing UI elements
- ✅ Three roles properly implemented: administrator, operator, read-only
- ✅ UI elements are hidden/shown based on permissions
---
### 8. ⚠️ Token Expiration Handling
**Severity:** LOW
**Location:** `api/main.py` and `web/static/js/app.js`
**Current Configuration:**
- JWT tokens expire in 30 minutes
- Session records expire in 24 hours (but not enforced in JWT)
**Potential Issue:**
- Users will be logged out after 30 minutes without warning
- No token refresh mechanism
**Recommendation (Future Enhancement):**
Consider implementing:
- Token refresh endpoint
- Warning before token expiration
- Silent token refresh in background
---
### 9. ✅ GOOD: SQL Injection Prevention
**Severity:** INFO
**Location:** `api/main.py`
**Assessment:**
- ✅ All SQL queries use parameterized statements with `%s` placeholders
- ✅ No string concatenation in SQL queries
- ✅ MySQL connector properly escapes parameters
---
### 10. ✅ GOOD: Password Security
**Severity:** INFO
**Location:** `api/main.py`
**Assessment:**
- ✅ Uses bcrypt for password hashing (`passlib.context.CryptContext`)
- ✅ Passwords are never stored in plain text
- ✅ Passwords are never logged or exposed in API responses
---
## Testing Recommendations
1. **Hard refresh browser cache** after updates (Ctrl+Shift+R)
2. **Test authentication flow:**
- Login with valid credentials
- Login with invalid credentials
- Token expiration after 30 minutes
- Logout functionality
3. **Test role-based access:**
- Administrator can see Users tab
- Operator can modify lists/members but not users
- Read-only can view but not modify
4. **Test bounce handling:**
- View bounce history
- Reset bounce status
5. **Test bulk import:**
- CSV upload
- Subscription assignment
---
## Summary
**3 Critical Bugs Fixed:**
1. Header merging bug in request method
2. Missing API client methods
3. Incorrect production API URL
**All Core Functionality Working:**
- Authentication and authorization
- CRUD operations for lists, members, and users
- Subscription management
- Bounce tracking
- Bulk import
⚠️ **Minor Improvements Suggested:**
1. Tighten CORS policy for production
2. Consider token refresh mechanism
3. Add user session timeout warnings
**Overall Assessment:** The codebase is now production-ready with proper security, error handling, and functionality. The critical bugs have been fixed and deployed.

View File

@@ -0,0 +1,243 @@
# Email-Based Bounce Handling Setup
This document explains the email-based bounce handling system implemented as an alternative to SNS webhooks for environments without SES production access.
## Overview
The system processes email bounces directly within the Postfix container by:
1. Rewriting return paths to direct bounces to a processing address
2. Processing bounce emails via Python script
3. Updating member bounce statistics in MySQL database
4. Automatically disabling members with excessive bounces
2. Setting up an alias that pipes bounce emails to a Python processing script
3. The script parses bounce emails, extracts bounced addresses, and updates the database
4. Members with hard bounces are automatically deactivated
5. Bounce history is tracked and displayed in the UI (same as SNS method)
## Advantages
- **Works with SES Sandbox**: No production SES access required
- **No External Dependencies**: Doesn't require SNS, webhooks, or HTTPS domains
- **Self-Contained**: All processing happens within the existing containers
- **Real-time Processing**: Bounces are processed as soon as emails arrive
- **Compatible**: Uses the same database schema and UI as SNS bounce handling
## Configuration
### 1. Enable Email Bounce Processing
In your `.env` file:
```bash
# Enable email-based bounce processing
ENABLE_EMAIL_BOUNCE_PROCESSING=true
# This will automatically enable bounce handling features
ENABLE_BOUNCE_HANDLING=true # Automatically set to true when email processing is enabled
```
### 2. Restart the System
```bash
sudo docker-compose down
sudo docker-compose up --build -d
```
### 3. Verify Configuration
Check that email bounce processing is enabled:
```bash
curl -s http://localhost:8000/config | jq .
```
Expected output:
```json
{
"bounce_handling_enabled": true,
"sns_webhooks_enabled": false,
"email_bounce_processing_enabled": true
}
```
## How It Works
### Postfix Configuration
The system configures Postfix with:
- `bounce_notice_recipient = bounces@lists.sasalliance.org`
- `2bounce_notice_recipient = bounces@lists.sasalliance.org`
- `error_notice_recipient = bounces@lists.sasalliance.org`
### Aliases Configuration
The `bounces` address is configured to pipe emails to the processing script:
```
bounces: "|/usr/local/bin/process-bounce.py"
```
### Bounce Processing Script
The Python script (`/usr/local/bin/process-bounce.py`):
1. **Reads bounce emails** from stdin (via pipe)
2. **Parses email content** using multiple regex patterns to extract bounced addresses
3. **Analyzes bounce type** based on SMTP error codes:
- 5xx codes = Permanent bounces
- 4xx codes = Transient bounces
- Unknown = Undetermined
4. **Updates database** using the same schema as SNS bounce handling:
- Logs bounce in `bounce_logs` table
- Updates member `bounce_count`, `bounce_status`, and `last_bounce_at`
- Deactivates members with permanent bounces
- Marks members with soft bounce status after 3 transient bounces
## Testing
### Test the Processing Script
You can test the bounce processing script in test mode:
```bash
# Test with sample bounce email
sudo docker-compose exec postfix /usr/local/bin/process-bounce.py --test
```
Expected output:
```
2025-10-14 15:49:16,041 - bounce-processor - INFO - Starting bounce processing
2025-10-14 15:49:16,041 - bounce-processor - INFO - Running in test mode with sample bounce email
2025-10-14 15:49:16,050 - bounce-processor - INFO - Extracted addresses: ['testuser@example.com']
2025-10-14 15:49:16,050 - bounce-processor - INFO - Test mode - would process 1 bounce(s):
2025-10-14 15:49:16,050 - bounce-processor - INFO - {'email': 'testuser@example.com', 'bounce_type': 'Permanent', 'bounce_subtype': 'General', 'diagnostic_code': '', 'timestamp': '2025-10-14 15:49:16'}
```
### Test with Real Bounce Email
To test with a real bounce:
1. Send an email to a non-existent address via your mailing list
2. Wait for the bounce to be processed
3. Check the database for bounce logs:
```bash
# Check bounce logs
sudo docker-compose exec mysql mysql -u maillist -pmaillist maillist -e "SELECT * FROM bounce_logs ORDER BY created_at DESC LIMIT 5;"
# Check member bounce status
sudo docker-compose exec mysql mysql -u maillist -pmaillist maillist -e "SELECT member_id, email, bounce_count, bounce_status, last_bounce_at FROM members WHERE bounce_count > 0;"
```
### View in Web Interface
1. Open http://localhost:3000
2. Navigate to the Members tab
3. Look for bounce badges and bounce counts next to member names
4. Click the "Bounces" button next to a member to view bounce history
## Supported Bounce Email Formats
The processing script recognizes these bounce patterns:
### SMTP Error Codes
- 550, 551, 553, 552, 554 (permanent failures)
- 450, 451, 452 (temporary failures)
### Delivery Status Notification (DSN)
- `Final-Recipient:` headers
- `Original-Recipient:` headers
### Common Bounce Messages
- "user unknown"
- "does not exist"
- "not found"
- "mailbox unavailable"
- "recipient rejected"
## Monitoring
### View Processing Logs
```bash
# View bounce processing logs
sudo docker-compose logs postfix | grep bounce-processor
# Follow logs in real-time
sudo docker-compose logs -f postfix | grep bounce-processor
```
### Check Aliases Configuration
```bash
# Verify aliases are configured correctly
sudo docker-compose exec postfix cat /etc/aliases
# Check alias database
sudo docker-compose exec postfix postmap -q bounces /etc/aliases
```
## Troubleshooting
### Bounces Not Being Processed
1. **Check aliases configuration:**
```bash
sudo docker-compose exec postfix cat /etc/aliases
```
2. **Verify script permissions:**
```bash
sudo docker-compose exec postfix ls -la /usr/local/bin/process-bounce.py
```
3. **Test script manually:**
```bash
sudo docker-compose exec postfix /usr/local/bin/process-bounce.py --test
```
4. **Check Postfix logs:**
```bash
sudo docker-compose logs postfix | grep -i bounce
```
### Database Connection Issues
1. **Check environment variables:**
```bash
sudo docker-compose exec postfix env | grep MYSQL
```
2. **Test database connection:**
```bash
sudo docker-compose exec postfix python3 -c "import pymysql; print('PyMySQL available')"
```
### Script Errors
View detailed error logs:
```bash
sudo docker-compose logs postfix | grep -A 10 -B 10 "bounce-processor.*ERROR"
```
## Comparison with SNS Webhooks
| Feature | Email-Based | SNS Webhooks |
|---------|-------------|--------------|
| **SES Requirement** | Sandbox OK | Production access required |
| **External Dependencies** | None | SNS, HTTPS domain |
| **Processing Speed** | Real-time | Real-time |
| **Setup Complexity** | Low | High |
| **Reliability** | High | High |
| **Bounce Detection** | Regex-based | AWS-provided |
| **Cost** | Free | SNS charges apply |
## Next Steps
1. **Monitor bounce processing** to ensure it's working correctly
2. **Review bounce patterns** in the logs to improve detection if needed
3. **Set up bounce notification alerts** (optional)
4. **Consider upgrading to SNS webhooks** when SES production access is available
Email-based bounce handling provides a robust alternative that works immediately with any SES configuration while providing the same bounce management features as the SNS webhook method.

View File

@@ -0,0 +1,212 @@
# Email-Based Bounce Handling Setup (Comprehensive)
This document explains the complete email-based bounce handling system implemented as an alternative to SNS webhooks for environments without SES production access.
## Overview
The system processes email bounces directly within the Postfix container by:
1. Rewriting return paths to direct bounces to a processing address
2. Processing bounce emails via Python script
3. Updating member bounce statistics in MySQL database
4. Automatically disabling members with excessive bounces
## Configuration
### Environment Variables (in .env)
```bash
# Bounce handling feature flags
ENABLE_BOUNCE_HANDLING=true # Master switch for bounce functionality
ENABLE_EMAIL_BOUNCE_PROCESSING=true # Enable email-based processing
ENABLE_SNS_WEBHOOKS=false # Disable SNS webhooks (optional)
# Database settings (required for bounce processing)
MYSQL_ROOT_PASSWORD=your_root_password
MYSQL_DATABASE=maillist
MYSQL_USER=maillist
MYSQL_PASSWORD=your_maillist_password
```
### Key Components
#### 1. Return Path Rewriting (`postfix/smtp_generic`)
Routes all bounces to the processing address:
```
@lists.sasalliance.org bounces@lists.sasalliance.org
@sasalliance.org bounces@lists.sasalliance.org
```
#### 2. Bounce Processing Script (`postfix/process-bounce.py`)
Python script that:
- Parses bounce emails for recipient addresses and bounce types
- Updates bounce counts in MySQL database
- Automatically disables members after 5 hard bounces
- Logs all bounce events
#### 3. Postfix Integration (`postfix/main.cf.template`)
```
# Return path rewriting for outbound mail
smtp_generic_maps = hash:/etc/postfix/smtp_generic
# Bounce processing
bounce_notice_recipient = bounces@lists.sasalliance.org
```
#### 4. Email Aliases (`postfix/entrypoint.sh`)
```
# Route bounces to processing script
bounces: "|/usr/local/bin/python3 /etc/postfix/process-bounce.py"
```
## How It Works
### Outbound Email Flow
1. User sends email to `community@lists.sasalliance.org`
2. Postfix expands to member list via MySQL
3. Email sent via SES with return path rewritten to `bounces@lists.sasalliance.org`
4. If delivery fails, bounce goes to bounce processing address
### Bounce Processing Flow
1. Bounce email arrives at `bounces@lists.sasalliance.org`
2. Postfix pipes email to `process-bounce.py` script
3. Script parses bounce for recipient and bounce type
4. Database updated with bounce information
5. Member automatically disabled if hard bounce threshold reached
### Bounce Types Detected
- **Hard Bounces**: Permanent failures (5.x.x SMTP codes)
- Invalid email addresses
- Domain doesn't exist
- Mailbox doesn't exist
- **Soft Bounces**: Temporary failures (4.x.x SMTP codes)
- Mailbox full
- Temporary server issues
- Rate limiting
## Database Schema
### Bounce Logs Table
```sql
CREATE TABLE bounce_logs (
id INT PRIMARY KEY AUTO_INCREMENT,
email VARCHAR(255) NOT NULL,
bounce_type ENUM('hard', 'soft', 'complaint') NOT NULL,
bounce_reason TEXT,
bounced_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
raw_message TEXT,
INDEX idx_email (email),
INDEX idx_bounced_at (bounced_at)
);
```
### Members Table (bounce tracking fields)
```sql
ALTER TABLE members ADD COLUMN bounce_count INT DEFAULT 0;
ALTER TABLE members ADD COLUMN last_bounce_at TIMESTAMP NULL;
ALTER TABLE members ADD COLUMN active BOOLEAN DEFAULT true;
```
## Testing Bounce Handling
### 1. Test Return Path Configuration
```bash
# Check that return path rewriting is working
sudo docker-compose exec postfix postconf smtp_generic_maps
sudo docker-compose exec postfix postmap -q "test@lists.sasalliance.org" hash:/etc/postfix/smtp_generic
```
### 2. Simulate Bounce Email
```bash
# Send test bounce to processing script
echo "Subject: Delivery Status Notification (Failure)
From: MAILER-DAEMON@ses.amazonaws.com
To: bounces@lists.sasalliance.org
The following message could not be delivered:
Recipient: test@example.com
Reason: 550 5.1.1 User unknown" | docker-compose exec -T postfix mail -s "Test Bounce" bounces@lists.sasalliance.org
```
### 3. Check Bounce Processing
```bash
# View bounce logs
sudo docker-compose exec mysql mysql -u maillist -pmaillist maillist -e "SELECT * FROM bounce_logs ORDER BY bounced_at DESC LIMIT 5;"
# Check member bounce counts
sudo docker-compose exec mysql mysql -u maillist -pmaillist maillist -e "SELECT email, bounce_count, last_bounce_at, active FROM members WHERE bounce_count > 0;"
```
## Monitoring and Maintenance
### View Processing Logs
```bash
# Monitor bounce processing
sudo docker-compose logs -f postfix | grep -E "(bounce|process-bounce)"
# Check API bounce handling status
curl -H "Authorization: Bearer $API_TOKEN" http://localhost:8000/config
```
### Reset Member Bounce Count
```bash
# Via API
curl -X POST http://localhost:8000/members/{member_id}/reset-bounces \
-H "Authorization: Bearer $API_TOKEN"
# Via Database
sudo docker-compose exec mysql mysql -u maillist -pmaillist maillist -e "UPDATE members SET bounce_count=0, active=true WHERE email='user@example.com';"
```
## Troubleshooting
### Common Issues
#### Bounces Not Being Processed
1. Check that `bounces` alias exists: `sudo docker-compose exec postfix cat /etc/aliases | grep bounces`
2. Verify Python script permissions: `sudo docker-compose exec postfix ls -la /etc/postfix/process-bounce.py`
3. Test script manually: `sudo docker-compose exec postfix python3 /etc/postfix/process-bounce.py --test`
#### Return Path Not Rewritten
1. Check smtp_generic configuration: `sudo docker-compose exec postfix postconf smtp_generic_maps`
2. Verify map file exists: `sudo docker-compose exec postfix ls -la /etc/postfix/smtp_generic*`
3. Test mapping: `sudo docker-compose exec postfix postmap -q "test@lists.sasalliance.org" hash:/etc/postfix/smtp_generic`
#### Database Connection Issues
1. Check PyMySQL installation: `sudo docker-compose exec postfix python3 -c "import pymysql; print('OK')"`
2. Test database connection: `sudo docker-compose exec postfix python3 -c "import pymysql; pymysql.connect(host='mysql', user='maillist', password='your_password', database='maillist')"`
3. Verify network connectivity: `sudo docker-compose exec postfix ping mysql`
### Log Analysis
```bash
# Postfix logs
sudo docker-compose logs postfix | grep -E "(bounce|MAILER-DAEMON|process-bounce)"
# MySQL connection logs
sudo docker-compose logs postfix | grep -E "(pymysql|mysql)"
# SES relay logs
sudo docker-compose logs postfix | grep -E "(relay|sent|deferred)"
```
## Security Considerations
- Bounce processing script runs with limited privileges
- Database credentials secured in environment variables
- Bounce emails contain sensitive delivery information - logs are rotated
- Return path rewriting prevents bounce loops
- Processing script validates email format before database updates
## Performance Notes
- Bounce processing is asynchronous (doesn't block email delivery)
- Database queries are indexed for bounce lookups
- Bounce logs should be periodically archived for large volumes
- SMTP generic maps are cached by Postfix for performance
## Advantages over SNS Webhooks
- **Works with SES Sandbox**: No production SES access required
- **No External Dependencies**: Doesn't require SNS, webhooks, or HTTPS domains
- **Self-Contained**: All processing happens within existing containers
- **Real-time Processing**: Bounces processed as emails arrive
- **Compatible**: Uses same database schema and UI as SNS method

375
README.md
View File

@@ -1,18 +1,23 @@
# Mail List Manager # Mail List Manager
A containerized mailing list management system built around Postfix as an SMTP relay through Amazon SES. A complete containerized mailing list management system with web interface, REST API, MySQL database, and Postfix mail server configured as an SMTP relay through Amazon SES.
## Architecture ## Architecture
**Current (Phase 1):** Static configuration with environment-based credentials **Multi-Service Architecture:**
- Postfix container configured as SES relay - **Web Frontend** - Modern responsive interface for managing lists and members (Port 3000)
- Static virtual aliases for mailing list distribution - **REST API** - FastAPI backend with token authentication (Port 8000)
- Environment variable configuration for security - **MySQL Database** - Stores lists, members, and subscriptions with real-time integration
- **Postfix Mail Server** - SMTP server with native MySQL integration for dynamic list expansion (Port 25)
- **Amazon SES** - Outbound mail relay for reliable delivery
**Planned (Phase 2+):** Web interface with SQL backend **Key Features:**
- Web frontend for list management (view/add/remove members) - Real-time list updates (no Postfix reload needed)
- SQL database for member storage - Native Postfix MySQL queries for instant list expansion
- Dynamic Postfix configuration generation - Token-based API authentication
- Sender whitelisting for authorized board members
- Private Docker network for security
- Complete web-based management interface
## Quick Start ## Quick Start
@@ -21,72 +26,239 @@ A containerized mailing list management system built around Postfix as an SMTP r
cp .env.example .env cp .env.example .env
``` ```
2. Edit `.env` with your SES credentials and configuration: 2. Edit `.env` with your credentials:
```bash ```bash
# Required: Your SES credentials # SES Credentials (Required)
SES_USER=your_ses_access_key SES_USER=your_ses_access_key
SES_PASS=your_ses_secret_key SES_PASS=your_ses_secret_key
# MySQL Credentials (Required)
MYSQL_ROOT_PASSWORD=secure_root_password
MYSQL_PASSWORD=secure_maillist_password
# API Authentication (Required)
API_TOKEN=your_secure_api_token
# Optional: SMTP configuration (defaults to EU West 2) # Optional: SMTP configuration (defaults to EU West 2)
SMTP_HOST=email-smtp.eu-west-2.amazonaws.com SMTP_HOST=email-smtp.eu-west-2.amazonaws.com
SMTP_PORT=587 SMTP_PORT=587
``` ```
3. Build and start the mail server: 3. Build and start all services:
```bash ```bash
docker-compose up --build sudo docker-compose up --build
``` ```
4. Test mail delivery: 4. Access the web interface:
```bash
# From inside container
docker-compose exec postfix bash
echo "Test message" | mail -s "Subject" community@lists.sasalliance.org
# Check logs
docker-compose logs -f postfix
``` ```
http://localhost:3000
```
Enter your API_TOKEN to authenticate and start managing lists!
5. Alternative: Use the REST API directly:
```bash
# Health check
curl http://localhost:8000/health
# Get all lists (requires authentication)
curl -H "Authorization: Bearer your_api_token" http://localhost:8000/lists
```
## Services
### Web Interface (Port 3000)
- Modern, responsive interface for managing mailing lists and members
- Token-based authentication
- Real-time updates and feedback
- See `web/README.md` for details
### REST API (Port 8000)
- Complete CRUD operations for lists, members, and subscriptions
- Bearer token authentication
- Interactive documentation at http://localhost:8000/docs
- See `api/README.md` for API reference
### MySQL Database (Internal Only)
- Stores all mailing list and member data
- Automatically initialized with schema
- Native Postfix integration for real-time queries
- See `database/README.md` for schema and management
### Postfix Mail Server (Port 25)
- Accepts mail for lists.sasalliance.org
- Queries MySQL for list expansion
- Relays through Amazon SES
- Sender whitelist: @sasalliance.org domain
- Changes take effect immediately (no restart needed)
## Configuration ## Configuration
### Adding Mailing Lists (Current) ### Managing Lists and Members
Edit `postfix/virtual_aliases.cf`: **Via Web Interface (Recommended):**
``` 1. Open http://localhost:3000
newlist@lists.sasalliance.org recipient1@domain.com, recipient2@domain.com 2. Enter your API token
``` 3. Use the intuitive interface to:
- Create/edit/delete mailing lists
- Add/edit/remove members
- Manage subscriptions with toggle switches
- View member counts and list details
Then rebuild the container: **Via REST API:**
```bash ```bash
docker-compose up --build # Create a new list
curl -X POST http://localhost:8000/lists \
-H "Authorization: Bearer your_token" \
-H "Content-Type: application/json" \
-d '{
"list_name": "Developers",
"list_email": "dev@lists.sasalliance.org",
"description": "Developer discussions",
"active": true
}'
# Add a new member
curl -X POST http://localhost:8000/members \
-H "Authorization: Bearer your_token" \
-H "Content-Type: application/json" \
-d '{
"name": "John Doe",
"email": "john@example.com",
"active": true
}'
# Subscribe member to list
curl -X POST http://localhost:8000/subscriptions \
-H "Authorization: Bearer your_token" \
-H "Content-Type: application/json" \
-d '{
"list_email": "dev@lists.sasalliance.org",
"member_email": "john@example.com"
}'
``` ```
### Domain Configuration **Via Direct Database Access:**
```bash
# Connect to MySQL
docker-compose exec mysql mysql -u maillist -p maillist
The system is configured for: # Run SQL queries (see database/README.md for examples)
- **Hostname**: `lists.sasalliance.org` (mailing lists) ```
### Mail Server Configuration
### Mail Server Configuration
The Postfix mail server is configured for:
- **Hostname**: `lists.sasalliance.org` (accepts mail for this domain)
- **Origin Domain**: `sasalliance.org` - **Origin Domain**: `sasalliance.org`
- **SES Region**: EU West 2 (configurable via `SMTP_HOST`) - **SES Region**: EU West 2 (configurable via `SMTP_HOST`)
- **Sender Whitelist**: @sasalliance.org (only authorized board members can send to lists)
- **Dynamic Lists**: Postfix queries MySQL in real-time for list members
### Testing Mail Delivery
```bash
# From inside Postfix container
docker-compose exec postfix bash
echo "Test message" | mail -s "Test Subject" community@lists.sasalliance.org
# Check delivery logs
docker-compose logs -f postfix | grep -E "(sent|bounced|deferred)"
# Verify MySQL query works
docker-compose exec postfix postmap -q "community@lists.sasalliance.org" \
mysql:/etc/postfix/mysql_virtual_alias_maps.cf
```
### Bounce Handling (Optional)
**Email bounce handling is optional and disabled by default.**
**Three Configuration Options:**
1. **SNS Webhooks** (Requires SES Production Access):
```bash
# In .env file
ENABLE_SNS_WEBHOOKS=true
ENABLE_BOUNCE_HANDLING=true
```
- Real-time bounce notifications via AWS SNS webhooks
- Automatic member deactivation for hard bounces
- Bounce history tracking and management in web UI
- Requires valid HTTPS domain and SES production access
- See `BOUNCE_HANDLING_SETUP.md` for complete setup
2. **Email-Based Processing** (Works with SES Sandbox):
```bash
# In .env file
ENABLE_EMAIL_BOUNCE_PROCESSING=true
ENABLE_BOUNCE_HANDLING=true # Automatically enabled
```
- Automatic bounce processing via email parsing
- Same bounce management features as SNS webhooks
- Works with SES sandbox accounts (no production access needed)
- Self-contained processing within existing containers
- See `EMAIL_BOUNCE_HANDLING_SETUP.md` for complete setup
3. **Disabled** (Manual Processing Only):
```bash
# In .env file (or leave these commented out)
ENABLE_SNS_WEBHOOKS=false
ENABLE_BOUNCE_HANDLING=false
ENABLE_EMAIL_BOUNCE_PROCESSING=false
```
- No automatic bounce processing
- Manual bounce management via email notifications
- Bounce-related UI elements are hidden
**When bounce handling is disabled:**
- `/webhooks/sns` endpoint is not registered
- Bounce history endpoints return empty results
- Web UI hides bounce badges and bounce management buttons
- No automatic member deactivation occurs
## Security ## Security
- SES credentials are stored in `.env` (git-ignored) - **Environment Variables**: All credentials stored in `.env` (git-ignored)
- SASL password files have restricted permissions (600) - **Private Network**: MySQL communicates with Postfix/API on internal Docker network only
- TLS encryption enforced for SES relay - **Token Authentication**: API requires Bearer token for all write operations
- Only localhost and configured hostname accepted for local delivery - **Sender Whitelist**: Only @sasalliance.org addresses can send to mailing lists
- **TLS Encryption**: Enforced for SES relay connections
- **File Permissions**: SASL password files have restricted permissions (600)
- **No External MySQL**: Database is not exposed to host (no port mapping)
## Development ## Development
### Project Structure ### Project Structure
``` ```
├── docker-compose.yaml # Service orchestration ├── docker-compose.yaml # Multi-service orchestration
├── .env # Environment configuration (not in git) ├── .env # Environment configuration (not in git)
├── postfix/ ├── web/ # Web frontend (Nginx + HTML/CSS/JS)
│ ├── Dockerfile # Postfix container build │ ├── index.html
│ ├── entrypoint.sh # Runtime configuration processing │ ├── static/
├── main.cf.template # Postfix main configuration template │ ├── css/style.css
├── sasl_passwd.template # SES authentication template │ └── js/
└── virtual_aliases.cf # Static mailing list definitions │ ├── api.js
│ │ ├── ui.js
│ │ └── app.js
│ ├── nginx.conf
│ └── Dockerfile
├── api/ # REST API (FastAPI)
│ ├── main.py
│ ├── requirements.txt
│ └── Dockerfile
├── database/ # MySQL schema and docs
│ ├── schema.sql
│ └── README.md
├── postfix/ # Mail server configuration
│ ├── Dockerfile
│ ├── entrypoint.sh
│ ├── main.cf.template
│ ├── mysql_virtual_alias_maps.cf
│ ├── sasl_passwd.template
│ └── sender_access
└── .github/ └── .github/
└── copilot-instructions.md # AI agent guidance └── copilot-instructions.md # AI agent guidance
``` ```
@@ -96,28 +268,131 @@ The system is configured for:
- `SES_PASS`: AWS SES secret access key - `SES_PASS`: AWS SES secret access key
- `SMTP_HOST`: SMTP server hostname (default: email-smtp.eu-west-2.amazonaws.com) - `SMTP_HOST`: SMTP server hostname (default: email-smtp.eu-west-2.amazonaws.com)
- `SMTP_PORT`: SMTP server port (default: 587) - `SMTP_PORT`: SMTP server port (default: 587)
- `MYSQL_ROOT_PASSWORD`: MySQL root password
- `MYSQL_DATABASE`: Database name (default: maillist)
- `MYSQL_USER`: MySQL user (default: maillist)
- `MYSQL_PASSWORD`: MySQL user password
- `API_TOKEN`: Bearer token for API authentication
### Service Dependencies
```
mysql (healthcheck) → postfix, api → web
```
- MySQL must be healthy before Postfix/API start
- Web frontend depends on API being available
- All services communicate on private Docker network
### Debugging ### Debugging
Monitor mail delivery: Monitor services:
```bash ```bash
# View all logs # View all service logs
docker-compose logs -f docker-compose logs -f
# Filter for delivery status # View specific service
docker-compose logs -f postfix
docker-compose logs -f api
docker-compose logs -f web
docker-compose logs -f mysql
# Filter for mail delivery status
docker-compose logs postfix | grep -E "(sent|bounced|deferred)" docker-compose logs postfix | grep -E "(sent|bounced|deferred)"
# Check Postfix queue # Check Postfix queue
docker-compose exec postfix postqueue -p docker-compose exec postfix postqueue -p
# Test MySQL connectivity from Postfix
docker-compose exec postfix postmap -q "community@lists.sasalliance.org" \
mysql:/etc/postfix/mysql_virtual_alias_maps.cf
# Check API health
curl http://localhost:8000/health
# Access MySQL directly
docker-compose exec mysql mysql -u maillist -p maillist
``` ```
### Development Workflow
1. **Make changes** to code or configuration
2. **Rebuild affected service**:
```bash
sudo docker-compose up --build -d web # Just web frontend
sudo docker-compose up --build -d api # Just API
sudo docker-compose up --build -d postfix # Just mail server
```
3. **Check logs** for errors:
```bash
docker-compose logs -f [service-name]
```
4. **Test changes** via web interface or API
### Common Tasks
**Reset database completely:**
```bash
docker-compose down -v # Removes volumes
docker-compose up -d # Reinitializes from schema.sql
```
**Update API token:**
1. Edit `.env` and change `API_TOKEN`
2. Restart API: `docker-compose restart api`
3. Use new token in web interface
**Add authorized sender domain:**
1. Edit `postfix/sender_access`
2. Add line: `newdomain.com OK`
3. Rebuild: `docker-compose up --build -d postfix`
## How It Works
1. **User manages lists** via web interface (http://localhost:3000)
2. **Web frontend** makes authenticated API calls to FastAPI backend
3. **API updates MySQL** database with new lists/members/subscriptions
4. **Email arrives** at Postfix for `someone@lists.sasalliance.org`
5. **Postfix queries MySQL** in real-time using native MySQL support
6. **MySQL returns** comma-separated list of active member emails
7. **Postfix expands** the alias and delivers to all members via SES
8. **Changes take effect immediately** - no restart or reload needed!
## Features
- ✅ **Real-time Updates** - Changes to lists/members take effect immediately
- ✅ **Native MySQL Integration** - Postfix queries database directly (no scripts)
- ✅ **Web Interface** - Modern, responsive UI for easy management
- ✅ **REST API** - Complete programmatic access with token auth
- ✅ **Sender Whitelist** - Only authorized domains can send to lists
- ✅ **SES Integration** - Reliable email delivery through Amazon SES
- ✅ **Bounce Handling** - Automatic tracking and management of email bounces via SNS
- ✅ **Secure** - Private Docker network, token auth, environment-based credentials
- ✅ **Flexible** - Manage via web, API, or direct database access
- ✅ **Scalable** - Database-driven architecture supports many lists and members
## Documentation
- **Web Interface**: See `web/README.md` for frontend features and usage
- **REST API**: See `api/README.md` for complete API reference
- **Database**: See `database/README.md` for schema and SQL examples
- **Bounce Handling**: See `BOUNCE_HANDLING_SETUP.md` for SNS configuration
- **AI Agents**: See `.github/copilot-instructions.md` for development guidance
## Roadmap ## Roadmap
- [ ] Web frontend for mailing list management - [x] Web frontend for mailing list management
- [ ] SQL database backend for member storage - [x] SQL database backend for member storage
- [ ] Dynamic configuration generation from database - [x] Dynamic configuration with native Postfix MySQL
- [ ] Multi-service Docker Compose architecture - [x] Multi-service Docker Compose architecture
- [ ] Migration tools for static → dynamic configuration - [x] REST API with authentication
- [x] Sender whitelist for authorized domains
- [x] Bounce handling with SES SNS integration
- [ ] Email verification workflow for new members
- [ ] Subscription confirmation (double opt-in)
- [ ] List archive functionality
- [ ] Unsubscribe links in emails
- [ ] Rate limiting and anti-spam measures
- [ ] HTTPS support for web interface
- [ ] Admin roles and permissions
## License ## License

398
SES_BOUNCE_TESTING_GUIDE.md Normal file
View File

@@ -0,0 +1,398 @@
# Testing Bounces from SES Sandbox
AWS SES provides **built-in bounce simulator addresses** that work even in sandbox mode. This guide shows you how to use them to test your bounce handling.
## Quick Answer
Send email to these special AWS addresses to simulate different bounce types:
### Hard Bounce (Permanent - Invalid Address)
```bash
bounce@simulator.amazonses.com
```
### Soft Bounce (Temporary - Mailbox Full)
```bash
ooto@simulator.amazonses.com
```
### Complaint (Spam Report)
```bash
complaint@simulator.amazonses.com
```
### Successful Delivery (No Bounce)
```bash
success@simulator.amazonses.com
```
## Step-by-Step Testing Guide
### Option 1: Using Your Mailing Lists (Recommended)
This tests the complete flow: Postfix → SES → SNS → API
1. **Add the simulator address as a member:**
```bash
# Using the API
curl -X POST http://localhost:8000/members \
-H "Authorization: Bearer $API_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"name": "Bounce Test",
"email": "bounce@simulator.amazonses.com",
"active": true
}'
```
Or use the Web UI:
- Go to http://localhost:3000
- Members tab → Add Member
- Name: `Bounce Test`
- Email: `bounce@simulator.amazonses.com`
2. **Subscribe to a test list:**
- Click "Subscriptions" button for the test member
- Toggle on one of your mailing lists
3. **Send email to the list:**
```bash
# From inside Postfix container
sudo docker compose exec postfix bash
echo "This will bounce" | mail -s "Test Bounce" community@lists.sasalliance.org
exit
```
Replace `community@lists.sasalliance.org` with your actual list email.
4. **Wait 30-60 seconds** for:
- Email to be sent via SES
- SES to process the bounce
- SNS to send notification to your webhook
5. **Check the results:**
**Watch API logs in real-time:**
```bash
sudo docker compose logs api -f
```
You should see:
```
============================================================
SNS Webhook Request Received
============================================================
Content-Type: text/plain; charset=UTF-8
User-Agent: Amazon Simple Notification Service Agent
✓ Notification Received
Notification Type: Bounce
✓ Processing Bounce
Bounce Type: Permanent
Recipients: ['bounce@simulator.amazonses.com']
✓ Bounce processed successfully
============================================================
```
**Check the database:**
```bash
# View bounce logs
sudo docker compose exec mysql mysql -u maillist -pmaillist maillist \
-e "SELECT * FROM bounce_logs ORDER BY created_at DESC LIMIT 5;"
# Check member status
sudo docker compose exec mysql mysql -u maillist -pmaillist maillist \
-e "SELECT email, active, bounce_count, bounce_status FROM members WHERE email='bounce@simulator.amazonses.com';"
```
**View in Web UI:**
- Open http://localhost:3000
- Go to Members tab
- Find "Bounce Test" member
- Should show: ❌ Inactive (red), bounce badge, last bounce timestamp
- Click "Bounces" button to see detailed history
### Option 2: Direct Email via Postfix (Simpler)
Send directly to the simulator without going through your list:
```bash
# Enter Postfix container
sudo docker compose exec postfix bash
# Send test email
echo "Testing hard bounce" | mail -s "Hard Bounce Test" bounce@simulator.amazonses.com
# Or soft bounce
echo "Testing soft bounce" | mail -s "Soft Bounce Test" ooto@simulator.amazonses.com
# Exit container
exit
```
### Option 3: Using AWS SES Console (For Non-Sandbox Testing)
If you have SES production access:
1. Go to AWS SES Console
2. Click "Send test email"
3. To: `bounce@simulator.amazonses.com`
4. From: Your verified email/domain
5. Subject: "Bounce test"
6. Body: "Testing bounce handling"
7. Click "Send test email"
## Testing Different Bounce Types
### 1. Hard Bounce (Permanent Failure)
```bash
echo "Test" | mail -s "Test" bounce@simulator.amazonses.com
```
**Expected Result:**
- Member marked as `hard_bounce`
- Member deactivated (`active = 0`)
- `bounce_count` incremented
- Entry in `bounce_logs` table
### 2. Soft Bounce (Transient Failure)
```bash
echo "Test" | mail -s "Test" ooto@simulator.amazonses.com
```
**Expected Result:**
- Member marked as `clean` (first time)
- After 3 soft bounces → `soft_bounce` status
- `bounce_count` incremented
- Member stays active
### 3. Complaint (Spam Report)
```bash
echo "Test" | mail -s "Test" complaint@simulator.amazonses.com
```
**Expected Result:**
- API receives complaint notification
- Currently logged but not processed (you can extend the handler)
## Monitoring the Test
### Real-Time Monitoring (Recommended)
Open 3 terminal windows:
**Terminal 1 - API Logs:**
```bash
sudo docker compose logs api -f
```
**Terminal 2 - Postfix Logs:**
```bash
sudo docker compose logs postfix -f
```
**Terminal 3 - Send Test Email:**
```bash
sudo docker compose exec postfix bash
echo "Test" | mail -s "Bounce Test" bounce@simulator.amazonses.com
```
### Timeline
Here's what happens and when:
- **T+0s**: Email sent to Postfix
- **T+1-3s**: Postfix relays to SES
- **T+5-10s**: SES processes and generates bounce
- **T+10-30s**: SNS sends notification to your webhook
- **T+30-60s**: API processes bounce and updates database
## Verifying the Complete Flow
### 1. Check Postfix Logs
```bash
sudo docker compose logs postfix | grep bounce@simulator
```
Should show:
```
postfix/smtp[xxx]: ... to=<bounce@simulator.amazonses.com>, relay=email-smtp.eu-west-2.amazonaws.com[...], ... status=sent
```
### 2. Check SNS Subscription Status
- Go to AWS SNS Console
- Find your topic
- Check "Subscriptions" tab
- Status should be "Confirmed"
- Messages delivered should be > 0
### 3. Check API Logs
```bash
sudo docker compose logs api | grep -A 20 "SNS Webhook"
```
Should show successful processing.
### 4. Check Database
```bash
sudo docker compose exec mysql mysql -u maillist -pmaillist maillist <<EOF
-- Show recent bounces
SELECT
b.bounce_id,
m.email,
b.bounce_type,
b.diagnostic_code,
b.timestamp
FROM bounce_logs b
JOIN members m ON b.member_id = m.member_id
ORDER BY b.timestamp DESC
LIMIT 5;
-- Show members with bounces
SELECT
email,
active,
bounce_count,
bounce_status,
last_bounce_at
FROM members
WHERE bounce_count > 0;
EOF
```
## Troubleshooting
### "Email not sent" or "Relay access denied"
**Problem**: Postfix not configured to relay via SES
**Check**:
```bash
sudo docker compose exec postfix postconf relayhost
sudo docker compose exec postfix postconf smtp_sasl_auth_enable
```
Should show:
```
relayhost = [email-smtp.eu-west-2.amazonaws.com]:587
smtp_sasl_auth_enable = yes
```
### "No bounce received after 5 minutes"
**Possible causes**:
1. **SNS subscription not confirmed**
- Check AWS SNS console
- Status should be "Confirmed", not "Pending"
2. **SNS topic not configured in SES**
- Check SES → Configuration Sets or Verified Identities → Notifications
- Bounce notifications should point to your SNS topic
3. **Webhook endpoint not accessible**
- SNS requires HTTPS
- Test: `curl https://your-domain.com:8000/health`
4. **API container not running**
```bash
sudo docker compose ps api
```
### "Bounce received but not in database"
**Check API logs for errors**:
```bash
sudo docker compose logs api | grep -i error
```
**Check database tables exist**:
```bash
sudo docker compose exec mysql mysql -u maillist -pmaillist maillist -e "SHOW TABLES;"
```
Should include: `bounce_logs`, `members`
## Testing Multiple Bounces
To test the "3 soft bounces = soft_bounce status" logic:
```bash
sudo docker compose exec postfix bash
# Send 3 emails to soft bounce simulator
for i in {1..3}; do
echo "Soft bounce test $i" | mail -s "Test $i" ooto@simulator.amazonses.com
sleep 70 # Wait between sends for SNS processing
done
```
After the 3rd bounce:
- Member's `bounce_status` should change from `clean` to `soft_bounce`
- `bounce_count` should be 3
## Cleanup After Testing
Remove test bounce data:
```bash
sudo docker compose exec mysql mysql -u maillist -pmaillist maillist <<EOF
-- Delete test bounce logs
DELETE FROM bounce_logs WHERE email IN ('bounce@simulator.amazonses.com', 'ooto@simulator.amazonses.com');
-- Reset test member
UPDATE members
SET bounce_count = 0,
bounce_status = 'clean',
last_bounce_at = NULL,
active = 1
WHERE email IN ('bounce@simulator.amazonses.com', 'ooto@simulator.amazonses.com');
EOF
```
Or use the Web UI:
- Go to Members tab
- Find the test member
- Click "Bounces" button
- Click "Reset Bounce Status"
## Alternative: Simulate Bounces Without SES
If SNS isn't set up yet, use the included script to simulate bounces directly in the database:
```bash
./simulate_bounce.sh
```
This is useful for:
- Testing the UI without AWS
- Development environments
- Demonstrating bounce handling to stakeholders
## Next Steps
Once bounce handling is working:
1. **Remove simulator addresses** from your member list
2. **Monitor real bounces** in production
3. **Set up alerts** for high bounce rates
4. **Review bounced members** regularly and update/remove invalid addresses
5. **Consider complaint handling** (similar to bounces, for spam reports)
## Summary Commands
**Quick test sequence**:
```bash
# 1. Watch logs
sudo docker compose logs api -f &
# 2. Send test bounce
echo "Test" | sudo docker compose exec -T postfix mail -s "Bounce Test" bounce@simulator.amazonses.com
# 3. Wait 60 seconds, then check database
sleep 60
sudo docker compose exec mysql mysql -u maillist -pmaillist maillist -e "SELECT * FROM bounce_logs ORDER BY created_at DESC LIMIT 1;"
```
That's it! The bounce simulator is the easiest way to test your bounce handling without needing real bounced emails.

132
SNS_DEBUG_GUIDE.md Normal file
View File

@@ -0,0 +1,132 @@
# SNS Webhook Debugging Guide
## Current Status
**Detailed logging is now active!** The API will log all incoming SNS requests with:
- Full headers (including SNS-specific headers)
- Request body content
- Parsed message structure
- Processing steps and results
## How to View Logs in Real-Time
```bash
# Watch API logs as SNS sends notifications
sudo docker compose logs api -f
```
## Testing with Real AWS SNS
### Step 1: Trigger a Test from SNS Console
1. Go to AWS SNS Console → Your Topic
2. Click "Publish message"
3. Send a test notification
### Step 2: Check the Logs
The logs will show you exactly what SNS sent:
```
============================================================
SNS Webhook Request Received
============================================================
Content-Type: text/plain; charset=UTF-8
User-Agent: Amazon Simple Notification Service Agent
X-Amz-SNS-Message-Type: Notification
X-Amz-SNS-Topic-Arn: arn:aws:sns:...
Message Type: Notification
Message Keys: ['Type', 'MessageId', 'TopicArn', 'Message', ...]
✓ Notification Received
Message (first 200 chars): {"notificationType":"Bounce",...
Notification Type: Bounce
✓ Processing Bounce
Bounce Type: Permanent
Recipients: ['bounce@example.com']
✓ Bounce processed successfully
============================================================
```
## What the Logs Tell You
### If you see this:
- **"SNS webhook received body: b''"** → SNS sent empty body (check SNS configuration)
- **"Missing SigningCertURL"** → Message missing required SNS fields
- **"Invalid certificate URL"** → Certificate URL not from amazonaws.com
- **"Invalid signature"** → Signature verification failed (message tampered or wrong cert)
### Successful Flow:
1. Headers logged → Shows SNS user agent and message type
2. Body logged → Shows the raw JSON
3. Message parsed → Shows the Type field
4. Notification processed → Shows bounce/complaint details
5. Database updated → Confirmation of processing
## Understanding Previous Errors
The error you saw earlier:
```
SNS webhook error: Expecting value: line 1 column 1 (char 0)
```
This means SNS was sending an **empty body** or the body was already consumed. Possible causes:
1. SNS subscription confirmation URL was opened in a browser (GET request, not POST)
2. Network issue causing body to be lost
3. SNS configuration issue
## Testing Bounce Handling
### Option 1: AWS SES Bounce Simulator
Send email to: `bounce@simulator.amazonses.com`
### Option 2: Verify Database Updates
After a bounce is processed:
```bash
# Check bounce logs
sudo docker compose exec mysql mysql -u maillist -pmaillist maillist -e "SELECT * FROM bounce_logs ORDER BY created_at DESC LIMIT 5;"
# Check member bounce status
sudo docker compose exec mysql mysql -u maillist -pmaillist maillist -e "SELECT member_id, email, bounce_count, bounce_status, last_bounce_at FROM members WHERE bounce_count > 0;"
```
## Common SNS Message Types
### 1. SubscriptionConfirmation
First message when you create the subscription. API auto-confirms by calling SubscribeURL.
### 2. Notification (Bounce)
```json
{
"Type": "Notification",
"Message": "{\"notificationType\":\"Bounce\",\"bounce\":{...}}"
}
```
### 3. Notification (Complaint)
```json
{
"Type": "Notification",
"Message": "{\"notificationType\":\"Complaint\",\"complaint\":{...}}"
}
```
## Next Steps
1. **Monitor logs while SNS sends**: `sudo docker compose logs api -f`
2. **Trigger a real bounce**: Send to `bounce@simulator.amazonses.com`
3. **Check the detailed logs** to see exactly what SNS is sending
4. **Verify database updates** to confirm bounces are being recorded
## If You Still See Errors
The enhanced logging will now show you:
- What headers SNS is sending
- What body content is arriving (or if it's empty)
- Where in the processing pipeline the error occurs
- Full stack traces for any exceptions
Copy the relevant log section and we can diagnose the exact issue!

View File

@@ -2,6 +2,15 @@
REST API for managing mailing lists and members with token-based authentication. REST API for managing mailing lists and members with token-based authentication.
## Overview
This FastAPI-based REST API provides programmatic access to the mailing list system. It's designed for:
- **Automation**: Scripts and integrations that need to manage lists/members
- **Web Interface**: Powers the frontend at http://localhost:3000
- **Third-party Apps**: Any application that needs mailing list management
**For most users**, the web interface (http://localhost:3000) is more convenient. Use this API when you need programmatic access or automation.
## Base URL ## Base URL
``` ```

View File

@@ -2,24 +2,58 @@
Mailing List Management API Mailing List Management API
FastAPI-based REST API for managing mailing lists and members FastAPI-based REST API for managing mailing lists and members
""" """
from fastapi import FastAPI, HTTPException, Depends, Header from fastapi import FastAPI, HTTPException, Depends, Header, status, Request
from fastapi.middleware.cors import CORSMiddleware from fastapi.middleware.cors import CORSMiddleware
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
from fastapi.responses import PlainTextResponse
from pydantic import BaseModel, EmailStr from pydantic import BaseModel, EmailStr
from typing import List, Optional from typing import List, Optional, Annotated
import mysql.connector import mysql.connector
from mysql.connector import Error from mysql.connector import Error
import os import os
import csv
import io
from contextlib import contextmanager from contextlib import contextmanager
from datetime import datetime, timedelta
import secrets
import bcrypt
from jose import JWTError, jwt
from passlib.context import CryptContext
from enum import Enum
import json
import base64
from urllib.parse import urlparse
import httpx
from cryptography import x509
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.asymmetric import padding
from cryptography.hazmat.primitives.serialization import load_pem_public_key
# Configuration # Configuration
API_TOKEN = os.getenv('API_TOKEN', 'change-this-token') API_TOKEN = os.getenv('API_TOKEN', 'change-this-token') # Keep for backward compatibility during transition
JWT_SECRET_KEY = os.getenv('JWT_SECRET_KEY', 'your-secret-key-change-this-in-production')
JWT_ALGORITHM = "HS256"
JWT_ACCESS_TOKEN_EXPIRE_MINUTES = 30
SESSION_EXPIRE_HOURS = 24
MYSQL_HOST = os.getenv('MYSQL_HOST', 'mysql') MYSQL_HOST = os.getenv('MYSQL_HOST', 'mysql')
MYSQL_PORT = int(os.getenv('MYSQL_PORT', 3306)) MYSQL_PORT = int(os.getenv('MYSQL_PORT', 3306))
MYSQL_DATABASE = os.getenv('MYSQL_DATABASE', 'maillist') MYSQL_DATABASE = os.getenv('MYSQL_DATABASE', 'maillist')
MYSQL_USER = os.getenv('MYSQL_USER', 'maillist') MYSQL_USER = os.getenv('MYSQL_USER', 'maillist')
MYSQL_PASSWORD = os.getenv('MYSQL_PASSWORD', '') MYSQL_PASSWORD = os.getenv('MYSQL_PASSWORD', '')
# Bounce handling configuration
ENABLE_SNS_WEBHOOKS = os.getenv('ENABLE_SNS_WEBHOOKS', 'false').lower() == 'true'
ENABLE_BOUNCE_HANDLING = os.getenv('ENABLE_BOUNCE_HANDLING', 'false').lower() == 'true'
ENABLE_EMAIL_BOUNCE_PROCESSING = os.getenv('ENABLE_EMAIL_BOUNCE_PROCESSING', 'false').lower() == 'true'
# Enable bounce handling if either method is enabled
ENABLE_BOUNCE_HANDLING = ENABLE_BOUNCE_HANDLING or ENABLE_SNS_WEBHOOKS or ENABLE_EMAIL_BOUNCE_PROCESSING
# Password hashing
pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto")
# FastAPI app # FastAPI app
app = FastAPI( app = FastAPI(
title="Mailing List Manager API", title="Mailing List Manager API",
@@ -28,12 +62,15 @@ app = FastAPI(
) )
# Add CORS middleware # Add CORS middleware
# Get allowed origins from environment or use secure defaults
ALLOWED_ORIGINS = os.getenv('ALLOWED_ORIGINS', 'http://localhost:3000,http://127.0.0.1:3000').split(',')
app.add_middleware( app.add_middleware(
CORSMiddleware, CORSMiddleware,
allow_origins=["*"], # In production, specify your frontend domain allow_origins=ALLOWED_ORIGINS, # Specific origins only
allow_credentials=True, allow_credentials=True,
allow_methods=["*"], allow_methods=["GET", "POST", "PUT", "PATCH", "DELETE", "OPTIONS"], # Specific methods
allow_headers=["*"], allow_headers=["Authorization", "Content-Type", "Accept", "Origin", "X-Requested-With"], # Specific headers
) )
security = HTTPBearer() security = HTTPBearer()
@@ -58,14 +95,144 @@ def get_db():
if connection and connection.is_connected(): if connection and connection.is_connected():
connection.close() connection.close()
# Authentication # Role-based access control
class UserRole(str, Enum):
ADMINISTRATOR = "administrator"
OPERATOR = "operator"
READ_ONLY = "read-only"
class CurrentUser(BaseModel):
user_id: int
username: str
role: UserRole
active: bool
# Authentication and authorization
def verify_password(plain_password: str, hashed_password: str) -> bool:
"""Verify a password against its hash"""
return pwd_context.verify(plain_password, hashed_password)
def get_password_hash(password: str) -> str:
"""Hash a password"""
return pwd_context.hash(password)
def create_access_token(data: dict, expires_delta: Optional[timedelta] = None):
"""Create a JWT access token"""
to_encode = data.copy()
if expires_delta:
expire = datetime.utcnow() + expires_delta
else:
expire = datetime.utcnow() + timedelta(minutes=15)
to_encode.update({"exp": expire})
encoded_jwt = jwt.encode(to_encode, JWT_SECRET_KEY, algorithm=JWT_ALGORITHM)
return encoded_jwt
def create_session_id() -> str:
"""Create a secure session ID"""
return secrets.token_urlsafe(32)
async def get_current_user(credentials: HTTPAuthorizationCredentials = Depends(security)) -> CurrentUser:
"""Get current authenticated user from JWT token"""
credentials_exception = HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Could not validate credentials",
headers={"WWW-Authenticate": "Bearer"},
)
try:
# Check if it's the old API token (for backward compatibility during transition)
if credentials.credentials == API_TOKEN:
# Return a fake admin user for legacy API token
return CurrentUser(
user_id=0,
username="legacy_admin",
role=UserRole.ADMINISTRATOR,
active=True
)
# Try to decode JWT token
payload = jwt.decode(credentials.credentials, JWT_SECRET_KEY, algorithms=[JWT_ALGORITHM])
username: str = payload.get("sub")
if username is None:
raise credentials_exception
except JWTError:
raise credentials_exception
# Get user from database
with get_db() as conn:
cursor = conn.cursor(dictionary=True)
cursor.execute("SELECT * FROM users WHERE username = %s AND active = 1", (username,))
user = cursor.fetchone()
cursor.close()
if user is None:
raise credentials_exception
return CurrentUser(
user_id=user["user_id"],
username=user["username"],
role=UserRole(user["role"]),
active=user["active"]
)
def require_role(required_roles: List[UserRole]):
"""Decorator factory for role-based access control"""
def role_checker(current_user: CurrentUser = Depends(get_current_user)) -> CurrentUser:
if current_user.role not in required_roles:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail=f"Insufficient permissions. Required roles: {[role.value for role in required_roles]}"
)
return current_user
return role_checker
# Convenience functions for common role requirements
def require_admin() -> CurrentUser:
return Depends(require_role([UserRole.ADMINISTRATOR]))
def require_write_access() -> CurrentUser:
return Depends(require_role([UserRole.ADMINISTRATOR, UserRole.OPERATOR]))
def require_read_access() -> CurrentUser:
return Depends(require_role([UserRole.ADMINISTRATOR, UserRole.OPERATOR, UserRole.READ_ONLY]))
# Legacy authentication (for backward compatibility)
def verify_token(credentials: HTTPAuthorizationCredentials = Depends(security)): def verify_token(credentials: HTTPAuthorizationCredentials = Depends(security)):
"""Verify API token""" """Legacy API token verification - deprecated, use get_current_user instead"""
if credentials.credentials != API_TOKEN: if credentials.credentials != API_TOKEN:
raise HTTPException(status_code=401, detail="Invalid authentication token") raise HTTPException(status_code=401, detail="Invalid authentication token")
return credentials.credentials return credentials.credentials
# Pydantic models # Pydantic models for authentication
class LoginRequest(BaseModel):
username: str
password: str
class TokenResponse(BaseModel):
access_token: str
token_type: str = "bearer"
expires_in: int
user: dict
class CreateUserRequest(BaseModel):
username: str
password: str
role: UserRole
class UpdateUserRequest(BaseModel):
password: Optional[str] = None
role: Optional[UserRole] = None
active: Optional[bool] = None
class UserResponse(BaseModel):
user_id: int
username: str
role: UserRole
created_at: datetime
last_login: Optional[datetime] = None
active: bool
# Pydantic models for mailing list functionality
class MailingList(BaseModel): class MailingList(BaseModel):
list_id: Optional[int] = None list_id: Optional[int] = None
list_name: str list_name: str
@@ -83,6 +250,9 @@ class Member(BaseModel):
name: str name: str
email: EmailStr email: EmailStr
active: bool = True active: bool = True
bounce_count: Optional[int] = 0
last_bounce_at: Optional[datetime] = None
bounce_status: Optional[str] = 'clean'
class MemberUpdate(BaseModel): class MemberUpdate(BaseModel):
name: Optional[str] = None name: Optional[str] = None
@@ -93,6 +263,191 @@ class Subscription(BaseModel):
member_email: EmailStr member_email: EmailStr
active: bool = True active: bool = True
class BulkImportRequest(BaseModel):
csv_data: str
list_ids: List[int]
class BulkImportResult(BaseModel):
total_rows: int
processed_rows: int
created_members: int
updated_members: int
subscriptions_added: int
errors: List[str]
# Authentication routes
@app.post("/auth/login", response_model=TokenResponse)
async def login(login_request: LoginRequest, request: Request):
"""Authenticate user and return JWT token"""
with get_db() as conn:
cursor = conn.cursor(dictionary=True)
cursor.execute("SELECT * FROM users WHERE username = %s AND active = 1", (login_request.username,))
user = cursor.fetchone()
if not user or not verify_password(login_request.password, user["password_hash"]):
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Incorrect username or password"
)
# Update last login
cursor.execute("UPDATE users SET last_login = %s WHERE user_id = %s",
(datetime.utcnow(), user["user_id"]))
conn.commit()
# Create session record
session_id = create_session_id()
expires_at = datetime.utcnow() + timedelta(hours=SESSION_EXPIRE_HOURS)
client_ip = request.client.host if request.client else None
user_agent = request.headers.get("user-agent", "")
cursor.execute("""
INSERT INTO user_sessions (session_id, user_id, expires_at, ip_address, user_agent)
VALUES (%s, %s, %s, %s, %s)
""", (session_id, user["user_id"], expires_at, client_ip, user_agent))
conn.commit()
cursor.close()
# Create JWT token
access_token_expires = timedelta(minutes=JWT_ACCESS_TOKEN_EXPIRE_MINUTES)
access_token = create_access_token(
data={"sub": user["username"]}, expires_delta=access_token_expires
)
return TokenResponse(
access_token=access_token,
expires_in=JWT_ACCESS_TOKEN_EXPIRE_MINUTES * 60,
user={
"user_id": user["user_id"],
"username": user["username"],
"role": user["role"],
"active": user["active"]
}
)
@app.post("/auth/logout")
async def logout(current_user: CurrentUser = Depends(get_current_user)):
"""Logout current user (invalidate sessions)"""
if current_user.user_id == 0: # Legacy admin user
return {"message": "Logged out successfully"}
with get_db() as conn:
cursor = conn.cursor()
cursor.execute("UPDATE user_sessions SET active = 0 WHERE user_id = %s", (current_user.user_id,))
conn.commit()
cursor.close()
return {"message": "Logged out successfully"}
@app.get("/users", response_model=List[UserResponse])
async def get_users(current_user: CurrentUser = require_admin()):
"""Get all users (admin only)"""
with get_db() as conn:
cursor = conn.cursor(dictionary=True)
cursor.execute("SELECT user_id, username, role, created_at, last_login, active FROM users ORDER BY username")
users = cursor.fetchall()
cursor.close()
return users
@app.post("/users", response_model=UserResponse, status_code=201)
async def create_user(user_request: CreateUserRequest, current_user: CurrentUser = require_admin()):
"""Create a new user (admin only)"""
# Check if username already exists
with get_db() as conn:
cursor = conn.cursor(dictionary=True)
cursor.execute("SELECT user_id FROM users WHERE username = %s", (user_request.username,))
existing_user = cursor.fetchone()
if existing_user:
raise HTTPException(status_code=400, detail="Username already exists")
# Hash password and create user
password_hash = get_password_hash(user_request.password)
cursor.execute("""
INSERT INTO users (username, password_hash, role)
VALUES (%s, %s, %s)
""", (user_request.username, password_hash, user_request.role.value))
conn.commit()
user_id = cursor.lastrowid
# Return created user
cursor.execute("""
SELECT user_id, username, role, created_at, last_login, active
FROM users WHERE user_id = %s
""", (user_id,))
new_user = cursor.fetchone()
cursor.close()
return new_user
@app.patch("/users/{user_id}", response_model=UserResponse)
async def update_user(user_id: int, updates: UpdateUserRequest, current_user: CurrentUser = require_admin()):
"""Update a user (admin only)"""
with get_db() as conn:
cursor = conn.cursor(dictionary=True)
# Build update query dynamically
update_fields = []
values = []
if updates.password is not None:
update_fields.append("password_hash = %s")
values.append(get_password_hash(updates.password))
if updates.role is not None:
update_fields.append("role = %s")
values.append(updates.role.value)
if updates.active is not None:
update_fields.append("active = %s")
values.append(updates.active)
if not update_fields:
raise HTTPException(status_code=400, detail="No fields to update")
values.append(user_id)
query = f"UPDATE users SET {', '.join(update_fields)} WHERE user_id = %s"
cursor.execute(query, values)
conn.commit()
# Return updated user
cursor.execute("""
SELECT user_id, username, role, created_at, last_login, active
FROM users WHERE user_id = %s
""", (user_id,))
updated_user = cursor.fetchone()
cursor.close()
if not updated_user:
raise HTTPException(status_code=404, detail="User not found")
return updated_user
@app.delete("/users/{user_id}", status_code=204)
async def delete_user(user_id: int, current_user: CurrentUser = require_admin()):
"""Delete a user (admin only)"""
if user_id == current_user.user_id:
raise HTTPException(status_code=400, detail="Cannot delete your own account")
with get_db() as conn:
cursor = conn.cursor()
cursor.execute("DELETE FROM users WHERE user_id = %s", (user_id,))
conn.commit()
if cursor.rowcount == 0:
raise HTTPException(status_code=404, detail="User not found")
cursor.close()
@app.get("/auth/me", response_model=dict)
async def get_current_user_info(current_user: CurrentUser = Depends(get_current_user)):
"""Get current user information"""
return {
"user_id": current_user.user_id,
"username": current_user.username,
"role": current_user.role.value,
"active": current_user.active
}
# Routes # Routes
@app.get("/") @app.get("/")
async def root(): async def root():
@@ -116,9 +471,18 @@ async def health():
except Exception as e: except Exception as e:
raise HTTPException(status_code=503, detail=f"Unhealthy: {str(e)}") raise HTTPException(status_code=503, detail=f"Unhealthy: {str(e)}")
@app.get("/config")
async def get_config():
"""Get public configuration settings"""
return {
"bounce_handling_enabled": ENABLE_BOUNCE_HANDLING,
"sns_webhooks_enabled": ENABLE_SNS_WEBHOOKS,
"email_bounce_processing_enabled": ENABLE_EMAIL_BOUNCE_PROCESSING
}
# Mailing Lists endpoints # Mailing Lists endpoints
@app.get("/lists", response_model=List[MailingList]) @app.get("/lists", response_model=List[MailingList])
async def get_lists(token: str = Depends(verify_token)): async def get_lists(current_user: CurrentUser = require_read_access()):
"""Get all mailing lists""" """Get all mailing lists"""
with get_db() as conn: with get_db() as conn:
cursor = conn.cursor(dictionary=True) cursor = conn.cursor(dictionary=True)
@@ -128,7 +492,7 @@ async def get_lists(token: str = Depends(verify_token)):
return lists return lists
@app.get("/lists/{list_id}", response_model=MailingList) @app.get("/lists/{list_id}", response_model=MailingList)
async def get_list(list_id: int, token: str = Depends(verify_token)): async def get_list(list_id: int, current_user: CurrentUser = require_read_access()):
"""Get a specific mailing list""" """Get a specific mailing list"""
with get_db() as conn: with get_db() as conn:
cursor = conn.cursor(dictionary=True) cursor = conn.cursor(dictionary=True)
@@ -141,7 +505,7 @@ async def get_list(list_id: int, token: str = Depends(verify_token)):
return mailing_list return mailing_list
@app.post("/lists", response_model=MailingList, status_code=201) @app.post("/lists", response_model=MailingList, status_code=201)
async def create_list(mailing_list: MailingList, token: str = Depends(verify_token)): async def create_list(mailing_list: MailingList, current_user: CurrentUser = require_write_access()):
"""Create a new mailing list""" """Create a new mailing list"""
with get_db() as conn: with get_db() as conn:
cursor = conn.cursor() cursor = conn.cursor()
@@ -158,7 +522,7 @@ async def create_list(mailing_list: MailingList, token: str = Depends(verify_tok
raise HTTPException(status_code=400, detail=f"Failed to create list: {str(e)}") raise HTTPException(status_code=400, detail=f"Failed to create list: {str(e)}")
@app.patch("/lists/{list_id}", response_model=MailingList) @app.patch("/lists/{list_id}", response_model=MailingList)
async def update_list(list_id: int, updates: MailingListUpdate, token: str = Depends(verify_token)): async def update_list(list_id: int, updates: MailingListUpdate, current_user: CurrentUser = require_write_access()):
"""Update a mailing list""" """Update a mailing list"""
with get_db() as conn: with get_db() as conn:
cursor = conn.cursor(dictionary=True) cursor = conn.cursor(dictionary=True)
@@ -196,7 +560,7 @@ async def update_list(list_id: int, updates: MailingListUpdate, token: str = Dep
return updated_list return updated_list
@app.delete("/lists/{list_id}", status_code=204) @app.delete("/lists/{list_id}", status_code=204)
async def delete_list(list_id: int, token: str = Depends(verify_token)): async def delete_list(list_id: int, current_user: CurrentUser = require_write_access()):
"""Delete a mailing list""" """Delete a mailing list"""
with get_db() as conn: with get_db() as conn:
cursor = conn.cursor() cursor = conn.cursor()
@@ -209,7 +573,7 @@ async def delete_list(list_id: int, token: str = Depends(verify_token)):
# Members endpoints # Members endpoints
@app.get("/members", response_model=List[Member]) @app.get("/members", response_model=List[Member])
async def get_members(token: str = Depends(verify_token)): async def get_members(current_user: CurrentUser = require_read_access()):
"""Get all members""" """Get all members"""
with get_db() as conn: with get_db() as conn:
cursor = conn.cursor(dictionary=True) cursor = conn.cursor(dictionary=True)
@@ -219,7 +583,7 @@ async def get_members(token: str = Depends(verify_token)):
return members return members
@app.get("/members/{member_id}", response_model=Member) @app.get("/members/{member_id}", response_model=Member)
async def get_member(member_id: int, token: str = Depends(verify_token)): async def get_member(member_id: int, current_user: CurrentUser = require_read_access()):
"""Get a specific member""" """Get a specific member"""
with get_db() as conn: with get_db() as conn:
cursor = conn.cursor(dictionary=True) cursor = conn.cursor(dictionary=True)
@@ -232,7 +596,7 @@ async def get_member(member_id: int, token: str = Depends(verify_token)):
return member return member
@app.post("/members", response_model=Member, status_code=201) @app.post("/members", response_model=Member, status_code=201)
async def create_member(member: Member, token: str = Depends(verify_token)): async def create_member(member: Member, current_user: CurrentUser = require_write_access()):
"""Create a new member""" """Create a new member"""
with get_db() as conn: with get_db() as conn:
cursor = conn.cursor() cursor = conn.cursor()
@@ -249,7 +613,7 @@ async def create_member(member: Member, token: str = Depends(verify_token)):
raise HTTPException(status_code=400, detail=f"Failed to create member: {str(e)}") raise HTTPException(status_code=400, detail=f"Failed to create member: {str(e)}")
@app.patch("/members/{member_id}", response_model=Member) @app.patch("/members/{member_id}", response_model=Member)
async def update_member(member_id: int, updates: MemberUpdate, token: str = Depends(verify_token)): async def update_member(member_id: int, updates: MemberUpdate, current_user: CurrentUser = require_write_access()):
"""Update a member""" """Update a member"""
with get_db() as conn: with get_db() as conn:
cursor = conn.cursor(dictionary=True) cursor = conn.cursor(dictionary=True)
@@ -282,7 +646,7 @@ async def update_member(member_id: int, updates: MemberUpdate, token: str = Depe
return updated_member return updated_member
@app.delete("/members/{member_id}", status_code=204) @app.delete("/members/{member_id}", status_code=204)
async def delete_member(member_id: int, token: str = Depends(verify_token)): async def delete_member(member_id: int, current_user: CurrentUser = require_write_access()):
"""Delete a member""" """Delete a member"""
with get_db() as conn: with get_db() as conn:
cursor = conn.cursor() cursor = conn.cursor()
@@ -295,7 +659,7 @@ async def delete_member(member_id: int, token: str = Depends(verify_token)):
# Subscription endpoints # Subscription endpoints
@app.get("/lists/{list_id}/members", response_model=List[Member]) @app.get("/lists/{list_id}/members", response_model=List[Member])
async def get_list_members(list_id: int, token: str = Depends(verify_token)): async def get_list_members(list_id: int, current_user: CurrentUser = require_read_access()):
"""Get all members of a specific list""" """Get all members of a specific list"""
with get_db() as conn: with get_db() as conn:
cursor = conn.cursor(dictionary=True) cursor = conn.cursor(dictionary=True)
@@ -311,7 +675,7 @@ async def get_list_members(list_id: int, token: str = Depends(verify_token)):
return members return members
@app.post("/subscriptions", status_code=201) @app.post("/subscriptions", status_code=201)
async def subscribe_member(subscription: Subscription, token: str = Depends(verify_token)): async def subscribe_member(subscription: Subscription, current_user: CurrentUser = require_write_access()):
"""Subscribe a member to a list""" """Subscribe a member to a list"""
with get_db() as conn: with get_db() as conn:
cursor = conn.cursor() cursor = conn.cursor()
@@ -345,7 +709,7 @@ async def subscribe_member(subscription: Subscription, token: str = Depends(veri
raise HTTPException(status_code=400, detail=f"Failed to create subscription: {str(e)}") raise HTTPException(status_code=400, detail=f"Failed to create subscription: {str(e)}")
@app.delete("/subscriptions") @app.delete("/subscriptions")
async def unsubscribe_member(list_email: EmailStr, member_email: EmailStr, token: str = Depends(verify_token)): async def unsubscribe_member(list_email: EmailStr, member_email: EmailStr, current_user: CurrentUser = require_write_access()):
"""Unsubscribe a member from a list""" """Unsubscribe a member from a list"""
with get_db() as conn: with get_db() as conn:
cursor = conn.cursor() cursor = conn.cursor()
@@ -365,6 +729,429 @@ async def unsubscribe_member(list_email: EmailStr, member_email: EmailStr, token
cursor.close() cursor.close()
return {"message": "Unsubscribed successfully"} return {"message": "Unsubscribed successfully"}
@app.post("/bulk-import", response_model=BulkImportResult)
async def bulk_import_members(bulk_request: BulkImportRequest, current_user: CurrentUser = require_write_access()):
"""Bulk import members from CSV data and subscribe them to specified lists"""
result = BulkImportResult(
total_rows=0,
processed_rows=0,
created_members=0,
updated_members=0,
subscriptions_added=0,
errors=[]
)
with get_db() as conn:
cursor = conn.cursor(dictionary=True)
try:
# Verify all list_ids exist
if bulk_request.list_ids:
placeholders = ','.join(['%s'] * len(bulk_request.list_ids))
cursor.execute(f"SELECT list_id FROM lists WHERE list_id IN ({placeholders})", bulk_request.list_ids)
existing_lists = [row['list_id'] for row in cursor.fetchall()]
invalid_lists = set(bulk_request.list_ids) - set(existing_lists)
if invalid_lists:
raise HTTPException(status_code=400, detail=f"Invalid list IDs: {list(invalid_lists)}")
# Parse CSV data
csv_reader = csv.DictReader(io.StringIO(bulk_request.csv_data))
# Validate CSV headers - we need at least Name and Email
if not csv_reader.fieldnames or 'Name' not in csv_reader.fieldnames or 'Email' not in csv_reader.fieldnames:
raise HTTPException(status_code=400, detail="CSV must contain 'Name' and 'Email' columns")
for row_num, row in enumerate(csv_reader, start=1):
result.total_rows += 1
try:
name = row.get('Name', '').strip()
email = row.get('Email', '').strip()
# Skip rows without email (name is optional)
if not email:
result.errors.append(f"Row {row_num}: Missing email address")
continue
# Basic email validation
if '@' not in email or '.' not in email.split('@')[1]:
result.errors.append(f"Row {row_num}: Invalid email format: {email}")
continue
# Use email as name if no name provided
if not name:
name = email.split('@')[0] # Use the part before @ as a default name
# Check if member exists
cursor.execute("SELECT member_id FROM members WHERE email = %s", (email,))
existing_member = cursor.fetchone()
if existing_member:
# Update existing member's name if we have a better name (not auto-generated from email)
should_update_name = (
row.get('Name', '').strip() and # We have a name in the CSV
name != email.split('@')[0] # It's not the auto-generated name
)
if should_update_name:
cursor.execute("UPDATE members SET name = %s WHERE member_id = %s", (name, existing_member['member_id']))
if cursor.rowcount > 0:
result.updated_members += 1
member_id = existing_member['member_id']
else:
# Create new member
cursor.execute(
"INSERT INTO members (name, email, active) VALUES (%s, %s, %s)",
(name, email, True)
)
member_id = cursor.lastrowid
result.created_members += 1
# Subscribe to selected lists
for list_id in bulk_request.list_ids:
try:
cursor.execute(
"INSERT INTO list_members (list_id, member_id, active) VALUES (%s, %s, %s)",
(list_id, member_id, True)
)
result.subscriptions_added += 1
except Error as e:
if "Duplicate entry" in str(e):
# Member already subscribed to this list - not an error
pass
else:
result.errors.append(f"Row {row_num}: Failed to subscribe to list {list_id}: {str(e)}")
result.processed_rows += 1
except Exception as e:
result.errors.append(f"Row {row_num}: {str(e)}")
continue
# Commit all changes
conn.commit()
cursor.close()
return result
except HTTPException:
raise
except Exception as e:
cursor.close()
raise HTTPException(status_code=500, detail=f"Bulk import failed: {str(e)}")
# SNS Webhook for Bounce Handling (conditionally enabled)
if ENABLE_SNS_WEBHOOKS:
async def verify_sns_signature(request: Request) -> dict:
"""Verify SNS message signature"""
try:
body = await request.body()
print(f"SNS webhook received body: {body}")
print(f"SNS webhook body length: {len(body)}")
print(f"SNS webhook headers: {dict(request.headers)}")
if not body:
print("ERROR: Empty body received")
raise HTTPException(status_code=400, detail="Empty request body")
message = json.loads(body.decode('utf-8'))
print(f"SNS webhook parsed message type: {message.get('Type')}")
# For SubscriptionConfirmation and UnsubscribeConfirmation, we don't validate signature
# AWS will send a URL to confirm
if message.get('Type') in ['SubscriptionConfirmation', 'UnsubscribeConfirmation']:
return message
# Get certificate URL and download certificate
cert_url = message.get('SigningCertURL')
if not cert_url:
raise HTTPException(status_code=400, detail="Missing SigningCertURL")
# Verify certificate URL is from AWS
parsed_url = urlparse(cert_url)
if not parsed_url.hostname.endswith('.amazonaws.com'):
raise HTTPException(status_code=400, detail="Invalid certificate URL")
# Download certificate
async with httpx.AsyncClient() as client:
cert_response = await client.get(cert_url)
cert_response.raise_for_status()
cert_pem = cert_response.content
# Load certificate and extract public key
cert = x509.load_pem_x509_certificate(cert_pem, default_backend())
public_key = cert.public_key()
# Build string to sign
if message.get('Type') == 'Notification':
string_to_sign = (
f"Message\n{message['Message']}\n"
f"MessageId\n{message['MessageId']}\n"
)
if 'Subject' in message:
string_to_sign += f"Subject\n{message['Subject']}\n"
string_to_sign += (
f"Timestamp\n{message['Timestamp']}\n"
f"TopicArn\n{message['TopicArn']}\n"
f"Type\n{message['Type']}\n"
)
else:
string_to_sign = (
f"Message\n{message['Message']}\n"
f"MessageId\n{message['MessageId']}\n"
f"SubscribeURL\n{message['SubscribeURL']}\n"
f"Timestamp\n{message['Timestamp']}\n"
f"Token\n{message['Token']}\n"
f"TopicArn\n{message['TopicArn']}\n"
f"Type\n{message['Type']}\n"
)
# Verify signature
signature = base64.b64decode(message['Signature'])
try:
public_key.verify(
signature,
string_to_sign.encode('utf-8'),
padding.PKCS1v15(),
hashes.SHA1()
)
except Exception as e:
raise HTTPException(status_code=400, detail=f"Invalid signature: {str(e)}")
return message
except json.JSONDecodeError:
raise HTTPException(status_code=400, detail="Invalid JSON")
except Exception as e:
raise HTTPException(status_code=400, detail=f"Signature verification failed: {str(e)}")
async def process_bounce(bounce_data: dict):
"""Process bounce notification and update database"""
try:
bounce_type = bounce_data.get('bounceType') # Permanent, Transient, Undetermined
bounce_subtype = bounce_data.get('bounceSubType', '')
timestamp_str = bounce_data.get('timestamp')
feedback_id = bounce_data.get('feedbackId', '')
# Convert ISO 8601 timestamp to MySQL datetime format
# SES sends: '2025-10-13T16:22:40.359Z'
# MySQL needs: '2025-10-13 16:22:40'
from datetime import datetime as dt
timestamp = dt.fromisoformat(timestamp_str.replace('Z', '+00:00')).strftime('%Y-%m-%d %H:%M:%S')
bounced_recipients = bounce_data.get('bouncedRecipients', [])
with get_db() as conn:
cursor = conn.cursor(dictionary=True)
for recipient in bounced_recipients:
email = recipient.get('emailAddress')
diagnostic_code = recipient.get('diagnosticCode', '')
if not email:
continue
# Find member by email
cursor.execute("SELECT member_id FROM members WHERE email = %s", (email,))
member = cursor.fetchone()
member_id = member['member_id'] if member else None
# Log the bounce
cursor.execute("""
INSERT INTO bounce_logs
(member_id, email, bounce_type, bounce_subtype, diagnostic_code, timestamp, feedback_id)
VALUES (%s, %s, %s, %s, %s, %s, %s)
""", (member_id, email, bounce_type, bounce_subtype, diagnostic_code, timestamp, feedback_id))
# Update member bounce status
if member_id:
# Determine bounce status
if bounce_type == 'Permanent':
new_status = 'hard_bounce'
# Deactivate member with hard bounce
cursor.execute("""
UPDATE members
SET bounce_count = bounce_count + 1,
last_bounce_at = %s,
bounce_status = %s,
active = 0
WHERE member_id = %s
""", (timestamp, new_status, member_id))
elif bounce_type == 'Transient':
# Check current bounce count
cursor.execute("SELECT bounce_count, bounce_status FROM members WHERE member_id = %s", (member_id,))
current = cursor.fetchone()
# If already hard bounced, don't change status
if current and current['bounce_status'] != 'hard_bounce':
new_count = current['bounce_count'] + 1
# After 3 soft bounces, mark as soft_bounce status
new_status = 'soft_bounce' if new_count >= 3 else 'clean'
cursor.execute("""
UPDATE members
SET bounce_count = %s,
last_bounce_at = %s,
bounce_status = %s
WHERE member_id = %s
""", (new_count, timestamp, new_status, member_id))
else: # Undetermined
cursor.execute("""
UPDATE members
SET bounce_count = bounce_count + 1,
last_bounce_at = %s
WHERE member_id = %s
""", (timestamp, member_id))
conn.commit()
cursor.close()
except Exception as e:
print(f"✗ Error processing bounce: {str(e)}")
print(f"Error type: {type(e).__name__}")
print(f"Bounce data: {bounce_data}")
import traceback
traceback.print_exc()
raise
@app.post("/webhooks/sns", response_class=PlainTextResponse)
async def sns_webhook(request: Request):
"""Handle SNS notifications for bounces and complaints"""
try:
print(f"=== SNS Webhook Request ===")
print(f"Headers: {dict(request.headers)}")
print(f"Content-Type: {request.headers.get('content-type')}")
print(f"User-Agent: {request.headers.get('user-agent')}")
# Verify SNS signature
message = await verify_sns_signature(request)
print(f"Message Type: {message.get('Type')}")
print(f"Message Keys: {list(message.keys())}")
message_type = message.get('Type')
# Handle subscription confirmation
if message_type == 'SubscriptionConfirmation':
subscribe_url = message.get('SubscribeURL')
print(f"Subscription confirmation received, URL: {subscribe_url}")
if subscribe_url:
# Confirm subscription
async with httpx.AsyncClient() as client:
response = await client.get(subscribe_url)
print(f"Subscription confirmation response: {response.status_code}")
return "Subscription confirmed"
# Handle notification
elif message_type == 'Notification':
# Parse the message
inner_message = message.get('Message', '{}')
print(f"Inner message (first 500 chars): {inner_message[:500]}")
notification = json.loads(inner_message)
# SES can send either 'notificationType' or 'eventType' depending on configuration
notification_type = notification.get('notificationType') or notification.get('eventType')
print(f"Notification type: {notification_type}")
if notification_type == 'Bounce':
bounce = notification.get('bounce', {})
print(f"\n✓ Processing Bounce")
print(f" Bounce Type: {bounce.get('bounceType')}")
print(f" Recipients: {[r.get('emailAddress') for r in bounce.get('bouncedRecipients', [])]}")
await process_bounce(bounce)
print(f" ✓ Bounce processed successfully")
return "Bounce processed"
elif notification_type == 'Complaint':
# We could also track complaints similarly to bounces
print(f"\n✓ Complaint received")
return "Complaint received"
print(f"=== End SNS Webhook Request ===")
return "OK"
except HTTPException:
raise
except Exception as e:
print(f"SNS webhook error: {str(e)}")
import traceback
traceback.print_exc()
raise HTTPException(status_code=500, detail=str(e))
else:
# Provide stub functions when SNS webhooks are disabled
print("SNS webhooks disabled - bounce handling via email only")
# Bounce management endpoints (conditionally enabled)
if ENABLE_BOUNCE_HANDLING:
class BounceLog(BaseModel):
bounce_id: int
email: str
bounce_type: str
bounce_subtype: Optional[str] = None
diagnostic_code: Optional[str] = None
timestamp: datetime
feedback_id: Optional[str] = None
created_at: datetime
class MemberWithBounces(BaseModel):
member_id: int
name: str
email: str
active: bool
bounce_count: int
last_bounce_at: Optional[datetime] = None
bounce_status: str
@app.get("/members/{member_id}/bounces", response_model=List[BounceLog])
async def get_member_bounces(member_id: int, current_user: CurrentUser = require_read_access()):
"""Get bounce history for a member"""
with get_db() as conn:
cursor = conn.cursor(dictionary=True)
cursor.execute("""
SELECT bounce_id, email, bounce_type, bounce_subtype, diagnostic_code,
timestamp, feedback_id, created_at
FROM bounce_logs
WHERE member_id = %s
ORDER BY timestamp DESC
""", (member_id,))
bounces = cursor.fetchall()
cursor.close()
return bounces
@app.patch("/members/{member_id}/bounce-status")
async def reset_bounce_status(member_id: int, current_user: CurrentUser = require_write_access()):
"""Reset bounce status for a member (e.g., after email address is corrected)"""
with get_db() as conn:
cursor = conn.cursor()
cursor.execute("""
UPDATE members
SET bounce_count = 0,
last_bounce_at = NULL,
bounce_status = 'clean'
WHERE member_id = %s
""", (member_id,))
conn.commit()
if cursor.rowcount == 0:
raise HTTPException(status_code=404, detail="Member not found")
cursor.close()
return {"message": "Bounce status reset successfully"}
else:
# When bounce handling is disabled, provide stub endpoints that return appropriate responses
@app.get("/members/{member_id}/bounces")
async def get_member_bounces_disabled(member_id: int, current_user: CurrentUser = require_read_access()):
"""Bounce history disabled - returns empty list"""
return []
@app.patch("/members/{member_id}/bounce-status")
async def reset_bounce_status_disabled(member_id: int, current_user: CurrentUser = require_write_access()):
"""Bounce status reset disabled"""
raise HTTPException(status_code=501, detail="Bounce handling is disabled")
if __name__ == "__main__": if __name__ == "__main__":
import uvicorn import uvicorn
uvicorn.run(app, host="0.0.0.0", port=8000) uvicorn.run(app, host="0.0.0.0", port=8000)

View File

@@ -5,3 +5,7 @@ pydantic==2.5.0
pydantic-settings==2.1.0 pydantic-settings==2.1.0
python-multipart==0.0.6 python-multipart==0.0.6
email-validator==2.1.0 email-validator==2.1.0
bcrypt==4.0.1
python-jose[cryptography]==3.3.0
passlib[bcrypt]==1.7.4
httpx==0.25.2

View File

@@ -2,6 +2,26 @@
This mailing list system uses MySQL with **Postfix's native MySQL support** for real-time dynamic list management. Postfix queries the database directly for each email - no scripts or reloads needed. This mailing list system uses MySQL with **Postfix's native MySQL support** for real-time dynamic list management. Postfix queries the database directly for each email - no scripts or reloads needed.
## Management Options
**1. Web Interface (Recommended for Non-Technical Users)**
- Access: http://localhost:3000
- Visual interface with tables and forms
- Toggle-based subscription management
- No SQL knowledge required
**2. REST API (Recommended for Automation)**
- Access: http://localhost:8000/docs
- Full CRUD operations via HTTP
- Token authentication
- Perfect for scripts and integrations
**3. Direct MySQL (Recommended for Advanced Users)**
- Full SQL access for complex queries
- Bulk operations and reporting
- Database administration tasks
- Described in detail below
## Database Schema ## Database Schema
Three-table design with many-to-many relationships: Three-table design with many-to-many relationships:
@@ -36,7 +56,38 @@ Three-table design with many-to-many relationships:
## Managing Lists and Members ## Managing Lists and Members
### Via MySQL Client ### Via Web Interface (Easiest)
1. Open http://localhost:3000 in your browser
2. Enter your API_TOKEN (from .env file)
3. Use the tabs to:
- **Lists Tab**: View, create, edit, delete mailing lists
- **Members Tab**: View, add, edit, remove members
- **Subscriptions**: Click "Subscriptions" button on any member to toggle their list memberships
### Via REST API (For Automation)
See `api/README.md` for complete API documentation, or visit http://localhost:8000/docs for interactive docs.
Quick examples:
```bash
# Get all lists
curl -H "Authorization: Bearer $API_TOKEN" http://localhost:8000/lists
# Create member
curl -X POST http://localhost:8000/members \
-H "Authorization: Bearer $API_TOKEN" \
-H "Content-Type: application/json" \
-d '{"name":"John Doe","email":"john@example.com","active":true}'
# Subscribe to list
curl -X POST http://localhost:8000/subscriptions \
-H "Authorization: Bearer $API_TOKEN" \
-H "Content-Type: application/json" \
-d '{"list_email":"community@lists.sasalliance.org","member_email":"john@example.com"}'
```
### Via MySQL Client (Advanced)
Connect to the database: Connect to the database:

View File

@@ -1,5 +1,37 @@
-- Mail List Manager Database Schema -- Mail List Manager Database Schema
-- Table: users
-- Stores user authentication and authorization information
CREATE TABLE IF NOT EXISTS users (
user_id INT AUTO_INCREMENT PRIMARY KEY,
username VARCHAR(50) NOT NULL UNIQUE,
password_hash VARCHAR(255) NOT NULL, -- bcrypt hash
role ENUM('administrator', 'operator', 'read-only') NOT NULL DEFAULT 'read-only',
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
last_login TIMESTAMP NULL,
active BOOLEAN DEFAULT TRUE,
INDEX idx_username (username),
INDEX idx_role (role),
INDEX idx_active (active)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
-- Table: user_sessions
-- Stores active user sessions for authentication
CREATE TABLE IF NOT EXISTS user_sessions (
session_id VARCHAR(64) PRIMARY KEY,
user_id INT NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
expires_at TIMESTAMP NOT NULL,
ip_address VARCHAR(45), -- Supports both IPv4 and IPv6
user_agent TEXT,
active BOOLEAN DEFAULT TRUE,
FOREIGN KEY (user_id) REFERENCES users(user_id) ON DELETE CASCADE,
INDEX idx_user_id (user_id),
INDEX idx_expires_at (expires_at),
INDEX idx_active (active)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
-- Table: lists -- Table: lists
-- Stores mailing list information -- Stores mailing list information
CREATE TABLE IF NOT EXISTS lists ( CREATE TABLE IF NOT EXISTS lists (
@@ -23,8 +55,12 @@ CREATE TABLE IF NOT EXISTS members (
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
active BOOLEAN DEFAULT TRUE, active BOOLEAN DEFAULT TRUE,
bounce_count INT DEFAULT 0,
last_bounce_at TIMESTAMP NULL,
bounce_status ENUM('clean', 'soft_bounce', 'hard_bounce') DEFAULT 'clean',
INDEX idx_email (email), INDEX idx_email (email),
INDEX idx_active (active) INDEX idx_active (active),
INDEX idx_bounce_status (bounce_status)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci; ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
-- Table: list_members -- Table: list_members
@@ -43,7 +79,33 @@ CREATE TABLE IF NOT EXISTS list_members (
INDEX idx_active (active) INDEX idx_active (active)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci; ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
-- Table: bounce_logs
-- Stores bounce notifications from SES SNS
CREATE TABLE IF NOT EXISTS bounce_logs (
bounce_id INT AUTO_INCREMENT PRIMARY KEY,
member_id INT,
email VARCHAR(255) NOT NULL,
bounce_type ENUM('Permanent', 'Transient', 'Undetermined') NOT NULL,
bounce_subtype VARCHAR(50),
diagnostic_code TEXT,
timestamp TIMESTAMP NOT NULL,
sns_message_id VARCHAR(255),
feedback_id VARCHAR(255),
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (member_id) REFERENCES members(member_id) ON DELETE SET NULL,
INDEX idx_member_id (member_id),
INDEX idx_email (email),
INDEX idx_timestamp (timestamp),
INDEX idx_bounce_type (bounce_type)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
-- Insert sample data -- Insert sample data
-- Create default admin user (password: 'password')
-- Hash generated with passlib CryptContext using bcrypt
INSERT INTO users (username, password_hash, role) VALUES
('admin', '$2b$12$6Lsp/i.clxyEE9O/4D9RLOfyFlxfjvJBhJ8VXT/t4H/QvrnKrE/YK', 'administrator');
INSERT INTO lists (list_name, list_email, description) VALUES INSERT INTO lists (list_name, list_email, description) VALUES
('Community', 'community@lists.sasalliance.org', 'General community announcements'), ('Community', 'community@lists.sasalliance.org', 'General community announcements'),
('Board', 'board@lists.sasalliance.org', 'Board members only'), ('Board', 'board@lists.sasalliance.org', 'Board members only'),

View File

@@ -0,0 +1,59 @@
# Example nginx configuration for production with SNS webhook support
server {
listen 80;
server_name yourdomain.com;
# Redirect HTTP to HTTPS in production
return 301 https://$server_name$request_uri;
}
server {
listen 443 ssl http2;
server_name yourdomain.com;
# SSL configuration (add your certificates)
# ssl_certificate /path/to/your/cert.pem;
# ssl_certificate_key /path/to/your/key.pem;
# Frontend (static files)
location / {
proxy_pass http://maillist-web:80;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
# API endpoints (including SNS webhook)
location /api/ {
# Remove /api prefix when forwarding to backend
rewrite ^/api/(.*) /$1 break;
proxy_pass http://maillist-api:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Important for SNS webhooks - increase timeouts and body size
proxy_connect_timeout 60s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
client_max_body_size 10M;
}
# Direct SNS webhook endpoint (alternative path)
location /webhooks/sns {
proxy_pass http://maillist-api:8000/webhooks/sns;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# SNS specific settings
proxy_connect_timeout 30s;
proxy_send_timeout 30s;
proxy_read_timeout 30s;
client_max_body_size 1M;
}
}

View File

@@ -9,6 +9,8 @@ RUN apt-get update && \
mailutils \ mailutils \
gettext-base \ gettext-base \
netcat-openbsd \ netcat-openbsd \
python3 \
python3-pymysql \
&& apt-get clean && rm -rf /var/lib/apt/lists/* && apt-get clean && rm -rf /var/lib/apt/lists/*
# Copy configs # Copy configs
@@ -16,11 +18,16 @@ COPY main.cf.template /etc/postfix/main.cf.template
COPY sasl_passwd.template /etc/postfix/sasl_passwd.template COPY sasl_passwd.template /etc/postfix/sasl_passwd.template
COPY mysql_virtual_alias_maps.cf /etc/postfix/mysql_virtual_alias_maps.cf.template COPY mysql_virtual_alias_maps.cf /etc/postfix/mysql_virtual_alias_maps.cf.template
COPY sender_access /etc/postfix/sender_access COPY sender_access /etc/postfix/sender_access
COPY smtp_generic /etc/postfix/smtp_generic
COPY aliases /etc/aliases
COPY process-bounce.py /usr/local/bin/process-bounce.py
COPY entrypoint.sh /entrypoint.sh COPY entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh RUN chmod +x /entrypoint.sh /usr/local/bin/process-bounce.py
# Generate Postfix maps for sender access # Generate Postfix maps for sender access, sender canonical, and aliases
RUN postmap /etc/postfix/sender_access RUN postmap /etc/postfix/sender_access && \
postmap /etc/postfix/smtp_generic && \
newaliases
# Expose SMTP # Expose SMTP
EXPOSE 25 EXPOSE 25

17
postfix/aliases Normal file
View File

@@ -0,0 +1,17 @@
# Postfix aliases for bounce handling
#
# This file defines how bounces are processed
# The bounces address pipes messages to our bounce processing script
# Bounce processing - pipe to Python script
bounces: "|/usr/bin/python3 /usr/local/bin/process-bounce.py"
# Standard aliases
postmaster: root
mailer-daemon: postmaster
abuse: postmaster
spam: postmaster
root: postmaster
# Default fallback
MAILER-DAEMON: bounces

View File

@@ -32,5 +32,28 @@ chmod 644 /etc/postfix/sender_access /etc/postfix/sender_access.db
# Set permissions on MySQL config # Set permissions on MySQL config
chmod 644 /etc/postfix/mysql_virtual_alias_maps.cf chmod 644 /etc/postfix/mysql_virtual_alias_maps.cf
# Configure bounce processing based on environment variable
if [ "${ENABLE_EMAIL_BOUNCE_PROCESSING:-false}" = "true" ]; then
echo "Email-based bounce processing enabled"
# Regenerate aliases database to enable bounce processing
newaliases
chmod 644 /etc/aliases /etc/aliases.db
# Generate smtp generic maps to ensure bounces come to our bounce address
postmap /etc/postfix/smtp_generic
chmod 644 /etc/postfix/smtp_generic /etc/postfix/smtp_generic.db
echo "Configured return path rewriting to bounces@lists.sasalliance.org"
else
echo "Email-based bounce processing disabled"
# Create minimal aliases without bounce processing
echo "postmaster: root" > /etc/aliases
echo "root: postmaster" >> /etc/aliases
newaliases
# Disable smtp generic maps
echo "# Email bounce processing disabled" > /etc/postfix/smtp_generic
postmap /etc/postfix/smtp_generic
fi
# Start Postfix in foreground # Start Postfix in foreground
exec postfix start-fg exec postfix start-fg

View File

@@ -33,3 +33,23 @@ smtpd_recipient_restrictions =
# Other recommended settings # Other recommended settings
alias_maps = hash:/etc/aliases alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases alias_database = hash:/etc/aliases
# Bounce handling configuration for email-based processing
# Configure bounce notification recipients
bounce_notice_recipient = bounces@lists.sasalliance.org
2bounce_notice_recipient = bounces@lists.sasalliance.org
delay_notice_recipient =
error_notice_recipient = bounces@lists.sasalliance.org
# Bounce settings
bounce_size_limit = 50000
bounce_queue_lifetime = 5d
maximal_bounce_delay = 1d
# Return path configuration - CRITICAL for bounce handling
# This ensures that when we relay emails via mailing lists through SES,
# bounces come back to our bounce processing address
#
# Use smtp_generic_maps instead of sender_canonical_maps because we only want
# to rewrite the return path for outbound SMTP (via SES), not for local delivery
smtp_generic_maps = hash:/etc/postfix/smtp_generic

348
postfix/process-bounce.py Normal file
View File

@@ -0,0 +1,348 @@
#!/usr/bin/env python3
"""
Email-based bounce processing script for Postfix
Parses bounce emails and updates the database with bounce information
This script is called by Postfix when bounce emails are received.
It reads the email from stdin, parses it for bounce information,
and updates the database accordingly.
"""
import sys
import os
import re
import email
from email.message import Message
import logging
from datetime import datetime
from typing import List, Tuple, Optional, Dict
import pymysql
# Configure logging
log_file = '/var/log/bounce-processor.log'
log_handlers = [logging.StreamHandler()]
# Add file handler if we can write to the log directory
try:
log_handlers.append(logging.FileHandler(log_file))
except (PermissionError, FileNotFoundError):
pass # Just use stdout if we can't write to log file
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
handlers=log_handlers
)
logger = logging.getLogger('bounce-processor')
# Database configuration from environment
MYSQL_HOST = os.getenv('MYSQL_HOST', 'mysql')
MYSQL_PORT = int(os.getenv('MYSQL_PORT', 3306))
MYSQL_DATABASE = os.getenv('MYSQL_DATABASE', 'maillist')
MYSQL_USER = os.getenv('MYSQL_USER', 'maillist')
MYSQL_PASSWORD = os.getenv('MYSQL_PASSWORD', '')
class BounceProcessor:
"""Processes bounce emails and updates the database"""
def __init__(self):
self.bounce_patterns = self._compile_bounce_patterns()
def _compile_bounce_patterns(self) -> List[re.Pattern]:
"""Compile regex patterns for detecting bounce information"""
patterns = [
# Standard bounce formats
re.compile(r'550.*?([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,})', re.IGNORECASE),
re.compile(r'554.*?([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,})', re.IGNORECASE),
re.compile(r'553.*?([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,})', re.IGNORECASE),
re.compile(r'552.*?([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,})', re.IGNORECASE),
re.compile(r'551.*?([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,})', re.IGNORECASE),
# Delivery Status Notification (DSN) format
re.compile(r'Final-Recipient:.*?([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,})', re.IGNORECASE),
re.compile(r'Original-Recipient:.*?([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,})', re.IGNORECASE),
# Common bounce message patterns
re.compile(r'user.*?([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}).*?unknown', re.IGNORECASE),
re.compile(r'([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}).*?does not exist', re.IGNORECASE),
re.compile(r'([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}).*?not found', re.IGNORECASE),
re.compile(r'([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}).*?mailbox.*?unavailable', re.IGNORECASE),
re.compile(r'([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}).*?recipient.*?rejected', re.IGNORECASE),
# Generic email extraction (fallback)
re.compile(r'([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,})', re.IGNORECASE),
]
return patterns
def parse_bounce_email(self, email_content: str) -> List[Dict]:
"""Parse bounce email and extract bounce information"""
try:
# Parse the email
msg = email.message_from_string(email_content)
bounces = []
# Get the email body
body = self._get_email_body(msg)
if not body:
logger.warning("No email body found")
return bounces
logger.info(f"Processing email body (first 500 chars): {body[:500]}")
# Extract bounced addresses
bounced_addresses = self._extract_bounced_addresses(body)
# Determine bounce type and create bounce records
for address in bounced_addresses:
bounce_info = self._analyze_bounce(body, address)
if bounce_info:
bounces.append(bounce_info)
return bounces
except Exception as e:
logger.error(f"Error parsing bounce email: {str(e)}")
return []
def _get_email_body(self, msg: Message) -> Optional[str]:
"""Extract the email body from the message"""
body = ""
if msg.is_multipart():
for part in msg.walk():
content_type = part.get_content_type()
if content_type in ['text/plain', 'text/html']:
payload = part.get_payload(decode=True)
if payload:
body += payload.decode('utf-8', errors='ignore') + "\n"
else:
payload = msg.get_payload(decode=True)
if payload:
body = payload.decode('utf-8', errors='ignore')
return body.strip() if body else None
def _extract_bounced_addresses(self, body: str) -> List[str]:
"""Extract bounced email addresses from the bounce message"""
addresses = []
for pattern in self.bounce_patterns:
matches = pattern.findall(body)
for match in matches:
email_addr = match.strip().lower()
if self._is_valid_email(email_addr) and email_addr not in addresses:
# Skip our own addresses
if not email_addr.endswith('@lists.sasalliance.org'):
addresses.append(email_addr)
logger.info(f"Extracted addresses: {addresses}")
return addresses
def _is_valid_email(self, email_addr: str) -> bool:
"""Validate email address format"""
email_pattern = re.compile(r'^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$')
return bool(email_pattern.match(email_addr))
def _analyze_bounce(self, body: str, email_addr: str) -> Optional[Dict]:
"""Analyze bounce message to determine bounce type and details"""
bounce_info = {
'email': email_addr,
'bounce_type': 'Undetermined',
'bounce_subtype': '',
'diagnostic_code': '',
'timestamp': datetime.now().strftime('%Y-%m-%d %H:%M:%S')
}
# Analyze bounce type based on SMTP codes and message content
if re.search(r'5[0-9]{2}', body): # 5xx codes are permanent failures
bounce_info['bounce_type'] = 'Permanent'
if re.search(r'550|551|553', body):
bounce_info['bounce_subtype'] = 'General'
elif re.search(r'552', body):
bounce_info['bounce_subtype'] = 'MailboxFull'
elif re.search(r'554', body):
bounce_info['bounce_subtype'] = 'MessageTooLarge'
elif re.search(r'4[0-9]{2}', body): # 4xx codes are temporary failures
bounce_info['bounce_type'] = 'Transient'
if re.search(r'450|451|452', body):
bounce_info['bounce_subtype'] = 'General'
elif re.search(r'452', body):
bounce_info['bounce_subtype'] = 'MailboxFull'
# Extract diagnostic code
smtp_code_match = re.search(r'([45][0-9]{2}.*?)[\r\n]', body)
if smtp_code_match:
bounce_info['diagnostic_code'] = smtp_code_match.group(1).strip()[:500] # Limit length
return bounce_info
def update_database(self, bounces: List[Dict]) -> None:
"""Update the database with bounce information"""
if not bounces:
logger.info("No bounces to process")
return
try:
connection = pymysql.connect(
host=MYSQL_HOST,
port=MYSQL_PORT,
database=MYSQL_DATABASE,
user=MYSQL_USER,
password=MYSQL_PASSWORD,
cursorclass=pymysql.cursors.DictCursor
)
cursor = connection.cursor()
for bounce in bounces:
try:
email_addr = bounce['email']
# Find member by email
cursor.execute("SELECT member_id FROM members WHERE email = %s", (email_addr,))
member = cursor.fetchone()
member_id = member['member_id'] if member else None
# Log the bounce
cursor.execute("""
INSERT INTO bounce_logs
(member_id, email, bounce_type, bounce_subtype, diagnostic_code, timestamp, created_at)
VALUES (%s, %s, %s, %s, %s, %s, NOW())
""", (member_id, email_addr, bounce['bounce_type'], bounce['bounce_subtype'],
bounce['diagnostic_code'], bounce['timestamp']))
logger.info(f"Logged bounce for {email_addr}: {bounce['bounce_type']}")
# Update member bounce status if member exists
if member_id:
if bounce['bounce_type'] == 'Permanent':
# Hard bounce - deactivate member
cursor.execute("""
UPDATE members
SET bounce_count = bounce_count + 1,
last_bounce_at = %s,
bounce_status = 'hard_bounce',
active = 0
WHERE member_id = %s
""", (bounce['timestamp'], member_id))
logger.info(f"Deactivated member {email_addr} due to hard bounce")
elif bounce['bounce_type'] == 'Transient':
# Soft bounce - increment counter and check threshold
cursor.execute("SELECT bounce_count, bounce_status FROM members WHERE member_id = %s", (member_id,))
current = cursor.fetchone()
if current and current['bounce_status'] != 'hard_bounce':
new_count = current['bounce_count'] + 1
new_status = 'soft_bounce' if new_count >= 3 else 'clean'
cursor.execute("""
UPDATE members
SET bounce_count = %s,
last_bounce_at = %s,
bounce_status = %s
WHERE member_id = %s
""", (new_count, bounce['timestamp'], new_status, member_id))
logger.info(f"Updated member {email_addr} bounce count to {new_count}, status: {new_status}")
else:
# Undetermined - just increment counter
cursor.execute("""
UPDATE members
SET bounce_count = bounce_count + 1,
last_bounce_at = %s
WHERE member_id = %s
""", (bounce['timestamp'], member_id))
logger.info(f"Updated member {email_addr} bounce count (undetermined)")
except Exception as e:
logger.error(f"Database error processing bounce for {bounce['email']}: {str(e)}")
continue
connection.commit()
cursor.close()
connection.close()
logger.info(f"Successfully processed {len(bounces)} bounces")
except pymysql.Error as e:
logger.error(f"Database connection error: {str(e)}")
except Exception as e:
logger.error(f"Unexpected error updating database: {str(e)}")
def main():
"""Main function - reads email from stdin and processes bounces"""
try:
logger.info("Starting bounce processing")
# Check if we're in test mode
test_mode = len(sys.argv) > 1 and sys.argv[1] == '--test'
if test_mode:
# Test mode - use sample bounce email
email_content = """From: Mail Delivery Subsystem <MAILER-DAEMON@example.com>
To: bounces@lists.sasalliance.org
Subject: Delivery Status Notification (Failure)
Content-Type: multipart/report; report-type=delivery-status; boundary="boundary123"
--boundary123
Content-Type: text/plain
This is a test bounce message.
The following address(es) failed:
testuser@example.com
SMTP error from remote mail server after RCPT TO:<testuser@example.com>:
550 5.1.1 User unknown
--boundary123
Content-Type: message/delivery-status
Reporting-MTA: dns; mail.example.com
Received-From-MTA: dns; localhost
Final-Recipient: rfc822; testuser@example.com
Action: failed
Status: 5.1.1
Diagnostic-Code: smtp; 550 5.1.1 User unknown
--boundary123--
"""
logger.info("Running in test mode with sample bounce email")
else:
# Read email from stdin
email_content = sys.stdin.read()
if not email_content.strip():
logger.warning("No email content received")
return
logger.info(f"Received email content ({len(email_content)} bytes)")
# Process the bounce
processor = BounceProcessor()
bounces = processor.parse_bounce_email(email_content)
if test_mode:
logger.info(f"Test mode - would process {len(bounces)} bounce(s):")
for bounce in bounces:
logger.info(f" {bounce}")
else:
if bounces:
processor.update_database(bounces)
logger.info(f"Processed {len(bounces)} bounce(s)")
else:
logger.info("No bounces detected in email")
except Exception as e:
logger.error(f"Error in main: {str(e)}")
import traceback
logger.error(f"Traceback: {traceback.format_exc()}")
sys.exit(1)
if __name__ == "__main__":
main()

23
postfix/smtp_generic Normal file
View File

@@ -0,0 +1,23 @@
# Postfix SMTP Generic Maps
# This file controls the return path (envelope sender) for outbound SMTP emails
# Only applies to emails being relayed through external SMTP (SES in our case)
#
# Format: original_sender rewritten_sender
#
# For mailing list emails, we want bounces to go to our bounce processing address
# This is critical for email-based bounce handling to work properly
# Rewrite envelope sender for all mailing list addresses to bounce address
# When emails are forwarded through mailing lists via SES, bounces come back to us
community@lists.sasalliance.org bounces@lists.sasalliance.org
board@lists.sasalliance.org bounces@lists.sasalliance.org
members@lists.sasalliance.org bounces@lists.sasalliance.org
announcements@lists.sasalliance.org bounces@lists.sasalliance.org
# Generic pattern - any @lists.sasalliance.org sender gets rewritten to bounce address
# This catches any new lists automatically
@lists.sasalliance.org bounces@lists.sasalliance.org
# Also rewrite any envelope sender that's sending through our system
# This ensures ALL outgoing mail via SES has our bounce address as return path
@sasalliance.org bounces@lists.sasalliance.org

View File

@@ -1,11 +0,0 @@
# Community mailing list - general announcements
community@lists.sasalliance.org james@pattinson.org, james.pattinson@sasalliance.org
# Board members mailing list
board@lists.sasalliance.org james.pattinson@sasalliance.org
# All members mailing list
members@lists.sasalliance.org james@pattinson.org, james.pattinson@sasalliance.org
# Announcements mailing list
announcements@lists.sasalliance.org james@pattinson.org, james.pattinson@sasalliance.org

206
simulate_bounce.sh Executable file
View File

@@ -0,0 +1,206 @@
#!/bin/bash
# Simulate Bounce Events for Testing
# This script inserts bounce data directly into the database to test the UI
# without needing to set up AWS SNS
set -e
echo "=== Bounce Simulation Script ==="
echo
echo "This script will create test bounce events for existing members"
echo
# Get database password from .env
DB_PASSWORD=$(grep MYSQL_ROOT_PASSWORD /home/jamesp/docker/maillist/.env | cut -d'=' -f2)
# Check if containers are running
if ! sudo docker-compose ps | grep -q "maillist-mysql.*Up"; then
echo "❌ MySQL container is not running. Starting containers..."
sudo docker-compose up -d
echo "Waiting for MySQL to be ready..."
sleep 5
fi
echo "1. Fetching existing members..."
echo
# Get list of members
MEMBERS=$(sudo docker-compose exec -T mysql mysql -u root -p"$DB_PASSWORD" maillist -N -e "SELECT member_id, name, email FROM members LIMIT 5;")
if [ -z "$MEMBERS" ]; then
echo "❌ No members found in database. Please add members first."
exit 1
fi
echo "Available members:"
echo "$MEMBERS" | while read -r line; do
MEMBER_ID=$(echo "$line" | awk '{print $1}')
NAME=$(echo "$line" | awk '{print $2, $3}')
EMAIL=$(echo "$line" | awk '{$1=""; $2=""; print $0}' | xargs)
echo " [$MEMBER_ID] $NAME - $EMAIL"
done
echo
# Prompt for member ID
read -p "Enter member ID to simulate bounce for (or press Enter for first member): " MEMBER_ID
if [ -z "$MEMBER_ID" ]; then
MEMBER_ID=$(echo "$MEMBERS" | head -1 | awk '{print $1}')
echo "Using member ID: $MEMBER_ID"
fi
# Get member details
MEMBER_INFO=$(sudo docker-compose exec -T mysql mysql -u root -p"$DB_PASSWORD" maillist -N -e "SELECT member_id, name, email FROM members WHERE member_id = $MEMBER_ID;")
if [ -z "$MEMBER_INFO" ]; then
echo "❌ Member ID $MEMBER_ID not found"
exit 1
fi
MEMBER_EMAIL=$(echo "$MEMBER_INFO" | awk '{$1=""; $2=""; print $0}' | xargs)
MEMBER_NAME=$(echo "$MEMBER_INFO" | awk '{print $2, $3}')
echo
echo "Selected member: $MEMBER_NAME ($MEMBER_EMAIL)"
echo
# Bounce type selection
echo "Select bounce type to simulate:"
echo " 1) Soft Bounce (Transient - e.g., mailbox full)"
echo " 2) Hard Bounce (Permanent - e.g., invalid address)"
echo " 3) Multiple Soft Bounces (3 bounces to trigger soft_bounce status)"
echo " 4) Undetermined Bounce"
echo
read -p "Enter choice (1-4): " BOUNCE_CHOICE
case $BOUNCE_CHOICE in
1)
BOUNCE_TYPE="Transient"
BOUNCE_SUBTYPE="MailboxFull"
DIAGNOSTIC="smtp; 452 4.2.2 Mailbox full"
COUNT=1
;;
2)
BOUNCE_TYPE="Permanent"
BOUNCE_SUBTYPE="General"
DIAGNOSTIC="smtp; 550 5.1.1 User unknown"
COUNT=1
;;
3)
BOUNCE_TYPE="Transient"
BOUNCE_SUBTYPE="General"
DIAGNOSTIC="smtp; 451 4.4.1 Temporary failure"
COUNT=3
;;
4)
BOUNCE_TYPE="Undetermined"
BOUNCE_SUBTYPE=""
DIAGNOSTIC="Unknown error occurred"
COUNT=1
;;
*)
echo "Invalid choice"
exit 1
;;
esac
echo
echo "Simulating $COUNT bounce event(s)..."
echo
# Insert bounce events
for i in $(seq 1 $COUNT); do
TIMESTAMP=$(date -u -d "-$((i * 24)) hours" '+%Y-%m-%d %H:%M:%S')
FEEDBACK_ID="test-feedback-$(date +%s)-$i"
echo "Creating bounce event $i/$COUNT (timestamp: $TIMESTAMP)..."
sudo docker-compose exec -T mysql mysql -u root -p"$DB_PASSWORD" maillist <<EOF
-- Insert bounce log
INSERT INTO bounce_logs (member_id, email, bounce_type, bounce_subtype, diagnostic_code, timestamp, feedback_id)
VALUES ($MEMBER_ID, '$MEMBER_EMAIL', '$BOUNCE_TYPE', '$BOUNCE_SUBTYPE', '$DIAGNOSTIC', '$TIMESTAMP', '$FEEDBACK_ID');
EOF
# Update member bounce status
if [ "$BOUNCE_TYPE" = "Permanent" ]; then
echo " → Hard bounce: Deactivating member..."
sudo docker-compose exec -T mysql mysql -u root -p"$DB_PASSWORD" maillist <<EOF
UPDATE members
SET bounce_count = bounce_count + 1,
last_bounce_at = '$TIMESTAMP',
bounce_status = 'hard_bounce',
active = 0
WHERE member_id = $MEMBER_ID;
EOF
elif [ "$BOUNCE_TYPE" = "Transient" ]; then
# Check current bounce count
CURRENT_COUNT=$(sudo docker-compose exec -T mysql mysql -u root -p"$DB_PASSWORD" maillist -N -e "SELECT bounce_count FROM members WHERE member_id = $MEMBER_ID;")
NEW_COUNT=$((CURRENT_COUNT + 1))
if [ $NEW_COUNT -ge 3 ]; then
echo " → Soft bounce threshold reached: Marking as soft_bounce..."
BOUNCE_STATUS="soft_bounce"
else
echo " → Soft bounce: Incrementing counter ($NEW_COUNT)..."
BOUNCE_STATUS="clean"
fi
sudo docker-compose exec -T mysql mysql -u root -p"$DB_PASSWORD" maillist <<EOF
UPDATE members
SET bounce_count = $NEW_COUNT,
last_bounce_at = '$TIMESTAMP',
bounce_status = '$BOUNCE_STATUS'
WHERE member_id = $MEMBER_ID;
EOF
else
echo " → Undetermined bounce: Incrementing counter..."
sudo docker-compose exec -T mysql mysql -u root -p"$DB_PASSWORD" maillist <<EOF
UPDATE members
SET bounce_count = bounce_count + 1,
last_bounce_at = '$TIMESTAMP'
WHERE member_id = $MEMBER_ID;
EOF
fi
done
echo
echo "✅ Bounce simulation complete!"
echo
# Show updated member status
echo "Updated member status:"
sudo docker-compose exec -T mysql mysql -u root -p"$DB_PASSWORD" maillist -e "
SELECT
member_id,
name,
email,
active,
bounce_count,
last_bounce_at,
bounce_status
FROM members
WHERE member_id = $MEMBER_ID\G
"
echo
echo "Bounce history for this member:"
sudo docker-compose exec -T mysql mysql -u root -p"$DB_PASSWORD" maillist -e "
SELECT
bounce_id,
bounce_type,
bounce_subtype,
diagnostic_code,
timestamp
FROM bounce_logs
WHERE member_id = $MEMBER_ID
ORDER BY timestamp DESC;
"
echo
echo "🎉 You can now view this bounce in the web UI:"
echo " 1. Open http://localhost:3000"
echo " 2. Go to the Members tab"
echo " 3. Look for $MEMBER_NAME"
echo " 4. Click the 'Bounces' button to see the history"
echo
echo "To simulate more bounces, run this script again!"

View File

@@ -0,0 +1,46 @@
<!DOCTYPE html>
<html>
<head>
<title>CORS Understanding Test</title>
</head>
<body>
<h1>CORS Test</h1>
<div id="results"></div>
<script>
const results = document.getElementById('results');
// Test 1: Same-origin request (should work without CORS headers)
console.log('Current origin:', window.location.origin);
// Test what your API client actually does
const protocol = window.location.protocol;
const hostname = window.location.hostname;
let baseURL;
if (hostname === 'localhost' || hostname === '127.0.0.1') {
baseURL = `${protocol}//${hostname}:8000`;
results.innerHTML += `<p><strong>Development Mode Detected</strong></p>`;
results.innerHTML += `<p>Frontend: ${window.location.origin}</p>`;
results.innerHTML += `<p>API: ${baseURL}</p>`;
results.innerHTML += `<p>This is a <strong>CROSS-ORIGIN</strong> request - CORS applies!</p>`;
} else {
baseURL = `${protocol}//${hostname}/api`;
results.innerHTML += `<p><strong>Production Mode Detected</strong></p>`;
results.innerHTML += `<p>Frontend: ${window.location.origin}</p>`;
results.innerHTML += `<p>API: ${baseURL}</p>`;
results.innerHTML += `<p>This is a <strong>SAME-ORIGIN</strong> request - NO CORS needed!</p>`;
}
// Test the actual request
fetch(`${baseURL}/health`)
.then(response => response.json())
.then(data => {
results.innerHTML += `<p style="color: green;">✅ API Request Successful: ${JSON.stringify(data)}</p>`;
})
.catch(error => {
results.innerHTML += `<p style="color: red;">❌ API Request Failed: ${error.message}</p>`;
});
</script>
</body>
</html>

45
test_bounce_quick.sh Normal file
View File

@@ -0,0 +1,45 @@
#!/bin/bash
# Quick SES Bounce Test - One-liner to test bounce handling
echo "🚀 Testing SES Bounce Handling"
echo "================================"
echo ""
# Check if API and Postfix are running
if ! sudo docker compose ps | grep -q "maillist-api.*Up"; then
echo "❌ API container not running. Start with: sudo docker compose up -d"
exit 1
fi
if ! sudo docker compose ps | grep -q "maillist-postfix.*Up"; then
echo "❌ Postfix container not running. Start with: sudo docker compose up -d"
exit 1
fi
echo "✅ Containers are running"
echo ""
# Start watching logs in background
echo "📋 Opening log viewer (press Ctrl+C to stop)..."
echo ""
sleep 2
# Show the command they can run
echo "Run this command to send a test bounce:"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "echo 'Test bounce' | sudo docker compose exec -T postfix mail -s 'Bounce Test' bounce@simulator.amazonses.com"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo ""
echo "Then wait 30-60 seconds and check:"
echo ""
echo "1. API logs (in this window)"
echo "2. Database: sudo docker compose exec mysql mysql -u maillist -pmaillist maillist -e 'SELECT * FROM bounce_logs ORDER BY created_at DESC LIMIT 5;'"
echo "3. Web UI: http://localhost:3000 → Members tab"
echo ""
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "Watching API logs now..."
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo ""
# Follow logs
sudo docker compose logs api -f --tail 20

76
test_bounce_webhook.sh Executable file
View File

@@ -0,0 +1,76 @@
#!/bin/bash
# Test SNS Bounce Webhook
# This script simulates an SNS bounce notification (without signature verification for testing)
API_URL="http://localhost:8000"
echo "Testing SNS Bounce Webhook..."
echo
# Test 1: Health check
echo "1. Testing API health..."
curl -s "$API_URL/health" | jq .
echo
# Test 2: Create a test subscription confirmation (the webhook will auto-confirm)
echo "2. Testing subscription confirmation handling..."
curl -s -X POST "$API_URL/webhooks/sns" \
-H "Content-Type: application/json" \
-d '{
"Type": "SubscriptionConfirmation",
"MessageId": "test-message-id",
"Token": "test-token",
"TopicArn": "arn:aws:sns:eu-west-2:123456789:test-topic",
"Message": "You have chosen to subscribe to the topic",
"SubscribeURL": "https://example.com/subscribe",
"Timestamp": "2025-01-01T12:00:00.000Z"
}' || echo "Note: Signature verification will fail for test messages (expected)"
echo
echo
# Test 3: Simulated bounce notification (will fail signature verification)
echo "3. Testing bounce notification structure..."
echo "Note: In production, AWS SNS will send properly signed messages."
echo "This test demonstrates the expected structure."
echo
cat << 'EOF' > /tmp/test_bounce.json
{
"Type": "Notification",
"MessageId": "test-notification-id",
"TopicArn": "arn:aws:sns:eu-west-2:123456789:test-topic",
"Subject": "Amazon SES Email Event Notification",
"Message": "{\"notificationType\":\"Bounce\",\"bounce\":{\"bounceType\":\"Permanent\",\"bounceSubType\":\"General\",\"bouncedRecipients\":[{\"emailAddress\":\"bounce@simulator.amazonses.com\",\"diagnosticCode\":\"smtp; 550 5.1.1 user unknown\"}],\"timestamp\":\"2025-01-01T12:00:00.000Z\",\"feedbackId\":\"test-feedback-id\"}}",
"Timestamp": "2025-01-01T12:00:00.000Z",
"SignatureVersion": "1",
"Signature": "test-signature",
"SigningCertURL": "https://sns.eu-west-2.amazonaws.com/test.pem"
}
EOF
cat /tmp/test_bounce.json | jq .
echo
echo "Expected behavior: Signature verification will fail without real AWS credentials"
echo "In production, AWS SNS will send properly signed messages that will be verified"
echo
# Test 4: Check if database schema has bounce tables
echo "4. Checking database schema for bounce tables..."
sudo docker-compose exec -T mysql mysql -u maillist -pmaillist maillist -e "SHOW TABLES LIKE '%bounce%';" 2>/dev/null
echo
# Test 5: Check members table for bounce columns
echo "5. Checking members table for bounce columns..."
sudo docker-compose exec -T mysql mysql -u maillist -pmaillist maillist -e "DESCRIBE members;" 2>/dev/null | grep -i bounce
echo
echo "✓ Setup complete!"
echo
echo "To test with real AWS SNS:"
echo "1. Set up SNS topic in AWS Console"
echo "2. Subscribe webhook: https://your-domain.com:8000/webhooks/sns"
echo "3. Configure SES to send bounce notifications to the SNS topic"
echo "4. Send test email to bounce@simulator.amazonses.com"
echo
echo "See BOUNCE_HANDLING_SETUP.md for detailed setup instructions"

View File

@@ -38,23 +38,29 @@ web/
## Usage ## Usage
### Development ### Accessing the Interface
1. Ensure the API is running on port 8000
2. Open `index.html` in a web browser
3. Enter your API token to authenticate
4. Start managing your mailing lists!
### Production (Docker)
The web frontend is served using Nginx and included in the Docker Compose setup.
**Via Docker (Recommended):**
```bash ```bash
# Build and start all services # Ensure all services are running
docker-compose up --build sudo docker-compose up -d
# Access the web interface # Access the web interface
open http://localhost:3000 open http://localhost:3000
``` ```
**For Development:**
You can also open `index.html` directly in a browser, but the API must be running on port 8000.
### Getting Started
1. **Authenticate**: Enter your API_TOKEN (from `.env` file) when prompted
2. **Manage Lists**: Click the "Lists" tab to view and manage mailing lists
3. **Manage Members**: Click the "Members" tab to add and edit member information
4. **Manage Subscriptions**: Click "Subscriptions" button on any member to toggle their list memberships
The interface saves your token in browser storage, so you won't need to re-enter it on subsequent visits.
## Features Overview ## Features Overview
### Authentication ### Authentication
@@ -65,24 +71,39 @@ open http://localhost:3000
### Subscription Management (New & Improved!) ### Subscription Management (New & Improved!)
#### Member-Centric Subscription Management #### Member-Centric Subscription Management
The subscription management has been completely overhauled for a much better user experience: The subscription management has been completely redesigned for the best user experience:
1. **Access Member Subscriptions**: In the Members tab, click the "Subscriptions" button next to any member **How to Use:**
2. **Visual Toggle Interface**: See all available mailing lists with modern toggle switches 1. Navigate to the **Members** tab
3. **Intuitive Controls**: 2. Find the member you want to manage
- Green toggle = Member is subscribed 3. Click the **"Subscriptions"** button next to their name
- Gray toggle = Member is not subscribed 4. A modal appears showing all available mailing lists with toggle switches
- Click anywhere on a list item to toggle subscription
4. **Batch Operations**: Make multiple changes and save them all at once
5. **Real-time Feedback**: The save button shows how many changes you've made
#### Benefits Over Previous System **Visual Interface:**
- **Much faster** - No need to add subscriptions one by one - **Green toggle** = Member is subscribed to this list
- **Gray toggle** = Member is not subscribed to this list
- **Click anywhere** on a list item to toggle subscription on/off
**Batch Operations:**
- Make multiple changes (subscribe to some lists, unsubscribe from others)
- The "Save Changes" button shows how many changes you've made (e.g., "Save 3 Changes")
- All changes are saved together when you click "Save Changes"
- Click "Cancel" to discard all changes
**Benefits:**
-**Fast** - Toggle multiple subscriptions at once
-**Visual** - See all subscriptions at a glance with color coding -**Visual** - See all subscriptions at a glance with color coding
-**Intuitive** - Toggle switches work like modern mobile apps -**Intuitive** - Works like modern mobile app switches
-**Batch operations** - Change multiple subscriptions simultaneously -**Smart** - Only saves actual changes, not unchanged items
-**Less error-prone** - Clear visual feedback prevents mistakes -**Clear** - Shows exactly how many changes you're about to save
-**Change tracking** - Only saves actual changes, not unchanged items
#### Alternative: Legacy Bulk Subscription
For power users who prefer the old approach:
1. Go to the **Subscriptions** tab
2. Select list and member from dropdowns
3. Click "Add Subscription"
This is still available but the member-centric approach is much more efficient for managing multiple subscriptions.
### Mailing Lists ### Mailing Lists
- View all mailing lists in a clean table - View all mailing lists in a clean table

View File

@@ -9,26 +9,92 @@
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.0.0/css/all.min.css"> <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.0.0/css/all.min.css">
</head> </head>
<body> <body>
<!-- Header --> <!-- Login Page -->
<header class="header"> <div class="login-page" id="loginPage">
<div class="container"> <div class="login-container">
<div class="header-content"> <div class="login-header">
<h1 class="logo"> <div class="login-logo">
<i class="fas fa-envelope"></i> <i class="fas fa-envelope"></i>
Mailing List Manager </div>
</h1> <h1>Mailing List Manager</h1>
<div class="auth-section"> <p>Sign in to manage your mailing lists</p>
<div class="auth-controls" id="authControls"> </div>
<input type="password" id="apiToken" placeholder="Enter API Token" class="token-input"> <form class="login-form" id="loginForm">
<button class="btn btn-primary" id="loginBtn">Login</button> <!-- Login Error Message -->
<div class="login-error" id="loginError" style="display: none;">
<i class="fas fa-exclamation-circle"></i>
<span id="loginErrorMessage">Invalid username or password</span>
</div>
<div class="form-group">
<label for="username">
<i class="fas fa-user"></i>
Username
</label>
<input type="text" id="username" placeholder="Enter your username" required autocomplete="username">
</div>
<div class="form-group">
<label for="password">
<i class="fas fa-lock"></i>
Password
</label>
<input type="password" id="password" placeholder="Enter your password" required autocomplete="current-password">
</div>
<button type="submit" class="btn btn-primary btn-block" id="loginBtn">
<span>Sign In</span>
<i class="fas fa-arrow-right"></i>
</button>
</form>
<div class="login-footer">
<p class="text-muted text-sm">
<i class="fas fa-shield-alt"></i>
Secure authentication required
</p>
</div>
</div>
</div>
<!-- Menu Bar Header (shown after login) -->
<header class="menu-bar" id="mainHeader" style="display: none;">
<div class="menu-bar-content">
<div class="app-title">
<i class="fas fa-envelope"></i>
<span>Mailing List Manager</span>
</div>
<div class="menu-spacer"></div>
<div class="user-dropdown" id="userDropdown">
<button class="user-dropdown-trigger" id="userDropdownTrigger">
<div class="user-avatar">
<i class="fas fa-user"></i>
</div> </div>
<div class="user-info" id="userInfo" style="display: none;"> <div class="user-details">
<span class="status-indicator"> <span class="user-name" id="currentUsername">User</span>
<i class="fas fa-check-circle"></i> <span class="user-role" id="currentUserRole">role</span>
Connected
</span>
<button class="btn btn-secondary" id="logoutBtn">Logout</button>
</div> </div>
<i class="fas fa-chevron-down dropdown-arrow"></i>
</button>
<div class="user-dropdown-menu" id="userDropdownMenu">
<div class="dropdown-header">
<div class="dropdown-user-info">
<div class="dropdown-avatar">
<i class="fas fa-user"></i>
</div>
<div class="dropdown-details">
<div class="dropdown-name" id="dropdownUsername">User</div>
<div class="dropdown-role" id="dropdownUserRole">role</div>
</div>
</div>
</div>
<div class="dropdown-divider"></div>
<button class="dropdown-item" id="userManagementBtn" style="display: none;">
<i class="fas fa-user-shield"></i>
<span>User Management</span>
</button>
<div class="dropdown-divider" id="userManagementDivider" style="display: none;"></div>
<button class="dropdown-item" id="logoutBtn">
<i class="fas fa-sign-out-alt"></i>
<span>Sign Out</span>
</button>
</div> </div>
</div> </div>
</div> </div>
@@ -38,7 +104,7 @@
<main class="main-content" id="mainContent" style="display: none;"> <main class="main-content" id="mainContent" style="display: none;">
<div class="container"> <div class="container">
<!-- Navigation Tabs --> <!-- Navigation Tabs -->
<nav class="tab-nav"> <nav class="tab-nav" id="tabNav">
<button class="tab-btn active" data-tab="lists"> <button class="tab-btn active" data-tab="lists">
<i class="fas fa-list"></i> <i class="fas fa-list"></i>
Mailing Lists Mailing Lists
@@ -89,17 +155,37 @@
<!-- Members Tab --> <!-- Members Tab -->
<div class="tab-content" id="members-tab"> <div class="tab-content" id="members-tab">
<div class="section-header"> <div class="section-header">
<div class="header-content"> <h2>Members</h2>
<h2>Members</h2> <div class="button-group">
<div class="header-help"> <button class="btn btn-primary" id="addMemberBtn">
<i class="fas fa-info-circle"></i> <i class="fas fa-plus"></i>
<span>Click the "Lists" button next to any member to manage their subscriptions</span> Add Member
</button>
<button class="btn btn-secondary" id="showBulkImportBtn">
<i class="fas fa-upload"></i>
Bulk Import CSV
</button>
</div>
</div>
<!-- Member Search -->
<div class="search-section">
<div class="search-container">
<div class="search-input-wrapper">
<i class="fas fa-search search-icon"></i>
<input type="text"
id="memberSearchInput"
class="search-input"
placeholder="Search members by name or email..."
autocomplete="off">
<button class="search-clear" id="memberSearchClear" style="display: none;">
<i class="fas fa-times"></i>
</button>
</div>
<div class="search-results-info" id="memberSearchInfo" style="display: none;">
<span id="memberSearchCount">0</span> members found
</div> </div>
</div> </div>
<button class="btn btn-primary" id="addMemberBtn">
<i class="fas fa-plus"></i>
Add Member
</button>
</div> </div>
<div class="data-table"> <div class="data-table">
@@ -120,6 +206,35 @@
</div> </div>
</div> </div>
<!-- Users Tab (Admin Only) -->
<div class="tab-content" id="users-tab">
<div class="section-header">
<h2>User Management</h2>
<button class="btn btn-primary" id="addUserBtn">
<i class="fas fa-plus"></i>
Add User
</button>
</div>
<div class="data-table">
<table class="table" id="usersTable">
<thead>
<tr>
<th>Username</th>
<th>Role</th>
<th>Created</th>
<th>Last Login</th>
<th>Status</th>
<th>Actions</th>
</tr>
</thead>
<tbody id="usersTableBody">
<!-- Dynamic content -->
</tbody>
</table>
</div>
</div>
</div> </div>
</main> </main>
@@ -273,6 +388,295 @@
</div> </div>
</div> </div>
<!-- Bulk Import Modal -->
<div class="modal modal-large" id="bulkImportModal">
<div class="modal-content">
<div class="modal-header">
<h3>Bulk Import Members from CSV</h3>
<button class="modal-close" id="bulkImportModalClose">
<i class="fas fa-times"></i>
</button>
</div>
<div class="modal-body">
<!-- Step 1: File Upload -->
<div class="import-step" id="importStep1">
<h4><i class="fas fa-upload"></i> Step 1: Upload CSV File</h4>
<p class="text-muted">Upload a CSV file containing member information. The file must have 'Name' and 'Email' columns.</p>
<div class="file-upload-area" id="fileUploadArea">
<div class="file-upload-content">
<i class="fas fa-cloud-upload-alt"></i>
<p>Click to select a CSV file or drag and drop</p>
<input type="file" id="csvFileInput" accept=".csv" style="display: none;">
</div>
</div>
<div class="file-info" id="fileInfo" style="display: none;">
<div class="file-details">
<i class="fas fa-file-csv"></i>
<span id="fileName">file.csv</span>
<span id="fileSize" class="text-muted">(0 KB)</span>
</div>
<button type="button" class="btn btn-sm btn-secondary" id="removeFileBtn">
<i class="fas fa-times"></i> Remove
</button>
</div>
<div class="csv-format-help">
<h5><i class="fas fa-info-circle"></i> CSV Format</h5>
<p>Your CSV file should look like this:</p>
<div class="code-example">
<code>Name,Email,Delivery,Member since<br>
Ahmed Ajzan,ahmedajzan@doctors.org.uk,,2025-04-06T18:44:26.819454<br>
Alan Bailey,abgower@icloud.com,,2025-04-06T18:44:26.824446</code>
</div>
<p class="text-muted text-sm">Only the 'Name' and 'Email' columns are required. Additional columns will be ignored.</p>
</div>
</div>
<!-- Step 2: Preview -->
<div class="import-step" id="importStep2" style="display: none;">
<h4><i class="fas fa-eye"></i> Step 2: Preview Data</h4>
<p class="text-muted">Review the data that will be imported:</p>
<div class="preview-stats" id="previewStats">
<div class="stat-item">
<span class="stat-value" id="totalRowsCount">0</span>
<span class="stat-label">Total Rows</span>
</div>
<div class="stat-item">
<span class="stat-value" id="validRowsCount">0</span>
<span class="stat-label">Valid Rows</span>
</div>
<div class="stat-item">
<span class="stat-value" id="errorRowsCount">0</span>
<span class="stat-label">Errors</span>
</div>
</div>
<div class="preview-table-container" id="previewTableContainer">
<table class="table table-sm" id="previewTable">
<thead>
<tr>
<th>Row</th>
<th>Name</th>
<th>Email</th>
<th>Status</th>
</tr>
</thead>
<tbody id="previewTableBody">
<!-- Dynamic content -->
</tbody>
</table>
</div>
<div class="error-list" id="errorList" style="display: none;">
<div class="error-header">
<h5><i class="fas fa-exclamation-triangle"></i> Errors Found</h5>
<button type="button" class="btn btn-sm btn-secondary" id="toggleErrorsBtn">
<span id="errorToggleText">Show All</span>
<i class="fas fa-chevron-down" id="errorToggleIcon"></i>
</button>
</div>
<div class="error-summary" id="errorSummary">
<!-- Summary will be shown here -->
</div>
<div class="error-details" id="errorDetails" style="display: none;">
<div class="error-table-container">
<table class="table table-sm" id="errorTable">
<thead>
<tr>
<th>Row</th>
<th>Issue</th>
<th>Name</th>
<th>Email</th>
<th>Raw Data</th>
</tr>
</thead>
<tbody id="errorTableBody">
<!-- Dynamic content -->
</tbody>
</table>
</div>
</div>
</div>
</div>
<!-- Step 3: List Selection -->
<div class="import-step" id="importStep3" style="display: none;">
<h4><i class="fas fa-list"></i> Step 3: Select Mailing Lists</h4>
<p class="text-muted">Choose which mailing lists these members should be subscribed to:</p>
<div class="list-selection" id="listSelection">
<!-- Dynamic content -->
</div>
<div class="selection-info">
<p class="text-muted text-sm">
<i class="fas fa-info-circle"></i>
If a member already exists, they will be added to the selected lists (existing subscriptions will be preserved).
</p>
</div>
</div>
<!-- Step 4: Import Results -->
<div class="import-step" id="importStep4" style="display: none;">
<h4><i class="fas fa-check-circle"></i> Step 4: Import Complete</h4>
<div class="import-results" id="importResults">
<div class="result-stats">
<div class="stat-item">
<span class="stat-value" id="processedCount">0</span>
<span class="stat-label">Processed</span>
</div>
<div class="stat-item">
<span class="stat-value" id="createdCount">0</span>
<span class="stat-label">Created</span>
</div>
<div class="stat-item">
<span class="stat-value" id="updatedCount">0</span>
<span class="stat-label">Updated</span>
</div>
<div class="stat-item">
<span class="stat-value" id="subscriptionsCount">0</span>
<span class="stat-label">Subscriptions</span>
</div>
</div>
<div class="result-errors" id="resultErrors" style="display: none;">
<h5><i class="fas fa-exclamation-triangle"></i> Import Errors</h5>
<ul id="resultErrorList">
<!-- Dynamic content -->
</ul>
</div>
</div>
</div>
</div>
<div class="modal-actions">
<button type="button" class="btn btn-secondary" id="bulkImportBackBtn" style="display: none;">
<i class="fas fa-arrow-left"></i> Back
</button>
<button type="button" class="btn btn-secondary" id="bulkImportCancelBtn">Cancel</button>
<button type="button" class="btn btn-primary" id="bulkImportNextBtn" disabled>
Next <i class="fas fa-arrow-right"></i>
</button>
<button type="button" class="btn btn-primary" id="bulkImportBtn" style="display: none;">
<i class="fas fa-upload"></i> Import Members
</button>
<button type="button" class="btn btn-success" id="bulkImportDoneBtn" style="display: none;">
<i class="fas fa-check"></i> Done
</button>
</div>
</div>
</div>
<!-- Add/Edit User Modal -->
<div class="modal" id="userModal">
<div class="modal-content">
<div class="modal-header">
<h3 id="userModalTitle">Add User</h3>
<button class="modal-close" id="userModalClose">
<i class="fas fa-times"></i>
</button>
</div>
<form id="userForm">
<div class="form-group">
<label for="userName">Username *</label>
<input type="text" id="userName" name="userName" required>
</div>
<div class="form-group">
<label for="userPassword">Password *</label>
<input type="password" id="userPassword" name="userPassword" required>
<div class="form-help" id="passwordHelp">
Leave blank to keep current password (when editing)
</div>
</div>
<div class="form-group">
<label for="userRole">Role *</label>
<select id="userRole" name="userRole" required>
<option value="">Select a role...</option>
<option value="administrator">Administrator - Full access including user management</option>
<option value="operator">Operator - Read/write access to lists and members</option>
<option value="read-only">Read-Only - View-only access</option>
</select>
</div>
<div class="form-group">
<label class="checkbox-label">
<input type="checkbox" id="userActive" name="userActive" checked>
<span class="checkmark"></span>
Active
</label>
</div>
<div class="modal-actions">
<button type="button" class="btn btn-secondary" id="userCancelBtn">Cancel</button>
<button type="submit" class="btn btn-primary" id="userSubmitBtn">Save</button>
</div>
</form>
</div>
</div>
<!-- Bounce History Modal -->
<div class="modal" id="bounceHistoryModal">
<div class="modal-content">
<div class="modal-header">
<h3 id="bounceHistoryTitle">Bounce History</h3>
<button class="modal-close" id="bounceHistoryModalClose">
<i class="fas fa-times"></i>
</button>
</div>
<div class="modal-body">
<div class="member-info-banner" id="bounceHistoryMemberInfo">
<div class="member-avatar">
<i class="fas fa-user"></i>
</div>
<div class="member-details">
<h4 id="bounceHistoryMemberName">Member Name</h4>
<p id="bounceHistoryMemberEmail">member@example.com</p>
</div>
</div>
<div class="bounce-summary">
<div class="bounce-stat">
<i class="fas fa-exclamation-triangle"></i>
<div>
<div class="bounce-stat-label">Total Bounces</div>
<div class="bounce-stat-value" id="bounceTotalCount">0</div>
</div>
</div>
<div class="bounce-stat">
<i class="fas fa-clock"></i>
<div>
<div class="bounce-stat-label">Last Bounce</div>
<div class="bounce-stat-value" id="bounceLastDate">Never</div>
</div>
</div>
<div class="bounce-stat">
<i class="fas fa-info-circle"></i>
<div>
<div class="bounce-stat-label">Status</div>
<div class="bounce-stat-value" id="bounceStatusText">Clean</div>
</div>
</div>
</div>
<div class="bounce-history-section">
<h5>Bounce Events</h5>
<div class="bounce-history-list" id="bounceHistoryList">
<!-- Dynamic content will be inserted here -->
</div>
</div>
</div>
<div class="modal-actions">
<button type="button" class="btn btn-secondary" id="bounceHistoryCloseBtn">Close</button>
<button type="button" class="btn btn-warning" id="bounceHistoryResetBtn" data-requires-write>
<i class="fas fa-redo"></i>
Reset Bounce Status
</button>
</div>
</div>
</div>
<!-- Confirmation Modal --> <!-- Confirmation Modal -->
<div class="modal" id="confirmModal"> <div class="modal" id="confirmModal">
<div class="modal-content"> <div class="modal-content">
@@ -293,7 +697,7 @@
</div> </div>
<script src="static/js/api.js"></script> <script src="static/js/api.js"></script>
<script src="static/js/ui.js"></script>
<script src="static/js/app.js"></script> <script src="static/js/app.js"></script>
<script src="static/js/ui.js"></script>
</body> </body>
</html> </html>

File diff suppressed because it is too large Load Diff

View File

@@ -25,8 +25,8 @@ class APIClient {
return `${protocol}//${hostname}:8000`; return `${protocol}//${hostname}:8000`;
} }
// If running in production, assume API is on port 8000 // If running in production behind a reverse proxy, use /api path
return `${protocol}//${hostname}:8000`; return `${protocol}//${hostname}/api`;
} }
/** /**
@@ -50,9 +50,14 @@ class APIClient {
*/ */
async request(endpoint, options = {}) { async request(endpoint, options = {}) {
const url = `${this.baseURL}${endpoint}`; const url = `${this.baseURL}${endpoint}`;
// Merge options, with custom headers taking precedence
const config = { const config = {
headers: { ...this.headers }, ...options,
...options headers: {
...this.headers,
...(options.headers || {})
}
}; };
try { try {
@@ -100,10 +105,48 @@ class APIClient {
return this.request('/health'); return this.request('/health');
} }
async getConfig() {
return this.request('/config');
}
async testAuth() { async testAuth() {
return this.request('/'); return this.request('/');
} }
async login(username, password) {
// Don't include Authorization header for login
const tempHeaders = { ...this.headers };
delete tempHeaders['Authorization'];
const response = await this.request('/auth/login', {
method: 'POST',
headers: tempHeaders,
body: JSON.stringify({ username, password })
});
// Set the token from the response
if (response.access_token) {
this.setToken(response.access_token);
}
return response;
}
async logout() {
try {
await this.request('/auth/logout', {
method: 'POST'
});
} finally {
// Clear token even if logout fails
this.clearToken();
}
}
async getCurrentUser() {
return this.request('/auth/me');
}
// Mailing Lists API // Mailing Lists API
async getLists() { async getLists() {
return this.request('/lists'); return this.request('/lists');
@@ -184,6 +227,55 @@ class APIClient {
method: 'DELETE' method: 'DELETE'
}); });
} }
/**
* Bulk import members from CSV data
*/
async bulkImportMembers(csvData, listIds) {
return this.request('/bulk-import', {
method: 'POST',
body: JSON.stringify({
csv_data: csvData,
list_ids: listIds
})
});
}
// User Management API
async getUsers() {
return this.request('/users');
}
async createUser(userData) {
return this.request('/users', {
method: 'POST',
body: JSON.stringify(userData)
});
}
async updateUser(userId, userData) {
return this.request(`/users/${userId}`, {
method: 'PATCH',
body: JSON.stringify(userData)
});
}
async deleteUser(userId) {
return this.request(`/users/${userId}`, {
method: 'DELETE'
});
}
// Bounce Management API
async getMemberBounces(memberId) {
return this.request(`/members/${memberId}/bounces`);
}
async resetBounceStatus(memberId) {
return this.request(`/members/${memberId}/bounce-status`, {
method: 'PATCH'
});
}
} }
/** /**

View File

@@ -6,8 +6,10 @@
class MailingListApp { class MailingListApp {
constructor() { constructor() {
this.isAuthenticated = false; this.isAuthenticated = false;
this.currentUser = null;
this.lists = []; this.lists = [];
this.members = []; this.members = [];
this.users = [];
this.subscriptions = new Map(); // list_id -> members[] this.subscriptions = new Map(); // list_id -> members[]
this.initializeApp(); this.initializeApp();
@@ -20,9 +22,10 @@ class MailingListApp {
this.setupEventListeners(); this.setupEventListeners();
// Check for saved token // Check for saved token
const savedToken = localStorage.getItem('apiToken'); const savedToken = localStorage.getItem('authToken');
if (savedToken) { if (savedToken) {
await this.login(savedToken, false); apiClient.setToken(savedToken);
await this.checkCurrentUser();
} }
} }
@@ -30,69 +33,215 @@ class MailingListApp {
* Setup event listeners * Setup event listeners
*/ */
setupEventListeners() { setupEventListeners() {
// Login/logout // Login form submission
document.getElementById('loginBtn').addEventListener('click', () => { document.getElementById('loginForm').addEventListener('submit', (e) => {
e.preventDefault();
this.handleLogin(); this.handleLogin();
}); });
// Clear error when user starts typing
document.getElementById('username').addEventListener('input', () => {
this.clearLoginError();
});
document.getElementById('password').addEventListener('input', () => {
this.clearLoginError();
});
document.getElementById('logoutBtn').addEventListener('click', () => { document.getElementById('logoutBtn').addEventListener('click', () => {
this.logout(); this.logout();
}); });
// Enter key in token input // User dropdown functionality
document.getElementById('apiToken').addEventListener('keypress', (e) => { const userDropdownTrigger = document.getElementById('userDropdownTrigger');
if (e.key === 'Enter') { const userDropdown = document.getElementById('userDropdown');
this.handleLogin();
userDropdownTrigger.addEventListener('click', (e) => {
e.preventDefault();
e.stopPropagation();
userDropdown.classList.toggle('active');
userDropdownTrigger.classList.toggle('active');
});
// Close dropdown when clicking outside
document.addEventListener('click', (e) => {
if (!userDropdown.contains(e.target)) {
userDropdown.classList.remove('active');
userDropdownTrigger.classList.remove('active');
} }
}); });
// Close dropdown on escape key
document.addEventListener('keydown', (e) => {
if (e.key === 'Escape') {
userDropdown.classList.remove('active');
userDropdownTrigger.classList.remove('active');
}
});
// Bulk import button
document.getElementById('showBulkImportBtn').addEventListener('click', () => {
uiManager.showBulkImportModal();
});
// Add user button (admin only)
document.getElementById('addUserBtn').addEventListener('click', () => {
uiManager.showUserModal();
});
// User management dropdown item
document.getElementById('userManagementBtn').addEventListener('click', () => {
// Close the dropdown
userDropdown.classList.remove('active');
userDropdownTrigger.classList.remove('active');
// Switch to users tab
this.switchToUsersTab();
});
// Member search functionality
const memberSearchInput = document.getElementById('memberSearchInput');
const memberSearchClear = document.getElementById('memberSearchClear');
memberSearchInput.addEventListener('input', (e) => {
this.filterMembers(e.target.value);
// Show/hide clear button
if (e.target.value.trim()) {
memberSearchClear.style.display = 'flex';
} else {
memberSearchClear.style.display = 'none';
}
});
memberSearchClear.addEventListener('click', () => {
memberSearchInput.value = '';
memberSearchClear.style.display = 'none';
this.filterMembers('');
memberSearchInput.focus();
});
// Clear search when switching tabs
document.querySelectorAll('.tab-btn').forEach(btn => {
btn.addEventListener('click', () => {
if (btn.dataset.tab !== 'members') {
memberSearchInput.value = '';
memberSearchClear.style.display = 'none';
}
});
});
}
/**
* Check if current user is still valid
*/
async checkCurrentUser() {
try {
const user = await apiClient.getCurrentUser();
this.currentUser = user;
this.isAuthenticated = true;
this.showAuthenticatedUI();
await this.loadData();
} catch (error) {
// Token is invalid, clear it
this.logout();
}
} }
/** /**
* Handle login button click * Handle login button click
*/ */
async handleLogin() { async handleLogin() {
const tokenInput = document.getElementById('apiToken'); const usernameInput = document.getElementById('username');
const token = tokenInput.value.trim(); const passwordInput = document.getElementById('password');
const username = usernameInput.value.trim();
const password = passwordInput.value.trim();
if (!token) { // Clear previous error states
uiManager.showNotification('Please enter an API token', 'error'); this.clearLoginError();
if (!username || !password) {
this.showLoginError('Please enter both username and password');
return; return;
} }
await this.login(token, true); await this.login(username, password);
}
/**
* Show login error message
*/
showLoginError(message) {
const errorDiv = document.getElementById('loginError');
const errorMessage = document.getElementById('loginErrorMessage');
const usernameInput = document.getElementById('username');
const passwordInput = document.getElementById('password');
errorMessage.textContent = message;
errorDiv.style.display = 'flex';
// Add error class to inputs
usernameInput.classList.add('error');
passwordInput.classList.add('error');
// Remove animation class and re-add to trigger animation
errorDiv.style.animation = 'none';
setTimeout(() => {
errorDiv.style.animation = '';
}, 10);
}
/**
* Clear login error message
*/
clearLoginError() {
const errorDiv = document.getElementById('loginError');
const usernameInput = document.getElementById('username');
const passwordInput = document.getElementById('password');
errorDiv.style.display = 'none';
usernameInput.classList.remove('error');
passwordInput.classList.remove('error');
} }
/** /**
* Authenticate with API * Authenticate with API
*/ */
async login(token, saveToken = true) { async login(username, password) {
try { try {
uiManager.setLoading(true); uiManager.setLoading(true);
// Set token and test authentication // Login and get token
apiClient.setToken(token); const response = await apiClient.login(username, password);
await apiClient.testAuth(); this.currentUser = response.user;
// Authentication successful
this.isAuthenticated = true; this.isAuthenticated = true;
if (saveToken) { // Save token
localStorage.setItem('apiToken', token); localStorage.setItem('authToken', response.access_token);
}
this.showAuthenticatedUI(); this.showAuthenticatedUI();
await this.loadData(); await this.loadData();
uiManager.showNotification('Successfully connected to API', 'success'); uiManager.showNotification(`Welcome back, ${this.currentUser.username}!`, 'success');
} catch (error) { } catch (error) {
this.isAuthenticated = false; this.isAuthenticated = false;
this.currentUser = null;
apiClient.clearToken(); apiClient.clearToken();
localStorage.removeItem('authToken');
if (saveToken) { // Show error on login form
localStorage.removeItem('apiToken'); let errorMessage = 'Login failed';
if (error.message) {
if (error.message.includes('401') || error.message.includes('Unauthorized') ||
error.message.includes('Invalid') || error.message.includes('credentials')) {
errorMessage = 'Invalid username or password';
} else if (error.message.includes('network') || error.message.includes('fetch')) {
errorMessage = 'Unable to connect to server';
} else {
errorMessage = error.message;
}
} }
uiManager.handleError(error, 'Authentication failed'); this.showLoginError(errorMessage);
} finally { } finally {
uiManager.setLoading(false); uiManager.setLoading(false);
} }
@@ -101,10 +250,17 @@ class MailingListApp {
/** /**
* Logout * Logout
*/ */
logout() { async logout() {
try {
await apiClient.logout();
} catch (error) {
// Ignore logout errors
}
this.isAuthenticated = false; this.isAuthenticated = false;
this.currentUser = null;
apiClient.clearToken(); apiClient.clearToken();
localStorage.removeItem('apiToken'); localStorage.removeItem('authToken');
this.showUnauthenticatedUI(); this.showUnauthenticatedUI();
uiManager.showNotification('Logged out successfully', 'info'); uiManager.showNotification('Logged out successfully', 'info');
@@ -114,23 +270,79 @@ class MailingListApp {
* Show authenticated UI * Show authenticated UI
*/ */
showAuthenticatedUI() { showAuthenticatedUI() {
document.getElementById('authControls').style.display = 'none'; document.getElementById('loginPage').style.display = 'none';
document.getElementById('userInfo').style.display = 'flex'; document.getElementById('mainHeader').style.display = 'block';
document.getElementById('mainContent').style.display = 'block'; document.getElementById('mainContent').style.display = 'block';
// Clear token input // Clear login inputs
document.getElementById('apiToken').value = ''; document.getElementById('username').value = '';
document.getElementById('password').value = '';
// Update user info display
if (this.currentUser) {
document.getElementById('currentUsername').textContent = this.currentUser.username;
document.getElementById('currentUserRole').textContent = this.currentUser.role;
document.getElementById('dropdownUsername').textContent = this.currentUser.username;
document.getElementById('dropdownUserRole').textContent = this.currentUser.role;
// Show/hide admin-only features
const isAdmin = this.currentUser.role === 'administrator';
document.getElementById('userManagementBtn').style.display = isAdmin ? 'block' : 'none';
document.getElementById('userManagementDivider').style.display = isAdmin ? 'block' : 'none';
// Show/hide write access features
const hasWriteAccess = this.currentUser.role === 'administrator' || this.currentUser.role === 'operator';
this.updateUIForPermissions(hasWriteAccess);
}
} }
/** /**
* Show unauthenticated UI * Show unauthenticated UI
*/ */
showUnauthenticatedUI() { showUnauthenticatedUI() {
document.getElementById('authControls').style.display = 'flex'; document.getElementById('loginPage').style.display = 'flex';
document.getElementById('userInfo').style.display = 'none'; document.getElementById('mainHeader').style.display = 'none';
document.getElementById('mainContent').style.display = 'none'; document.getElementById('mainContent').style.display = 'none';
} }
/**
* Update UI elements based on user permissions
*/
updateUIForPermissions(hasWriteAccess) {
// Disable/enable write action buttons
const writeButtons = document.querySelectorAll('[data-requires-write]');
writeButtons.forEach(button => {
button.style.display = hasWriteAccess ? '' : 'none';
});
// Update button attributes for later reference
document.getElementById('addListBtn').setAttribute('data-requires-write', '');
document.getElementById('addMemberBtn').setAttribute('data-requires-write', '');
document.getElementById('showBulkImportBtn').setAttribute('data-requires-write', '');
}
/**
* Switch to users tab (triggered from user dropdown)
*/
switchToUsersTab() {
// Switch to users tab programmatically
const tabButtons = document.querySelectorAll('.tab-btn');
const tabContents = document.querySelectorAll('.tab-content');
// Remove active class from all tabs and contents
tabButtons.forEach(btn => btn.classList.remove('active'));
tabContents.forEach(content => content.classList.remove('active'));
// Show users tab content
const usersTab = document.getElementById('users-tab');
if (usersTab) {
usersTab.classList.add('active');
}
// Load users data if needed
this.loadUsers();
}
/** /**
* Load all data from API * Load all data from API
*/ */
@@ -140,21 +352,37 @@ class MailingListApp {
try { try {
uiManager.setLoading(true); uiManager.setLoading(true);
// Load lists and members in parallel // Load configuration, lists and members in parallel
const [lists, members] = await Promise.all([ const [config, lists, members] = await Promise.all([
apiClient.getConfig(),
apiClient.getLists(), apiClient.getLists(),
apiClient.getMembers() apiClient.getMembers()
]); ]);
this.config = config;
this.lists = lists; this.lists = lists;
this.members = members; this.members = members;
// Load users if admin
if (this.currentUser && this.currentUser.role === 'administrator') {
try {
this.users = await apiClient.getUsers();
} catch (error) {
console.warn('Failed to load users:', error);
this.users = [];
}
}
// Load subscriptions for each list // Load subscriptions for each list
await this.loadSubscriptions(); await this.loadSubscriptions();
// Render all views // Render all views
this.renderLists(); this.renderLists();
this.renderMembers(); this.renderMembers();
if (this.currentUser && this.currentUser.role === 'administrator') {
this.renderUsers();
}
} catch (error) { } catch (error) {
uiManager.handleError(error, 'Failed to load data'); uiManager.handleError(error, 'Failed to load data');
@@ -205,6 +433,8 @@ class MailingListApp {
return; return;
} }
const hasWriteAccess = this.currentUser && (this.currentUser.role === 'administrator' || this.currentUser.role === 'operator');
this.lists.forEach(list => { this.lists.forEach(list => {
const row = document.createElement('tr'); const row = document.createElement('tr');
const memberCount = this.subscriptions.get(list.list_id)?.length || 0; const memberCount = this.subscriptions.get(list.list_id)?.length || 0;
@@ -237,24 +467,27 @@ class MailingListApp {
const statusCell = row.cells[4]; const statusCell = row.cells[4];
statusCell.appendChild(uiManager.createStatusBadge(list.active)); statusCell.appendChild(uiManager.createStatusBadge(list.active));
// Add action buttons // Add action buttons only for users with write access
const actionsCell = row.cells[5].querySelector('.action-buttons'); const actionsCell = row.cells[5].querySelector('.action-buttons');
const editBtn = uiManager.createActionButton('Edit', 'edit', 'btn-secondary', () => { if (hasWriteAccess) {
uiManager.showListModal(list); const editBtn = uiManager.createActionButton('Edit', 'edit', 'btn-secondary', () => {
}); uiManager.showListModal(list);
});
const deleteBtn = uiManager.createActionButton('Delete', 'trash', 'btn-danger', () => {
uiManager.showConfirmation(
`Are you sure you want to delete the mailing list "${list.list_name}"? This action cannot be undone.`,
async () => {
await this.deleteList(list.list_id);
}
);
});
actionsCell.appendChild(editBtn);
actionsCell.appendChild(deleteBtn);
}
const deleteBtn = uiManager.createActionButton('Delete', 'trash', 'btn-danger', () => {
uiManager.showConfirmation(
`Are you sure you want to delete the mailing list "${list.list_name}"? This action cannot be undone.`,
async () => {
await this.deleteList(list.list_id);
}
);
});
actionsCell.appendChild(editBtn);
actionsCell.appendChild(deleteBtn);
tbody.appendChild(row); tbody.appendChild(row);
}); });
} }
@@ -282,6 +515,8 @@ class MailingListApp {
return; return;
} }
const hasWriteAccess = this.currentUser && (this.currentUser.role === 'administrator' || this.currentUser.role === 'operator');
this.members.forEach(member => { this.members.forEach(member => {
const row = document.createElement('tr'); const row = document.createElement('tr');
@@ -299,6 +534,7 @@ class MailingListApp {
row.innerHTML = ` row.innerHTML = `
<td> <td>
<div class="font-medium">${uiManager.escapeHtml(member.name)}</div> <div class="font-medium">${uiManager.escapeHtml(member.name)}</div>
${this.config?.bounce_handling_enabled && member.bounce_count > 0 ? `<div class="text-xs text-muted" style="margin-top: 2px;"></div>` : ''}
</td> </td>
<td> <td>
<a href="mailto:${member.email}" style="color: var(--primary-color)"> <a href="mailto:${member.email}" style="color: var(--primary-color)">
@@ -319,6 +555,15 @@ class MailingListApp {
</td> </td>
`; `;
// Add bounce badge if member has bounces (only if bounce handling is enabled)
if (this.config?.bounce_handling_enabled && member.bounce_count > 0) {
const bounceInfoDiv = row.cells[0].querySelector('.text-xs');
const bounceBadge = uiManager.createBounceStatusBadge(member.bounce_status, member.bounce_count);
if (bounceBadge) {
bounceInfoDiv.appendChild(bounceBadge);
}
}
// Add status badge // Add status badge
const statusCell = row.cells[3]; const statusCell = row.cells[3];
statusCell.appendChild(uiManager.createStatusBadge(member.active)); statusCell.appendChild(uiManager.createStatusBadge(member.active));
@@ -335,23 +580,120 @@ class MailingListApp {
uiManager.showMemberSubscriptionsModal(member); uiManager.showMemberSubscriptionsModal(member);
}); });
// Create Bounces button (show if bounce handling is enabled and member has bounces or for admins/operators)
if (this.config?.bounce_handling_enabled && (member.bounce_count > 0 || hasWriteAccess)) {
const bouncesBtn = document.createElement('button');
bouncesBtn.className = `btn btn-sm ${member.bounce_count > 0 ? 'btn-warning' : 'btn-secondary'}`;
bouncesBtn.innerHTML = `<i class="fas fa-exclamation-triangle"></i> Bounces${member.bounce_count > 0 ? ` (${member.bounce_count})` : ''}`;
bouncesBtn.title = 'View Bounce History';
bouncesBtn.addEventListener('click', () => {
uiManager.showBounceHistoryModal(member);
});
actionsCell.appendChild(bouncesBtn);
}
// Only show edit/delete buttons for users with write access
if (hasWriteAccess) {
const editBtn = uiManager.createActionButton('Edit', 'edit', 'btn-secondary', () => {
uiManager.showMemberModal(member);
});
const deleteBtn = uiManager.createActionButton('Delete', 'trash', 'btn-danger', () => {
uiManager.showConfirmation(
`Are you sure you want to delete the member "${member.name}"? This will remove them from all mailing lists.`,
async () => {
await this.deleteMember(member.member_id);
}
);
});
actionsCell.appendChild(subscriptionsBtn);
actionsCell.appendChild(editBtn);
actionsCell.appendChild(deleteBtn);
} else {
actionsCell.appendChild(subscriptionsBtn);
}
tbody.appendChild(row);
});
}
/**
* Render users table (admin only)
*/
renderUsers() {
const tbody = document.getElementById('usersTableBody');
tbody.innerHTML = '';
if (this.users.length === 0) {
tbody.innerHTML = `
<tr>
<td colspan="6" class="text-center text-muted">
No users found. <a href="#" id="createFirstUser">Create your first user</a>
</td>
</tr>
`;
document.getElementById('createFirstUser').addEventListener('click', (e) => {
e.preventDefault();
uiManager.showUserModal();
});
return;
}
this.users.forEach(user => {
const row = document.createElement('tr');
// Format dates
const createdAt = new Date(user.created_at).toLocaleDateString();
const lastLogin = user.last_login ? new Date(user.last_login).toLocaleDateString() : 'Never';
row.innerHTML = `
<td>
<div class="font-medium">${uiManager.escapeHtml(user.username)}</div>
</td>
<td>
<span class="role-badge role-${user.role}">${user.role.replace('-', ' ')}</span>
</td>
<td>
<div class="text-sm text-muted">${createdAt}</div>
</td>
<td>
<div class="text-sm text-muted">${lastLogin}</div>
</td>
<td></td>
<td>
<div class="action-buttons"></div>
</td>
`;
// Add status badge
const statusCell = row.cells[4];
statusCell.appendChild(uiManager.createStatusBadge(user.active));
// Add action buttons
const actionsCell = row.cells[5].querySelector('.action-buttons');
const editBtn = uiManager.createActionButton('Edit', 'edit', 'btn-secondary', () => { const editBtn = uiManager.createActionButton('Edit', 'edit', 'btn-secondary', () => {
uiManager.showMemberModal(member); uiManager.showUserModal(user);
}); });
const deleteBtn = uiManager.createActionButton('Delete', 'trash', 'btn-danger', () => { // Don't allow deletion of current user
uiManager.showConfirmation( if (user.user_id !== this.currentUser.user_id) {
`Are you sure you want to delete the member "${member.name}"? This will remove them from all mailing lists.`, const deleteBtn = uiManager.createActionButton('Delete', 'trash', 'btn-danger', () => {
async () => { uiManager.showConfirmation(
await this.deleteMember(member.member_id); `Are you sure you want to delete the user "${user.username}"? This action cannot be undone.`,
} async () => {
); await this.deleteUser(user.user_id);
}); }
);
});
actionsCell.appendChild(editBtn);
actionsCell.appendChild(deleteBtn);
} else {
actionsCell.appendChild(editBtn);
}
// Append all buttons
actionsCell.appendChild(subscriptionsBtn);
actionsCell.appendChild(editBtn);
actionsCell.appendChild(deleteBtn);
tbody.appendChild(row); tbody.appendChild(row);
}); });
} }
@@ -405,6 +747,165 @@ class MailingListApp {
uiManager.setLoading(false); uiManager.setLoading(false);
} }
} }
/**
* Delete a user
*/
async deleteUser(userId) {
try {
uiManager.setLoading(true);
await apiClient.deleteUser(userId);
uiManager.showNotification('User deleted successfully', 'success');
await this.loadData();
} catch (error) {
uiManager.handleError(error, 'Failed to delete user');
} finally {
uiManager.setLoading(false);
}
}
/**
* Filter members by search term (name or email)
*/
filterMembers(searchTerm) {
const searchInfo = document.getElementById('memberSearchInfo');
const searchCount = document.getElementById('memberSearchCount');
const tbody = document.getElementById('membersTableBody');
if (!this.members || !Array.isArray(this.members)) {
return;
}
// If no search term, show all members
if (!searchTerm || searchTerm.trim() === '') {
this.renderMembers();
searchInfo.style.display = 'none';
return;
}
const normalizedSearch = searchTerm.trim().toLowerCase();
// Filter members by name or email
const filteredMembers = this.members.filter(member => {
const name = (member.name || '').toLowerCase();
const email = (member.email || '').toLowerCase();
return name.includes(normalizedSearch) || email.includes(normalizedSearch);
});
// Update search results info
searchCount.textContent = filteredMembers.length;
searchInfo.style.display = 'block';
// Render filtered results
this.renderFilteredMembers(filteredMembers);
}
/**
* Render filtered members (similar to renderMembers but with filtered data)
*/
renderFilteredMembers(filteredMembers) {
const tbody = document.getElementById('membersTableBody');
tbody.innerHTML = '';
if (filteredMembers.length === 0) {
// Show no results message
const row = tbody.insertRow();
const cell = row.insertCell();
cell.colSpan = 5;
cell.className = 'no-results';
cell.innerHTML = `
<i class="fas fa-search"></i>
<h3>No members found</h3>
<p>Try adjusting your search terms or check the spelling.</p>
`;
return;
}
filteredMembers.forEach(member => {
const row = tbody.insertRow();
// Name
const nameCell = row.insertCell();
nameCell.textContent = member.name;
// Email
const emailCell = row.insertCell();
const emailLink = document.createElement('a');
emailLink.href = `mailto:${member.email}`;
emailLink.textContent = member.email;
emailLink.style.color = 'var(--primary-color)';
emailCell.appendChild(emailLink);
// Lists (show member's subscriptions)
const listsCell = row.insertCell();
const memberLists = [];
this.subscriptions.forEach((members, listId) => {
if (members.some(m => m.member_id === member.member_id)) {
const list = this.lists.find(l => l.list_id === listId);
if (list) {
memberLists.push(list.list_name);
}
}
});
listsCell.textContent = memberLists.length > 0 ? memberLists.join(', ') : 'None';
// Status
const statusCell = row.insertCell();
const statusBadge = document.createElement('span');
statusBadge.className = `status-badge ${member.active ? 'active' : 'inactive'}`;
statusBadge.innerHTML = `<i class="fas fa-${member.active ? 'check' : 'times'}"></i> ${member.active ? 'Active' : 'Inactive'}`;
// Add bounce status if exists and bounce handling is enabled
if (this.config?.bounce_handling_enabled && member.bounce_status && member.bounce_status !== 'clean') {
const bounceIndicator = document.createElement('span');
bounceIndicator.className = `bounce-badge bounce-${member.bounce_status === 'hard_bounce' ? 'hard' : 'soft'}`;
bounceIndicator.innerHTML = `<i class="fas fa-exclamation-triangle"></i> Bounces`;
bounceIndicator.title = `${member.bounce_count || 0} bounces - ${member.bounce_status}`;
statusCell.appendChild(document.createElement('br'));
statusCell.appendChild(bounceIndicator);
}
statusCell.appendChild(statusBadge);
// Actions
const actionsCell = row.insertCell();
actionsCell.className = 'action-buttons';
// Bounces button (if bounce handling is enabled and member has bounce data)
if (this.config?.bounce_handling_enabled && member.bounce_count > 0) {
const bouncesBtn = uiManager.createActionButton('Bounces', 'exclamation-triangle', 'btn-warning', () => {
uiManager.showBounceHistory(member);
});
actionsCell.appendChild(bouncesBtn);
}
const subscriptionsBtn = uiManager.createActionButton('Lists', 'list', 'btn-secondary', () => {
uiManager.showMemberSubscriptions(member);
});
actionsCell.appendChild(subscriptionsBtn);
const editBtn = uiManager.createActionButton('Edit', 'edit', 'btn-primary', () => {
uiManager.showMemberModal(member);
});
editBtn.setAttribute('data-requires-write', '');
actionsCell.appendChild(editBtn);
const deleteBtn = uiManager.createActionButton('Delete', 'trash', 'btn-danger', () => {
uiManager.showConfirmation(
`Are you sure you want to delete member "${member.name}"? This will also remove them from all mailing lists.`,
async () => {
await this.deleteMember(member.member_id);
}
);
});
deleteBtn.setAttribute('data-requires-write', '');
actionsCell.appendChild(deleteBtn);
});
// Update UI permissions for the filtered results
const hasWriteAccess = this.currentUser && (this.currentUser.role === 'administrator' || this.currentUser.role === 'operator');
this.updateUIForPermissions(hasWriteAccess);
}
} }
// Initialize the application when DOM is loaded // Initialize the application when DOM is loaded

View File

@@ -50,10 +50,17 @@ class UIManager {
this.showListModal(); this.showListModal();
}); });
// Initialize bulk import listeners
this.initializeBulkImportListeners();
document.getElementById('addMemberBtn').addEventListener('click', () => { document.getElementById('addMemberBtn').addEventListener('click', () => {
this.showMemberModal(); this.showMemberModal();
}); });
document.getElementById('addUserBtn').addEventListener('click', () => {
this.showUserModal();
});
// Member subscriptions modal // Member subscriptions modal
@@ -69,6 +76,19 @@ class UIManager {
this.handleMemberSubscriptionsSave(); this.handleMemberSubscriptionsSave();
}); });
// Bounce history modal
document.getElementById('bounceHistoryModalClose').addEventListener('click', () => {
this.closeModal(document.getElementById('bounceHistoryModal'));
});
document.getElementById('bounceHistoryCloseBtn').addEventListener('click', () => {
this.closeModal(document.getElementById('bounceHistoryModal'));
});
document.getElementById('bounceHistoryResetBtn').addEventListener('click', () => {
this.handleBounceStatusReset();
});
// Form submissions // Form submissions
document.getElementById('listForm').addEventListener('submit', (e) => { document.getElementById('listForm').addEventListener('submit', (e) => {
e.preventDefault(); e.preventDefault();
@@ -85,6 +105,11 @@ class UIManager {
this.handleSubscriptionFormSubmit(); this.handleSubscriptionFormSubmit();
}); });
document.getElementById('userForm').addEventListener('submit', (e) => {
e.preventDefault();
this.handleUserFormSubmit();
});
// Confirmation modal // Confirmation modal
document.getElementById('confirmOkBtn').addEventListener('click', () => { document.getElementById('confirmOkBtn').addEventListener('click', () => {
if (this.confirmCallback) { if (this.confirmCallback) {
@@ -98,6 +123,19 @@ class UIManager {
this.confirmCallback = null; this.confirmCallback = null;
this.closeModal(document.getElementById('confirmModal')); this.closeModal(document.getElementById('confirmModal'));
}); });
// Bounce history modal
document.getElementById('bounceHistoryModalClose').addEventListener('click', () => {
this.closeModal(document.getElementById('bounceHistoryModal'));
});
document.getElementById('bounceHistoryCloseBtn').addEventListener('click', () => {
this.closeModal(document.getElementById('bounceHistoryModal'));
});
document.getElementById('bounceHistoryResetBtn').addEventListener('click', () => {
this.handleResetBounceStatus();
});
} }
/** /**
@@ -242,6 +280,42 @@ class UIManager {
this.showModal(modal); this.showModal(modal);
} }
/**
* Show user modal (add/edit)
*/
showUserModal(userData = null) {
const modal = document.getElementById('userModal');
const title = document.getElementById('userModalTitle');
const form = document.getElementById('userForm');
const passwordHelp = document.getElementById('passwordHelp');
const passwordField = document.getElementById('userPassword');
if (userData) {
// Edit mode
title.textContent = 'Edit User';
document.getElementById('userName').value = userData.username;
document.getElementById('userName').readOnly = true; // Can't change username
passwordField.placeholder = 'Leave blank to keep current password';
passwordField.required = false;
passwordHelp.style.display = 'block';
document.getElementById('userRole').value = userData.role;
document.getElementById('userActive').checked = userData.active;
this.currentEditingItem = userData;
} else {
// Add mode
title.textContent = 'Add User';
form.reset();
document.getElementById('userName').readOnly = false;
passwordField.placeholder = 'Password';
passwordField.required = true;
passwordHelp.style.display = 'none';
document.getElementById('userActive').checked = true;
this.currentEditingItem = null;
}
this.showModal(modal);
}
/** /**
* Show subscription modal * Show subscription modal
*/ */
@@ -397,6 +471,50 @@ class UIManager {
} }
} }
/**
* Handle user form submission
*/
async handleUserFormSubmit() {
const form = document.getElementById('userForm');
const formData = new FormData(form);
const userData = {
username: formData.get('userName'),
role: formData.get('userRole'),
active: formData.get('userActive') === 'on'
};
// Only include password if it's provided (for updates, empty means no change)
const password = formData.get('userPassword');
if (password) {
userData.password = password;
}
try {
this.setLoading(true);
if (this.currentEditingItem) {
// Update existing user
await apiClient.updateUser(this.currentEditingItem.user_id, userData);
this.showNotification('User updated successfully', 'success');
} else {
// Create new user
if (!userData.password) {
throw new Error('Password is required for new users');
}
await apiClient.createUser(userData);
this.showNotification('User created successfully', 'success');
}
this.closeModal(document.getElementById('userModal'));
await window.app.loadData();
} catch (error) {
this.handleError(error, 'Failed to save user');
} finally {
this.setLoading(false);
}
}
/** /**
* Show member subscriptions modal * Show member subscriptions modal
*/ */
@@ -572,6 +690,117 @@ class UIManager {
} }
} }
/**
* Show bounce history modal for a member
*/
async showBounceHistoryModal(member) {
const modal = document.getElementById('bounceHistoryModal');
// Update member info
document.getElementById('bounceHistoryTitle').textContent = `Bounce History - ${member.name}`;
document.getElementById('bounceHistoryMemberName').textContent = member.name;
document.getElementById('bounceHistoryMemberEmail').textContent = member.email;
// Update summary stats
document.getElementById('bounceTotalCount').textContent = member.bounce_count || 0;
document.getElementById('bounceLastDate').textContent = this.formatDateTime(member.last_bounce_at);
const statusText = document.getElementById('bounceStatusText');
statusText.className = 'bounce-stat-value';
if (member.bounce_status === 'hard_bounce') {
statusText.textContent = 'Hard Bounce';
statusText.classList.add('text-danger');
} else if (member.bounce_status === 'soft_bounce') {
statusText.textContent = 'Soft Bounce';
statusText.classList.add('text-warning');
} else {
statusText.textContent = 'Clean';
statusText.classList.add('text-success');
}
try {
this.currentMemberForBounces = member;
// Load bounce history
const bounces = await apiClient.getMemberBounces(member.member_id);
this.renderBounceHistory(bounces);
this.showModal(modal);
} catch (error) {
this.handleError(error, 'Failed to load bounce history');
}
}
/**
* Render bounce history list
*/
renderBounceHistory(bounces) {
const container = document.getElementById('bounceHistoryList');
container.innerHTML = '';
if (bounces.length === 0) {
container.innerHTML = `
<div class="empty-state">
<i class="fas fa-check-circle"></i>
<p>No bounces recorded for this member</p>
</div>
`;
return;
}
bounces.forEach(bounce => {
const item = document.createElement('div');
item.className = 'bounce-history-item';
let typeClass = 'bounce-type-';
if (bounce.bounce_type === 'Permanent') {
typeClass += 'permanent';
} else if (bounce.bounce_type === 'Transient') {
typeClass += 'transient';
} else {
typeClass += 'undetermined';
}
item.innerHTML = `
<div class="bounce-header">
<span class="bounce-type ${typeClass}">
<i class="fas fa-${bounce.bounce_type === 'Permanent' ? 'times-circle' : bounce.bounce_type === 'Transient' ? 'exclamation-circle' : 'question-circle'}"></i>
${bounce.bounce_type}
</span>
<span class="bounce-date">${this.formatDateTime(bounce.timestamp)}</span>
</div>
${bounce.bounce_subtype ? `<div class="bounce-subtype">Subtype: ${this.escapeHtml(bounce.bounce_subtype)}</div>` : ''}
${bounce.diagnostic_code ? `<div class="bounce-diagnostic">${this.escapeHtml(bounce.diagnostic_code)}</div>` : ''}
`;
container.appendChild(item);
});
}
/**
* Handle bounce status reset
*/
async handleResetBounceStatus() {
if (!this.currentMemberForBounces) return;
this.showConfirmation(
`Are you sure you want to reset the bounce status for "${this.currentMemberForBounces.name}"? This will clear the bounce count and allow emails to be sent to this address again.`,
async () => {
try {
this.setLoading(true);
await apiClient.resetBounceStatus(this.currentMemberForBounces.member_id);
this.showNotification('Bounce status reset successfully', 'success');
this.closeModal(document.getElementById('bounceHistoryModal'));
await window.app.loadData();
} catch (error) {
this.handleError(error, 'Failed to reset bounce status');
} finally {
this.setLoading(false);
}
}
);
}
/** /**
* Handle API errors * Handle API errors
*/ */
@@ -623,6 +852,49 @@ class UIManager {
return badge; return badge;
} }
/**
* Create bounce status badge
*/
createBounceStatusBadge(bounceStatus, bounceCount) {
const badge = document.createElement('span');
if (bounceStatus === 'hard_bounce') {
badge.className = 'bounce-badge bounce-hard';
badge.innerHTML = `<i class="fas fa-exclamation-triangle"></i> Hard Bounce`;
badge.title = `${bounceCount} bounce(s) - Email permanently failed`;
} else if (bounceStatus === 'soft_bounce') {
badge.className = 'bounce-badge bounce-soft';
badge.innerHTML = `<i class="fas fa-exclamation-circle"></i> Soft Bounce`;
badge.title = `${bounceCount} bounce(s) - Temporary delivery issues`;
} else if (bounceCount > 0) {
badge.className = 'bounce-badge bounce-warning';
badge.innerHTML = `<i class="fas fa-info-circle"></i> ${bounceCount} bounce(s)`;
badge.title = `${bounceCount} bounce(s) recorded`;
} else {
return null; // No badge for clean status
}
return badge;
}
/**
* Format date and time
*/
formatDateTime(dateString) {
if (!dateString) return 'Never';
const date = new Date(dateString);
return date.toLocaleString();
}
/**
* Format date only
*/
formatDate(dateString) {
if (!dateString) return 'Never';
const date = new Date(dateString);
return date.toLocaleDateString();
}
/** /**
* Format email as mailto link * Format email as mailto link
*/ */
@@ -634,6 +906,546 @@ class UIManager {
return link; return link;
} }
/**
* Show bulk import modal
*/
showBulkImportModal() {
this.resetBulkImportModal();
this.showModal(document.getElementById('bulkImportModal'));
}
/**
* Reset bulk import modal state
*/
resetBulkImportModal() {
// Reset to step 1
document.querySelectorAll('.import-step').forEach(step => {
step.style.display = 'none';
});
document.getElementById('importStep1').style.display = 'block';
// Reset button states
document.getElementById('bulkImportBackBtn').style.display = 'none';
document.getElementById('bulkImportNextBtn').style.display = 'inline-block';
document.getElementById('bulkImportNextBtn').disabled = true;
document.getElementById('bulkImportBtn').style.display = 'none';
document.getElementById('bulkImportDoneBtn').style.display = 'none';
// Clear file input
document.getElementById('csvFileInput').value = '';
document.getElementById('fileInfo').style.display = 'none';
// Reset parsed data
this.csvData = null;
this.parsedRows = [];
this.currentStep = 1;
}
/**
* Initialize bulk import event listeners
*/
initializeBulkImportListeners() {
const fileUploadArea = document.getElementById('fileUploadArea');
const csvFileInput = document.getElementById('csvFileInput');
const nextBtn = document.getElementById('bulkImportNextBtn');
const backBtn = document.getElementById('bulkImportBackBtn');
const importBtn = document.getElementById('bulkImportBtn');
const doneBtn = document.getElementById('bulkImportDoneBtn');
const removeFileBtn = document.getElementById('removeFileBtn');
// File upload area click
fileUploadArea.addEventListener('click', () => {
csvFileInput.click();
});
// File input change
csvFileInput.addEventListener('change', (e) => {
this.handleFileSelection(e.target.files[0]);
});
// Remove file button
removeFileBtn.addEventListener('click', () => {
this.removeSelectedFile();
});
// Drag and drop
fileUploadArea.addEventListener('dragover', (e) => {
e.preventDefault();
fileUploadArea.classList.add('dragover');
});
fileUploadArea.addEventListener('dragleave', () => {
fileUploadArea.classList.remove('dragover');
});
fileUploadArea.addEventListener('drop', (e) => {
e.preventDefault();
fileUploadArea.classList.remove('dragover');
const files = e.dataTransfer.files;
if (files.length > 0) {
this.handleFileSelection(files[0]);
}
});
// Navigation buttons
nextBtn.addEventListener('click', () => {
this.handleBulkImportNext();
});
backBtn.addEventListener('click', () => {
this.handleBulkImportBack();
});
importBtn.addEventListener('click', () => {
this.handleBulkImportSubmit();
});
doneBtn.addEventListener('click', () => {
this.closeModal(document.getElementById('bulkImportModal'));
// Reload data to show imported members
if (window.app) {
window.app.loadData();
}
});
}
/**
* Handle file selection
*/
handleFileSelection(file) {
if (!file) return;
if (!file.name.toLowerCase().endsWith('.csv')) {
this.showNotification('Please select a CSV file', 'error');
return;
}
const reader = new FileReader();
reader.onload = (e) => {
this.csvData = e.target.result;
this.displayFileInfo(file);
this.parseCSVPreview();
document.getElementById('bulkImportNextBtn').disabled = false;
};
reader.readAsText(file);
}
/**
* Display file information
*/
displayFileInfo(file) {
document.getElementById('fileName').textContent = file.name;
document.getElementById('fileSize').textContent = `(${this.formatFileSize(file.size)})`;
document.getElementById('fileInfo').style.display = 'flex';
}
/**
* Remove selected file
*/
removeSelectedFile() {
document.getElementById('csvFileInput').value = '';
document.getElementById('fileInfo').style.display = 'none';
document.getElementById('bulkImportNextBtn').disabled = true;
this.csvData = null;
this.parsedRows = [];
}
/**
* Format file size in human readable format
*/
formatFileSize(bytes) {
if (bytes === 0) return '0 B';
const k = 1024;
const sizes = ['B', 'KB', 'MB', 'GB'];
const i = Math.floor(Math.log(bytes) / Math.log(k));
return parseFloat((bytes / Math.pow(k, i)).toFixed(1)) + ' ' + sizes[i];
}
/**
* Parse CSV for preview
*/
parseCSVPreview() {
if (!this.csvData) return;
const lines = this.csvData.trim().split('\n');
if (lines.length < 2) {
this.showNotification('CSV file must contain at least a header and one data row', 'error');
return;
}
// Parse header
const headers = this.parseCSVLine(lines[0]);
const nameIndex = headers.findIndex(h => h.toLowerCase().includes('name'));
const emailIndex = headers.findIndex(h => h.toLowerCase().includes('email'));
if (nameIndex === -1 || emailIndex === -1) {
this.showNotification('CSV must contain Name and Email columns', 'error');
return;
}
this.parsedRows = [];
this.csvHeaders = headers;
let validRows = 0;
let errorRows = 0;
// Parse data rows
for (let i = 1; i < lines.length; i++) {
const rawLine = lines[i].trim();
if (!rawLine) continue; // Skip empty lines
const values = this.parseCSVLine(rawLine);
const name = (values[nameIndex] || '').trim();
const email = (values[emailIndex] || '').trim();
// Determine error type and message
let error = null;
let isValid = true;
if (!email) {
error = 'Missing email address';
isValid = false;
} else if (!this.isValidEmail(email)) {
error = 'Invalid email format';
isValid = false;
}
// Note: Missing name is OK - we'll use the email as display name if needed
const row = {
index: i,
name,
email,
valid: isValid,
error,
rawLine,
parsedValues: values,
headers: headers
};
this.parsedRows.push(row);
if (isValid) validRows++;
else errorRows++;
}
// Update preview stats
document.getElementById('totalRowsCount').textContent = this.parsedRows.length;
document.getElementById('validRowsCount').textContent = validRows;
document.getElementById('errorRowsCount').textContent = errorRows;
}
/**
* Parse a CSV line handling quoted values
*/
parseCSVLine(line) {
const result = [];
let current = '';
let inQuotes = false;
for (let i = 0; i < line.length; i++) {
const char = line[i];
if (char === '"') {
inQuotes = !inQuotes;
} else if (char === ',' && !inQuotes) {
result.push(current.trim());
current = '';
} else {
current += char;
}
}
result.push(current.trim());
return result;
}
/**
* Simple email validation
*/
isValidEmail(email) {
const emailRegex = /^[^\s@]+@[^\s@]+\.[^\s@]+$/;
return emailRegex.test(email);
}
/**
* Handle next button click
*/
handleBulkImportNext() {
if (this.currentStep === 1) {
this.showStep2Preview();
} else if (this.currentStep === 2) {
this.showStep3ListSelection();
}
}
/**
* Handle back button click
*/
handleBulkImportBack() {
if (this.currentStep === 2) {
this.showStep1Upload();
} else if (this.currentStep === 3) {
this.showStep2Preview();
}
}
/**
* Show step 1 (upload)
*/
showStep1Upload() {
document.querySelectorAll('.import-step').forEach(step => {
step.style.display = 'none';
});
document.getElementById('importStep1').style.display = 'block';
document.getElementById('bulkImportBackBtn').style.display = 'none';
document.getElementById('bulkImportNextBtn').style.display = 'inline-block';
document.getElementById('bulkImportBtn').style.display = 'none';
this.currentStep = 1;
}
/**
* Show step 2 (preview)
*/
showStep2Preview() {
document.querySelectorAll('.import-step').forEach(step => {
step.style.display = 'none';
});
document.getElementById('importStep2').style.display = 'block';
document.getElementById('bulkImportBackBtn').style.display = 'inline-block';
document.getElementById('bulkImportNextBtn').style.display = 'inline-block';
document.getElementById('bulkImportBtn').style.display = 'none';
this.renderPreviewTable();
this.currentStep = 2;
}
/**
* Show step 3 (list selection)
*/
showStep3ListSelection() {
document.querySelectorAll('.import-step').forEach(step => {
step.style.display = 'none';
});
document.getElementById('importStep3').style.display = 'block';
document.getElementById('bulkImportBackBtn').style.display = 'inline-block';
document.getElementById('bulkImportNextBtn').style.display = 'none';
document.getElementById('bulkImportBtn').style.display = 'inline-block';
this.renderListSelection();
this.currentStep = 3;
}
/**
* Render preview table
*/
renderPreviewTable() {
const tbody = document.getElementById('previewTableBody');
tbody.innerHTML = '';
// Show first 10 rows for preview
const previewRows = this.parsedRows.slice(0, 10);
previewRows.forEach(row => {
const tr = document.createElement('tr');
tr.innerHTML = `
<td>${row.index}</td>
<td>${this.escapeHtml(row.name)}</td>
<td>${this.escapeHtml(row.email)}</td>
<td>
<span class="row-status ${row.valid ? 'valid' : 'error'}">
${row.valid ? 'Valid' : 'Error'}
</span>
</td>
`;
tbody.appendChild(tr);
});
// Show errors if any
const errors = this.parsedRows.filter(row => !row.valid);
if (errors.length > 0) {
document.getElementById('errorList').style.display = 'block';
this.renderErrorDisplay(errors);
} else {
document.getElementById('errorList').style.display = 'none';
}
}
/**
* Render error display with expandable details
*/
renderErrorDisplay(errors) {
// Update error summary
const errorSummary = document.getElementById('errorSummary');
const errorTypes = {};
errors.forEach(error => {
const type = error.error;
errorTypes[type] = (errorTypes[type] || 0) + 1;
});
const summaryText = Object.entries(errorTypes)
.map(([type, count]) => `${count} ${type.toLowerCase()}`)
.join(', ');
errorSummary.innerHTML = `
<p><strong>${errors.length}</strong> rows have issues: ${summaryText}</p>
<p class="text-muted text-sm">These rows will be skipped during import. Click "Show All" to see details.</p>
`;
// Render error table
const errorTableBody = document.getElementById('errorTableBody');
errorTableBody.innerHTML = '';
errors.forEach(row => {
const tr = document.createElement('tr');
tr.innerHTML = `
<td class="font-medium">${row.index + 1}</td>
<td class="font-medium">${this.escapeHtml(row.error)}</td>
<td>${this.escapeHtml(row.name || '-')}</td>
<td>${this.escapeHtml(row.email || '-')}</td>
<td class="raw-data-cell" title="${this.escapeHtml(row.rawLine)}">${this.escapeHtml(row.rawLine)}</td>
`;
errorTableBody.appendChild(tr);
});
// Setup toggle functionality
this.setupErrorToggle();
}
/**
* Setup error toggle functionality
*/
setupErrorToggle() {
const toggleBtn = document.getElementById('toggleErrorsBtn');
const errorDetails = document.getElementById('errorDetails');
const toggleText = document.getElementById('errorToggleText');
const toggleIcon = document.getElementById('errorToggleIcon');
// Remove existing listener to prevent duplicates
const newToggleBtn = toggleBtn.cloneNode(true);
toggleBtn.parentNode.replaceChild(newToggleBtn, toggleBtn);
newToggleBtn.addEventListener('click', () => {
const isHidden = errorDetails.style.display === 'none';
if (isHidden) {
errorDetails.style.display = 'block';
toggleText.textContent = 'Hide Details';
toggleIcon.className = 'fas fa-chevron-up';
} else {
errorDetails.style.display = 'none';
toggleText.textContent = 'Show All';
toggleIcon.className = 'fas fa-chevron-down';
}
});
}
/**
* Render list selection checkboxes
*/
renderListSelection() {
const container = document.getElementById('listSelection');
container.innerHTML = '';
if (!window.app || !window.app.lists) {
container.innerHTML = '<p class="text-muted">No mailing lists available</p>';
return;
}
window.app.lists.forEach(list => {
const div = document.createElement('div');
div.className = 'list-checkbox';
div.innerHTML = `
<input type="checkbox" id="list_${list.list_id}" value="${list.list_id}">
<div class="list-checkbox-label">
<span class="list-checkbox-name">${this.escapeHtml(list.list_name)}</span>
<span class="list-checkbox-email">${this.escapeHtml(list.list_email)}</span>
</div>
`;
container.appendChild(div);
// Make the whole div clickable
div.addEventListener('click', (e) => {
if (e.target.type !== 'checkbox') {
const checkbox = div.querySelector('input[type="checkbox"]');
checkbox.checked = !checkbox.checked;
}
});
});
}
/**
* Handle bulk import submission
*/
async handleBulkImportSubmit() {
const selectedLists = Array.from(document.querySelectorAll('#listSelection input[type="checkbox"]:checked'))
.map(cb => parseInt(cb.value));
if (selectedLists.length === 0) {
this.showNotification('Please select at least one mailing list', 'error');
return;
}
try {
this.setLoading(true);
const result = await apiClient.bulkImportMembers(this.csvData, selectedLists);
this.showStep4Results(result);
} catch (error) {
this.handleError(error, 'Bulk import failed');
} finally {
this.setLoading(false);
}
}
/**
* Show step 4 (results)
*/
showStep4Results(result) {
document.querySelectorAll('.import-step').forEach(step => {
step.style.display = 'none';
});
document.getElementById('importStep4').style.display = 'block';
document.getElementById('bulkImportBackBtn').style.display = 'none';
document.getElementById('bulkImportNextBtn').style.display = 'none';
document.getElementById('bulkImportBtn').style.display = 'none';
document.getElementById('bulkImportDoneBtn').style.display = 'inline-block';
// Update result stats
document.getElementById('processedCount').textContent = result.processed_rows;
document.getElementById('createdCount').textContent = result.created_members;
document.getElementById('updatedCount').textContent = result.updated_members;
document.getElementById('subscriptionsCount').textContent = result.subscriptions_added;
// Show errors if any
if (result.errors && result.errors.length > 0) {
document.getElementById('resultErrors').style.display = 'block';
const errorList = document.getElementById('resultErrorList');
errorList.innerHTML = '';
result.errors.forEach(error => {
const li = document.createElement('li');
li.textContent = error;
errorList.appendChild(li);
});
} else {
document.getElementById('resultErrors').style.display = 'none';
}
this.currentStep = 4;
// Show success notification
this.showNotification(
`Successfully imported ${result.processed_rows} members with ${result.subscriptions_added} subscriptions`,
'success'
);
}
/** /**
* Escape HTML to prevent XSS * Escape HTML to prevent XSS
*/ */