169 lines
6.3 KiB
Markdown
169 lines
6.3 KiB
Markdown
# Copilot Instructions: Mail List Manager (Postfix + SES + MySQL + API + Web)
|
|
|
|
## Architecture Overview
|
|
|
|
This is a complete containerized mailing list management system with four services:
|
|
|
|
1. **Web Frontend** (Port 3000) - Nginx serving modern HTML/CSS/JS interface
|
|
2. **REST API** (Port 8000) - FastAPI backend with token authentication
|
|
3. **MySQL Database** (Internal) - Stores lists, members, and subscriptions
|
|
4. **Postfix Mail Server** (Port 25) - SMTP server with native MySQL integration
|
|
|
|
**Key Architecture Principles:**
|
|
- Real-time operation: Changes take effect immediately without Postfix reload
|
|
- Native integration: Postfix queries MySQL directly (no Python scripts)
|
|
- Security-first: Private Docker network, token auth, sender whitelist
|
|
- User-friendly: Web interface for easy management by non-technical users
|
|
|
|
## Configuration Pattern
|
|
|
|
**Dynamic Database-Driven Architecture:**
|
|
|
|
1. **Data Flow**: Web UI → REST API → MySQL ← Postfix (real-time queries)
|
|
2. **Credentials**: All sensitive data in `.env` file (SES, MySQL, API_TOKEN)
|
|
3. **Mail Processing**:
|
|
- Email arrives for `list@lists.sasalliance.org`
|
|
- Postfix queries MySQL via `mysql_virtual_alias_maps.cf`
|
|
- MySQL returns comma-separated member emails
|
|
- Postfix expands alias and delivers via SES
|
|
4. **Security**: Private Docker network (maillist-internal), sender whitelist (@sasalliance.org)
|
|
|
|
**No Static Configuration:**
|
|
- Virtual aliases are stored in MySQL database, not config files
|
|
- Changes take effect immediately without container restarts
|
|
- Postfix caches query results for performance
|
|
|
|
## Key Files and Their Roles
|
|
|
|
### Docker & Environment
|
|
- `docker-compose.yaml`: Multi-service orchestration (mysql, postfix, api, web)
|
|
- `.env`: All credentials and configuration (SES, MySQL, API_TOKEN)
|
|
|
|
### Web Frontend (`web/`)
|
|
- `index.html`: Single-page application interface
|
|
- `static/js/api.js`: API client with authentication
|
|
- `static/js/ui.js`: UI helpers and modal management
|
|
- `static/js/app.js`: Main application controller
|
|
- `static/css/style.css`: Complete styling system
|
|
- `nginx.conf`: Nginx configuration for static file serving
|
|
|
|
### REST API (`api/`)
|
|
- `main.py`: FastAPI application with all CRUD endpoints
|
|
- `requirements.txt`: Python dependencies (fastapi, mysql-connector-python, etc.)
|
|
- `Dockerfile`: Python 3.11 container build
|
|
|
|
### Database (`database/`)
|
|
- `schema.sql`: Database initialization with tables and sample data
|
|
- Tables: `lists`, `members`, `list_members` (many-to-many junction)
|
|
|
|
### Postfix (`postfix/`)
|
|
- `main.cf.template`: Core Postfix config with MySQL virtual alias maps
|
|
- `mysql_virtual_alias_maps.cf`: Native Postfix MySQL query configuration
|
|
- `sender_access`: Sender whitelist (sasalliance.org domain allowed)
|
|
- `sasl_passwd.template`: SES authentication template
|
|
- `entrypoint.sh`: Container startup with MySQL health check and config generation
|
|
|
|
## Development Workflows
|
|
|
|
**Building and Running:**
|
|
|
|
Always use sudo for Docker commands. Don't bother opening the simple browser UI for Docker as it is not very useful.
|
|
|
|
```bash
|
|
sudo docker-compose up --build # Build and start all services
|
|
sudo docker-compose logs -f # Monitor all service logs
|
|
sudo docker-compose logs -f api # Monitor specific service
|
|
```
|
|
|
|
**Managing Lists and Members:**
|
|
|
|
**Via Web Interface (Recommended):**
|
|
1. Open http://localhost:3000
|
|
2. Enter API_TOKEN from .env file
|
|
3. Use intuitive interface to manage lists, members, and subscriptions
|
|
4. Click "Subscriptions" button on any member for toggle-based subscription management
|
|
|
|
**Via REST API:**
|
|
```bash
|
|
# Get all lists
|
|
curl -H "Authorization: Bearer $API_TOKEN" http://localhost:8000/lists
|
|
|
|
# Create new list
|
|
curl -X POST http://localhost:8000/lists \
|
|
-H "Authorization: Bearer $API_TOKEN" \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"list_name":"Test","list_email":"test@lists.sasalliance.org","active":true}'
|
|
|
|
# Subscribe member to list
|
|
curl -X POST http://localhost:8000/subscriptions \
|
|
-H "Authorization: Bearer $API_TOKEN" \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"list_email":"test@lists.sasalliance.org","member_email":"user@example.com"}'
|
|
```
|
|
|
|
**Via Direct Database Access:**
|
|
```bash
|
|
docker-compose exec mysql mysql -u maillist -p maillist
|
|
# See database/README.md for SQL examples
|
|
```
|
|
|
|
**Testing Mail Delivery:**
|
|
```bash
|
|
# From inside container
|
|
docker-compose exec postfix bash
|
|
echo "Test message" | mail -s "Subject" community@lists.sasalliance.org
|
|
|
|
# Check logs for SES relay status
|
|
docker-compose logs postfix | grep -E "(sent|bounced|deferred)"
|
|
|
|
# Verify MySQL query works
|
|
docker-compose exec postfix postmap -q "community@lists.sasalliance.org" \
|
|
mysql:/etc/postfix/mysql_virtual_alias_maps.cf
|
|
```
|
|
|
|
## Security Considerations
|
|
|
|
- All credentials stored in `.env` file (git-ignored)
|
|
- MySQL on private Docker network (no host port mapping)
|
|
- API requires Bearer token authentication
|
|
- Sender whitelist restricts who can send to lists (@sasalliance.org only)
|
|
- SASL password file permissions set to 600 in entrypoint
|
|
- TLS encryption enforced for SES relay (`smtp_tls_security_level = encrypt`)
|
|
- Web interface requires token for all operations
|
|
|
|
## Configuration Conventions
|
|
|
|
- **Hostname Pattern**: `lists.sasalliance.org` for mailing lists, origin domain `sasalliance.org`
|
|
- **Virtual Aliases**: Dynamically queried from MySQL database in real-time
|
|
- **SES Region**: EU West 2 (`email-smtp.eu-west-2.amazonaws.com`)
|
|
- **Port Mapping**: Standard SMTP port 25 exposed to host
|
|
- **Database Tables**: `lists`, `members`, `list_members` (many-to-many design)
|
|
- **API Port**: 8000 for REST API
|
|
- **Web Port**: 3000 for web interface
|
|
|
|
## Common Modifications
|
|
|
|
**Adding Recipients to a List:**
|
|
Use web interface or API - changes are immediate!
|
|
|
|
**Creating New Mailing List:**
|
|
1. Via Web: Click "Add List" button, fill in form
|
|
2. Via API: POST to /lists endpoint with list details
|
|
3. Via MySQL: INSERT into lists table
|
|
|
|
**Adding New Member:**
|
|
1. Via Web: Click "Add Member" button in Members tab
|
|
2. Via API: POST to /members endpoint
|
|
3. Via MySQL: INSERT into members table
|
|
|
|
**Managing Subscriptions:**
|
|
1. Via Web: Click "Subscriptions" button on member, toggle lists
|
|
2. Via API: POST/DELETE to /subscriptions endpoint
|
|
3. Via MySQL: INSERT/DELETE in list_members table
|
|
|
|
**Credential Updates:**
|
|
1. Edit `.env` file with new credentials
|
|
2. Restart affected services:
|
|
- SES: `docker-compose restart postfix`
|
|
- MySQL: `docker-compose restart mysql`
|
|
- API: `docker-compose restart api` |