sasa-maillist/README.md

# Mail List Manager

A complete containerized mailing list management system with web interface, REST API, MySQL database, and Postfix mail server configured as an SMTP relay through Amazon SES.

## Architecture

**Multi-Service Architecture:**
- **Web Frontend** - Modern responsive interface for managing lists and members (Port 3000)
- **REST API** - FastAPI backend with token authentication (Port 8000)
- **MySQL Database** - Stores lists, members, and subscriptions with real-time integration
- **Postfix Mail Server** - SMTP server with native MySQL integration for dynamic list expansion (Port 25)
- **Amazon SES** - Outbound mail relay for reliable delivery

**Key Features:**
- Real-time list updates (no Postfix reload needed)
- Native Postfix MySQL queries for instant list expansion
- Token-based API authentication
- Sender whitelisting for authorized board members
- Private Docker network for security
- Complete web-based management interface

## Quick Start

1. Copy the environment template:
   ```bash
   cp .env.example .env
   ```

2. Edit `.env` with your credentials:
   ```bash
   # SES Credentials (Required)
   SES_USER=your_ses_access_key
   SES_PASS=your_ses_secret_key
   
   # MySQL Credentials (Required)
   MYSQL_ROOT_PASSWORD=secure_root_password
   MYSQL_PASSWORD=secure_maillist_password
   
   # API Authentication (Required)
   API_TOKEN=your_secure_api_token
   
   # Optional: SMTP configuration (defaults to EU West 2)
   SMTP_HOST=email-smtp.eu-west-2.amazonaws.com
   SMTP_PORT=587
   ```

3. Build and start all services:
   ```bash
   sudo docker-compose up --build
   ```

4. Access the web interface:
   ```
   http://localhost:3000
   ```
   
   Enter your API_TOKEN to authenticate and start managing lists!

5. Alternative: Use the REST API directly:
   ```bash
   # Health check
   curl http://localhost:8000/health
   
   # Get all lists (requires authentication)
   curl -H "Authorization: Bearer your_api_token" http://localhost:8000/lists
   ```

## Services

### Web Interface (Port 3000)
- Modern, responsive interface for managing mailing lists and members
- Token-based authentication
- Real-time updates and feedback
- See `web/README.md` for details

### REST API (Port 8000)
- Complete CRUD operations for lists, members, and subscriptions
- Bearer token authentication
- Interactive documentation at http://localhost:8000/docs
- See `api/README.md` for API reference

### MySQL Database (Internal Only)
- Stores all mailing list and member data
- Automatically initialized with schema
- Native Postfix integration for real-time queries
- See `database/README.md` for schema and management

### Postfix Mail Server (Port 25)
- Accepts mail for lists.sasalliance.org
- Queries MySQL for list expansion
- Relays through Amazon SES
- Sender whitelist: @sasalliance.org domain
- Changes take effect immediately (no restart needed)

## Configuration

### Managing Lists and Members

**Via Web Interface (Recommended):**
1. Open http://localhost:3000
2. Enter your API token
3. Use the intuitive interface to:
   - Create/edit/delete mailing lists
   - Add/edit/remove members
   - Manage subscriptions with toggle switches
   - View member counts and list details

**Via REST API:**
```bash
# Create a new list
curl -X POST http://localhost:8000/lists \
  -H "Authorization: Bearer your_token" \
  -H "Content-Type: application/json" \
  -d '{
    "list_name": "Developers",
    "list_email": "dev@lists.sasalliance.org",
    "description": "Developer discussions",
    "active": true
  }'

# Add a new member
curl -X POST http://localhost:8000/members \
  -H "Authorization: Bearer your_token" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "John Doe",
    "email": "john@example.com",
    "active": true
  }'

# Subscribe member to list
curl -X POST http://localhost:8000/subscriptions \
  -H "Authorization: Bearer your_token" \
  -H "Content-Type: application/json" \
  -d '{
    "list_email": "dev@lists.sasalliance.org",
    "member_email": "john@example.com"
  }'
```

**Via Direct Database Access:**
```bash
# Connect to MySQL
docker-compose exec mysql mysql -u maillist -p maillist

# Run SQL queries (see database/README.md for examples)
```

### Mail Server Configuration

### Mail Server Configuration

The Postfix mail server is configured for:
- **Hostname**: `lists.sasalliance.org` (accepts mail for this domain)
- **Origin Domain**: `sasalliance.org`
- **SES Region**: EU West 2 (configurable via `SMTP_HOST`)
- **Sender Whitelist**: @sasalliance.org (only authorized board members can send to lists)
- **Dynamic Lists**: Postfix queries MySQL in real-time for list members

### Testing Mail Delivery

```bash
# From inside Postfix container
docker-compose exec postfix bash
echo "Test message" | mail -s "Test Subject" community@lists.sasalliance.org

# Check delivery logs
docker-compose logs -f postfix | grep -E "(sent|bounced|deferred)"

# Verify MySQL query works
docker-compose exec postfix postmap -q "community@lists.sasalliance.org" \
  mysql:/etc/postfix/mysql_virtual_alias_maps.cf
```

## Security

- **Environment Variables**: All credentials stored in `.env` (git-ignored)
- **Private Network**: MySQL communicates with Postfix/API on internal Docker network only
- **Token Authentication**: API requires Bearer token for all write operations
- **Sender Whitelist**: Only @sasalliance.org addresses can send to mailing lists
- **TLS Encryption**: Enforced for SES relay connections
- **File Permissions**: SASL password files have restricted permissions (600)
- **No External MySQL**: Database is not exposed to host (no port mapping)

## Development

### Project Structure
```
├── docker-compose.yaml          # Multi-service orchestration
├── .env                        # Environment configuration (not in git)
├── web/                        # Web frontend (Nginx + HTML/CSS/JS)
│   ├── index.html
│   ├── static/
│   │   ├── css/style.css
│   │   └── js/
│   │       ├── api.js
│   │       ├── ui.js
│   │       └── app.js
│   ├── nginx.conf
│   └── Dockerfile
├── api/                        # REST API (FastAPI)
│   ├── main.py
│   ├── requirements.txt
│   └── Dockerfile
├── database/                   # MySQL schema and docs
│   ├── schema.sql
│   └── README.md
├── postfix/                    # Mail server configuration
│   ├── Dockerfile
│   ├── entrypoint.sh
│   ├── main.cf.template
│   ├── mysql_virtual_alias_maps.cf
│   ├── sasl_passwd.template
│   └── sender_access
└── .github/
    └── copilot-instructions.md # AI agent guidance
```

### Environment Variables
- `SES_USER`: AWS SES access key ID
- `SES_PASS`: AWS SES secret access key  
- `SMTP_HOST`: SMTP server hostname (default: email-smtp.eu-west-2.amazonaws.com)
- `SMTP_PORT`: SMTP server port (default: 587)
- `MYSQL_ROOT_PASSWORD`: MySQL root password
- `MYSQL_DATABASE`: Database name (default: maillist)
- `MYSQL_USER`: MySQL user (default: maillist)
- `MYSQL_PASSWORD`: MySQL user password
- `API_TOKEN`: Bearer token for API authentication

### Service Dependencies
```
mysql (healthcheck) → postfix, api → web
```
- MySQL must be healthy before Postfix/API start
- Web frontend depends on API being available
- All services communicate on private Docker network

### Debugging

Monitor services:
```bash
# View all service logs
docker-compose logs -f

# View specific service
docker-compose logs -f postfix
docker-compose logs -f api
docker-compose logs -f web
docker-compose logs -f mysql

# Filter for mail delivery status
docker-compose logs postfix | grep -E "(sent|bounced|deferred)"

# Check Postfix queue
docker-compose exec postfix postqueue -p

# Test MySQL connectivity from Postfix
docker-compose exec postfix postmap -q "community@lists.sasalliance.org" \
  mysql:/etc/postfix/mysql_virtual_alias_maps.cf

# Check API health
curl http://localhost:8000/health

# Access MySQL directly
docker-compose exec mysql mysql -u maillist -p maillist
```

### Development Workflow

1. **Make changes** to code or configuration
2. **Rebuild affected service**:
   ```bash
   sudo docker-compose up --build -d web   # Just web frontend
   sudo docker-compose up --build -d api   # Just API
   sudo docker-compose up --build -d postfix  # Just mail server
   ```
3. **Check logs** for errors:
   ```bash
   docker-compose logs -f [service-name]
   ```
4. **Test changes** via web interface or API

### Common Tasks

**Reset database completely:**
```bash
docker-compose down -v  # Removes volumes
docker-compose up -d    # Reinitializes from schema.sql
```

**Update API token:**
1. Edit `.env` and change `API_TOKEN`
2. Restart API: `docker-compose restart api`
3. Use new token in web interface

**Add authorized sender domain:**
1. Edit `postfix/sender_access`
2. Add line: `newdomain.com    OK`
3. Rebuild: `docker-compose up --build -d postfix`

## How It Works

1. **User manages lists** via web interface (http://localhost:3000)
2. **Web frontend** makes authenticated API calls to FastAPI backend
3. **API updates MySQL** database with new lists/members/subscriptions
4. **Email arrives** at Postfix for `someone@lists.sasalliance.org`
5. **Postfix queries MySQL** in real-time using native MySQL support
6. **MySQL returns** comma-separated list of active member emails
7. **Postfix expands** the alias and delivers to all members via SES
8. **Changes take effect immediately** - no restart or reload needed!

## Features

- ✅ **Real-time Updates** - Changes to lists/members take effect immediately
- ✅ **Native MySQL Integration** - Postfix queries database directly (no scripts)
- ✅ **Web Interface** - Modern, responsive UI for easy management
- ✅ **REST API** - Complete programmatic access with token auth
- ✅ **Sender Whitelist** - Only authorized domains can send to lists
- ✅ **SES Integration** - Reliable email delivery through Amazon SES
- ✅ **Secure** - Private Docker network, token auth, environment-based credentials
- ✅ **Flexible** - Manage via web, API, or direct database access
- ✅ **Scalable** - Database-driven architecture supports many lists and members

## Documentation

- **Web Interface**: See `web/README.md` for frontend features and usage
- **REST API**: See `api/README.md` for complete API reference
- **Database**: See `database/README.md` for schema and SQL examples
- **AI Agents**: See `.github/copilot-instructions.md` for development guidance

## Roadmap

- [x] Web frontend for mailing list management
- [x] SQL database backend for member storage
- [x] Dynamic configuration with native Postfix MySQL
- [x] Multi-service Docker Compose architecture
- [x] REST API with authentication
- [x] Sender whitelist for authorized domains
- [ ] Email verification workflow for new members
- [ ] Subscription confirmation (double opt-in)
- [ ] List archive functionality
- [ ] Unsubscribe links in emails
- [ ] Rate limiting and anti-spam measures
- [ ] HTTPS support for web interface
- [ ] Admin roles and permissions

## License

MIT License - see LICENSE file for details.