This directory contains setup scripts for configuring Microsoft SQL Server to work with the Weather Connect application. The schema mirrors the Oracle implementation with a two-table structure.

Prerequisites

Microsoft SQL Server 2017 or later
SQL Server Management Studio (SSMS) or Azure Data Studio (optional but recommended)
Appropriate permissions to create databases and users
For Always On Availability Groups: SQL Server Enterprise Edition

Database Schema Overview

The database consists of two tables:

weather_data - Main table storing weather measurements
- id (BIGINT, IDENTITY, PRIMARY KEY)
- timestamp (DATETIME2(6))
- topic (NVARCHAR(255))
- location (NVARCHAR(100))
- temperature (DECIMAL(5,2))
- humidity (DECIMAL(5,2))
- pressure (DECIMAL(7,2))
- wind_speed (DECIMAL(5,2))
- wind_direction (DECIMAL(5,2))
- rainfall (DECIMAL(6,2))
- created_at (DATETIME2(6))
raw_data - Table storing raw JSON data
- weather_data_id (BIGINT, PRIMARY KEY, FOREIGN KEY)
- raw_data (NVARCHAR(MAX) with JSON validation)

Setup Instructions

Standard SQL Server Setup

Run the scripts in order:

1. Create Database

sqlcmd -S your_server -U sa -P your_password -i 01_create_database.sql

Or in SSMS/Azure Data Studio, open and execute 01_create_database.sql.

This script:

Creates the WeatherDB database
Sets the recovery model to FULL (required for Always On AG)
Takes a full backup

2. Create User and Grant Permissions

sqlcmd -S your_server -U sa -P your_password -i 02_create_user.sql

This script:

Creates SQL Server login weather_user with password Weather123!
Creates database user in WeatherDB
Grants necessary permissions (db_datareader, db_datawriter, CREATE TABLE, etc.)

Security Note: Change the default password in production!

3. Create Tables

sqlcmd -S your_server -U weather_user -P Weather123! -d WeatherDB -i 03_create_tables.sql

This script:

Creates weather_data table with indexes
Creates raw_data table with foreign key constraint
Adds JSON validation constraint on raw_data.raw_data column

Alternatively, the application can create tables automatically on first run if the user has appropriate permissions.

Always On Availability Group Setup

If you're using SQL Server Always On Availability Groups for high availability:

Prerequisites

Windows Server Failover Clustering (WSFC) configured
Always On Availability Groups enabled on all SQL Server instances
Full backup of the database taken (handled by 01_create_database.sql)

Steps to Add Database to AG

Take a transaction log backup (after running the setup scripts):

USE master;
GO
BACKUP LOG WeatherDB 
TO DISK = '/var/opt/mssql/data/WeatherDB_Log.trk'
WITH FORMAT, INIT, NAME = 'WeatherDB Log Backup';
GO

Add database to Availability Group (on primary replica):

ALTER AVAILABILITY GROUP AAGONE ADD DATABASE WeatherDB;
GO

Restore on secondary replicas:

For each secondary replica, restore the database with NORECOVERY:

-- On each secondary replica
RESTORE DATABASE WeatherDB 
FROM DISK = 'WeatherDB_Full.bak'
WITH NORECOVERY;
GO

RESTORE LOG WeatherDB 
FROM DISK = '/path/to/WeatherDB_Log.trk'
WITH NORECOVERY;
GO

-- Join the database to the AG
ALTER DATABASE WeatherDB SET HADR AVAILABILITY GROUP = AAGONE;
GO

Connection String for AG

When connecting to an Always On Availability Group, use the listener name:

Server=AG_Listener_Name,1433;Database=WeatherDB;User Id=weather_user;Password=Weather123!;
ApplicationIntent=ReadWrite;MultiSubnetFailover=True;

Application Configuration

Update your .env file to use MS SQL Server:

# Database Configuration - MS SQL Server
DB_TYPE=mssql
DB_HOST=your_sql_server  # Or AG listener name
DB_PORT=1433
DB_NAME=WeatherDB
DB_USERNAME=weather_user
DB_PASSWORD=Weather123!

# MS SQL specific
MSSQL_DRIVER=ODBC Driver 17 for SQL Server

ODBC Driver Installation

Ubuntu/Debian:

curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
curl https://packages.microsoft.com/config/ubuntu/$(lsb_release -rs)/prod.list > /etc/apt/sources.list.d/mssql-release.list
apt-get update
ACCEPT_EULA=Y apt-get install -y msodbcsql17

Red Hat/CentOS:

curl https://packages.microsoft.com/config/rhel/8/prod.repo > /etc/yum.repos.d/mssql-release.repo
yum remove unixODBC-utf16 unixODBC-utf16-devel
ACCEPT_EULA=Y yum install -y msodbcsql17

Windows:

Download and install from: https://docs.microsoft.com/en-us/sql/connect/odbc/download-odbc-driver-for-sql-server

Verification

After setup, verify the tables were created:

USE WeatherDB;
GO

-- Check tables
SELECT * FROM INFORMATION_SCHEMA.TABLES 
WHERE TABLE_NAME IN ('weather_data', 'raw_data');

-- Check foreign keys
SELECT * FROM INFORMATION_SCHEMA.REFERENTIAL_CONSTRAINTS
WHERE CONSTRAINT_NAME = 'fk_raw_data_weather_data';

-- Check indexes
SELECT * FROM sys.indexes
WHERE object_id = OBJECT_ID('weather_data');

Testing

Test the connection from the application:

# Start the application
python src/wxconnect/main.py

The application should:

Connect to MS SQL Server
Create tables if they don't exist (or verify existing tables)
Start receiving MQTT messages
Insert data into both weather_data and raw_data tables

Troubleshooting

Connection Issues

Check SQL Server is listening on TCP/IP:
- SQL Server Configuration Manager → SQL Server Network Configuration → Protocols
- Enable TCP/IP protocol
- Restart SQL Server service

Firewall rules:

# Linux
sudo firewall-cmd --add-port=1433/tcp --permanent
sudo firewall-cmd --reload

# Windows
New-NetFirewallRule -DisplayName "SQL Server" -Direction Inbound -Protocol TCP -LocalPort 1433 -Action Allow

Test connection with sqlcmd:

sqlcmd -S your_server,1433 -U weather_user -P Weather123! -d WeatherDB

Permission Issues

If the application can't create tables, grant explicit permissions:

USE WeatherDB;
GO
GRANT CREATE TABLE TO weather_user;
GRANT ALTER ON SCHEMA::dbo TO weather_user;
GO

Always On AG Issues

Database not synchronizing:
- Check AG health: SELECT * FROM sys.dm_hadr_database_replica_states;
- Verify network connectivity between replicas
- Check SQL Server error log
Automatic failover not working:
- Verify AG is configured with automatic failover mode
- Check WSFC quorum configuration
- Ensure database is in SYNCHRONIZED state

Comparison with Oracle Schema

The MS SQL schema exactly mirrors the Oracle implementation:

Feature	Oracle	MS SQL
Identity column	SEQUENCE	IDENTITY
Timestamp	TIMESTAMP(6)	DATETIME2(6)
String	VARCHAR2	NVARCHAR
JSON storage	CLOB with IS JSON check	NVARCHAR(MAX) with ISJSON check
FK cascade	ON DELETE CASCADE	ON DELETE CASCADE
Index	Standard B-tree	Standard B-tree

Both implementations provide:

Automatic ID generation
Foreign key relationship between tables
JSON validation on raw data
Indexed timestamp and topic columns
Identical application-level interface

Support

For issues or questions:

Check application logs: LOG_LEVEL=DEBUG in .env
Review SQL Server error log
Consult Microsoft SQL Server documentation