Skip to content

Tax Practice AI - Operations Runbook

Last updated: 2025-12-28 Version: 0.14

This runbook provides operational procedures for running Tax Practice AI components in development and production environments.

Table of Contents

  1. Quick Reference
  2. Prerequisites
  3. Development Environment
  4. Python Components
  5. CI/CD Pipeline
  6. Progress Tracking
  7. Java Components
  8. Database Operations
  9. External Services
  10. Field-Level Encryption (COMP-006)
  11. Troubleshooting

Quick Reference

Most Common Operations

# Start local database
docker-compose up -d

# Activate Python environment
source .venv/bin/activate

# Load environment variables
source .env

# Connect to local database
psql -h localhost -U dev_user -d tax_practice

# Run API server
S3_ENDPOINT_URL=http://localhost:4566 uvicorn src.api.main:app --reload --port 8000

# Update progress documentation (after completing a sequence)
python scripts/update_status.py
mkdocs build
wrangler pages deploy site --project-name=ali-ai-acctg
wrangler pages deploy site/client_facing_docs --project-name=tax-practice-ai-client

# Frontend development
cd frontend && npm run dev  # Start staff-app on :3001

# Frontend tests (Playwright)
cd frontend && npx playwright test              # Run all tests headless
cd frontend && npx playwright test --headed     # Run with visible browser
cd frontend && npx playwright test --ui         # Interactive UI mode

Emergency Contacts

  • Primary Developer: Don McCarty
  • Architecture Documentation: ARCHITECTURE.md
  • Backlog/Issues: backlog.md

Prerequisites

1. Python Environment

Required Python version: 3.10+

Virtual environment setup: 

cd /Users/donmccarty/ali-ai-acctg
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

2. Java Environment

Required Java version: 17+ (LTS)

# Verify Java installation
java --version

# Build Java components (when implemented)
# cd parsers/document-extractor
# mvn clean package

3. Docker (for Local Database)

Required: Docker Desktop or Docker Engine

# Verify Docker installation
docker --version
docker-compose --version

4. Environment Variables

Required for all operations: 

# Database connection (local dev uses defaults)
export DB_HOST=localhost
export DB_PORT=5432
export DB_NAME=tax_practice
export DB_USER=dev_user
export DB_PASSWORD=dev_password

# AWS (for S3, Bedrock)
export AWS_REGION=us-east-1
export AWS_ACCESS_KEY_ID=your_key
export AWS_SECRET_ACCESS_KEY=your_secret

# Anthropic API (for AI features)
export ANTHROPIC_API_KEY=sk-ant-...

Using .env file (recommended for local development): 

# .env file is gitignored - never commit credentials
source .env

5. Local Database Setup

One-time setup:

Create docker-compose.yml in project root: 

# docker-compose.yml
services:
  postgres:
    image: postgres:15
    environment:
      POSTGRES_DB: tax_practice
      POSTGRES_USER: dev_user
      POSTGRES_PASSWORD: dev_password
    ports:
      - "5432:5432"
    volumes:
      - postgres_data:/var/lib/postgresql/data

volumes:
  postgres_data:

Start database: 

docker-compose up -d

Verify connection: 

psql -h localhost -U dev_user -d tax_practice -c "SELECT 1"
# Password: dev_password

Stop database: 

docker-compose down          # Stop but keep data
docker-compose down -v       # Stop and delete data

5.1 Bootstrap Script

The bootstrap script (scripts/bootstrap.py) sets up the local development database:

Basic usage: 

# Truncate default tables and reload seed data
python scripts/bootstrap.py

# Full reset: drop all tables, recreate schema, seed
python scripts/bootstrap.py --reset

# Only reload seed data (don't touch schema)
python scripts/bootstrap.py --seed

# Only apply DATABASE_SCHEMA.sql (don't seed)
python scripts/bootstrap.py --schema

Table group filtering: 

# List available table groups
python scripts/bootstrap.py --list-groups

# Exclude specific groups (e.g., auth tables)
python scripts/bootstrap.py --reset --exclude auth,audit

# Include only specific groups
python scripts/bootstrap.py --reset --only core,billing

Available table groups:

Group Description Default
core Client, returns, documents, workflow Yes
billing Invoices, payments, estimated taxes Yes
auth Magic links, login attempts, identity verification No
chat AI chat sessions and messages Yes
integration SmartVault, SurePrep, Google, Stripe No
efiling E-filing transmissions, acknowledgements No
staging Pending documents, metadata cache Yes
communication Questions, messages, notes Yes
compliance Engagements, 7216 consent, PTIN Yes
audit Audit log (protected - append-only) No
system Schema version (protected) No

Schema management: - DDL is defined in docs/DATABASE_SCHEMA.sql (single source of truth) - Table names are extracted dynamically via regex parsing - Schema is applied via psql (local or Docker container) - Falls back to minimal schema if psql unavailable

6. LocalStack (S3 / AWS Services)

LocalStack provides local S3 for development without AWS credentials.

Start LocalStack: 

docker compose up -d localstack

Verify S3 is available: 

curl http://localhost:4566/_localstack/health | python3 -m json.tool
# Should show "s3": "available"

Configure for LocalStack (in .env or shell): 

export S3_ENDPOINT_URL=http://localhost:4566

Create test buckets (first time): 

# Using AWS CLI with LocalStack endpoint
aws --endpoint-url=http://localhost:4566 s3 mb s3://tax-practice-documents-dev
aws --endpoint-url=http://localhost:4566 s3 mb s3://tax-practice-archive-dev

# Or let the application create them (S3Service.ensure_bucket_exists)

List buckets: 

aws --endpoint-url=http://localhost:4566 s3 ls

Development Environment

Initial Setup

# 1. Clone repository (if not already done)
cd /Users/donmccarty/ali-ai-acctg

# 2. Run setup script
./setup_dev_environment.sh

# 3. Verify environment
source .venv/bin/activate
python3 -c "import yaml; print('Python OK')"

Launch Claude Code

./launch_claude.sh
# Select account (personal/work/api)

Python Components

API Server

Status: Implemented

# Start development server
uvicorn src.api.main:app --reload --port 8000

# Production
uvicorn src.api.main:app --host 0.0.0.0 --port 8000

# Verify health
curl http://localhost:8000/health

# API documentation
# Open http://localhost:8000/docs in browser

Middleware Stack: - CorrelationIdMiddleware - Request tracing via X-Correlation-ID - RequestLoggingMiddleware - Logs request/response with timing - CORSMiddleware - Configured from config.yaml

Logging: - Development: Console output with colors - Production: JSON structured logging

Workflow Engine

Status: Not yet implemented

# Run workflow (example)
# python -m src.workflows.intake.registration --client-id 123

Running Tests

Status: Full test pyramid implemented (1,522 tests) - Sequences S2-S11 complete

Test Type Count Target Description
Unit 1,290 85% Fast, no I/O, mocked dependencies
Integration 182 12% Real PostgreSQL + S3 (Docker/LocalStack)
E2E 50 3% Full HTTP stack with real DB 
# Activate virtual environment first
source .venv/bin/activate

# Run all tests
python -m pytest tests/ -v

# Run by test type
python -m pytest tests/unit -v           # Unit tests only (fast)
python -m pytest tests/integration -v    # Integration tests (requires Docker DB)
python -m pytest tests/e2e -v            # E2E tests (requires Docker DB)

# Run tests with coverage
python -m pytest tests/ --cov=src --cov-report=html

# Run specific test file
python -m pytest tests/unit/test_exceptions.py -v

# Run tests matching pattern
python -m pytest -k "test_validation" -v

# Run with short traceback (cleaner output)
python -m pytest tests/ -v --tb=short

Prerequisites for Integration/E2E Tests:

Integration and E2E tests require PostgreSQL running in Docker: 

# Ensure Docker database is running (port 5433)
docker ps | grep postgres

# If not running, start it
docker-compose up -d

Test Structure: 

tests/
├── conftest.py                          # Global fixtures (pytest_plugins)
├── unit/                                # 80% - Fast, no I/O
│   ├── test_exceptions.py
│   └── middleware/
│       ├── test_auth.py                 # JWT auth (25 tests)
│       ├── test_correlation.py
│       ├── test_error_handler.py
│       └── test_request_logging.py
├── integration/                         # 15% - Real DB
│   ├── conftest.py                      # DB fixtures, test schema
│   └── repositories/
│       └── test_client_repository.py    # CRUD operations (21 tests)
└── e2e/                                 # 5% - Full HTTP stack
    ├── conftest.py                      # API client fixtures
    └── test_client_api.py               # API endpoints (18 tests)

Test Fixtures: - test_db: Creates fresh PostgreSQL database per test - aurora_service: AuroraService with test database pool - api_client: httpx AsyncClient with dependency injection - mock_aurora: Mocked AuroraService for unit tests - demo_staff_users: Demo staff user data from YAML - sample_user_id: Primary demo user UUID for FK constraints

Demo Users (Single Source of Truth): - Defined in: tests/fixtures/demo_users.yaml - Used by: staff_auth.py, bootstrap.py, test fixtures - See TEST_PROTOCOL.md Section 10 for details

CI/CD Pipeline

Status: Implemented via GitHub Actions

Overview

The CI pipeline runs automatically on: - Push to main branch - Pull requests targeting main

Workflow file: .github/workflows/ci.yml

Jobs

Backend Jobs:

Job Duration Description
lint ~30s Ruff linting and formatting checks
unit-tests ~30s Fast tests with coverage reporting
integration-tests ~2m Tests with PostgreSQL service container
e2e-tests ~2m Full stack with PostgreSQL + LocalStack

Frontend Jobs:

Job Duration Description
frontend-typecheck ~1m TypeScript type validation
frontend-build ~2m Verify production bundle compiles
frontend-playwright ~5m Playwright UI tests with real API

Dependency Chain

lint ────────────────────┐
unit-tests ──────────────┼──► integration-tests ──► e2e-tests ──► frontend-playwright
frontend-typecheck ──────┤
                         ▼
                 frontend-build

Viewing Results

# View workflow runs (requires gh CLI)
gh run list

# View specific run details
gh run view <run-id>

# View workflow status on GitHub
# https://github.com/<owner>/<repo>/actions

Troubleshooting CI Failures

Lint failures: 

# Run locally to see issues
ruff check src/ tests/

# Auto-fix formatting issues
ruff format src/ tests/

Test failures: 

# Run the specific test tier locally
python -m pytest tests/unit/ -v        # Unit tests
python -m pytest tests/integration/ -v # Requires Docker
python -m pytest tests/e2e/ -v         # Requires Docker

Frontend failures: 

# TypeCheck locally
cd frontend && npm run typecheck

# Build locally
cd frontend && npm run build

# Run Playwright tests locally (requires API + frontend running)
cd frontend && npx playwright test --grep-invert "demo"

Service container issues: - PostgreSQL: Check container health in Actions logs - LocalStack: Wait for S3 service to be ready (health check in workflow) - Frontend server: Ensure port 3001 is available and Vite starts correctly

Progress Tracking

Status: Single source of truth via STATUS.yaml

Overview

Progress metrics are centralized in docs/STATUS.yaml. After updating this file, run the update script to propagate changes to all documentation.

Files managed by the system: - docs/STATUS.yaml - Single source of truth (edit this) - docs/backlog.md - Header line with version and progress - docs/client_facing_docs/Tax_Practice_AI_Presentation.html - Slide 16 - docs/client_facing_docs/index.html - Progress section

Updating Progress

After completing a sequence or milestone:

# 1. Edit STATUS.yaml with new values
#    - version: increment (e.g., "0.11" -> "0.12")
#    - sequences_complete: increment
#    - stories_complete: update count
#    - tests_passing: update count (rounded to nearest 10)
#    - progress_percent: recalculate (sequences_complete / 18 * 100)
#    - Add new sequence to completed_sequences list

# 2. Run the update script
python scripts/update_status.py

# 3. Rebuild and deploy to both Cloudflare Pages projects
mkdocs build
wrangler pages deploy site --project-name=ali-ai-acctg
wrangler pages deploy site/client_facing_docs --project-name=tax-practice-ai-client

# Optional: dry-run first to verify changes
python scripts/update_status.py --dry-run

STATUS.yaml Fields

Field Description
version Document version (e.g., "0.11")
sequences_complete Number of sequences done (1-18)
sequences_total Total sequences (18)
stories_complete MVP user stories completed
stories_total Total MVP stories (80)
tests_passing Test count (rounded, used with + suffix)
progress_percent Calculated: sequences_complete / 18 * 100
next_sequence Next sequence to implement
last_updated Date in YYYY-MM-DD format
completed_sequences List of completed sequences with descriptions

Java Components

Document Extractor

Status: Not yet implemented

# Build
# cd parsers/document-extractor
# mvn clean package

# Run
# java -jar target/document-extractor-1.0.0.jar --input /path/to/docs

Database Operations

Local PostgreSQL (Development)

# Interactive session
psql -h localhost -U dev_user -d tax_practice

# Single query
psql -h localhost -U dev_user -d tax_practice -c "SELECT COUNT(*) FROM clients"

# Run SQL file
psql -h localhost -U dev_user -d tax_practice -f scripts/setup.sql

# List tables
psql -h localhost -U dev_user -d tax_practice -c "\dt"

# Describe table
psql -h localhost -U dev_user -d tax_practice -c "\d clients"

Aurora PostgreSQL (AWS)

Status: Not yet configured

# Connection via psql (when configured)
# psql -h aurora-endpoint -U app_user -d tax_practice

External Services

Service Implementation Status

Service Status Notes
EmailService (SES) Stubbed Awaiting AWS SES credentials
SMSService (Twilio) Stubbed Awaiting Twilio credentials
StripeService Implemented Full checkout, ACH, webhooks
PersonaService Stubbed Awaiting Persona API credentials
SmartVaultService Stubbed Awaiting OAuth credentials
SurePrepService Stubbed Awaiting API credentials
GoogleService Stubbed Awaiting service account
S3Service Implemented Works with LocalStack (dev) or AWS (prod)
AuroraService Implemented Works with local Postgres (dev) or Aurora (prod)
PDF Generation Implemented ReportLab for tax package PDFs

Stripe (Payments)

Status: Implemented

StripeService provides: - Checkout session creation - ACH bank transfer support - Customer management - Webhook handling (payment_intent.succeeded, etc.)

# Test mode uses Stripe test keys
export STRIPE_SECRET_KEY=sk_test_...
export STRIPE_WEBHOOK_SECRET=whsec_...

Google Workspace (Signatures)

Status: Stubbed - awaiting service account

Persona (Identity Verification)

Status: Stubbed - awaiting API credentials

Field-Level Encryption (COMP-006)

Field-level encryption protects sensitive PII data at rest using AES-256-GCM encryption with HMAC-SHA256 blind indexing for searchable encrypted fields.

Encrypted Fields

Table Fields
client ssn_last4, ein, date_of_birth, prior_year_agi
client_contact ssn_last4, date_of_birth
users ptin
ptin_record ptin

Key Generation

Generate new encryption keys (32 bytes each, base64 encoded):

# Generate master key (AES-256 encryption)
python3 -c "import base64,os; print(base64.b64encode(os.urandom(32)).decode())"

# Generate HMAC key (blind indexing)
python3 -c "import base64,os; print(base64.b64encode(os.urandom(32)).decode())"

Environment Variables

Variable Required Description
ENCRYPTION_MASTER_KEY Yes 32-byte base64-encoded AES-256 key
ENCRYPTION_HMAC_KEY Yes 32-byte base64-encoded HMAC key
ENCRYPTION_KEY_VERSION No Key version (default: 1)
ENCRYPTION_ENABLED No Toggle on/off (default: true)

Initial Setup

  1. Generate keys using the commands above
  2. Store keys securely in AWS Secrets Manager or environment
  3. Run database migration: 
    psql -h localhost -U dev_user -d tax_practice -f scripts/migrations/006_field_encryption.sql
  4. Encrypt existing data: 
    # Preview changes
python scripts/encrypt_existing_data.py --dry-run

# Execute migration
python scripts/encrypt_existing_data.py

Key Rotation Procedure

  1. Generate new keys
  2. Increment ENCRYPTION_KEY_VERSION
  3. Deploy with new keys (old data still decryptable)
  4. Re-run data migration script to re-encrypt with new key
  5. Old key version stored in ciphertext enables graceful transition

Verification

# Verify encryption service health
python3 -c "
from src.services import services
from src.config.settings import get_config
services.initialize(get_config())
print('Encryption enabled:', services.encryption.is_enabled)
print('Health check:', services.encryption.health_check())
"

Troubleshooting Encryption

Error: Encryption service is disabled - Cause: Keys not set in environment - Solution: Set ENCRYPTION_MASTER_KEY and ENCRYPTION_HMAC_KEY

Error: Master key must be 32 bytes - Cause: Invalid key length - Solution: Regenerate key using generation command above

Error: Decryption failed - Cause: Data encrypted with different key or tampered - Solution: Verify correct keys are configured

Troubleshooting

Common Errors

Error: ModuleNotFoundError

Cause: Virtual environment not activated or dependencies not installed

Solution: 

source .venv/bin/activate
pip install -r requirements.txt

Error: Environment variable not set

Cause: Required environment variables missing

Solution: 

source .env
# Or export individually
export SNOWFLAKE_ACCOUNT=...

Error: Snowflake connection failed

Cause: Invalid credentials or network issue

Solution: 1. Verify .snowsql/config has correct credentials 2. Check network connectivity 3. Verify account/warehouse names

Getting Help

  1. Check ARCHITECTURE.md for system design
  2. Check CLAUDE.md for development standards
  3. Review backlog.md for known issues

Version History

Version Date Changes
0.1 2024-12-22 Initial runbook structure
0.2 2024-12-23 Added API server, testing docs
0.3 2024-12-24 Full test pyramid (187 tests: unit, integration, E2E)
0.4 2024-12-24 LocalStack setup, S3Service (207 tests)
0.5 2024-12-24 Sequence 4 complete (495 tests)
0.6 2024-12-24 S2-S11 complete (1,522 tests), service status table
0.7 2024-12-24 Non-API tech debt fixes (ReportLab PDF, citation resolution)
0.8 2024-12-24 CI/CD pipeline with GitHub Actions (TD-002)
0.9 2024-12-24 Added Progress Tracking section (STATUS.yaml)
0.14 2025-12-28 Added frontend CI jobs (typecheck, build, Playwright)

End of Runbook