Skip to content

UI Stack Architecture

The management console is a React frontend backed by a Rust Axum server that calls Python directly via PyO3 (embedded Python interpreter).

Data Flow

graph LR
    React["React UI<br/>:5173 (dev) / :3000 (prod)"]
    Axum["Axum REST<br/>:3000"]
    PyO3["PyO3<br/>(embedded Python)"]
    SQLite["SQLite"]
    Spark["Spark/Iceberg"]

    React -->|"fetch /api/*"| Axum
    Axum -->|"spawn_blocking + GIL"| PyO3
    Axum -->|pipelines, config| SQLite
    PyO3 --> Spark

Rust Backend

Stack: Axum 0.8 + PyO3 0.23 + SQLx (SQLite) + tower (middleware) + tracing

ui/backend/
├── Cargo.toml
├── rust-toolchain.toml         # stable
├── migrations/                 # SQLx migrations (001–010)
└── src/
    ├── main.rs                 # Server setup, routes, middleware
    ├── python/                 # PyO3 bridge to Python runtime
    │   ├── mod.rs              # PyBridge, spawn_py, pyany_to_json
    │   ├── catalog.rs          # list_tables, get_schema
    │   ├── health.rs           # get_table_health
    │   ├── operations.rs       # compact, expire, add/rename/drop column
    │   └── agent.rs            # agent_chat
    ├── api/                    # REST handlers (JSON ↔ PyO3)
    │   ├── catalog.rs          # list_tables, get_schema
    │   ├── health.rs           # get_table_health
    │   ├── operations.rs       # compact, expire, add/rename/drop column
    │   ├── streaming.rs        # start/stop/status/list pipelines (pure Rust)
    │   ├── pipeline_defs.rs    # Pipeline definitions, versions, environments, approvals
    │   ├── agent.rs            # chat
    │   ├── config.rs           # get/set config (pure Rust, ~/.drls/config.json)
    │   └── ws.rs               # WebSocket pipeline monitoring
    ├── db/                     # SQLx layer
    │   ├── mod.rs
    │   ├── models.rs           # Pipeline, Config, AuditLog, PipelineDefinition, PipelineVersion, etc.
    │   └── queries.rs          # CRUD queries
    ├── notebook/               # Marimo notebook service
    └── middleware/              # Tower middleware

REST API Routes

Method Path Handler Description
GET /api/health health Health check
GET /api/tables list_tables List Iceberg tables
GET /api/table/{name}/schema get_schema Get table schema
GET /api/table/{name}/health get_table_health Table health report
POST /api/compact compact Run compaction
POST /api/expire expire Expire snapshots
POST /api/evolve/add-column add_column Add column
POST /api/evolve/rename-column rename_column Rename column
POST /api/evolve/drop-column drop_column Drop column
POST /api/pipeline/start start_pipeline Start streaming pipeline
POST /api/pipeline/{id}/stop stop_pipeline Stop pipeline
GET /api/pipeline/{id}/status pipeline_status Pipeline status
GET /api/pipelines list_pipelines List all pipelines
POST /api/agent/chat chat Agent chat
GET /api/pipeline/monitor WebSocket Live metrics stream
GET /api/config get_config Get configuration
PUT /api/config set_config Set configuration
POST /api/pipeline-defs create_definition Create pipeline definition
GET /api/pipeline-defs list_definitions List definitions
GET /api/pipeline-defs/{id} get_definition Get definition detail
PUT /api/pipeline-defs/{id} update_definition Update definition
DELETE /api/pipeline-defs/{id} delete_definition Delete definition
POST /api/pipeline-defs/{id}/versions create_version Create new version
GET /api/pipeline-defs/{id}/versions list_versions List versions
GET .../versions/{vid} get_version Get version detail
POST .../versions/{vid}/promote promote_version Promote to next env
POST .../versions/{vid}/demote demote_version Demote UAT → QA
POST .../versions/{vid}/retire retire_version Retire from production
GET .../versions/{vid}/export export_version Export as JSON
POST /api/pipeline-defs/import import_definition Import from JSON
GET /api/approvals list_approvals List approval requests
GET /api/approvals/{id} get_approval Get approval detail
POST /api/approvals/{id}/approve approve_request Approve promotion
POST /api/approvals/{id}/reject reject_request Reject promotion
GET /api/approvals/{id}/comments list_comments List comments
POST /api/approvals/{id}/comments add_comment Add comment
GET /api/notifications/pending-approvals pending_approvals Pending count
POST /api/notebook/start start_notebook Create notebook pod
POST /api/notebook/stop stop_notebook Delete notebook pod
GET /api/notebook/status notebook_status Poll pod phase
ANY /notebook/{user_id}/* notebook_proxy Reverse proxy (HTTP + WS)

React Frontend

Stack: React 18 + Vite 6 + TypeScript 5.6 + TailwindCSS v4 + React Flow (@xyflow/react v12)

ui/frontend/
├── package.json
├── vite.config.ts              # Dev proxy: /api → localhost:3000
├── eslint.config.js            # ESLint 9 flat config
├── .prettierrc
└── src/
    ├── components/
    │   ├── tables/             # TableExplorer, TableDetail, HealthIndicator, TableOpsPanel
    │   ├── pipelines/          # PipelineBuilder, PipelineList, VersionHistory, PromoteModal, NodePalette
    │   ├── approvals/          # ApprovalQueue, ApprovalReview
    │   ├── common/             # EnvironmentBadge
    │   ├── monitor/            # StreamingDashboard (WebSocket), PipelineStatus
    │   ├── agent/              # ChatPanel, LLMSelector, ToolCallDisplay
    │   └── setup/              # SetupWizard (4-step), ConnectionTest
    ├── api/client.ts           # API client matching REST routes
    └── hooks/                  # useApi, useWebSocket, useApprovalNotifications

Development Workflow

Run two terminals:

cd ui/backend && cargo run
Starts on :3000. Embeds Python via PyO3.

cd ui/frontend && npm run dev
Starts on :5173, proxies /api to :3000

Production Deployment

In production, the Rust server serves the built React assets directly:

# Build frontend
cd ui/frontend && npm run build

# Run Rust server (serves frontend from ../frontend/dist)
cd ui/backend && cargo run --release

The FRONTEND_DIR environment variable controls the path to the built assets (default: ../frontend/dist).