gRPC Services¶

The gRPC contract between the Rust backend and Python runtime is defined in ui/proto/lakehouse.proto. It contains 6 services with 17 RPCs.

Service Overview¶

graph LR
    Axum["Axum REST"] --> CatalogService
    Axum --> HealthService
    Axum --> OperationsService
    Axum --> StreamingService
    Axum --> AgentService
    Axum --> ConfigService

CatalogService¶

Table discovery and schema introspection.

RPC	Request	Response	Description
`ListTables`	`ListTablesRequest`	`ListTablesResponse`	List all namespaces and tables
`GetSchema`	`GetSchemaRequest`	`SchemaResponse`	Get table schema with column details

HealthService¶

Table health analysis.

RPC	Request	Response	Description
`GetTableHealth`	`GetTableHealthRequest`	`TableHealthResponse`	Snapshot count, file stats, recommendations

OperationsService¶

Table maintenance and schema evolution.

RPC	Request	Response	Description
`Compact`	`CompactRequest`	`OperationResponse`	Run file compaction
`Expire`	`ExpireRequest`	`OperationResponse`	Expire old snapshots
`RemoveOrphans`	`RemoveOrphansRequest`	`OperationResponse`	Remove orphan files
`AddColumn`	`AddColumnRequest`	`OperationResponse`	Add a column
`RenameColumn`	`RenameColumnRequest`	`OperationResponse`	Rename a column
`DropColumn`	`DropColumnRequest`	`OperationResponse`	Drop a column

StreamingService¶

Pipeline lifecycle and monitoring.

RPC	Request	Response	Description
`StartPipeline`	`StartPipelineRequest`	`PipelineStatusResponse`	Start a streaming pipeline
`StopPipeline`	`StopPipelineRequest`	`StopPipelineResponse`	Stop a pipeline
`GetPipelineStatus`	`GetPipelineStatusRequest`	`PipelineStatusResponse`	Get pipeline status
`ListPipelines`	`ListPipelinesRequest`	`ListPipelinesResponse`	List all pipelines
`MonitorMetrics`	`MonitorMetricsRequest`	stream `PipelineMetrics`	Server-streaming metrics

AgentService¶

AI-driven lakehouse management.

RPC	Request	Response	Description
`Chat`	`ChatRequest`	`ChatResponse`	Single-turn agent chat
`ChatStream`	`ChatRequest`	stream `ChatStreamEvent`	Streaming agent chat

ConfigService¶

Catalog configuration management.

RPC	Request	Response	Description
`GetConfig`	`GetConfigRequest`	`ConfigResponse`	Get current configuration
`SetConfig`	`SetConfigRequest`	`SetConfigResponse`	Update configuration

Key Message Types¶

CompactRequest¶

message CompactRequest {
  string table = 1;
  string strategy = 2;           // binpack, sort, zorder
  map<string, string> options = 3;
}

TableHealthResponse¶

message TableHealthResponse {
  bool success = 1;
  string table = 2;
  int32 snapshot_count = 3;
  int32 data_file_count = 4;
  int64 total_size_bytes = 5;
  int64 avg_file_size_bytes = 6;
  int32 small_file_count = 7;
  string recommendation = 8;
  optional string error = 9;
}

ChatRequest¶

message ChatRequest {
  string prompt = 1;
  string provider = 2;
  string model = 3;
  optional string api_base = 4;
}

PipelineMetrics (server-streaming)¶

message PipelineMetrics {
  string pipeline_id = 1;
  string status = 2;
  int64 batches_processed = 3;
  int64 rows_processed = 4;
}

Stubs Generation¶

Python gRPC stubs are generated from the proto file:

drls grpc-codegen

Rust stubs are compiled at build time by build.rs using tonic-build.