independent project · 01AI developer platform
Skyeline
A platform for versioning prompts, routing LLM requests across providers, and observing the latency and token usage of agent and RAG applications.
- intended user
- Engineering teams that need a controlled way to develop prompts and operate LLM-backed applications across environments.
- current status
- Deployed prototype. Deployed on Google Cloud Run, although the service may currently be disabled. It has no users or sales.
01
Ownership
I originated and built this project from scratch, from the product idea and architecture through implementation. I use Claude Code and Codex as implementation tools; no human collaborator directed the work.
02
Architecture
A public-safe view of the system boundaries and the paths between them.
- interface
Next.js dashboard
Human interface for prompt lifecycle, approvals, playground runs, and trace inspection.
- interface
SDK consumer
An application-scoped TypeScript client using bearer authentication.
- control
NestJS API boundary
Fastify HTTP layer with validation, authentication, workspace context, and authorization guards.
- service
Prompt control plane
Versioning, lifecycle transitions, diffs, rollbacks, and approval workflows.
- service
LLM gateway
Provider registry, request normalization, streaming, and trace capture.
- service
Provider adapters
A common boundary for supported model providers and provider-specific translation.
- data
PostgreSQL
Workspace-scoped prompts, credentials, approvals, and observability records behind repositories.
- output
External model providers
Third-party inference endpoints reached only through the gateway adapters.
- Next.js dashboard → NestJS API boundary · session-authenticated HTTP
- SDK consumer → NestJS API boundary · allowlisted API + API key
- NestJS API boundary → Prompt control plane · manage and approve
- NestJS API boundary → LLM gateway · execute and stream
- Prompt control plane → PostgreSQL · versioned state
- LLM gateway → PostgreSQL · credentials and traces
- LLM gateway → Provider adapters · normalized request
- Provider adapters → External model providers · provider-specific call
03
Evidence boundary
What this case study establishes, and what it deliberately does not claim.
verified
Prompt lifecycle
The repository implements versioned prompts, diffs, rollbacks, tags, template variables, lifecycle states, and approval workflows.
basis · Private repository structure, service modules, and tests reviewed for this case study.
Multi-provider runtime
A provider registry and adapters normalize requests, support streaming, and persist latency and token metadata for inspection.
basis · Gateway architecture, adapter modules, and observability test inventory.
Tenant and credential controls
Workspaces and applications form the tenancy boundary, with role checks and per-workspace encryption for provider credentials.
basis · Authentication, authorization, workspace, application, and crypto modules.
Contract-driven SDK
The public TypeScript SDK is generated from an explicit OpenAPI operation allowlist, then wrapped with application scoping, typed errors, retry behavior, templates, streaming, and webhook verification utilities.
basis · OpenAPI generation pipeline, SDK package, and SDK test suites.
Cloud deployment design
The deployment workflow builds a container, runs database migrations as a separate Cloud Run job, and deploys the API service with managed secrets.
basis · Repository deployment workflow and architecture documentation.
not claimed
Current service state
The prototype was deployed to Google Cloud Run, but its current availability and full hosted flow have not been re-verified.
Outcome boundary
This case study reports implemented engineering surfaces only. It makes no claims about adoption, traffic, commercial outcomes, or current uptime.
Private implementation
The source repository and service configuration remain private. This module contains no copied source, private endpoints, identifiers, credentials, or operational data.
Verification boundary
The repository contains broad automated test coverage, but this portfolio content does not claim that every suite or hosted workflow is currently passing.
04
Engineering decisions
The design choices, reasoning, and tradeoffs that shaped the system.
Make the public API opt-in
Generate the SDK from an allowlist of public OpenAPI operation identifiers.
rationaleInternal endpoints stay private unless they are deliberately reviewed and added to the contract.
tradeoffEvery intended public operation requires an explicit contract update and regeneration step.
Keep provider logic behind adapters
Route model requests through a registry of provider adapters and expose a normalized gateway surface.
rationaleApplication code can use one execution path while provider-specific translation remains isolated.
tradeoffA normalized interface cannot erase every provider difference, so unsupported capabilities still need clear runtime errors.
Treat prompts as governed artifacts
Model prompt changes as immutable versions with draft, active, and archived states plus approval workflows.
rationaleA prompt change becomes reviewable and reversible instead of silently replacing the current behavior.
tradeoffThe lifecycle adds workflow overhead for simple experiments, which the playground must keep lightweight.
Separate schema changes from startup
Run database migrations as a dedicated deployment job before updating the application service.
rationaleStartup remains deterministic and failed migrations stop deployment before new application code receives traffic.
tradeoffThe deployment pipeline must coordinate two artifacts and surface migration failures clearly.
Scope credentials to the tenant boundary
Encrypt provider credentials with AES-256-GCM using keys derived per workspace through HKDF, and decrypt them only for execution.
rationaleA stored credential is not plaintext, and derived keys reduce cross-workspace blast radius.
tradeoffMaster-key protection and rotation remain operational responsibilities beyond application-layer encryption.
05
Reliability & safety
Controls that make failures visible, bounded, and recoverable.
Testing
Unit boundaries
Focused suites cover authorization guards, crypto helpers, provider adapters, prompt behavior, gateway orchestration, error mapping, and SDK utilities.
Testing
Database-backed integration
Integration and end-to-end suites exercise service boundaries with isolated PostgreSQL infrastructure through Testcontainers.
Safety boundary
Public-surface contract checks
Contract tests compare the generated SDK surface with the allowlisted public OpenAPI document to catch accidental exposure or drift.
Failure handling
Transport failure handling
The SDK has dedicated resilience tests for retry classification, backoff, retry guidance, streaming, and typed error behavior.
Observability
Structured diagnostics
Backend flows use structured logging with stable context, duration fields, sensitive-header redaction, and explicit outcome logging.
Testing
Deployment gates
Repository workflows define lint, type-check, unit, integration, end-to-end, build, contract, and packaging checks before deployment or release steps.
06
Synthetic evaluation
A six-row, public-safe fixture demonstrates how the case study can explain trace aggregation without presenting synthetic values as product performance.
Synthetic observability fixturesynthetic · not production or customer data
- Synthetic traces
- 6
- Fixture row count, not traffic.
- Synthetic success rate
- 83.3%
- 5 success rows and 1 error row in the fixture.
- Synthetic median latency
- 895 ms
- Median of the six invented latency values.
- Synthetic token volume
- 2,640
- Sum of invented input and output token values.
set · 6 synthetic traces
method · Define six fictional traces across three supported provider adapters with explicit latency, token, and outcome fields. Compute counts, success rate, median latency, and total tokens in this module from those exported fixture rows. Use the output only to demonstrate the observability model and the UI's aggregation concepts.
limits · The fixture is not a benchmark, load test, production sample, or customer dataset. Provider, network, model, prompt, and cache behavior are intentionally uncontrolled, so the values cannot support performance comparisons. A credible future evaluation would define a repeatable workload, pin models and regions, record warm and cold runs separately, and report distributions rather than a single aggregate.
07 – 09
State, material, next
current state
- The system was deployed to Google Cloud Run, but the deployment may need to be re-enabled before a live walkthrough.
- There are currently no users, sales, or adoption metrics, so this is presented as a deployed independent prototype.
- The repository remains private while architecture and public-safe product material can be shared.
demo & public material
- Architecture and case studyavailable
Public-safe system flow, engineering decisions, security boundaries, and limitations.
- Synthetic trace fixtureavailable
Six fictional rows plus exported aggregation logic for reproducible display metrics.
- Recorded walkthroughplanned
Publish only after the hosted flow is re-verified and demo data is reviewed for disclosure safety.
- Source repositorynot public
Implementation remains private. No repository link or copied source is included.
next steps
- 01Re-enable and verify the full hosted flow before publishing any live link.
- 02Run design-partner sessions to validate the prompt approval and observability workflows.
- 03Add usage-based feedback loops before treating the current feature set as a stable product surface.