# GHN Gap Closure Plan

Date: 2026-03-07
Source of truth: `GHN Migration_Requirements.xlsx`

Companion checklist: `docs/PRODUCTION-EVIDENCE-CHECKLIST.md`

## Summary

- Goal: move all `Gap`/`Partial` items to `Done` where technically feasible.
- Constraint: several rows require real datacenter/network/operations capability and cannot be fully closed by repository code alone.

## Normalized Status Baseline

- `Done`: 5 rows
- `Done/Partial`: 8 rows (includes row 10 Multi-NIC and row 20 IAM deep-role uplift)
- `Partial`: 3 rows
- `Gap`: 9 rows

## Row-Level Closure Map

| Row | Requirement (short) | Current | Owner | Effort | ETA | Closure path |
|---|---|---|---|---|---|---|
| 2 | DBaaS architecture + pooling | Gap | Platform + DBA | L | 3-5 weeks | Add `dbaas-service`, PgBouncer layer, operational runbook |
| 3 | Migration MySQL/Postgres/Mongo no-downtime | Partial | Platform + Data Eng | M | 2-3 weeks | CDC playbooks + cutover/rollback scripts |
| 4 | Manual master failover | Done/Partial | SQL Service + DBA | S->M | 1-2 weeks | API implemented; complete with real primary/replica control-plane |
| 5 | Migration support readiness | Partial | PM + Platform | S | 1 week | Publish rehearsal templates and support SOPs |
| 6 | MongoDB Enterprise DBaaS roadmap | Gap | Product + Platform | M | 2 weeks (roadmap) | Product roadmap + PoC plan |
| 7 | Hub-and-spoke VPC | Partial | Network Service | M | 2-3 weeks | Transit hub model + route policy APIs |
| 8 | Site-to-site VPN | Partial | Network + Infra Ops | M | 2 weeks | BGP/IPSec templates + validation scripts |
| 9 | Scale bandwidth 1->10Gbps no downtime | Gap | Infra Ops | L | 3-4 weeks | Hardware benchmark, traffic migration runbook |
| 10 | VM Multi-NIC | Done/Partial | Compute + Network | M | 2-3 weeks | Implemented NIC attach/detach + create-time NIC specs + primary NIC protections; remaining step is cross-VPC runtime validation in production networking |
| 11 | Overlay/Underlay transparency | Gap | Infra Docs | S | 3-5 days | Publish architecture notes and diagrams |
| 12 | OpenStack Neutron/SDN detail | Gap | Infra Docs | S | 3-5 days | Add SDN implementation document |
| 13 | VPC peering perf metrics | Gap | SRE + Network | M | 2 weeks | Latency/throughput benchmark jobs + dashboard |
| 14 | Billing by SKU/resource | Done | Billing Service | Done | Done | Implemented |
| 15 | Project isolation + billing per team | Done/Partial | IAM + Resource Mgmt | M | 1-2 weeks | Extend guardrails to all auxiliary services |
| 16 | Bucket-level RBAC/ACL | Done/Partial | IAM + Gateway | M | 1-2 weeks | Extend from API-key scope to role-level ACL matrix |
| 17 | Workload identity | Done | IAM Service | Done | Done | Implemented |
| 18 | Access key over-permission reduction | Done/Partial | IAM + Gateway | M | 1-2 weeks | Add quotas/rotation policy for keys |
| 19 | Key scoped to bucket | Done/Partial | IAM + Gateway | M | 1 week | Implemented base flow; add richer action scopes |
| 20 | Deeper IAM roles | Done/Partial | IAM Service | M | 2 weeks | Implemented role inheritance (`parent_role_id`) + delegation workflow + gateway authorize route; remaining step is broader role catalog hardening and audit policy templates |
| 21 | Multi-cluster 2-site + GLB | Gap | K8s + Infra Ops | L | 4-6 weeks | Multi-site cluster deployment + weighted global LB |
| 22 | K8s node provisioning SLA | Gap | SRE | M | 1-2 weeks | Provisioning benchmark and SLA dashboard |
| 23 | Backup/restore automation API | Done | Backup Service | Done | Done | Implemented |
| 24 | Cross-zone backup target | Done | Backup Service | Done | Done | Implemented |
| 25 | Centralized activity log/audit | Done | Gateway + Monitor | Done | Done | Implemented |
| 26 | Unified service status page | Done/Partial | Monitor + Frontend | S | 1 week | Extend to DC-level aggregation and incident timeline |

## Next Coding Sprint (Repo-only feasible)

1. VPC benchmark jobs and metrics endpoint generation.
2. Project guardrail propagation to remaining auxiliary services.
3. Role catalog hardening (custom roles, deny policies, reusable assignment templates).
4. Migration automation playbooks (CDC/cutover/rollback) for row 3/5 partials.

## Progress Update (This Commit Window)

- VM Multi-NIC API/model: implemented in compute + network service layers (API, model, repository, tests).
- IAM role depth: implemented delegation endpoint wiring plus recursive role inheritance in authorization checks.
- API gateway updated for new routes and RBAC action mapping.
- IAM authorize route parity: `POST /api/v1/auth/authorize` now exposed via API gateway and validated in smoke flow.

## Infra/Partner Track (Non-code prerequisite)

1. Build 2-site production-like staging and validate failover drills.
2. Establish DB HA cluster runtime with controlled promotion/rollback.
3. Execute network bandwidth scaling and VPC peering performance certification.
4. Publish formal SLA report for K8s provisioning and datacenter reliability.

## Evidence Needed To Upgrade Done/Partial -> Done

### Row 10 (VM Multi-NIC)

- Production validation report for cross-VPC and cross-zone NIC attachment scenarios.
- Test evidence for failover behavior when primary/secondary NIC path changes.
- Runtime metrics: attach latency, packet loss, route convergence after NIC operations.
- Signed UAT from Network + Compute owners.

### Row 11/12 (Overlay/Underlay transparency, Neutron/SDN detail)

- Final topology diagrams per environment (staging, production) with node-level mapping.
- Data-plane packet path document for east-west and north-south traffic.
- MTU and encapsulation validation report (VXLAN/Geneve path, fragmentation policy).
- Incident troubleshooting matrix (symptom -> component -> runbook).

### Row 13 (VPC peering performance)

- Benchmark execution logs from at least 3 time windows (peak, off-peak, mixed).
- Dashboard snapshots: p50/p95 latency, throughput, packet loss, jitter.
- Comparison baseline between same-zone, cross-zone, and cross-region paths.
- Acceptance thresholds approved by SRE/Network.

### Row 20 (Enterprise role catalog)

- Role matrix finalized for common enterprise personas (platform-admin, secops, billing-admin, project-admin, auditor, readonly).
- Policy template pack with inheritance + explicit deny examples.
- Audit evidence: delegation + authorization decisions traceable in logs.
- Security sign-off for privilege boundaries and separation of duties.

### Other Infra-heavy Rows (9, 21, 22)

- Row 9: real 1->10Gbps traffic migration drill with no-downtime evidence.
- Row 21: two-site multi-cluster + GLB failover report with RTO/RPO measurements.
- Row 22: provisioning SLA report from repeated benchmark runs and weekly trend.

## Target System Design (Production Validation Scope)

```mermaid
flowchart LR
	U[Users / Enterprise Apps] --> GLB[Global Load Balancer]
	GLB --> DC1[Datacenter A]
	GLB --> DC2[Datacenter B]

	subgraph DC1 [Site A - Primary]
		APIGW1[API Gateway]
		IAM1[IAM Service\nDelegation + Inheritance]
		CMP1[Compute Service\nVM + Multi-NIC]
		NET1[Network Service\nVPC/Peering/Attachments]
		OBS1[Observability\nMetrics/Logs/Tracing]
		DB1[(Primary DB Cluster)]
		APIGW1 --> IAM1
		APIGW1 --> CMP1
		APIGW1 --> NET1
		IAM1 --> DB1
		CMP1 --> DB1
		NET1 --> DB1
		APIGW1 --> OBS1
		IAM1 --> OBS1
		CMP1 --> OBS1
		NET1 --> OBS1
	end

	subgraph DC2 [Site B - Secondary/Failover]
		APIGW2[API Gateway]
		IAM2[IAM Service Replica]
		CMP2[Compute Service Replica]
		NET2[Network Service Replica]
		OBS2[Observability Replica]
		DB2[(Replica DB Cluster)]
		APIGW2 --> IAM2
		APIGW2 --> CMP2
		APIGW2 --> NET2
		IAM2 --> DB2
		CMP2 --> DB2
		NET2 --> DB2
		APIGW2 --> OBS2
	end

	DB1 <-- Sync/CDC --> DB2
	NET1 <-- Inter-site Peering --> NET2
```

## Minimal Acceptance Gates For Final Done

1. Production-like test evidence is stored in docs and linked to each row.
2. Metrics dashboards are available for at least 14 consecutive days.
3. Failover drills are executed and signed off by Platform, Network, and SRE.
4. IAM role catalog and audit trail are reviewed by Security.