Compare commits
No commits in common. "main" and "tolaria-discovery" have entirely different histories.
main
...
tolaria-di
221 changed files with 65 additions and 44339 deletions
|
|
@ -1,11 +1,3 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/f4e559f5-403c-f241-be50-0962d8174644
|
||||
type: arch
|
||||
role: reference
|
||||
representation: source
|
||||
state: current
|
||||
---
|
||||
# Infrastruktur & Betriebs-Handbuch — Trading-System VPS
|
||||
|
||||
> Stand: 20.08.2026 · VPS: `187.124.31.123`
|
||||
|
|
|
|||
|
|
@ -1,11 +1,3 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/1761a726-55af-f97b-d509-84aa4577f102
|
||||
type: arch
|
||||
role: module
|
||||
representation: source
|
||||
state: current
|
||||
---
|
||||
# Modul-03-Market-Data — Betriebsdokumentation
|
||||
|
||||
> Erstellt: 20.08.2026 (Rain Ocampo) · Status: ✅ In Betrieb (healthy)
|
||||
|
|
|
|||
|
|
@ -1,11 +1,3 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/6fca3e3a-70b2-0927-d7fb-e650a8f0e7f8
|
||||
type: arch
|
||||
role: module
|
||||
representation: source
|
||||
state: current
|
||||
---
|
||||
# Modul-04-Market-Regime — Betriebsdokumentation
|
||||
|
||||
> Erstellt: 20.08.2026 (Rain Ocampo) · Status: ✅ Freigegeben (E2E bestanden)
|
||||
|
|
|
|||
|
|
@ -1,11 +1,3 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/0aef122a-790b-b17a-9176-cc32df98c5a1
|
||||
type: arch
|
||||
role: module
|
||||
representation: source
|
||||
state: current
|
||||
---
|
||||
# Modul-05-Strategy-Engine — Betriebsdokumentation
|
||||
|
||||
> Erstellt: 20.08.2026 (Rain Ocampo) · Status: ✅ Freigegeben (E2E bestanden)
|
||||
|
|
|
|||
|
|
@ -1,11 +1,3 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/2b4c4170-b5b2-8a2b-fdee-8ea5eff191b5
|
||||
type: arch
|
||||
role: module
|
||||
representation: source
|
||||
state: current
|
||||
---
|
||||
# Modul-06-Signal-Ranking — Betriebsdokumentation
|
||||
|
||||
> Erstellt: 20.08.2026 (Rain Ocampo) · Status: ✅ **Freigegeben** (E2E+Idempotenz+Reconnect+Doku grün)
|
||||
|
|
|
|||
|
|
@ -1,11 +1,3 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/645f8ff7-5a1a-bcf3-8dfe-0a8cfdf15dbc
|
||||
type: arch
|
||||
role: module
|
||||
representation: source
|
||||
state: current
|
||||
---
|
||||
# Modul-07: Risk-Manager
|
||||
|
||||
**Status:** ✅ **Freigegeben** (20.08.2026)
|
||||
|
|
|
|||
|
|
@ -1,11 +1,3 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/603e6b87-115a-e258-5bfb-7d94d6d78c97
|
||||
type: arch
|
||||
role: module
|
||||
representation: source
|
||||
state: current
|
||||
---
|
||||
# Modul-08: Portfolio-Manager
|
||||
|
||||
**Status: FREIGEGEBEN** (20.08.2026)
|
||||
|
|
|
|||
|
|
@ -1,11 +1,3 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/48bd264f-607b-15f1-5f73-3e922af9b19d
|
||||
type: arch
|
||||
role: module
|
||||
representation: source
|
||||
state: current
|
||||
---
|
||||
# Modul-09: Execution-Service
|
||||
|
||||
**Status: FREIGEGEBEN** (20.08.2026)
|
||||
|
|
|
|||
|
|
@ -1,11 +1,3 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/0b393153-53e5-99fe-1098-e8bb4e0b6c7c
|
||||
type: journal
|
||||
role: module
|
||||
representation: source
|
||||
state: current
|
||||
---
|
||||
# Modul-10: Trade-Journal
|
||||
|
||||
**Status: FREIGEGEBEN** (20.08.2026)
|
||||
|
|
|
|||
|
|
@ -1,11 +1,3 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/31cf1506-0ab6-5f7a-cecc-0e3a7a2e6572
|
||||
type: arch
|
||||
role: module
|
||||
representation: source
|
||||
state: current
|
||||
---
|
||||
# Modul-11: Analytics
|
||||
|
||||
**Status: FREIGEGEBEN** · Container `Modul-11-Analytics` · Image `analytics-service:0.1.0`
|
||||
|
|
|
|||
|
|
@ -1,11 +1,3 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/302e9929-e186-c930-2406-ad4a8f17c6fd
|
||||
type: arch
|
||||
role: module
|
||||
representation: source
|
||||
state: current
|
||||
---
|
||||
# Modul-12: Backtesting
|
||||
|
||||
**Status: FREIGEGEBEN** · Container `Modul-12-Backtesting` · Image `backtesting:0.1.2` (V1 + V1.1 parallel)
|
||||
|
|
|
|||
|
|
@ -1,11 +1,3 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/bde121b1-121f-590d-2b54-fedc13b02173
|
||||
type: arch
|
||||
role: module
|
||||
representation: source
|
||||
state: current
|
||||
---
|
||||
# Modul-13: Optimization
|
||||
|
||||
**Status: FREIGEGEBEN ✅** · Container `Modul-13-Optimization` · Image `optimization:0.1.0`
|
||||
|
|
|
|||
|
|
@ -1,11 +1,3 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/0a0c9ca4-1cf4-aee1-3072-e1c3e8cfd29e
|
||||
type: arch
|
||||
role: module
|
||||
representation: source
|
||||
state: current
|
||||
---
|
||||
# Modul-14: Notification-Service
|
||||
|
||||
**Status: FREIGEGEBEN ✅** · Container `Modul-14-Notification` · Image `notification:0.1.0`
|
||||
|
|
|
|||
|
|
@ -1,11 +1,3 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/1d6c2323-a31a-8b50-fcdd-3c9268175a69
|
||||
type: design
|
||||
role: design
|
||||
representation: source
|
||||
state: current
|
||||
---
|
||||
# Design-Vorschlag: Modul-15 → Modul-09 Anbindung (Safety-/Trading-Control)
|
||||
|
||||
**Status: NUR DESIGN-VORSCHLAG — keine Implementierung.**
|
||||
|
|
|
|||
|
|
@ -1,11 +1,3 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/a3cffdc1-41fd-ab84-c211-7b586f153e8d
|
||||
type: arch
|
||||
role: implementation
|
||||
representation: source
|
||||
state: current
|
||||
---
|
||||
# Modul-15 → Modul-09 Control-Anbindung — Implementierung & Live-E2E
|
||||
|
||||
**Status: FREIGEGEBEN / PRODUKTIV VERIFIZIERT** (20.08.2026, Nutzer-Bestätigung).
|
||||
|
|
|
|||
|
|
@ -1,11 +1,3 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/c2641bbf-0612-0737-86fd-622d899109ca
|
||||
type: arch
|
||||
role: module
|
||||
representation: source
|
||||
state: current
|
||||
---
|
||||
# Modul-15: Monitoring-Control
|
||||
|
||||
**Status: FREIGEGEBEN** (20.08.2026) · Container `Modul-15-Monitoring-Control` · Image `monitoring-control:0.1.0`
|
||||
|
|
|
|||
|
|
@ -1,11 +1,3 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/2ad746ca-67f1-ce90-1304-4a1f4b052eb7
|
||||
type: arch
|
||||
role: module
|
||||
representation: source
|
||||
state: current
|
||||
---
|
||||
# Modul-18: Position-Manager
|
||||
|
||||
**Status: FREIGEGEBEN (21.08.2026)** · Container `Modul-18-Position-Manager` · Port `55018` (nur intern/expose)
|
||||
|
|
|
|||
|
|
@ -1,11 +1,3 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/3f834840-867d-9e9f-c7d2-8f1caf1550a4
|
||||
type: arch
|
||||
role: module
|
||||
representation: source
|
||||
state: current
|
||||
---
|
||||
# Modul-19: Broker-Reconciliation
|
||||
|
||||
**Status: FREIGEGEBEN (21.08.2026)** · Container `Modul-19-Broker-Reconciliation` · Port `55019` (nur intern/expose)
|
||||
|
|
|
|||
|
|
@ -1,17 +1,11 @@
|
|||
---
|
||||
id: object/fb050d49-41a1-a58b-3baf-e95a1765c7ed
|
||||
type: arch
|
||||
role: hub
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Agent
|
||||
title: AI-Agents Übersicht
|
||||
tags: [ai, agents, home]
|
||||
created: 2026-08-20
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
||||
# AI-Agents
|
||||
|
||||
Multi-Agent-System von Christian List (Resident-Evil-Theme).
|
||||
|
|
|
|||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/6923ab6b-5132-55d7-4fd5-66890048e12b
|
||||
type: arch
|
||||
role: note
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Agent
|
||||
title: Alice Betrieb
|
||||
tags: [ai, alice, openclaw, betrieb]
|
||||
created: 2026-08-20
|
||||
|
|
|
|||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/7e4314d1-6c2c-df92-e40d-c3bc04d88182
|
||||
type: arch
|
||||
role: index
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Note
|
||||
title: Inbox
|
||||
tags: [inbox]
|
||||
created: 2026-08-20
|
||||
|
|
|
|||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/2325c9b5-a668-9a04-3cf3-38be914b170c
|
||||
type: arch
|
||||
role: index
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Projekt
|
||||
title: Projekte
|
||||
tags: [projects, home]
|
||||
created: 2026-08-20
|
||||
|
|
|
|||
|
|
@ -1,17 +1,11 @@
|
|||
---
|
||||
id: object/9bd5b7db-bb0f-ae22-3a65-9410b77ad3de
|
||||
type: arch
|
||||
role: reference
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Referenz
|
||||
title: VPS Infrastruktur
|
||||
tags: [vps, infrastruktur, reference]
|
||||
created: 2026-08-20
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
||||
# VPS-Infrastruktur
|
||||
|
||||
## Kern-Container
|
||||
|
|
|
|||
|
|
@ -1,17 +1,11 @@
|
|||
---
|
||||
id: object/4dd3d5ea-da35-5265-a501-a2108253464e
|
||||
type: arch
|
||||
role: index
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Start
|
||||
title: Vault-Start
|
||||
tags: [home, index]
|
||||
created: 2026-08-20
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
||||
# Vault-Start
|
||||
|
||||
Willkommen im Tolaria-Second-Brain von Christian List.
|
||||
|
|
|
|||
|
|
@ -1,14 +1,8 @@
|
|||
---
|
||||
id: object/deea2ed6-cd6f-b006-2e4e-949abea60904
|
||||
type: arch
|
||||
role: history
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Note
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
||||
# Phase 10c — Deterministische Slippage (DETERMINISTIC_FIXED)
|
||||
|
||||
**Geändert von:** Rain Ocampo (Hermes)
|
||||
|
|
|
|||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/799b279e-9709-3c05-972b-3751d1dc8a8d
|
||||
type: arch
|
||||
role: note
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Notiz
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,16 +1,10 @@
|
|||
---
|
||||
id: object/879703ed-5d7c-5446-397a-d2ec7b1a1d22
|
||||
type: journal
|
||||
role: journal
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
date: 2026-07-20
|
||||
summary: T212 Tages-Recap
|
||||
type: Journal
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
||||
# 📊 Tages-Recap — 20.07.2026
|
||||
|
||||
**📊 T212 Tages-Recap — 20.07.2026**
|
||||
|
|
|
|||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/99e784de-4e66-c9d3-7d3e-1a62afc91d90
|
||||
type: log
|
||||
role: log
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Log
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/d334663d-ec48-f6e7-03a9-e9715571955f
|
||||
type: log
|
||||
role: log
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Log
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/48e725ec-b9cb-43f1-f719-0e977fb11570
|
||||
type: log
|
||||
role: log
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Log
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/b8d1e818-00ba-c4de-c12b-819b9bc92c48
|
||||
type: log
|
||||
role: log
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Log
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/69dce38b-d44c-8361-ee76-128f4639c165
|
||||
type: log
|
||||
role: log
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Log
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/10f3cd5a-aa77-9cfd-cfc0-f4ab7abd0906
|
||||
type: log
|
||||
role: log
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Log
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/1748b435-164d-5a9b-674c-233b1a0a1cff
|
||||
type: log
|
||||
role: log
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Log
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/03b4a75f-71b1-b33f-c960-c12e948560e1
|
||||
type: log
|
||||
role: log
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Log
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/acf5b5da-0138-6bf8-b470-f80ba8ca7e03
|
||||
type: log
|
||||
role: log
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Log
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/730534ed-b44a-46de-0082-268a476003c7
|
||||
type: log
|
||||
role: log
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Log
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/1b4b6582-403a-d1d3-6f4c-a8f5ea868ddb
|
||||
type: log
|
||||
role: log
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Log
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/866ae1ee-7fd7-4175-c6a7-5a5c0e663d8d
|
||||
type: log
|
||||
role: log
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Log
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/6d67c1dc-044b-9e40-b519-b3dc57a85d3a
|
||||
type: log
|
||||
role: log
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Log
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/286f4f60-ab5d-f872-ae6b-cb997c214dc5
|
||||
type: log
|
||||
role: log
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Log
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/73fdf272-78c3-1c55-1198-84181898f9f0
|
||||
type: log
|
||||
role: log
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Log
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/5a6cf599-8023-e788-1d6c-eb72bfa2ca58
|
||||
type: log
|
||||
role: log
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Log
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/3fbb2b3c-921b-3db5-73f3-7de26261ef2f
|
||||
type: log
|
||||
role: log
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Log
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/badaaaea-79fb-3c65-c1d3-f0028040204b
|
||||
type: log
|
||||
role: log
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Log
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,14 +1,8 @@
|
|||
---
|
||||
id: object/43673951-0706-29e0-12bb-55b652456962
|
||||
type: log
|
||||
role: log
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Log
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
||||
# T212 Logs — 14.08.2026 23:45 UTC
|
||||
## Offen (4) | Unrealisiert: +0.0%
|
||||
- XRX_US_EQ | +0.0% | SL: 2.98€
|
||||
|
|
|
|||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/3b9de5f6-784d-a2dd-a9ea-d154450c3586
|
||||
type: log
|
||||
role: log
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Log
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/5811e85a-191f-c7a3-8f2e-e3768ae15044
|
||||
type: log
|
||||
role: log
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Log
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/3479a3df-f762-0cd1-bf71-2e29fff56cca
|
||||
type: log
|
||||
role: log
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Log
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,105 +0,0 @@
|
|||
# D1/D2 — Canonical Schema Contract & Fingerprint Tooling (SoT)
|
||||
|
||||
**Status:** FORMALLY PERSISTED — authoritative origin/main
|
||||
**Datum:** 2026-09-08
|
||||
**Verdict:** D1+D2 FORMALLY PERSISTED TO AUTHORITATIVE SOT
|
||||
|
||||
---
|
||||
|
||||
## 1. Zweck
|
||||
|
||||
Dieses Verzeichnis materialisiert den **CURRENT VERIFIED PRODUCTION SCHEMA CONTRACT (L0)**
|
||||
als kanonische, deterministisch fingerprintbare Autorität. Es ist das Ergebnis der
|
||||
D1/D2-Remediation (Schema-Contract + deterministischer Fingerprint) und wurde durch den
|
||||
unabhängigen D1+D2-Checker mit **Verdict A** bestätigt.
|
||||
|
||||
## 2. Canonical Fingerprint
|
||||
|
||||
```
|
||||
b49f64337ffafc0e73d46e009aa802cd0172d59421b7df61d3eb198775c1986d
|
||||
```
|
||||
|
||||
Dieser Fingerprint ist deterministisch (H1==H2==H3), sensitiv auf alle semantischen
|
||||
Schema-Änderungen (type, nullable, default, PK, UNIQUE, FK, CHECK, index), und fail-closed.
|
||||
|
||||
## 3. DR-Reproducibility (ehrlich getrennt)
|
||||
|
||||
| Aspekt | Status |
|
||||
|--------|--------|
|
||||
| **CURRENT-STATE SCHEMA REPRODUCIBLE FROM L0** | **YES** — frische DB aus L0-SQL rekonstruiert, REBUILT_HASH == PRODUCTION_HASH == b49f6433…, Semantic-Diff TOTAL_MISMATCH=0 |
|
||||
| **HISTORICAL EVOLUTION REPRODUCIBLE** | **NO** — historische DDL-Chronologie (Wave1→WaveD→Closure) bleibt nicht aus L0 rekonstruierbar |
|
||||
|
||||
Der L0-Contract beansprucht ausschliesslich **CURRENT PRODUCTION TRUTH**, nicht historische
|
||||
Migrationschronologie. Historische Artefakte bleiben immutable.
|
||||
|
||||
## 4. Canonical Contract Semantics
|
||||
|
||||
| Tabelle | Spalten |
|
||||
|---------|---------|
|
||||
| historical_bar | 41 |
|
||||
| derived_bar | 35 |
|
||||
| data_ingestion_run | 29 |
|
||||
| dataset_version | 23 |
|
||||
| ingestion_provenance | 41 |
|
||||
|
||||
**Semantic Overrides (Owner-Canonical):**
|
||||
- `last_successful_chunk` = **timestamp with time zone / timestamptz** (Production + Owner-Entscheidung)
|
||||
- `interrupt_reason` = **text** · PRESERVED · origin unknown · do not remove
|
||||
- `technical_quality` = **text** · nullable · default `'VALID'`
|
||||
|
||||
## 5. Historische Wahrheit (nicht rückwirkend ändern)
|
||||
|
||||
- **WaveD historisches Artefakt:** `last_successful_chunk = text` (immutable Evidence)
|
||||
- **Owner/current canonical contract:** `last_successful_chunk = timestamptz`
|
||||
- Historische Artefakte: Wave1 `6983526da3ca51fd2ececffbef0ccbd66b048a3d11c228000011aaba3e1d6bc5`,
|
||||
WaveD `7aa4e2a10c98fa327e000a49f6a497efb5814c8808dba429dfebfc90c544a02b`,
|
||||
Closure `e56567f7cfae68da70c11ece6f576e974488f343` — **unverändert**.
|
||||
|
||||
## 6. Open Debts (nicht fälschlich geschlossen)
|
||||
|
||||
| Debt | Status |
|
||||
|------|--------|
|
||||
| **Debt A** — Current-state DR/schema reproducibility gap | **CLOSED** by D1/D2 |
|
||||
| **Debt A** — Historical DDL evolution provenance gap | **REMAINS OPEN** |
|
||||
| **Debt B** — interrupt_reason origin unknown | **OPEN** (non-blocking) |
|
||||
| **Debt C** — WaveD text vs current Owner-canonical timestamptz | **OPEN** (non-blocking) |
|
||||
|
||||
## 7. SoT-Layout
|
||||
|
||||
```
|
||||
d1d2-schema-contract/
|
||||
├── SCHEMA_AUTHORITY.md # dieses Dokument
|
||||
├── contract/
|
||||
│ ├── historical_v2_schema.sql # L0-Schema-Contract (schema-only)
|
||||
│ ├── historical_v2_schema.json # maschinenlesbares kanonisches Modell
|
||||
│ ├── historical_v2_schema.sha256 # deterministischer Fingerprint
|
||||
│ └── README.md # Contract-Dokumentation
|
||||
└── tools/
|
||||
├── canonical_schema.py # Kanonisierungs-Implementierung
|
||||
├── capture_schema.py # Capture-Parser (read-only)
|
||||
├── compare_schema.py # Semantic-Diff-CLI
|
||||
├── fingerprint_schema.py # Fingerprint-CLI
|
||||
└── gen_l0_sql.py # L0-SQL-Generator
|
||||
```
|
||||
|
||||
**Nicht persistiert:** Maker-Evidence, Checker-Evidence, temporäre DBs, /tmp-Dateien,
|
||||
raw Scratch Outputs, runtime-spezifische Dateien, Credentials/Secrets, Transport-Manifeste,
|
||||
`gen_evidence.py` (enthält hardcodierte Maker-Pfade, nicht environment-neutral).
|
||||
|
||||
## 8. Tooling-Nutzung
|
||||
|
||||
```bash
|
||||
# Fingerprint aus machine contract
|
||||
python3 tools/fingerprint_schema.py contract/historical_v2_schema.json
|
||||
# → b49f64337ffafc0e73d46e009aa802cd0172d59421b7df61d3eb198775c1986d
|
||||
|
||||
# Semantic-Diff (expected vs actual)
|
||||
python3 tools/compare_schema.py contract/historical_v2_schema.json <actual_model.json>
|
||||
# → TOTAL_MISMATCH=0 bei Übereinstimmung; Exit 1 bei Drift (fail-closed)
|
||||
|
||||
# L0-SQL generieren
|
||||
python3 tools/gen_l0_sql.py contract/historical_v2_schema.json --out /tmp/l0.sql
|
||||
```
|
||||
|
||||
Alle Tools sind environment-neutral (keine absoluten Pfade, keine DB-Credentials,
|
||||
keine mutierenden Production-SQL-Pfade). Read-only.
|
||||
|
|
@ -1,20 +0,0 @@
|
|||
# Historical V2 — Canonical Current Schema Contract (L0)
|
||||
|
||||
Diese Artefakte materialisieren den **CURRENT VERIFIED PRODUCTION SCHEMA CONTRACT**
|
||||
als kanonische L0-Autorität (D1/D2 Remediation, forward-only).
|
||||
|
||||
- `historical_v2_schema.sql` — schema-only PostgreSQL-Contract (kein DML/Secrets/volatile Artefakte)
|
||||
- `historical_v2_schema.json` — maschinenlesbares kanonisches Modell
|
||||
- `historical_v2_schema.sha256` — deterministischer SHA-256-Fingerprint
|
||||
- Kanonischer Fingerprint: `b49f64337ffafc0e73d46e009aa802cd0172d59421b7df61d3eb198775c1986d`
|
||||
|
||||
Dies ist KEINE historische Migrationskette. DO NOT INVENT HISTORY: der L0-Contract
|
||||
beansprucht ausschliesslich CURRENT PRODUCTION TRUTH (§1).
|
||||
Historische Artefakte (Wave1 `6983526da3ca51fd2ececffbef0ccbd66b048a3d11c228000011aaba3e1d6bc5`, WaveD `7aa4e2a10c98fa327e000a49f6a497efb5814c8808dba429dfebfc90c544a02b`) bleiben unverändert.
|
||||
|
||||
**Semantic Overrides (§4):**
|
||||
- last_successful_chunk = timestamp with time zone (Production+Owner; WaveD-historical text bleibt immutable Evidence)
|
||||
- interrupt_reason = text (preserved, origin unknown, do not remove)
|
||||
- technical_quality = text nullable default 'VALID' (historical provenance debt OPEN)
|
||||
|
||||
**SoT:** origin/main = `e56567f7cfae68da70c11ece6f576e974488f343`
|
||||
File diff suppressed because it is too large
Load diff
|
|
@ -1 +0,0 @@
|
|||
b49f64337ffafc0e73d46e009aa802cd0172d59421b7df61d3eb198775c1986d
|
||||
|
|
@ -1,601 +0,0 @@
|
|||
-- ============================================================
|
||||
-- CANONICAL CURRENT PRODUCTION SCHEMA CONTRACT (L0)
|
||||
-- db: historical · schema: public
|
||||
-- Auto-generated by gen_l0_sql.py from read-only capture
|
||||
-- Modus: CURRENT VERIFIED PRODUCTION TRUTH — NICHT historische Migration
|
||||
-- ============================================================
|
||||
|
||||
-- SEMANTIC OVERRIDES (Owner-Canonical)
|
||||
-- last_successful_chunk = timestamp with time zone (Production + Owner decision)
|
||||
-- interrupt_reason = text · PRESERVED CURRENT CONTRACT · origin unknown · do not remove
|
||||
-- technical_quality = text · nullable · default 'VALID' · historical provenance debt OPEN
|
||||
|
||||
CREATE SEQUENCE IF NOT EXISTS "calendar_event_event_id_seq";
|
||||
CREATE SEQUENCE IF NOT EXISTS "corporate_action_action_id_seq";
|
||||
CREATE SEQUENCE IF NOT EXISTS "daily_quality_report_report_id_seq";
|
||||
CREATE SEQUENCE IF NOT EXISTS "data_gap_gap_id_seq";
|
||||
CREATE SEQUENCE IF NOT EXISTS "data_ingestion_run_ingestion_id_seq";
|
||||
CREATE SEQUENCE IF NOT EXISTS "data_quality_issue_issue_id_seq";
|
||||
CREATE SEQUENCE IF NOT EXISTS "dataset_dataset_id_seq";
|
||||
CREATE SEQUENCE IF NOT EXISTS "dataset_version_dataset_version_id_seq";
|
||||
CREATE SEQUENCE IF NOT EXISTS "derived_bar_derived_bar_id_seq";
|
||||
CREATE SEQUENCE IF NOT EXISTS "historical_bar_bar_id_seq";
|
||||
CREATE SEQUENCE IF NOT EXISTS "historical_tick_tick_id_seq";
|
||||
CREATE SEQUENCE IF NOT EXISTS "ingestion_provenance_provenance_id_seq";
|
||||
CREATE SEQUENCE IF NOT EXISTS "instrument_instrument_id_seq";
|
||||
CREATE SEQUENCE IF NOT EXISTS "market_calendar_calendar_id_seq";
|
||||
CREATE SEQUENCE IF NOT EXISTS "provider_provider_id_seq";
|
||||
|
||||
-- Table: calendar_event (9 cols)
|
||||
CREATE TABLE IF NOT EXISTS "calendar_event" (
|
||||
"event_id" bigint NOT NULL DEFAULT nextval('calendar_event_event_id_seq'::regclass),
|
||||
"calendar_id" bigint NOT NULL,
|
||||
"event_date" date NOT NULL,
|
||||
"kind" text NOT NULL,
|
||||
"reason" text,
|
||||
"session_phase_override" text,
|
||||
"source" text,
|
||||
"notes" text,
|
||||
"created_at" timestamp with time zone NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
-- Table: corporate_action (9 cols)
|
||||
CREATE TABLE IF NOT EXISTS "corporate_action" (
|
||||
"action_id" bigint NOT NULL DEFAULT nextval('corporate_action_action_id_seq'::regclass),
|
||||
"instrument_id" bigint NOT NULL,
|
||||
"action_type" text NOT NULL,
|
||||
"ex_date" date NOT NULL,
|
||||
"ratio_num" double precision,
|
||||
"ratio_denom" double precision,
|
||||
"dividend" double precision,
|
||||
"currency" text,
|
||||
"provider_id" bigint
|
||||
);
|
||||
|
||||
-- Table: daily_quality_report (23 cols)
|
||||
CREATE TABLE IF NOT EXISTS "daily_quality_report" (
|
||||
"report_id" bigint NOT NULL DEFAULT nextval('daily_quality_report_report_id_seq'::regclass),
|
||||
"instrument_id" bigint NOT NULL,
|
||||
"provider_id" bigint NOT NULL,
|
||||
"report_date" date NOT NULL,
|
||||
"feed_type" text DEFAULT 'REFERENCE'::text,
|
||||
"expected_m1" bigint,
|
||||
"received_m1" bigint,
|
||||
"eligible" bigint,
|
||||
"excluded" bigint,
|
||||
"conditional" bigint,
|
||||
"stale_count" bigint,
|
||||
"zero_volume" bigint,
|
||||
"spread_median_price" double precision,
|
||||
"spread_p95_price" double precision,
|
||||
"spread_p99_price" double precision,
|
||||
"gaps_count" bigint,
|
||||
"coverage_ratio" double precision,
|
||||
"session_anomalies" bigint,
|
||||
"status" text NOT NULL,
|
||||
"quality_model_version" text,
|
||||
"session_model_version" text,
|
||||
"calendar_version" text,
|
||||
"generated_at" timestamp with time zone NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
-- Table: data_gap (9 cols)
|
||||
CREATE TABLE IF NOT EXISTS "data_gap" (
|
||||
"gap_id" bigint NOT NULL DEFAULT nextval('data_gap_gap_id_seq'::regclass),
|
||||
"provider_id" bigint NOT NULL,
|
||||
"instrument_id" bigint NOT NULL,
|
||||
"timeframe" text NOT NULL,
|
||||
"start_utc" timestamp with time zone NOT NULL,
|
||||
"end_utc" timestamp with time zone NOT NULL,
|
||||
"kind" text NOT NULL,
|
||||
"backfilled" boolean NOT NULL DEFAULT false,
|
||||
"filled_at" timestamp with time zone
|
||||
);
|
||||
|
||||
-- Table: data_ingestion_run (29 cols)
|
||||
CREATE TABLE IF NOT EXISTS "data_ingestion_run" (
|
||||
"ingestion_id" bigint NOT NULL DEFAULT nextval('data_ingestion_run_ingestion_id_seq'::regclass),
|
||||
"provider_id" bigint NOT NULL,
|
||||
"instrument_id" bigint NOT NULL,
|
||||
"timeframe" text NOT NULL,
|
||||
"start_utc" timestamp with time zone,
|
||||
"end_utc" timestamp with time zone,
|
||||
"rows_inserted" bigint DEFAULT 0,
|
||||
"rows_skipped" bigint DEFAULT 0,
|
||||
"status" text NOT NULL DEFAULT 'RUNNING'::text,
|
||||
"started_at" timestamp with time zone NOT NULL DEFAULT now(),
|
||||
"finished_at" timestamp with time zone,
|
||||
"checkpoint" jsonb,
|
||||
"status_fsm" text DEFAULT 'PENDING'::text,
|
||||
"rows_downloaded" bigint DEFAULT 0,
|
||||
"rows_decoded" bigint DEFAULT 0,
|
||||
"rows_valid" bigint DEFAULT 0,
|
||||
"rows_suspect" bigint DEFAULT 0,
|
||||
"rows_excluded" bigint DEFAULT 0,
|
||||
"rows_derived" bigint DEFAULT 0,
|
||||
"last_successful_chunk" timestamp with time zone,
|
||||
"last_timestamp" timestamp with time zone,
|
||||
"updated_at" timestamp with time zone DEFAULT now(),
|
||||
"interrupt_reason" text,
|
||||
"rows_existing" bigint DEFAULT 0,
|
||||
"rows_deduplicated" bigint DEFAULT 0,
|
||||
"rows_404" bigint DEFAULT 0,
|
||||
"rows_503" bigint DEFAULT 0,
|
||||
"rows_provider_unavailable" bigint DEFAULT 0,
|
||||
"rows_failed" bigint DEFAULT 0
|
||||
);
|
||||
|
||||
-- Table: data_quality_issue (9 cols)
|
||||
CREATE TABLE IF NOT EXISTS "data_quality_issue" (
|
||||
"issue_id" bigint NOT NULL DEFAULT nextval('data_quality_issue_issue_id_seq'::regclass),
|
||||
"instrument_id" bigint NOT NULL,
|
||||
"timeframe" text NOT NULL,
|
||||
"ts_utc" timestamp with time zone,
|
||||
"severity" text NOT NULL,
|
||||
"issue_type" text NOT NULL,
|
||||
"detail" jsonb,
|
||||
"resolved" boolean NOT NULL DEFAULT false,
|
||||
"created_at" timestamp with time zone NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
-- Table: dataset (6 cols)
|
||||
CREATE TABLE IF NOT EXISTS "dataset" (
|
||||
"dataset_id" bigint NOT NULL DEFAULT nextval('dataset_dataset_id_seq'::regclass),
|
||||
"name" text NOT NULL,
|
||||
"provider_id" bigint NOT NULL,
|
||||
"feed_type" text NOT NULL,
|
||||
"description" text,
|
||||
"created_at" timestamp with time zone NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
-- Table: dataset_version (23 cols)
|
||||
CREATE TABLE IF NOT EXISTS "dataset_version" (
|
||||
"dataset_version_id" bigint NOT NULL DEFAULT nextval('dataset_version_dataset_version_id_seq'::regclass),
|
||||
"dataset_id" bigint NOT NULL,
|
||||
"version_label" text NOT NULL,
|
||||
"provider_id" bigint NOT NULL,
|
||||
"feed_type" text NOT NULL,
|
||||
"instrument_id" bigint NOT NULL,
|
||||
"timeframe" text NOT NULL,
|
||||
"start_utc" timestamp with time zone NOT NULL,
|
||||
"end_utc" timestamp with time zone NOT NULL,
|
||||
"raw_source" text NOT NULL,
|
||||
"normalization_version" text NOT NULL,
|
||||
"aggregation_version" text,
|
||||
"quality_version" text NOT NULL,
|
||||
"dataset_version_hash" text NOT NULL,
|
||||
"data_as_of" timestamp with time zone NOT NULL,
|
||||
"notes" text,
|
||||
"created_at" timestamp with time zone NOT NULL DEFAULT now(),
|
||||
"quality_model_version" text,
|
||||
"stale_model_version" text,
|
||||
"session_model_version" text,
|
||||
"calendar_version" text,
|
||||
"eligibility_model_version" text,
|
||||
"is_fixture" boolean NOT NULL DEFAULT false
|
||||
);
|
||||
|
||||
-- Table: derived_bar (35 cols)
|
||||
CREATE TABLE IF NOT EXISTS "derived_bar" (
|
||||
"derived_bar_id" bigint NOT NULL DEFAULT nextval('derived_bar_derived_bar_id_seq'::regclass),
|
||||
"provider_id" bigint NOT NULL,
|
||||
"instrument_id" bigint NOT NULL,
|
||||
"feed_type" text NOT NULL DEFAULT 'REFERENCE'::text,
|
||||
"price_basis" text NOT NULL DEFAULT 'bid_ask'::text,
|
||||
"timeframe" text NOT NULL,
|
||||
"ts_utc" timestamp with time zone NOT NULL,
|
||||
"source_timeframe" text NOT NULL,
|
||||
"aggregation_version" text NOT NULL,
|
||||
"bid_open" double precision,
|
||||
"bid_high" double precision,
|
||||
"bid_low" double precision,
|
||||
"bid_close" double precision,
|
||||
"ask_open" double precision,
|
||||
"ask_high" double precision,
|
||||
"ask_low" double precision,
|
||||
"ask_close" double precision,
|
||||
"mid_open" double precision,
|
||||
"mid_high" double precision,
|
||||
"mid_low" double precision,
|
||||
"mid_close" double precision,
|
||||
"volume" double precision,
|
||||
"volume_type" text DEFAULT 'tick'::text,
|
||||
"quality_status" text NOT NULL DEFAULT 'VALID'::text,
|
||||
"quality_flags" _text,
|
||||
"dataset_version_id" bigint NOT NULL,
|
||||
"expected_bar_count" bigint,
|
||||
"actual_bar_count" bigint,
|
||||
"eligible_bar_count" bigint,
|
||||
"excluded_bar_count" bigint,
|
||||
"coverage_ratio" double precision,
|
||||
"aggregation_quality" text,
|
||||
"eligibility_model_version" text DEFAULT 'eligibility_v1'::text,
|
||||
"session_model_version" text DEFAULT 'fx_session_v1'::text,
|
||||
"calendar_version" text DEFAULT 'fx_cal_v1'::text
|
||||
);
|
||||
|
||||
-- Table: historical_bar (41 cols)
|
||||
CREATE TABLE IF NOT EXISTS "historical_bar" (
|
||||
"bar_id" bigint NOT NULL DEFAULT nextval('historical_bar_bar_id_seq'::regclass),
|
||||
"provider_id" bigint NOT NULL,
|
||||
"instrument_id" bigint NOT NULL,
|
||||
"feed_type" text NOT NULL DEFAULT 'REFERENCE'::text,
|
||||
"price_basis" text NOT NULL DEFAULT 'bid_ask'::text,
|
||||
"timeframe" text NOT NULL,
|
||||
"ts_utc" timestamp with time zone NOT NULL,
|
||||
"raw_or_derived" text NOT NULL DEFAULT 'raw'::text,
|
||||
"source_timeframe" text,
|
||||
"aggregation_version" text,
|
||||
"bid_open" double precision,
|
||||
"bid_high" double precision,
|
||||
"bid_low" double precision,
|
||||
"bid_close" double precision,
|
||||
"ask_open" double precision,
|
||||
"ask_high" double precision,
|
||||
"ask_low" double precision,
|
||||
"ask_close" double precision,
|
||||
"mid_open" double precision,
|
||||
"mid_high" double precision,
|
||||
"mid_low" double precision,
|
||||
"mid_close" double precision,
|
||||
"volume" double precision,
|
||||
"volume_type" text DEFAULT 'tick'::text,
|
||||
"quality_status" text NOT NULL DEFAULT 'VALID'::text,
|
||||
"quality_flags" _text,
|
||||
"source_timestamp" timestamp with time zone,
|
||||
"ingested_at" timestamp with time zone NOT NULL DEFAULT now(),
|
||||
"dataset_version_id" bigint NOT NULL,
|
||||
"technical_quality" text DEFAULT 'VALID'::text,
|
||||
"market_quality" text,
|
||||
"session_state" text,
|
||||
"session_phase" text,
|
||||
"eligibility" text DEFAULT 'ELIGIBLE'::text,
|
||||
"exclusion_reason" text,
|
||||
"quality_model_version" text DEFAULT 'quality_v1'::text,
|
||||
"session_model_version" text,
|
||||
"calendar_version" text,
|
||||
"eligibility_model_version" text DEFAULT 'eligibility_v1'::text,
|
||||
"effective_eligibility" text,
|
||||
"stale_model_version" text
|
||||
);
|
||||
|
||||
-- Table: historical_tick (10 cols)
|
||||
CREATE TABLE IF NOT EXISTS "historical_tick" (
|
||||
"tick_id" bigint NOT NULL DEFAULT nextval('historical_tick_tick_id_seq'::regclass),
|
||||
"provider_id" bigint NOT NULL,
|
||||
"instrument_id" bigint NOT NULL,
|
||||
"feed_type" text NOT NULL DEFAULT 'REFERENCE'::text,
|
||||
"ts_utc" timestamp with time zone NOT NULL,
|
||||
"bid" double precision,
|
||||
"ask" double precision,
|
||||
"last" double precision,
|
||||
"volume" double precision,
|
||||
"dataset_version_id" bigint NOT NULL
|
||||
);
|
||||
|
||||
-- Table: ingestion_provenance (41 cols)
|
||||
CREATE TABLE IF NOT EXISTS "ingestion_provenance" (
|
||||
"provenance_id" bigint NOT NULL,
|
||||
"ingestion_run_id" bigint NOT NULL,
|
||||
"provider_id" bigint NOT NULL,
|
||||
"provider_name_snapshot" text NOT NULL,
|
||||
"provider_environment_class" text NOT NULL,
|
||||
"provider_dataset" text NOT NULL,
|
||||
"provider_feed" text NOT NULL,
|
||||
"provider_instrument_id" text NOT NULL,
|
||||
"canonical_instrument_id" bigint NOT NULL,
|
||||
"asset_class" text NOT NULL,
|
||||
"request_start" timestamp with time zone NOT NULL,
|
||||
"request_end" timestamp with time zone NOT NULL,
|
||||
"actual_start" timestamp with time zone NOT NULL,
|
||||
"actual_end" timestamp with time zone NOT NULL,
|
||||
"request_timestamp" timestamp with time zone NOT NULL,
|
||||
"response_timestamp" timestamp with time zone NOT NULL,
|
||||
"http_status" integer NOT NULL,
|
||||
"provider_metadata" jsonb,
|
||||
"provider_version" text,
|
||||
"raw_payload_sha256" text NOT NULL,
|
||||
"raw_content_length" bigint NOT NULL,
|
||||
"compression" text NOT NULL,
|
||||
"content_type" text,
|
||||
"parser_version" text NOT NULL,
|
||||
"adapter_version" text NOT NULL,
|
||||
"normalization_version" text NOT NULL,
|
||||
"timezone_source" text NOT NULL,
|
||||
"timezone_target" text NOT NULL,
|
||||
"timezone_transform_version" text NOT NULL,
|
||||
"price_type" text NOT NULL,
|
||||
"granularity" text NOT NULL,
|
||||
"correction_status" text NOT NULL DEFAULT 'ORIGINAL'::text,
|
||||
"license_class" text NOT NULL DEFAULT 'UNKNOWN'::text,
|
||||
"software_git_commit" text NOT NULL,
|
||||
"software_image_id" text NOT NULL,
|
||||
"software_build_input_sha" text,
|
||||
"request_identity_hash" text NOT NULL,
|
||||
"supersedes_provenance_id" bigint,
|
||||
"raw_storage_ref" text,
|
||||
"extra" jsonb,
|
||||
"created_at" timestamp with time zone NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
-- Table: instrument (13 cols)
|
||||
CREATE TABLE IF NOT EXISTS "instrument" (
|
||||
"instrument_id" bigint NOT NULL DEFAULT nextval('instrument_instrument_id_seq'::regclass),
|
||||
"symbol" text NOT NULL,
|
||||
"asset_class" text NOT NULL,
|
||||
"name" text,
|
||||
"currency" text,
|
||||
"exchange" text,
|
||||
"mic" text,
|
||||
"isin" text,
|
||||
"tick_size" double precision,
|
||||
"contract_size" double precision,
|
||||
"min_deal_size" double precision,
|
||||
"active" boolean NOT NULL DEFAULT true,
|
||||
"created_at" timestamp with time zone NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
-- Table: market_calendar (13 cols)
|
||||
CREATE TABLE IF NOT EXISTS "market_calendar" (
|
||||
"calendar_id" bigint NOT NULL DEFAULT nextval('market_calendar_calendar_id_seq'::regclass),
|
||||
"calendar_name" text NOT NULL,
|
||||
"asset_class" text NOT NULL,
|
||||
"venue_exchange" text,
|
||||
"timezone" text NOT NULL,
|
||||
"week_open_utc" text,
|
||||
"week_close_utc" text,
|
||||
"rollover_utc" text,
|
||||
"session_open_utc" text,
|
||||
"session_close_utc" text,
|
||||
"notes" text,
|
||||
"calendar_version" text NOT NULL,
|
||||
"created_at" timestamp with time zone NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
-- Table: provider (6 cols)
|
||||
CREATE TABLE IF NOT EXISTS "provider" (
|
||||
"provider_id" bigint NOT NULL DEFAULT nextval('provider_provider_id_seq'::regclass),
|
||||
"provider_key" text NOT NULL,
|
||||
"name" text NOT NULL,
|
||||
"data_type" text NOT NULL,
|
||||
"pricing_plan" text,
|
||||
"license_terms" text
|
||||
);
|
||||
|
||||
-- Table: provider_instrument (6 cols)
|
||||
CREATE TABLE IF NOT EXISTS "provider_instrument" (
|
||||
"provider_id" bigint NOT NULL,
|
||||
"instrument_id" bigint NOT NULL,
|
||||
"provider_symbol" text NOT NULL,
|
||||
"broker_epic" text,
|
||||
"extra_metadata" jsonb,
|
||||
"active" boolean NOT NULL DEFAULT true
|
||||
);
|
||||
|
||||
ALTER TABLE "calendar_event" ADD CONSTRAINT "calendar_event_pkey" PRIMARY KEY ("event_id");
|
||||
ALTER TABLE "corporate_action" ADD CONSTRAINT "corporate_action_pkey" PRIMARY KEY ("action_id");
|
||||
ALTER TABLE "daily_quality_report" ADD CONSTRAINT "daily_quality_report_pkey" PRIMARY KEY ("report_id");
|
||||
ALTER TABLE "data_gap" ADD CONSTRAINT "data_gap_pkey" PRIMARY KEY ("gap_id");
|
||||
ALTER TABLE "data_ingestion_run" ADD CONSTRAINT "data_ingestion_run_pkey" PRIMARY KEY ("ingestion_id");
|
||||
ALTER TABLE "data_quality_issue" ADD CONSTRAINT "data_quality_issue_pkey" PRIMARY KEY ("issue_id");
|
||||
ALTER TABLE "dataset" ADD CONSTRAINT "dataset_pkey" PRIMARY KEY ("dataset_id");
|
||||
ALTER TABLE "dataset_version" ADD CONSTRAINT "dataset_version_pkey" PRIMARY KEY ("dataset_version_id");
|
||||
ALTER TABLE "derived_bar" ADD CONSTRAINT "derived_bar_pkey" PRIMARY KEY ("derived_bar_id");
|
||||
ALTER TABLE "historical_bar" ADD CONSTRAINT "historical_bar_pkey" PRIMARY KEY ("bar_id");
|
||||
ALTER TABLE "historical_tick" ADD CONSTRAINT "historical_tick_pkey" PRIMARY KEY ("tick_id");
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "ingestion_provenance_pkey" PRIMARY KEY ("provenance_id");
|
||||
ALTER TABLE "instrument" ADD CONSTRAINT "instrument_pkey" PRIMARY KEY ("instrument_id");
|
||||
ALTER TABLE "market_calendar" ADD CONSTRAINT "market_calendar_pkey" PRIMARY KEY ("calendar_id");
|
||||
ALTER TABLE "provider" ADD CONSTRAINT "provider_pkey" PRIMARY KEY ("provider_id");
|
||||
ALTER TABLE "provider_instrument" ADD CONSTRAINT "provider_instrument_pkey" PRIMARY KEY ("provider_id", "instrument_id");
|
||||
|
||||
ALTER TABLE "calendar_event" ADD CONSTRAINT "calendar_event_calendar_id_event_date_kind_key" UNIQUE ("calendar_id", "event_date", "kind");
|
||||
ALTER TABLE "corporate_action" ADD CONSTRAINT "corporate_action_instrument_id_action_type_ex_date_provider_key" UNIQUE ("instrument_id", "action_type", "ex_date", "provider_id");
|
||||
ALTER TABLE "daily_quality_report" ADD CONSTRAINT "daily_quality_report_instrument_id_provider_id_report_date__key" UNIQUE ("instrument_id", "provider_id", "report_date", "feed_type");
|
||||
ALTER TABLE "data_gap" ADD CONSTRAINT "data_gap_unique" UNIQUE ("provider_id", "instrument_id", "timeframe", "start_utc", "end_utc", "kind");
|
||||
ALTER TABLE "data_ingestion_run" ADD CONSTRAINT "data_ingestion_run_provider_id_instrument_id_timeframe_star_key" UNIQUE ("provider_id", "instrument_id", "timeframe", "started_at");
|
||||
ALTER TABLE "dataset" ADD CONSTRAINT "dataset_name_key" UNIQUE ("name");
|
||||
ALTER TABLE "dataset_version" ADD CONSTRAINT "dataset_version_dataset_version_hash_key" UNIQUE ("dataset_version_hash");
|
||||
ALTER TABLE "derived_bar" ADD CONSTRAINT "derived_bar_provider_id_instrument_id_feed_type_price_basis_key" UNIQUE ("provider_id", "instrument_id", "feed_type", "price_basis", "timeframe", "ts_utc", "aggregation_version");
|
||||
ALTER TABLE "historical_bar" ADD CONSTRAINT "historical_bar_provider_id_instrument_id_feed_type_price_ba_key" UNIQUE ("provider_id", "instrument_id", "feed_type", "price_basis", "timeframe", "ts_utc", "raw_or_derived");
|
||||
ALTER TABLE "historical_tick" ADD CONSTRAINT "historical_tick_provider_id_instrument_id_feed_type_ts_utc__key" UNIQUE ("provider_id", "instrument_id", "feed_type", "ts_utc", "bid", "ask");
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "ingestion_provenance_ingestion_run_id_key" UNIQUE ("ingestion_run_id");
|
||||
ALTER TABLE "instrument" ADD CONSTRAINT "instrument_symbol_key" UNIQUE ("symbol");
|
||||
ALTER TABLE "market_calendar" ADD CONSTRAINT "market_calendar_calendar_name_key" UNIQUE ("calendar_name");
|
||||
ALTER TABLE "provider" ADD CONSTRAINT "provider_provider_key_key" UNIQUE ("provider_key");
|
||||
ALTER TABLE "provider_instrument" ADD CONSTRAINT "provider_instrument_provider_id_provider_symbol_key" UNIQUE ("provider_id", "provider_symbol");
|
||||
|
||||
ALTER TABLE "calendar_event" ADD CONSTRAINT "calendar_event_calendar_id_fkey" FOREIGN KEY ("calendar_id") REFERENCES "market_calendar" ("calendar_id");
|
||||
ALTER TABLE "corporate_action" ADD CONSTRAINT "corporate_action_instrument_id_fkey" FOREIGN KEY ("instrument_id") REFERENCES "instrument" ("instrument_id");
|
||||
ALTER TABLE "corporate_action" ADD CONSTRAINT "corporate_action_provider_id_fkey" FOREIGN KEY ("provider_id") REFERENCES "provider" ("provider_id");
|
||||
ALTER TABLE "daily_quality_report" ADD CONSTRAINT "daily_quality_report_instrument_id_fkey" FOREIGN KEY ("instrument_id") REFERENCES "instrument" ("instrument_id");
|
||||
ALTER TABLE "daily_quality_report" ADD CONSTRAINT "daily_quality_report_provider_id_fkey" FOREIGN KEY ("provider_id") REFERENCES "provider" ("provider_id");
|
||||
ALTER TABLE "data_gap" ADD CONSTRAINT "data_gap_instrument_id_fkey" FOREIGN KEY ("instrument_id") REFERENCES "instrument" ("instrument_id");
|
||||
ALTER TABLE "data_gap" ADD CONSTRAINT "data_gap_provider_id_fkey" FOREIGN KEY ("provider_id") REFERENCES "provider" ("provider_id");
|
||||
ALTER TABLE "data_ingestion_run" ADD CONSTRAINT "data_ingestion_run_instrument_id_fkey" FOREIGN KEY ("instrument_id") REFERENCES "instrument" ("instrument_id");
|
||||
ALTER TABLE "data_ingestion_run" ADD CONSTRAINT "data_ingestion_run_provider_id_fkey" FOREIGN KEY ("provider_id") REFERENCES "provider" ("provider_id");
|
||||
ALTER TABLE "data_quality_issue" ADD CONSTRAINT "data_quality_issue_instrument_id_fkey" FOREIGN KEY ("instrument_id") REFERENCES "instrument" ("instrument_id");
|
||||
ALTER TABLE "dataset" ADD CONSTRAINT "dataset_provider_id_fkey" FOREIGN KEY ("provider_id") REFERENCES "provider" ("provider_id");
|
||||
ALTER TABLE "dataset_version" ADD CONSTRAINT "dataset_version_dataset_id_fkey" FOREIGN KEY ("dataset_id") REFERENCES "dataset" ("dataset_id");
|
||||
ALTER TABLE "dataset_version" ADD CONSTRAINT "dataset_version_instrument_id_fkey" FOREIGN KEY ("instrument_id") REFERENCES "instrument" ("instrument_id");
|
||||
ALTER TABLE "dataset_version" ADD CONSTRAINT "dataset_version_provider_id_fkey" FOREIGN KEY ("provider_id") REFERENCES "provider" ("provider_id");
|
||||
ALTER TABLE "derived_bar" ADD CONSTRAINT "derived_bar_dataset_version_id_fkey" FOREIGN KEY ("dataset_version_id") REFERENCES "dataset_version" ("dataset_version_id");
|
||||
ALTER TABLE "derived_bar" ADD CONSTRAINT "derived_bar_instrument_id_fkey" FOREIGN KEY ("instrument_id") REFERENCES "instrument" ("instrument_id");
|
||||
ALTER TABLE "derived_bar" ADD CONSTRAINT "derived_bar_provider_id_fkey" FOREIGN KEY ("provider_id") REFERENCES "provider" ("provider_id");
|
||||
ALTER TABLE "historical_bar" ADD CONSTRAINT "historical_bar_dataset_version_id_fkey" FOREIGN KEY ("dataset_version_id") REFERENCES "dataset_version" ("dataset_version_id");
|
||||
ALTER TABLE "historical_bar" ADD CONSTRAINT "historical_bar_instrument_id_fkey" FOREIGN KEY ("instrument_id") REFERENCES "instrument" ("instrument_id");
|
||||
ALTER TABLE "historical_bar" ADD CONSTRAINT "historical_bar_provider_id_fkey" FOREIGN KEY ("provider_id") REFERENCES "provider" ("provider_id");
|
||||
ALTER TABLE "historical_tick" ADD CONSTRAINT "historical_tick_dataset_version_id_fkey" FOREIGN KEY ("dataset_version_id") REFERENCES "dataset_version" ("dataset_version_id");
|
||||
ALTER TABLE "historical_tick" ADD CONSTRAINT "historical_tick_instrument_id_fkey" FOREIGN KEY ("instrument_id") REFERENCES "instrument" ("instrument_id");
|
||||
ALTER TABLE "historical_tick" ADD CONSTRAINT "historical_tick_provider_id_fkey" FOREIGN KEY ("provider_id") REFERENCES "provider" ("provider_id");
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "ingestion_provenance_canonical_instrument_id_fkey" FOREIGN KEY ("canonical_instrument_id") REFERENCES "instrument" ("instrument_id");
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "ingestion_provenance_provider_id_fkey" FOREIGN KEY ("provider_id") REFERENCES "provider" ("provider_id");
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "ingestion_provenance_supersedes_fkey" FOREIGN KEY ("supersedes_provenance_id") REFERENCES "ingestion_provenance" ("provenance_id");
|
||||
ALTER TABLE "provider_instrument" ADD CONSTRAINT "provider_instrument_instrument_id_fkey" FOREIGN KEY ("instrument_id") REFERENCES "instrument" ("instrument_id");
|
||||
ALTER TABLE "provider_instrument" ADD CONSTRAINT "provider_instrument_provider_id_fkey" FOREIGN KEY ("provider_id") REFERENCES "provider" ("provider_id");
|
||||
|
||||
ALTER TABLE "calendar_event" ADD CONSTRAINT "2200_16679_1_not_null" CHECK (event_id IS NOT NULL);
|
||||
ALTER TABLE "calendar_event" ADD CONSTRAINT "2200_16679_2_not_null" CHECK (calendar_id IS NOT NULL);
|
||||
ALTER TABLE "calendar_event" ADD CONSTRAINT "2200_16679_3_not_null" CHECK (event_date IS NOT NULL);
|
||||
ALTER TABLE "calendar_event" ADD CONSTRAINT "2200_16679_4_not_null" CHECK (kind IS NOT NULL);
|
||||
ALTER TABLE "calendar_event" ADD CONSTRAINT "2200_16679_9_not_null" CHECK (created_at IS NOT NULL);
|
||||
ALTER TABLE "corporate_action" ADD CONSTRAINT "2200_16567_1_not_null" CHECK (action_id IS NOT NULL);
|
||||
ALTER TABLE "corporate_action" ADD CONSTRAINT "2200_16567_2_not_null" CHECK (instrument_id IS NOT NULL);
|
||||
ALTER TABLE "corporate_action" ADD CONSTRAINT "2200_16567_3_not_null" CHECK (action_type IS NOT NULL);
|
||||
ALTER TABLE "corporate_action" ADD CONSTRAINT "2200_16567_4_not_null" CHECK (ex_date IS NOT NULL);
|
||||
ALTER TABLE "daily_quality_report" ADD CONSTRAINT "2200_16696_19_not_null" CHECK (status IS NOT NULL);
|
||||
ALTER TABLE "daily_quality_report" ADD CONSTRAINT "2200_16696_1_not_null" CHECK (report_id IS NOT NULL);
|
||||
ALTER TABLE "daily_quality_report" ADD CONSTRAINT "2200_16696_23_not_null" CHECK (generated_at IS NOT NULL);
|
||||
ALTER TABLE "daily_quality_report" ADD CONSTRAINT "2200_16696_2_not_null" CHECK (instrument_id IS NOT NULL);
|
||||
ALTER TABLE "daily_quality_report" ADD CONSTRAINT "2200_16696_3_not_null" CHECK (provider_id IS NOT NULL);
|
||||
ALTER TABLE "daily_quality_report" ADD CONSTRAINT "2200_16696_4_not_null" CHECK (report_date IS NOT NULL);
|
||||
ALTER TABLE "data_gap" ADD CONSTRAINT "2200_16613_1_not_null" CHECK (gap_id IS NOT NULL);
|
||||
ALTER TABLE "data_gap" ADD CONSTRAINT "2200_16613_2_not_null" CHECK (provider_id IS NOT NULL);
|
||||
ALTER TABLE "data_gap" ADD CONSTRAINT "2200_16613_3_not_null" CHECK (instrument_id IS NOT NULL);
|
||||
ALTER TABLE "data_gap" ADD CONSTRAINT "2200_16613_4_not_null" CHECK (timeframe IS NOT NULL);
|
||||
ALTER TABLE "data_gap" ADD CONSTRAINT "2200_16613_5_not_null" CHECK (start_utc IS NOT NULL);
|
||||
ALTER TABLE "data_gap" ADD CONSTRAINT "2200_16613_6_not_null" CHECK (end_utc IS NOT NULL);
|
||||
ALTER TABLE "data_gap" ADD CONSTRAINT "2200_16613_7_not_null" CHECK (kind IS NOT NULL);
|
||||
ALTER TABLE "data_gap" ADD CONSTRAINT "2200_16613_8_not_null" CHECK (backfilled IS NOT NULL);
|
||||
ALTER TABLE "data_ingestion_run" ADD CONSTRAINT "2200_16588_10_not_null" CHECK (started_at IS NOT NULL);
|
||||
ALTER TABLE "data_ingestion_run" ADD CONSTRAINT "2200_16588_1_not_null" CHECK (ingestion_id IS NOT NULL);
|
||||
ALTER TABLE "data_ingestion_run" ADD CONSTRAINT "2200_16588_2_not_null" CHECK (provider_id IS NOT NULL);
|
||||
ALTER TABLE "data_ingestion_run" ADD CONSTRAINT "2200_16588_3_not_null" CHECK (instrument_id IS NOT NULL);
|
||||
ALTER TABLE "data_ingestion_run" ADD CONSTRAINT "2200_16588_4_not_null" CHECK (timeframe IS NOT NULL);
|
||||
ALTER TABLE "data_ingestion_run" ADD CONSTRAINT "2200_16588_9_not_null" CHECK (status IS NOT NULL);
|
||||
ALTER TABLE "data_quality_issue" ADD CONSTRAINT "2200_16633_1_not_null" CHECK (issue_id IS NOT NULL);
|
||||
ALTER TABLE "data_quality_issue" ADD CONSTRAINT "2200_16633_2_not_null" CHECK (instrument_id IS NOT NULL);
|
||||
ALTER TABLE "data_quality_issue" ADD CONSTRAINT "2200_16633_3_not_null" CHECK (timeframe IS NOT NULL);
|
||||
ALTER TABLE "data_quality_issue" ADD CONSTRAINT "2200_16633_5_not_null" CHECK (severity IS NOT NULL);
|
||||
ALTER TABLE "data_quality_issue" ADD CONSTRAINT "2200_16633_6_not_null" CHECK (issue_type IS NOT NULL);
|
||||
ALTER TABLE "data_quality_issue" ADD CONSTRAINT "2200_16633_8_not_null" CHECK (resolved IS NOT NULL);
|
||||
ALTER TABLE "data_quality_issue" ADD CONSTRAINT "2200_16633_9_not_null" CHECK (created_at IS NOT NULL);
|
||||
ALTER TABLE "dataset" ADD CONSTRAINT "2200_16430_1_not_null" CHECK (dataset_id IS NOT NULL);
|
||||
ALTER TABLE "dataset" ADD CONSTRAINT "2200_16430_2_not_null" CHECK (name IS NOT NULL);
|
||||
ALTER TABLE "dataset" ADD CONSTRAINT "2200_16430_3_not_null" CHECK (provider_id IS NOT NULL);
|
||||
ALTER TABLE "dataset" ADD CONSTRAINT "2200_16430_4_not_null" CHECK (feed_type IS NOT NULL);
|
||||
ALTER TABLE "dataset" ADD CONSTRAINT "2200_16430_6_not_null" CHECK (created_at IS NOT NULL);
|
||||
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_10_not_null" CHECK (raw_source IS NOT NULL);
|
||||
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_11_not_null" CHECK (normalization_version IS NOT NULL);
|
||||
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_13_not_null" CHECK (quality_version IS NOT NULL);
|
||||
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_14_not_null" CHECK (dataset_version_hash IS NOT NULL);
|
||||
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_15_not_null" CHECK (data_as_of IS NOT NULL);
|
||||
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_17_not_null" CHECK (created_at IS NOT NULL);
|
||||
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_1_not_null" CHECK (dataset_version_id IS NOT NULL);
|
||||
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_23_not_null" CHECK (is_fixture IS NOT NULL);
|
||||
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_2_not_null" CHECK (dataset_id IS NOT NULL);
|
||||
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_3_not_null" CHECK (version_label IS NOT NULL);
|
||||
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_4_not_null" CHECK (provider_id IS NOT NULL);
|
||||
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_5_not_null" CHECK (feed_type IS NOT NULL);
|
||||
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_6_not_null" CHECK (instrument_id IS NOT NULL);
|
||||
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_7_not_null" CHECK (timeframe IS NOT NULL);
|
||||
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_8_not_null" CHECK (start_utc IS NOT NULL);
|
||||
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_9_not_null" CHECK (end_utc IS NOT NULL);
|
||||
ALTER TABLE "derived_bar" ADD CONSTRAINT "2200_16508_1_not_null" CHECK (derived_bar_id IS NOT NULL);
|
||||
ALTER TABLE "derived_bar" ADD CONSTRAINT "2200_16508_24_not_null" CHECK (quality_status IS NOT NULL);
|
||||
ALTER TABLE "derived_bar" ADD CONSTRAINT "2200_16508_26_not_null" CHECK (dataset_version_id IS NOT NULL);
|
||||
ALTER TABLE "derived_bar" ADD CONSTRAINT "2200_16508_2_not_null" CHECK (provider_id IS NOT NULL);
|
||||
ALTER TABLE "derived_bar" ADD CONSTRAINT "2200_16508_3_not_null" CHECK (instrument_id IS NOT NULL);
|
||||
ALTER TABLE "derived_bar" ADD CONSTRAINT "2200_16508_4_not_null" CHECK (feed_type IS NOT NULL);
|
||||
ALTER TABLE "derived_bar" ADD CONSTRAINT "2200_16508_5_not_null" CHECK (price_basis IS NOT NULL);
|
||||
ALTER TABLE "derived_bar" ADD CONSTRAINT "2200_16508_6_not_null" CHECK (timeframe IS NOT NULL);
|
||||
ALTER TABLE "derived_bar" ADD CONSTRAINT "2200_16508_7_not_null" CHECK (ts_utc IS NOT NULL);
|
||||
ALTER TABLE "derived_bar" ADD CONSTRAINT "2200_16508_8_not_null" CHECK (source_timeframe IS NOT NULL);
|
||||
ALTER TABLE "derived_bar" ADD CONSTRAINT "2200_16508_9_not_null" CHECK (aggregation_version IS NOT NULL);
|
||||
ALTER TABLE "historical_bar" ADD CONSTRAINT "2200_16474_1_not_null" CHECK (bar_id IS NOT NULL);
|
||||
ALTER TABLE "historical_bar" ADD CONSTRAINT "2200_16474_25_not_null" CHECK (quality_status IS NOT NULL);
|
||||
ALTER TABLE "historical_bar" ADD CONSTRAINT "2200_16474_28_not_null" CHECK (ingested_at IS NOT NULL);
|
||||
ALTER TABLE "historical_bar" ADD CONSTRAINT "2200_16474_29_not_null" CHECK (dataset_version_id IS NOT NULL);
|
||||
ALTER TABLE "historical_bar" ADD CONSTRAINT "2200_16474_2_not_null" CHECK (provider_id IS NOT NULL);
|
||||
ALTER TABLE "historical_bar" ADD CONSTRAINT "2200_16474_3_not_null" CHECK (instrument_id IS NOT NULL);
|
||||
ALTER TABLE "historical_bar" ADD CONSTRAINT "2200_16474_4_not_null" CHECK (feed_type IS NOT NULL);
|
||||
ALTER TABLE "historical_bar" ADD CONSTRAINT "2200_16474_5_not_null" CHECK (price_basis IS NOT NULL);
|
||||
ALTER TABLE "historical_bar" ADD CONSTRAINT "2200_16474_6_not_null" CHECK (timeframe IS NOT NULL);
|
||||
ALTER TABLE "historical_bar" ADD CONSTRAINT "2200_16474_7_not_null" CHECK (ts_utc IS NOT NULL);
|
||||
ALTER TABLE "historical_bar" ADD CONSTRAINT "2200_16474_8_not_null" CHECK (raw_or_derived IS NOT NULL);
|
||||
ALTER TABLE "historical_tick" ADD CONSTRAINT "2200_16539_10_not_null" CHECK (dataset_version_id IS NOT NULL);
|
||||
ALTER TABLE "historical_tick" ADD CONSTRAINT "2200_16539_1_not_null" CHECK (tick_id IS NOT NULL);
|
||||
ALTER TABLE "historical_tick" ADD CONSTRAINT "2200_16539_2_not_null" CHECK (provider_id IS NOT NULL);
|
||||
ALTER TABLE "historical_tick" ADD CONSTRAINT "2200_16539_3_not_null" CHECK (instrument_id IS NOT NULL);
|
||||
ALTER TABLE "historical_tick" ADD CONSTRAINT "2200_16539_4_not_null" CHECK (feed_type IS NOT NULL);
|
||||
ALTER TABLE "historical_tick" ADD CONSTRAINT "2200_16539_5_not_null" CHECK (ts_utc IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_10_not_null" CHECK (asset_class IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_11_not_null" CHECK (request_start IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_12_not_null" CHECK (request_end IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_13_not_null" CHECK (actual_start IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_14_not_null" CHECK (actual_end IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_15_not_null" CHECK (request_timestamp IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_16_not_null" CHECK (response_timestamp IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_17_not_null" CHECK (http_status IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_1_not_null" CHECK (provenance_id IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_20_not_null" CHECK (raw_payload_sha256 IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_21_not_null" CHECK (raw_content_length IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_22_not_null" CHECK (compression IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_24_not_null" CHECK (parser_version IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_25_not_null" CHECK (adapter_version IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_26_not_null" CHECK (normalization_version IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_27_not_null" CHECK (timezone_source IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_28_not_null" CHECK (timezone_target IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_29_not_null" CHECK (timezone_transform_version IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_2_not_null" CHECK (ingestion_run_id IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_30_not_null" CHECK (price_type IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_31_not_null" CHECK (granularity IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_32_not_null" CHECK (correction_status IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_33_not_null" CHECK (license_class IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_34_not_null" CHECK (software_git_commit IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_35_not_null" CHECK (software_image_id IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_37_not_null" CHECK (request_identity_hash IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_3_not_null" CHECK (provider_id IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_41_not_null" CHECK (created_at IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_4_not_null" CHECK (provider_name_snapshot IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_5_not_null" CHECK (provider_environment_class IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_6_not_null" CHECK (provider_dataset IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_7_not_null" CHECK (provider_feed IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_8_not_null" CHECK (provider_instrument_id IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_9_not_null" CHECK (canonical_instrument_id IS NOT NULL);
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "chk_prov_actual_range" CHECK (((actual_end >= actual_start)));
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "chk_prov_compression" CHECK (((compression = ANY (ARRAY['none'::text, 'lzma'::text, 'gzip'::text, 'zstd'::text, 'deflate'::text, 'unknown'::text]))));
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "chk_prov_content_length" CHECK (((raw_content_length >= 0)));
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "chk_prov_correction" CHECK (((correction_status = ANY (ARRAY['ORIGINAL'::text, 'CORRECTED'::text, 'SUPERSEDED'::text, 'UNKNOWN'::text]))));
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "chk_prov_env_class" CHECK (((provider_environment_class = ANY (ARRAY['PROD'::text, 'SANDBOX'::text, 'TEST'::text, 'UNKNOWN'::text]))));
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "chk_prov_http_status" CHECK ((((http_status >= 100) AND (http_status <= 599))));
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "chk_prov_license" CHECK (((license_class = ANY (ARRAY['UNKNOWN'::text, 'PRIVATE_INTERNAL'::text, 'INTERNAL_DERIVED_ALLOWED'::text, 'STORAGE_RESTRICTED'::text, 'REDISTRIBUTION_RESTRICTED'::text, 'EXPIRED'::text, 'REVOKED'::text]))));
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "chk_prov_price_type" CHECK (((price_type = ANY (ARRAY['bid'::text, 'ask'::text, 'mid'::text, 'bid_ask'::text, 'last'::text, 'unknown'::text]))));
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "chk_prov_sha256" CHECK (((raw_payload_sha256 ~ '^[0-9a-f]{64}$'::text)));
|
||||
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "chk_prov_time_range" CHECK (((request_end >= request_start)));
|
||||
ALTER TABLE "instrument" ADD CONSTRAINT "2200_16386_12_not_null" CHECK (active IS NOT NULL);
|
||||
ALTER TABLE "instrument" ADD CONSTRAINT "2200_16386_13_not_null" CHECK (created_at IS NOT NULL);
|
||||
ALTER TABLE "instrument" ADD CONSTRAINT "2200_16386_1_not_null" CHECK (instrument_id IS NOT NULL);
|
||||
ALTER TABLE "instrument" ADD CONSTRAINT "2200_16386_2_not_null" CHECK (symbol IS NOT NULL);
|
||||
ALTER TABLE "instrument" ADD CONSTRAINT "2200_16386_3_not_null" CHECK (asset_class IS NOT NULL);
|
||||
ALTER TABLE "market_calendar" ADD CONSTRAINT "2200_16667_12_not_null" CHECK (calendar_version IS NOT NULL);
|
||||
ALTER TABLE "market_calendar" ADD CONSTRAINT "2200_16667_13_not_null" CHECK (created_at IS NOT NULL);
|
||||
ALTER TABLE "market_calendar" ADD CONSTRAINT "2200_16667_1_not_null" CHECK (calendar_id IS NOT NULL);
|
||||
ALTER TABLE "market_calendar" ADD CONSTRAINT "2200_16667_2_not_null" CHECK (calendar_name IS NOT NULL);
|
||||
ALTER TABLE "market_calendar" ADD CONSTRAINT "2200_16667_3_not_null" CHECK (asset_class IS NOT NULL);
|
||||
ALTER TABLE "market_calendar" ADD CONSTRAINT "2200_16667_5_not_null" CHECK (timezone IS NOT NULL);
|
||||
ALTER TABLE "provider" ADD CONSTRAINT "2200_16399_1_not_null" CHECK (provider_id IS NOT NULL);
|
||||
ALTER TABLE "provider" ADD CONSTRAINT "2200_16399_2_not_null" CHECK (provider_key IS NOT NULL);
|
||||
ALTER TABLE "provider" ADD CONSTRAINT "2200_16399_3_not_null" CHECK (name IS NOT NULL);
|
||||
ALTER TABLE "provider" ADD CONSTRAINT "2200_16399_4_not_null" CHECK (data_type IS NOT NULL);
|
||||
ALTER TABLE "provider_instrument" ADD CONSTRAINT "2200_16409_1_not_null" CHECK (provider_id IS NOT NULL);
|
||||
ALTER TABLE "provider_instrument" ADD CONSTRAINT "2200_16409_2_not_null" CHECK (instrument_id IS NOT NULL);
|
||||
ALTER TABLE "provider_instrument" ADD CONSTRAINT "2200_16409_3_not_null" CHECK (provider_symbol IS NOT NULL);
|
||||
ALTER TABLE "provider_instrument" ADD CONSTRAINT "2200_16409_6_not_null" CHECK (active IS NOT NULL);
|
||||
|
||||
CREATE INDEX idx_bar_dsver ON public.historical_bar USING btree (dataset_version_id);
|
||||
CREATE INDEX idx_bar_elig ON public.historical_bar USING btree (eligibility);
|
||||
CREATE INDEX idx_bar_inst_tf_ts ON public.historical_bar USING btree (instrument_id, timeframe, ts_utc);
|
||||
CREATE INDEX idx_bar_sess_ts ON public.historical_bar USING btree (session_state, instrument_id, ts_utc);
|
||||
CREATE INDEX idx_bar_session ON public.historical_bar USING btree (session_state, session_phase);
|
||||
CREATE INDEX idx_bar_techq ON public.historical_bar USING btree (technical_quality);
|
||||
CREATE INDEX idx_derived_inst_tf_ts ON public.derived_bar USING btree (instrument_id, timeframe, ts_utc);
|
||||
CREATE INDEX idx_historical_bar_market_quality ON public.historical_bar USING btree (instrument_id, market_quality);
|
||||
CREATE INDEX idx_historical_bar_session_state ON public.historical_bar USING btree (instrument_id, session_state);
|
||||
CREATE INDEX idx_prov_adapter_version ON public.ingestion_provenance USING btree (adapter_version);
|
||||
CREATE INDEX idx_prov_instrument ON public.ingestion_provenance USING btree (canonical_instrument_id);
|
||||
CREATE INDEX idx_prov_provider_feed ON public.ingestion_provenance USING btree (provider_id, provider_feed);
|
||||
CREATE INDEX idx_prov_raw_sha ON public.ingestion_provenance USING btree (raw_payload_sha256);
|
||||
CREATE INDEX idx_prov_request_identity ON public.ingestion_provenance USING btree (request_identity_hash);
|
||||
CREATE INDEX idx_prov_time_range ON public.ingestion_provenance USING btree (request_start, request_end);
|
||||
CREATE INDEX idx_tick_inst_ts ON public.historical_tick USING btree (instrument_id, ts_utc);
|
||||
|
|
@ -1,174 +0,0 @@
|
|||
#!/usr/bin/env python3
|
||||
"""
|
||||
D2 — canonical_schema.py
|
||||
Deterministischer Schema-Kanonisierer + Fingerprint für das Historical V2 Subsystem.
|
||||
|
||||
Liefert EINE autoritative Kanonisierungsfunktion, die aus einem Captured-Schema-Modell
|
||||
(JSON, wie von capture_schema.py erzeugt) eine deterministische semantische
|
||||
Serialisierung baut und daraus den SHA-256-Fingerprint berechnet.
|
||||
|
||||
Kanonisierungsregeln (siehe Mission §7):
|
||||
- Schemas/Tabellen alphabetisch
|
||||
- Spalten nach ordinal_position
|
||||
- Constraints deterministisch nach (type, name, content)
|
||||
- Indexe deterministisch
|
||||
- Whitespace/Identifier-Quoting normalisiert, aber keine semantischen Unterschiede weggewaschen
|
||||
|
||||
Fingerprint-Coverage (§8): Tabellenexistenz, Spaltenname, ordinal_position,
|
||||
Datentyp, Nullability, Default, Primary Key, UNIQUE, FK, CHECK, Index-Definitionen.
|
||||
"""
|
||||
import hashlib
|
||||
import json
|
||||
|
||||
|
||||
def _norm_whitespace(s):
|
||||
"""Normalisiere Whitespace; reduziere Wiederholungen, aber erhalte Wörter/Zeichen."""
|
||||
if s is None:
|
||||
return None
|
||||
return " ".join(str(s).split())
|
||||
|
||||
|
||||
def sorted_cols(columns):
|
||||
"""Spalten einer Tabelle, sortiert nach ordinal_position."""
|
||||
return sorted(columns, key=lambda c: (c.get("ordinal", 0), c.get("column", "")))
|
||||
|
||||
|
||||
def canonical_columns(columns):
|
||||
"""Spalten → deterministische Liste von Tuplen."""
|
||||
out = []
|
||||
for c in sorted_cols(columns):
|
||||
out.append((
|
||||
c["column"],
|
||||
int(c.get("ordinal", 0)),
|
||||
_norm_whitespace(c.get("type")),
|
||||
_norm_whitespace(c.get("udt")),
|
||||
bool(c.get("nullable")),
|
||||
_norm_whitespace(c.get("default")),
|
||||
))
|
||||
return out
|
||||
|
||||
|
||||
def canonical_constraints(constraints, kind):
|
||||
"""Constraints eines Typs (PRIMARY KEY, UNIQUE) → sortierte Liste."""
|
||||
out = []
|
||||
for con in constraints:
|
||||
cols = tuple(c for c in con.get("columns") or [])
|
||||
out.append((
|
||||
con.get("table", ""),
|
||||
con.get("constraint", ""),
|
||||
cols,
|
||||
))
|
||||
return sorted(out, key=lambda x: (x[0], x[1], x[2]))
|
||||
|
||||
|
||||
def canonical_fks(fks):
|
||||
"""FK → sortierte Liste inkl. ref_table und ref_columns."""
|
||||
out = []
|
||||
for fk in fks:
|
||||
out.append((
|
||||
fk.get("table", ""),
|
||||
fk.get("constraint", ""),
|
||||
tuple(fk.get("columns") or []),
|
||||
fk.get("ref_table", ""),
|
||||
tuple(fk.get("ref_columns") or []),
|
||||
))
|
||||
return sorted(out, key=lambda x: (x[0], x[1]))
|
||||
|
||||
|
||||
def canonical_checks(checks):
|
||||
"""CHECK-Constraints → sortierte Liste mit normierter Definition.
|
||||
|
||||
Auto-generierte PostgreSQL 'NOT NULL'-Checks (CONSTRAINT-Name endet auf
|
||||
'_not_null', von PostgreSQL automatisch pro NOT-NULL-Spalte erzeugt) werden
|
||||
AUSGESCHLOSSEN: sie sind redundant zu 'nullable' und ihre Definition
|
||||
unterscheidet sich zwischen DBs (Production 'x IS NOT NULL' vs. Checker
|
||||
'((x IS NOT NULL))') → semantisches Rauschen. Zuverlässige Erkennung NUR
|
||||
über den Namenssuffix '_not_null'. Explizit benannte User-Checks
|
||||
(chk_* etc.) bleiben erhalten.
|
||||
"""
|
||||
out = []
|
||||
for chk in checks:
|
||||
name = chk.get("constraint", "")
|
||||
if name.endswith("_not_null"):
|
||||
continue
|
||||
out.append((
|
||||
chk.get("table", ""),
|
||||
name,
|
||||
_norm_whitespace(chk.get("definition")) or "",
|
||||
))
|
||||
return sorted(out, key=lambda x: (x[0], x[1], x[2]))
|
||||
|
||||
|
||||
def canonical_indexes(indexes):
|
||||
"""Index-Definitionen → sortierte Liste mit normierter Definition."""
|
||||
out = []
|
||||
for idx in indexes:
|
||||
out.append((
|
||||
idx.get("table", ""),
|
||||
idx.get("index", ""),
|
||||
_norm_whitespace(idx.get("definition")),
|
||||
))
|
||||
return sorted(out, key=lambda x: (x[0], x[1], x[2]))
|
||||
|
||||
|
||||
def canonical_sequences(sequences):
|
||||
"""Sequenzen → sortierte Liste (nur Existenz/Name)."""
|
||||
return sorted(sequences or [])
|
||||
|
||||
|
||||
def canonical_document(model):
|
||||
"""Baue die vollständige deterministische semantische Serialisierung."""
|
||||
tables = sorted(model.get("tables") or [])
|
||||
cols_by_table = {}
|
||||
for c in model.get("columns") or []:
|
||||
cols_by_table.setdefault(c["table"], []).append(c)
|
||||
|
||||
pks = canonical_constraints(model.get("primary_keys") or [], "PRIMARY KEY")
|
||||
uniques = canonical_constraints(model.get("unique_constraints") or [], "UNIQUE")
|
||||
fks = canonical_fks(model.get("foreign_keys") or [])
|
||||
checks = canonical_checks(model.get("check_constraints") or [])
|
||||
indexes = canonical_indexes(model.get("indexes") or [])
|
||||
sequences = canonical_sequences(model.get("sequences") or [])
|
||||
|
||||
doc = []
|
||||
doc.append("schema=public")
|
||||
doc.append("sequences=" + "|".join(sequences))
|
||||
doc.append("tables=" + "|".join(tables))
|
||||
for t in tables:
|
||||
doc.append(f"table:{t}:cols=" + ";".join(
|
||||
f"{name}|{ordp}|{typ}|{udt}|{'n' if nul else 'y'}|{dflt}"
|
||||
for (name, ordp, typ, udt, nul, dflt) in canonical_columns(cols_by_table.get(t, []))
|
||||
))
|
||||
doc.append("pk=" + ";".join(
|
||||
f"{t}|{n}|{','.join(c)}"
|
||||
for (t, n, c) in pks
|
||||
))
|
||||
doc.append("unique=" + ";".join(
|
||||
f"{t}|{n}|{','.join(c)}"
|
||||
for (t, n, c) in uniques
|
||||
))
|
||||
doc.append("fk=" + ";".join(
|
||||
f"{t}|{n}|{','.join(c)}->{rt}|{','.join(rc)}"
|
||||
for (t, n, c, rt, rc) in fks
|
||||
))
|
||||
doc.append("check=" + ";".join(
|
||||
f"{t}|{n}|{d}"
|
||||
for (t, n, d) in checks
|
||||
))
|
||||
doc.append("index=" + ";".join(
|
||||
f"{t}|{n}|{d}"
|
||||
for (t, n, d) in indexes
|
||||
))
|
||||
return "\n".join(doc)
|
||||
|
||||
|
||||
def fingerprint(model):
|
||||
"""SHA-256 über die kanonische Serialisierung."""
|
||||
doc = canonical_document(model)
|
||||
return hashlib.sha256(doc.encode("utf-8")).hexdigest(), doc
|
||||
|
||||
|
||||
def load_model(path):
|
||||
"""Lade ein Captured-Schema-Modell (JSON)."""
|
||||
with open(path) as f:
|
||||
return json.load(f)
|
||||
|
|
@ -1,95 +0,0 @@
|
|||
#!/usr/bin/env python3
|
||||
"""
|
||||
D2 — capture_schema.py
|
||||
Liest Schema-Metadaten (read-only) von einer DB und erzeugt das kanonische Modell-JSON.
|
||||
|
||||
Quellenmodus: Erwartet eine Datei mit den 7 JSON-Aggregaten aus dem Capture-SQL
|
||||
(der read-only psql-Ausgabe), ODER ein bereits geparstes Modell-JSON.
|
||||
|
||||
Usage:
|
||||
capture_schema.py --from-capture <capture.json> --out <model.json>
|
||||
capture_schema.py --from-model <model.json> --out <model.json> # Normalisierung
|
||||
|
||||
Der DB-Zugriff selbst erfolgt ausschließlich über das read-only Capture-SQL
|
||||
(BEGIN TRANSACTION READ ONLY). Dieses Tool verarbeitet nur die Ergebnisse.
|
||||
"""
|
||||
import json
|
||||
import sys
|
||||
import argparse
|
||||
|
||||
AGG_NAMES = ["columns", "primary_keys", "unique_constraints",
|
||||
"foreign_keys", "check_constraints", "indexes", "tables",
|
||||
"sequences"]
|
||||
|
||||
|
||||
def strip_psql_noise(text):
|
||||
"""Entferne psql-Kontrollzeilen (BEGIN/on/COMMIT/leer/--)."""
|
||||
out = []
|
||||
for line in text.splitlines():
|
||||
s = line.strip()
|
||||
if s and s not in ("BEGIN", "on", "COMMIT") and not s.startswith("--"):
|
||||
out.append(s)
|
||||
return out
|
||||
|
||||
|
||||
def parse_raw_capture(text):
|
||||
"""Extrahiere die N JSON-Aggregate aus der psql-Ausgabe.
|
||||
|
||||
psql splittet große JSON-Arrays über mehrere Zeilen; daher wird nicht
|
||||
zeilenzählend, sondern per JSON-Streaming (raw_decode) geparst.
|
||||
"""
|
||||
data = "\n".join(strip_psql_noise(text))
|
||||
decoder = json.JSONDecoder()
|
||||
aggs = []
|
||||
i = 0
|
||||
n = len(data)
|
||||
while i < n and len(aggs) < len(AGG_NAMES):
|
||||
while i < n and data[i] in " \t\r\n":
|
||||
i += 1
|
||||
if i >= n:
|
||||
break
|
||||
if data[i] not in "[{":
|
||||
# non-JSON noise (z.B. Fehlermeldung) -> nächste Position
|
||||
i += 1
|
||||
continue
|
||||
try:
|
||||
val, end = decoder.raw_decode(data, i)
|
||||
except json.JSONDecodeError as e:
|
||||
raise SystemExit(f"FEHLER: JSON-Parse bei {i}: {e}")
|
||||
aggs.append(val if val is not None else [])
|
||||
i = end
|
||||
if len(aggs) != len(AGG_NAMES):
|
||||
raise SystemExit(
|
||||
f"FEHLER: erwartete {len(AGG_NAMES)} JSON-Aggregate, fand {len(aggs)}")
|
||||
return dict(zip(AGG_NAMES, aggs))
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--from-capture", help="Rohe read-only psql-Capture-Datei (7 Aggregate)")
|
||||
ap.add_argument("--from-model", help="Bereits geparstes Modell-JSON")
|
||||
ap.add_argument("--out", required=True, help="Ziel-Modell-JSON")
|
||||
args = ap.parse_args()
|
||||
|
||||
if args.from_capture:
|
||||
with open(args.from_capture) as f:
|
||||
model = parse_raw_capture(f.read())
|
||||
elif args.from_model:
|
||||
with open(args.from_model) as f:
|
||||
model = json.load(f)
|
||||
else:
|
||||
raise SystemExit("FEHLER: --from-capture oder --from-model erforderlich")
|
||||
|
||||
# Konsistenz: columns/tables müssen übereinstimmen
|
||||
col_tables = sorted({c["table"] for c in model["columns"]})
|
||||
model["tables"] = sorted(model.get("tables") or col_tables)
|
||||
|
||||
with open(args.out, "w") as f:
|
||||
json.dump(model, f, indent=2, sort_keys=True)
|
||||
print(f"Modell geschrieben: {args.out}")
|
||||
print(f"Tabellen: {len(model['tables'])} | Spalten: {len(model['columns'])}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
|
|
@ -1,108 +0,0 @@
|
|||
#!/usr/bin/env python3
|
||||
"""
|
||||
D2 — compare_schema.py
|
||||
Vergleicht zwei Schema-Modelle (Soll = kanonischer L0-Contract, Ist = tatsächlich).
|
||||
|
||||
Semantische Diff-Dimensionen (§12):
|
||||
fehlende/extras Tabellen, Spalten, Typ-Mismatches, Nullable-Mismatches,
|
||||
Default-Mismatches, PK, UNIQUE, FK, CHECK, Index-Mismatches.
|
||||
|
||||
Fail-closed: exit-Nonzero bei beliebigem semantischem Mismatch.
|
||||
Acceptance: alle Mismatch-Counts == 0.
|
||||
"""
|
||||
import sys
|
||||
import json
|
||||
import argparse
|
||||
from canonical_schema import (canonical_columns, canonical_constraints,
|
||||
canonical_fks, canonical_checks, canonical_indexes)
|
||||
|
||||
|
||||
def diff_counts(actual, expected):
|
||||
"""Liefert dict mit Mismatch-Counts. 0 = keine."""
|
||||
counts = {
|
||||
"missing_tables": 0, "extra_tables": 0,
|
||||
"missing_columns": 0, "extra_columns": 0,
|
||||
"type_mismatch": 0, "nullable_mismatch": 0, "default_mismatch": 0,
|
||||
"pk_mismatch": 0, "unique_mismatch": 0, "fk_mismatch": 0,
|
||||
"check_mismatch": 0, "index_mismatch": 0,
|
||||
}
|
||||
# Tabellen
|
||||
a_tbl = set((actual.get("tables") or []))
|
||||
e_tbl = set((expected.get("tables") or []))
|
||||
counts["missing_tables"] = len(e_tbl - a_tbl)
|
||||
counts["extra_tables"] = len(a_tbl - e_tbl)
|
||||
|
||||
# Spalten je Tabelle
|
||||
a_cols = {}
|
||||
for c in actual.get("columns") or []:
|
||||
a_cols.setdefault(c["table"], []).append(c)
|
||||
e_cols = {}
|
||||
for c in expected.get("columns") or []:
|
||||
e_cols.setdefault(c["table"], []).append(c)
|
||||
for tbl in e_tbl:
|
||||
ec = {c["column"]: c for c in e_cols.get(tbl, [])}
|
||||
ac = {c["column"]: c for c in a_cols.get(tbl, [])}
|
||||
for cname, cexp in ec.items():
|
||||
if cname not in ac:
|
||||
counts["missing_columns"] += 1
|
||||
continue
|
||||
cact = ac[cname]
|
||||
if _n(cact.get("type")) != _n(cexp.get("type")):
|
||||
counts["type_mismatch"] += 1
|
||||
if bool(cact.get("nullable")) != bool(cexp.get("nullable")):
|
||||
counts["nullable_mismatch"] += 1
|
||||
if _n(cact.get("default")) != _n(cexp.get("default")):
|
||||
counts["default_mismatch"] += 1
|
||||
for cname in ac:
|
||||
if cname not in ec:
|
||||
counts["extra_columns"] += 1
|
||||
|
||||
# Constraints
|
||||
if canonical_constraints(actual.get("primary_keys") or [], "PK") != \
|
||||
canonical_constraints(expected.get("primary_keys") or [], "PK"):
|
||||
counts["pk_mismatch"] += 1
|
||||
if canonical_constraints(actual.get("unique_constraints") or [], "UNIQUE") != \
|
||||
canonical_constraints(expected.get("unique_constraints") or [], "UNIQUE"):
|
||||
counts["unique_mismatch"] += 1
|
||||
if canonical_fks(actual.get("foreign_keys") or []) != \
|
||||
canonical_fks(expected.get("foreign_keys") or []):
|
||||
counts["fk_mismatch"] += 1
|
||||
if canonical_checks(actual.get("check_constraints") or []) != \
|
||||
canonical_checks(expected.get("check_constraints") or []):
|
||||
counts["check_mismatch"] += 1
|
||||
if canonical_indexes(actual.get("indexes") or []) != \
|
||||
canonical_indexes(expected.get("indexes") or []):
|
||||
counts["index_mismatch"] += 1
|
||||
|
||||
return counts
|
||||
|
||||
|
||||
def _n(x):
|
||||
return None if x is None else " ".join(str(x).split())
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("expected", help="Kanonisches L0-Contract-Modell")
|
||||
ap.add_argument("actual", help="Tatsächlich gecaptures Modell")
|
||||
args = ap.parse_args()
|
||||
|
||||
with open(args.expected) as f:
|
||||
expected = json.load(f)
|
||||
with open(args.actual) as f:
|
||||
actual = json.load(f)
|
||||
|
||||
counts = diff_counts(actual, expected)
|
||||
total = sum(counts.values())
|
||||
print(json.dumps(counts, indent=2))
|
||||
print(f"TOTAL_MISMATCH={total}")
|
||||
|
||||
if total == 0:
|
||||
print("PASS: Semantic diff = 0")
|
||||
return 0
|
||||
print("FAIL: semantic drift detected", file=sys.stderr)
|
||||
return 1 # FAIL CLOSED
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
|
|
@ -1,47 +0,0 @@
|
|||
#!/usr/bin/env python3
|
||||
"""
|
||||
D2 — fingerprint_schema.py
|
||||
Deterministischer SHA-256 über ein Captured-Schema-Modell.
|
||||
|
||||
Usage:
|
||||
fingerprint_schema.py <model.json>
|
||||
→ schreibt nur den Hex-Digest auf stdout (fail-closed: exit 0 bei Erfolg)
|
||||
fingerprint_schema.py <model.json> --expect <digest>
|
||||
→ exit 0 wenn MATCH, sonst exit 1 (FAIL CLOSED)
|
||||
|
||||
Der Fingerprint deckt ab: Tabellenexistenz, Spaltenname, ordinal_position,
|
||||
Datentyp, Nullability, Default, PK, UNIQUE, FK, CHECK, Index-Definitionen (§8).
|
||||
"""
|
||||
import sys
|
||||
import argparse
|
||||
from canonical_schema import fingerprint, load_model
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("model")
|
||||
ap.add_argument("--expect", default=None, help="Erwarteter Digest (fail-closed Vergleich)")
|
||||
ap.add_argument("--no-hash", action="store_true", help="Nur kanonische Serialisierung ausgeben")
|
||||
args = ap.parse_args()
|
||||
|
||||
model = load_model(args.model)
|
||||
digest, doc = fingerprint(model)
|
||||
|
||||
if args.no_hash:
|
||||
print(doc)
|
||||
return 0
|
||||
|
||||
if args.expect:
|
||||
if digest == args.expect:
|
||||
print(f"MATCH {digest}")
|
||||
return 0
|
||||
else:
|
||||
print(f"MISMATCH expected={args.expect} actual={digest}", file=sys.stderr)
|
||||
return 1 # FAIL CLOSED
|
||||
|
||||
print(digest)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
|
|
@ -1,136 +0,0 @@
|
|||
#!/usr/bin/env python3
|
||||
"""
|
||||
D2 — gen_l0_sql.py
|
||||
Erzeugt den kanonischen L0-SQL-Contract (schema-only) aus einem Captured-Schema-Modell.
|
||||
|
||||
Repräsentiert die aktuelle kanonische Production-Truth. Enthält KEINE:
|
||||
data rows, credentials, volatile timestamps, container IDs, host paths, secrets.
|
||||
Kein CREATE DATABASE, keine Grant/GRANT-relevanten Owner, keine Umgebungs-Sequenzen.
|
||||
|
||||
Semantische Overrides (§4) werden als Kommentar-Blöcke dokumentiert:
|
||||
last_successful_chunk = timestamp with time zone (Owner-canonical)
|
||||
interrupt_reason = text (preserved current contract)
|
||||
technical_quality = text nullable default 'VALID'
|
||||
"""
|
||||
import json
|
||||
import sys
|
||||
import argparse
|
||||
|
||||
TYPE_MAP = {
|
||||
'bigint': 'bigint', 'integer': 'integer', 'smallint': 'smallint',
|
||||
'numeric': 'numeric', 'timestamp with time zone': 'timestamp with time zone',
|
||||
'boolean': 'boolean', 'text': 'text', 'date': 'date', 'double precision': 'double precision',
|
||||
'jsonb': 'jsonb', 'real': 'real', 'character varying': 'character varying',
|
||||
}
|
||||
|
||||
|
||||
def ident(s):
|
||||
return '"' + str(s).replace('"', '""') + '"'
|
||||
|
||||
|
||||
def col_def(c):
|
||||
typ = TYPE_MAP.get(c.get('type'), c.get('udt', c.get('type', 'text')))
|
||||
nullable = '' if not c.get('nullable') else ''
|
||||
notnull = ' NOT NULL' if not c.get('nullable') else ''
|
||||
dflt = c.get('default')
|
||||
default_part = ''
|
||||
if dflt:
|
||||
default_part = ' DEFAULT ' + dflt
|
||||
return f" {ident(c['column'])} {typ}{notnull}{default_part}"
|
||||
|
||||
|
||||
def build_sql(model, dbname="historical"):
|
||||
out = []
|
||||
out.append("-- ============================================================")
|
||||
out.append(f"-- CANONICAL CURRENT PRODUCTION SCHEMA CONTRACT (L0)")
|
||||
out.append(f"-- db: {dbname} · schema: public")
|
||||
out.append("-- Auto-generated by gen_l0_sql.py from read-only capture")
|
||||
out.append("-- Modus: CURRENT VERIFIED PRODUCTION TRUTH — NICHT historische Migration")
|
||||
out.append("-- ============================================================")
|
||||
out.append("")
|
||||
out.append("-- SEMANTIC OVERRIDES (Owner-Canonical)")
|
||||
out.append("-- last_successful_chunk = timestamp with time zone (Production + Owner decision)")
|
||||
out.append("-- interrupt_reason = text · PRESERVED CURRENT CONTRACT · origin unknown · do not remove")
|
||||
out.append("-- technical_quality = text · nullable · default 'VALID' · historical provenance debt OPEN")
|
||||
out.append("")
|
||||
# Sequenzen (für nextval-Defaults)
|
||||
for seq in sorted(model.get("sequences") or []):
|
||||
out.append(f"CREATE SEQUENCE IF NOT EXISTS {ident(seq)};")
|
||||
if model.get("sequences"):
|
||||
out.append("")
|
||||
|
||||
tables = sorted(model.get("tables") or [])
|
||||
cols_by_table = {}
|
||||
for c in model.get("columns") or []:
|
||||
cols_by_table.setdefault(c["table"], []).append(c)
|
||||
|
||||
pks = {}
|
||||
for pk in model.get("primary_keys") or []:
|
||||
cols = sorted(pk.get("columns") or [], key=lambda x: 0)
|
||||
pks[pk["table"]] = [pk.get("columns") or []]
|
||||
|
||||
for t in tables:
|
||||
cols = sorted(cols_by_table.get(t, []), key=lambda c: c.get("ordinal", 0))
|
||||
out.append(f"-- Table: {t} ({len(cols)} cols)")
|
||||
out.append(f"CREATE TABLE IF NOT EXISTS {ident(t)} (")
|
||||
body_lines = [col_def(c) for c in cols]
|
||||
out.append(",\n".join(body_lines))
|
||||
out.append(");")
|
||||
out.append("")
|
||||
|
||||
# PKs
|
||||
for t in tables:
|
||||
for pk in model.get("primary_keys") or []:
|
||||
if pk["table"] == t:
|
||||
colstr = ", ".join(ident(c) for c in pk.get("columns") or [])
|
||||
out.append(f"ALTER TABLE {ident(t)} ADD CONSTRAINT {ident(pk['constraint'])} PRIMARY KEY ({colstr});")
|
||||
out.append("")
|
||||
|
||||
# UNIQUE
|
||||
for uq in sorted(model.get("unique_constraints") or [], key=lambda x: x["table"]):
|
||||
colstr = ", ".join(ident(c) for c in uq.get("columns") or [])
|
||||
out.append(f"ALTER TABLE {ident(uq['table'])} ADD CONSTRAINT {ident(uq['constraint'])} UNIQUE ({colstr});")
|
||||
out.append("")
|
||||
|
||||
# FK
|
||||
for fk in sorted(model.get("foreign_keys") or [], key=lambda x: x["table"]):
|
||||
colstr = ", ".join(ident(c) for c in fk.get("columns") or [])
|
||||
rcolstr = ", ".join(ident(c) for c in fk.get("ref_columns") or [])
|
||||
out.append(f"ALTER TABLE {ident(fk['table'])} ADD CONSTRAINT {ident(fk['constraint'])} FOREIGN KEY ({colstr}) REFERENCES {ident(fk['ref_table'])} ({rcolstr});")
|
||||
out.append("")
|
||||
|
||||
# CHECK - semantische Definition
|
||||
for chk in sorted(model.get("check_constraints") or [], key=lambda x: (x["table"], x["constraint"])):
|
||||
out.append(f"ALTER TABLE {ident(chk['table'])} ADD CONSTRAINT {ident(chk['constraint'])} CHECK ({chk.get('definition','')});")
|
||||
out.append("")
|
||||
|
||||
# Indexe (non-PK/UNIQUE)
|
||||
idx_set = set()
|
||||
for pk in model.get("primary_keys") or []:
|
||||
idx_set.add(pk["constraint"])
|
||||
for uq in model.get("unique_constraints") or []:
|
||||
idx_set.add(uq["constraint"])
|
||||
for idx in sorted(model.get("indexes") or [], key=lambda x: x["index"]):
|
||||
if idx["index"] in idx_set:
|
||||
continue
|
||||
out.append(f"{idx['definition']};")
|
||||
out.append("")
|
||||
return "\n".join(out)
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("model")
|
||||
ap.add_argument("--out", required=True)
|
||||
args = ap.parse_args()
|
||||
with open(args.model) as f:
|
||||
model = json.load(f)
|
||||
sql = build_sql(model)
|
||||
with open(args.out, "w") as f:
|
||||
f.write(sql)
|
||||
print(f"L0 SQL-Contract geschrieben: {args.out} ({len(sql)} B)")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/90957134-8e8b-021b-717c-e9c7ca6cd887
|
||||
type: arch
|
||||
role: history
|
||||
representation: canonical
|
||||
state: historical
|
||||
knowledge_schema: 1
|
||||
type: Trading-Modul-System
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,11 +1,3 @@
|
|||
---
|
||||
id: object/ac1df780-019b-bb11-cb91-4edaa49e1843
|
||||
type: arch
|
||||
role: history
|
||||
representation: canonical
|
||||
state: historical
|
||||
knowledge_schema: 1
|
||||
---
|
||||
# Phase 8 — M12 Historical-V2 DatasetGate + Run-Audit
|
||||
|
||||
Autor: Rain Ocampo (Hermes) | Datum: 2026-08-22 | Status: IMPLEMENTIERT + GETESTET
|
||||
|
|
|
|||
|
|
@ -1,17 +1,9 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/5fcf4885-a361-f49f-2164-f4e10709da34
|
||||
type: arch
|
||||
role: reference
|
||||
representation: canonical
|
||||
state: current
|
||||
derived_from: object/f4e559f5-403c-f241-be50-0962d8174644
|
||||
|
||||
type: Trading-Modul-System
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
||||
# Infrastruktur & Betriebs-Handbuch — Trading-System VPS
|
||||
# Infrastruktur & Betriebs-Handbuch — Modul-Trading-System VPS
|
||||
|
||||
> Stand: 20.08.2026 · VPS: `187.124.31.123`
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/ebdc6b2b-4b30-c839-729b-c240ad433a60
|
||||
type: arch
|
||||
role: module
|
||||
representation: canonical
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Trading-Modul-System
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/b909ccbc-f7cf-7f71-46ea-817f0b697d35
|
||||
type: arch
|
||||
role: module
|
||||
representation: canonical
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Trading-Modul-System
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,22 +1,14 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/cf54abfc-4c8b-7477-1b4f-21e76ca7b055
|
||||
type: arch
|
||||
role: module
|
||||
representation: canonical
|
||||
state: current
|
||||
derived_from: object/1761a726-55af-f97b-d509-84aa4577f102
|
||||
|
||||
type: Trading-Modul-System
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
||||
# Modul-03-Market-Data — Betriebsdokumentation
|
||||
|
||||
> Erstellt: 20.08.2026 (Rain Ocampo) · Status: ✅ In Betrieb (healthy)
|
||||
|
||||
## Zweck
|
||||
Zentrale Marktdatenquelle des Trading-Systems. Pipeline:
|
||||
Zentrale Marktdatenquelle des Modul-Trading-Systems. Pipeline:
|
||||
`Marktdaten → Validierung → Normalisierung → PostgreSQL → RabbitMQ-Event`
|
||||
Keine Strategie, keine Orders, keine KI — nur zuverlässige Marktdaten.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,16 +1,8 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/36d6dc60-1965-ed65-a002-cdb477ccd638
|
||||
type: arch
|
||||
role: module
|
||||
representation: canonical
|
||||
state: current
|
||||
derived_from: object/6fca3e3a-70b2-0927-d7fb-e650a8f0e7f8
|
||||
|
||||
type: Trading-Modul-System
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
||||
# Modul-04-Market-Regime — Betriebsdokumentation
|
||||
|
||||
> Erstellt: 20.08.2026 (Rain Ocampo) · Status: ✅ Freigegeben (E2E bestanden)
|
||||
|
|
|
|||
|
|
@ -1,16 +1,8 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/58f1bc40-5b12-87a0-e95f-5288b91ebc87
|
||||
type: arch
|
||||
role: module
|
||||
representation: canonical
|
||||
state: current
|
||||
derived_from: object/0aef122a-790b-b17a-9176-cc32df98c5a1
|
||||
|
||||
type: Trading-Modul-System
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
||||
# Modul-05-Strategy-Engine — Betriebsdokumentation
|
||||
|
||||
> Erstellt: 20.08.2026 (Rain Ocampo) · Status: ✅ Freigegeben (E2E bestanden)
|
||||
|
|
|
|||
|
|
@ -1,16 +1,8 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/5c04acf8-baf8-0771-f36b-05bf5cf58138
|
||||
type: arch
|
||||
role: module
|
||||
representation: canonical
|
||||
state: current
|
||||
derived_from: object/2b4c4170-b5b2-8a2b-fdee-8ea5eff191b5
|
||||
|
||||
type: Trading-Modul-System
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
||||
# Modul-06-Signal-Ranking — Betriebsdokumentation
|
||||
|
||||
> Erstellt: 20.08.2026 (Rain Ocampo) · Status: ✅ **Freigegeben** (E2E+Idempotenz+Reconnect+Doku grün)
|
||||
|
|
|
|||
|
|
@ -1,16 +1,8 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/d87e2842-ed93-9f5b-e8ea-3b4de31f4bc6
|
||||
type: arch
|
||||
role: module
|
||||
representation: canonical
|
||||
state: current
|
||||
derived_from: object/645f8ff7-5a1a-bcf3-8dfe-0a8cfdf15dbc
|
||||
|
||||
type: Trading-Modul-System
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
||||
# Modul-07: Risk-Manager
|
||||
|
||||
**Status:** ✅ **Freigegeben** (20.08.2026)
|
||||
|
|
|
|||
|
|
@ -1,16 +1,8 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/7ff050c3-ebaa-d0ae-5aa4-6e9da67d4773
|
||||
type: arch
|
||||
role: module
|
||||
representation: canonical
|
||||
state: current
|
||||
derived_from: object/603e6b87-115a-e258-5bfb-7d94d6d78c97
|
||||
|
||||
type: Trading-Modul-System
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
||||
# Modul-08: Portfolio-Manager
|
||||
|
||||
**Status: FREIGEGEBEN** (20.08.2026)
|
||||
|
|
|
|||
|
|
@ -1,16 +1,8 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/ffa8db13-8aa6-d4a4-6756-69eabd0c8d77
|
||||
type: arch
|
||||
role: module
|
||||
representation: canonical
|
||||
state: current
|
||||
derived_from: object/48bd264f-607b-15f1-5f73-3e922af9b19d
|
||||
|
||||
type: Trading-Modul-System
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
||||
# Modul-09: Execution-Service
|
||||
|
||||
**Status: FREIGEGEBEN** (20.08.2026)
|
||||
|
|
@ -58,65 +50,10 @@ RabbitMQPublisher (market.execution)
|
|||
`submit_order()`, `get_order_status()`
|
||||
- `PaperBrokerAdapter` (app/broker/paper.py): V1, simuliert realistischen
|
||||
Lifecycle (SUBMITTED → FILLED, Partial Fill, Reject, Timeout) für Tests
|
||||
- `IgDemoAdapter` (app/broker/ig_demo.py): IG-Markets-Demo-Broker (IG_DEMO-Modus),
|
||||
echter externer Broker-Adapter. Liest/schreibt über IG REST API (demo-api.ig.com).
|
||||
- `BrokerRegistry` (app/broker/registry.py): wählt Adapter anhand
|
||||
`DEFAULT_BROKER` (V1: paper). Echte Broker-Adapter später austauschbar,
|
||||
keine Brokerlogik im Core-Code.
|
||||
|
||||
## IG-Demo-Adapter (IG_DEMO-Modus, 21.08.2026)
|
||||
|
||||
**Status: FREIGEGEBEN / PRODUKTIV VERIFIZIERT** — erster echter externer Broker-Adapter.
|
||||
|
||||
- **Eigener Modus `IG_DEMO`** (nicht LIVE+Gate); PAPER/IG_DEMO/LIVE strikt getrennt.
|
||||
- **Demo-Basis-URL erzwungen** (`demo-api.ig.com`), LIVE-Endpunkt (`api.ig.com`)
|
||||
technisch blockiert, kein IG_DEMO→LIVE-Autowechsel.
|
||||
- **Credentials ausschließlich VPS-Secret/ENV** (`.env.ig`, chmod 600/root-only);
|
||||
nie in Code/Forgejo/Tolaria/DB/Logs.
|
||||
- **Mengen instrumentabhängig** über `broker_mapping` (PostgreSQL); kein globales 1:1.
|
||||
- **Epic-Mapping produktiv in PostgreSQL `broker_mapping`**; Forgejo nur Schema/Doku/Beispiele.
|
||||
|
||||
### Order-Aktions-Auswertung (CLOSE-Pfad-Fix, 21.08.2026)
|
||||
`BrokerOrderRequest` trägt `action_type` (Default `OPEN`) + `broker_order_id`.
|
||||
Der Service reicht beides durch; der Adapter wertet `action_type` aus:
|
||||
|
||||
| action_type | IG-Pfad |
|
||||
|-------------|---------|
|
||||
| `OPEN` / `INCREASE` | `POST /positions/otc` (create_position) |
|
||||
| `CLOSE` / `REDUCE` | `POST /positions/otc` mit `_method:DELETE` (close_position) |
|
||||
| unbekannt/ungültig | **FAIL-CLOSED** (`IG_UNSUPPORTED_ACTION`) |
|
||||
|
||||
**CLOSE-Direction korrekt umkehren:** CLOSE einer LONG/BUY-Position muss mit
|
||||
**SELL** erfolgen (Gegenseite), nicht mit der Positionsrichtung. `_close_direction()`:
|
||||
LONG→SELL, SHORT→BUY.
|
||||
|
||||
**broker_order_id/dealId sauber durchreichen:** `broker_order_id` wird vom Service
|
||||
in die Order übernommen und an den Adapter durchgereicht; IG `dealId` wird als
|
||||
`broker_order_id` persistiert.
|
||||
|
||||
**Unbekannte/ungültige Aktionen FAIL-CLOSED:** `_derive_action_type()` setzt
|
||||
unbekannte Aktionen NICHT mehr auf OPEN zurück, sondern reicht sie durch →
|
||||
Control blockt mit `UNKNOWN_ACTION_TYPE`, Adapter mit `IG_UNSUPPORTED_ACTION`.
|
||||
|
||||
### IG-Adapter-Tests (6/6 grün)
|
||||
1. `test_unsupported_action_fail_closed` — unbekannte Aktion → FAIL-CLOSED
|
||||
2. `test_close_direction_inverted` — CLOSE einer BUY-Position → SELL
|
||||
3. `test_open_uses_create_position` — OPEN → create_position
|
||||
4. `test_close_uses_close_position` — CLOSE → close_position mit `_method:DELETE`
|
||||
5. `test_broker_order_id_passthrough` — broker_order_id/dealId durchgereicht
|
||||
6. `test_fail_closed_without_credentials` — ohne Credentials → nicht konfiguriert
|
||||
|
||||
### Erster erfolgreicher IG-DEMO OPEN→CLOSE-E2E (21.08.2026)
|
||||
- **OPEN** `IGE2E-US500-OPEN-1787292734` → dealId `DIAAAAYB46KMNAE`, size 1.0, BUY,
|
||||
openLevel 7652.58 → **FILLED**
|
||||
- **CLOSE-Bug gefunden + gefixt:** erster CLOSE sendete fälschlich als zweite OPEN
|
||||
(action_type ignoriert) + falsche direction (BUY statt SELL)
|
||||
- **CLOSE2** `IGE2E-US500-CLOSE2-1787293768` für `DIAAAAYB463S4AB` → **FILLED**
|
||||
(filled_quantity 1, avg_fill_price 7652.19)
|
||||
- **IG GET /positions: Anzahl 0** — alle Positionen geschlossen
|
||||
- **Safety-Reset:** M09 zurück auf PAPER (`EXECUTION_MODE=PAPER`,
|
||||
`TRADING_ENABLED=false`, `DEFAULT_BROKER=paper`)
|
||||
|
||||
## Idempotenz
|
||||
|
||||
- Unique-Index `uq_execution_order_src` auf `source_portfolio_decision_id`
|
||||
|
|
@ -152,11 +89,10 @@ sowie `REJECTED`, `CANCELLED`, `FAILED`.
|
|||
|
||||
## Tests
|
||||
|
||||
- **31/31 Unit-Tests** (test_execution.py + test_ig_demo.py + test_control.py):
|
||||
gültige Order → PAPER FILLED, Idempotenz, ungültige Quantity → REJECTED,
|
||||
Kill-Switch, fehlende Broker-Credentials → FAIL-CLOSED, Broker-Reject,
|
||||
Timeout → keine blinde Doppelorder, Partial Fill, Reconnect-Backoff,
|
||||
parallele identische Events, Control-Gate (M15), IG-Adapter (6/6)
|
||||
- **21/21 Unit-Tests** (test_execution.py): gültige Order → PAPER FILLED,
|
||||
Idempotenz, ungültige Quantity → REJECTED, Kill-Switch, fehlende
|
||||
Broker-Credentials → FAIL-CLOSED, Broker-Reject, Timeout → keine blinde
|
||||
Doppelorder, Partial Fill, Reconnect-Backoff, parallele identische Events
|
||||
- **E2E 6/6 Checks** (Kette 03→04→05→06→07→08→09): Execution-Order in DB,
|
||||
exakt 1 Paper-Order, Pflichtfelder, Idempotenz, ORDER_SUBMITTED+ORDER_FILLED,
|
||||
gültiger Trade → PAPER FILLED
|
||||
|
|
|
|||
|
|
@ -1,16 +1,8 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/e2beb080-49cc-0b05-64fb-291440fe9abb
|
||||
type: journal
|
||||
role: module
|
||||
representation: canonical
|
||||
state: current
|
||||
derived_from: object/0b393153-53e5-99fe-1098-e8bb4e0b6c7c
|
||||
|
||||
type: Trading-Modul-System
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
||||
# Modul-10: Trade-Journal
|
||||
|
||||
**Status: FREIGEGEBEN** (20.08.2026)
|
||||
|
|
|
|||
|
|
@ -1,16 +1,8 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/c86dba29-835c-6b6a-d4f9-ac7cec69b8c4
|
||||
type: arch
|
||||
role: module
|
||||
representation: canonical
|
||||
state: current
|
||||
derived_from: object/31cf1506-0ab6-5f7a-cecc-0e3a7a2e6572
|
||||
|
||||
type: Trading-Modul-System
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
||||
# Modul-11: Analytics
|
||||
|
||||
**Status: FREIGEGEBEN** · Container `Modul-11-Analytics` · Image `analytics-service:0.1.0`
|
||||
|
|
|
|||
|
|
@ -1,14 +1,8 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/47b8029c-5874-e73f-0279-d335785b359a
|
||||
type: arch
|
||||
role: module
|
||||
representation: canonical
|
||||
state: current
|
||||
derived_from: object/302e9929-e186-c930-2406-ad4a8f17c6fd
|
||||
|
||||
type: Trading-Modul-System
|
||||
_organized: true
|
||||
---
|
||||
|
||||
# Modul-12: Backtesting
|
||||
|
||||
**Status: FREIGEGEBEN** · Container `Modul-12-Backtesting` · Image `backtesting:0.1.2` (V1 + V1.1 parallel)
|
||||
|
|
@ -127,62 +121,10 @@ Jede Tabelle FK auf `backtest_run(run_id) ON DELETE CASCADE`.
|
|||
- **Modul-13: Optimization** — Image `optimization:0.1.0`, E2E-verifiziert (12/12 Punkte), **FREIGEGEBEN** (20.08.2026). Details: `modul-13-optimization.md`.
|
||||
- **V1.1-Strategie**: `trend_pullback_v1.1` (Image `backtesting:0.1.2`) — Spezifikation und Parameter: `trend-pullback-v1.1-spec.md`.
|
||||
|
||||
---
|
||||
## Phase 10e — Intrabar Execution + Gap Execution Realism (23.08.2026)
|
||||
|
||||
Versionierung der Gap-/Intrabar-Semantik (KEIN separates `gap_execution_version`):
|
||||
- Träger: `intrabar_policy` + `intrabar_policy_version = "intrabar_policy_v1"` im
|
||||
`ExecutionContext` (`shared/historical/execution_context.py`). Dadurch ist die neue
|
||||
Semantik eindeutig versioniert und ändert den V2-`run_hash` bei Policy-Wechsel
|
||||
(PESSIMISTIC ≠ OPTIMISTIC → anderer run_hash).
|
||||
- Legacy-Pfad bleibt unangetastet (PESSIMISTIC-Default, identisches Verhalten).
|
||||
|
||||
Intrabar-Policy (nur V2/BID_ASK): `PESSIMISTIC` | `OPTIMISTIC` | `STOP_FIRST` |
|
||||
`TARGET_FIRST` | `TICK_RESOLUTION` | `UNKNOWN`. **Fail-closed** §18: `UNKNOWN`/
|
||||
`TICK_RESOLUTION`/`BOGUS` → `ValueError`/`NotImplementedError`, kein Auto-Default.
|
||||
|
||||
Gap-Semantik (deterministisch, kein Lookahead):
|
||||
- LONG: Stop/Target/Exit auf **BID**; SHORT auf **ASK**.
|
||||
- Gap-Stop → Fill zum Gap-Open (schlechter, NICHT auf Stop-Level geclampt).
|
||||
- Target-Gap → Fill zum besseren Gap-Open (nicht geclampt auf Target).
|
||||
- Intrabar-Target füllt auf `target` (nicht low/high); Intrabar-Ambiguity (Stop UND
|
||||
Target in einer Bar) per Policy (PESSIMISTIC→Stop, OPTIMISTIC→Target).
|
||||
- Slippage wird adversial auf den tatsächlichen Fill (Trigger ≠ Market ≠ Final Fill
|
||||
bei Gap + Slippage); Commission auf finalem Fill (rate × qty × fill pro Seite).
|
||||
- **Kein Doppelspread/Slippage/Commission** (Vier-Ebenen-Trennung 10b/10c/10d/10e).
|
||||
|
||||
Audit (18 Felder, forensisch rekonstruierbar): `reason`, `intrabar_policy`,
|
||||
`trigger_price`, `market_execution_price`, `exit_fill_price`, `gap_execution`,
|
||||
`gap_open_price`, `execution_model`, `price_basis`, `spread_model`, `slippage_model`,
|
||||
`slippage_value`, `cost_model`, `entry_fee`, `exit_fee`, `gross_pnl`, `net_pnl`
|
||||
(verwendete vorhandene Feldnamen, keine künstlichen).
|
||||
|
||||
Verifikation 23.08.2026:
|
||||
- Unit-Suite `test_phase10e_intrabar_gap.py`: 40/40 grün (Intrabar, Gap, Slippage
|
||||
nach Gap, Commission auf Final Fill, BID/ASK-Seiten, Doppelzählung, Audit-18,
|
||||
pathologische Fälle, 4 numerische PnL-E2E-Beispiele).
|
||||
- Fixture-E2E (`test_phase10e_fixture_e2e.py`): 7/7 grün (A LONG Stop-Gap, B SHORT
|
||||
Stop-Gap, C Intrabar PESSIMISTIC, D Intrabar OPTIMISTIC run_hash-Differenz,
|
||||
E LONG Target-Gap, F SHORT Target-Gap, ENV-RESET); numerische Kette
|
||||
Trigger → Gap-Open → Slippage → Final Fill → Commission → Gross → Net nachgewiesen.
|
||||
- Regression Lauf 1 + Lauf 2: vollständig grün (M12 169 inkl. 10e, Phase5 20,
|
||||
Phase6 14, Phase7 A–J Legacy-Hash `8a5760ae…`, M13 Shared A–J, M13 Compat 5/5).
|
||||
- Legacy Production Smoke 2×: A==B==VORHER (run_hash `bc6e2553…`, data_hash
|
||||
`d76a7549…`, net_pnl 196.586062, 1× LONG target). DB-Wahrheit verifiziert
|
||||
(backtest_run/backtest_trade konsistent). Legacy unverändert durch Phase 10e.
|
||||
- Production Gates M12+M13: `HISTORICAL_DATA_SOURCE`/`ALLOW_FIXTURE_DATA` UNSET.
|
||||
- Deployt: `core/engine.py`, `core/service.py`, `api/schemas.py` (Container-Hashes
|
||||
aktualisiert); `shared/historical/execution_context.py` + `pricing.py` unverändert.
|
||||
|
||||
---
|
||||
## Geändert
|
||||
```
|
||||
Geändert von: Rain Ocampo
|
||||
Datum: 23.08.2026
|
||||
Grund: Modul-12-Doku um Phase 10e (Intrabar Execution + Gap Realism) ergänzt — Versionierung, Intrabar-Policy, Gap-Semantik, Audit-18, Verifikation, Deploy, Smoke, Production Gates.
|
||||
```
|
||||
```
|
||||
Geändert von: Rain Ocampo
|
||||
Datum: 20.08.2026
|
||||
Grund: Modul-12-Backtesting als FREIGEGEBEN markiert (deployt + E2E verifiziert: deterministisch, kein Lookahead, Stop/Target/Gap, LONG/SHORT, run_hash+data_hash).
|
||||
```
|
||||
|
|
|
|||
|
|
@ -1,16 +1,8 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/2c48489e-0fb9-bdbf-8000-6bbba26771fb
|
||||
type: arch
|
||||
role: module
|
||||
representation: canonical
|
||||
state: current
|
||||
derived_from: object/bde121b1-121f-590d-2b54-fedc13b02173
|
||||
|
||||
type: Trading-Modul-System
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
||||
# Modul-13: Optimization
|
||||
|
||||
**Status: FREIGEGEBEN ✅** · Container `Modul-13-Optimization` · Image `optimization:0.1.0`
|
||||
|
|
|
|||
|
|
@ -1,16 +1,8 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/ea077314-9e29-6c7f-1c17-008fa8e720ae
|
||||
type: arch
|
||||
role: module
|
||||
representation: canonical
|
||||
state: current
|
||||
derived_from: object/0a0c9ca4-1cf4-aee1-3072-e1c3e8cfd29e
|
||||
|
||||
type: Trading-Modul-System
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
||||
# Modul-14: Notification-Service
|
||||
|
||||
**Status: FREIGEGEBEN ✅** · Container `Modul-14-Notification` · Image `notification:0.1.0`
|
||||
|
|
|
|||
|
|
@ -1,16 +1,8 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/5f28e1fd-19f4-e5af-1c92-a17ec80b08f0
|
||||
type: design
|
||||
role: design
|
||||
representation: canonical
|
||||
state: current
|
||||
derived_from: object/1d6c2323-a31a-8b50-fcdd-3c9268175a69
|
||||
|
||||
type: Trading-Modul-System
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
||||
# Design-Vorschlag: Modul-15 → Modul-09 Anbindung (Safety-/Trading-Control)
|
||||
|
||||
**Status: NUR DESIGN-VORSCHLAG — keine Implementierung.**
|
||||
|
|
|
|||
|
|
@ -1,16 +1,8 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/42b89fc3-4572-b493-c05e-dfb85a639df2
|
||||
type: arch
|
||||
role: implementation
|
||||
representation: canonical
|
||||
state: current
|
||||
derived_from: object/a3cffdc1-41fd-ab84-c211-7b586f153e8d
|
||||
|
||||
type: Trading-Modul-System
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
||||
# Modul-15 → Modul-09 Control-Anbindung — Implementierung & Live-E2E
|
||||
|
||||
**Status: FREIGEGEBEN / PRODUKTIV VERIFIZIERT** (20.08.2026, Nutzer-Bestätigung).
|
||||
|
|
|
|||
|
|
@ -1,22 +1,14 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/a50e821b-9c56-fe60-7b46-a0b607ef9e00
|
||||
type: arch
|
||||
role: module
|
||||
representation: canonical
|
||||
state: current
|
||||
derived_from: object/c2641bbf-0612-0737-86fd-622d899109ca
|
||||
|
||||
type: Trading-Modul-System
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
||||
# Modul-15: Monitoring-Control
|
||||
|
||||
**Status: FREIGEGEBEN** (20.08.2026) · Container `Modul-15-Monitoring-Control` · Image `monitoring-control:0.1.0`
|
||||
|
||||
## Zweck
|
||||
Zentrale **technische Überwachung und Sicherheits-/Kontrollebene** des Trading-Systems.
|
||||
Zentrale **technische Überwachung und Sicherheits-/Kontrollebene** des Modul-Trading-Systems.
|
||||
Überwacht die Dienste M03–M14 (Health/Readiness/Erreichbarkeit) plus RabbitMQ-Queues/Consumer
|
||||
und PostgreSQL und leitet daraus deterministisch einen **globalen Trading-Status**
|
||||
(`ENABLED` / `PAUSED` / `HALTED` / `UNKNOWN`) ab.
|
||||
|
|
|
|||
|
|
@ -1,11 +1,3 @@
|
|||
---
|
||||
id: object/eb0ef2f8-1ed9-aaf1-1afb-2fdbdaa15c74
|
||||
type: arch
|
||||
role: module
|
||||
representation: canonical
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
---
|
||||
# Modul-16: Paperclip (Agent-Orchestrierungsebene)
|
||||
|
||||
**Status: FREIGEGEBEN (21.08.2026)** · Container `Modul-16-Paperclip` · Image `trading-modules-modul16-paperclip:latest`
|
||||
|
|
|
|||
|
|
@ -1,11 +1,3 @@
|
|||
---
|
||||
id: object/3063a680-c35e-a62c-91b4-f382027930a5
|
||||
type: arch
|
||||
role: module
|
||||
representation: canonical
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
---
|
||||
# Modul-17: Hermes-Agent (autonomer Research-/Analyse-Worker unter Paperclip)
|
||||
|
||||
**Status: FREIGEGEBEN (21.08.2026)** · Container `Modul-17-Hermes-Agent` · Image `trading-modules-modul17-hermes-agent:latest`
|
||||
|
|
|
|||
|
|
@ -1,11 +1,3 @@
|
|||
---
|
||||
id: object/3b41fc80-9d60-e640-a19c-1107114d8495
|
||||
type: arch
|
||||
role: module
|
||||
representation: canonical
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
---
|
||||
# Modul-26-Telegram-Gateway
|
||||
|
||||
**Status:** FREIGEGEBEN (2026-08-21)
|
||||
|
|
|
|||
|
|
@ -1,11 +1,3 @@
|
|||
---
|
||||
id: object/086b7673-c183-6ca2-52fc-6f394107c9cf
|
||||
type: arch
|
||||
role: history
|
||||
representation: canonical
|
||||
state: historical
|
||||
knowledge_schema: 1
|
||||
---
|
||||
# Phase 10a — ExecutionContext-Dataclass + Versionierung + Tests (M12/M13)
|
||||
|
||||
> Autoren: Rain Ocampo (Hermes) | Datum: 2026-08-23
|
||||
|
|
|
|||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/da874dfe-edd4-a6ba-1587-a145ae045ddc
|
||||
type: arch
|
||||
role: history
|
||||
representation: canonical
|
||||
state: historical
|
||||
knowledge_schema: 1
|
||||
type: Note
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/dd4f96c5-6cc8-28e4-c8d4-7069c949fa3f
|
||||
type: arch
|
||||
role: history
|
||||
representation: canonical
|
||||
state: historical
|
||||
knowledge_schema: 1
|
||||
type: Note
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,90 +0,0 @@
|
|||
# Phase 13.5 — Runner Remediation + Control Plane + TrustGate Canonical Path (FORMAL CLOSURE)
|
||||
|
||||
**Status:** FORMALLY CLOSED (Dokumentation/SoT abgeschlossen, Runtime unverändert)
|
||||
**Datum:** 31.08.2026
|
||||
**Autor:** Red Queen (Rain Ocampo / Hermes)
|
||||
**Freigabe:** Christian List (Owner)
|
||||
|
||||
---
|
||||
|
||||
## 1. Finales Verdict
|
||||
|
||||
> **PHASE 13.5 CLOSED — RUNNER REMEDIATION + CONTROL PLANE + TRUSTGATE CANONICAL PATH VERIFIED**
|
||||
|
||||
Phase 13.5 ist formal geschlossen. Die autoritative Closure-Evidence wurde in Red Queens Ausführungs-/Workspace-Kontext erzeugt und dort verifiziert:
|
||||
|
||||
- **Closure Record:** `/opt/data/rq_phase135_FORMAL_CLOSURE.md` (Red-Queen-Workspace-Kontext)
|
||||
- **Authoritative Runner/TrustGate Evidence:** `/opt/data/rq_phase135_runner_trustgate_final/final-evidence.md` (SHA `5538ecf57ce602afaddab58534ccda05ffcdf311ed6f840532d9bad8283ef097`, Red-Queen-Workspace-Kontext)
|
||||
|
||||
---
|
||||
|
||||
## 2. Runtime (unverändert, verifiziert 31.08.2026)
|
||||
|
||||
| Feld | Wert |
|
||||
|---|---|
|
||||
| Container | `513b5b814d8d245db3f4db57d169d24cc9a69def655a2d66de7ffe4460452c7f` |
|
||||
| Image | `sha256:f72b27e72fc6418edde5810090dad62538f2f85903424992691d79cfc69b8884` |
|
||||
| Running / RestartCount | `true` / `0` |
|
||||
| health / ready | `200` / `200` |
|
||||
| Wrapper SHA | `109a5258bc306084f6909b3903294870b356eb3a6b244ad02b3599d4d02e8e1d` |
|
||||
| Commit | `738af54` (NO PUSH) |
|
||||
| Runner SHA (deployt) | `6cca3c55bb559d75be2f58e59c231003902f124a54ada25582970524599f0752` |
|
||||
| repository.py SHA | `8fff892f52a612f464b5be21c66ef7787a47f2d7d049f815ef87837553c2a228` |
|
||||
|
||||
---
|
||||
|
||||
## 3. Kernbefunde (verifiziert)
|
||||
|
||||
- **Runner → BackfillOrchestrator:** bewiesen (Thin CLI Facade, kein direkter insert_*).
|
||||
- **TrustGate:** mandatory im produktiven Pfad.
|
||||
- **UNKNOWN / UNTRUSTED / CONDITIONAL / MISSING / EXCEPTION:** FAIL CLOSED.
|
||||
- **Legacy Direct-Write Bypass:** über produktiven Wrapper/Runner unerreichbar.
|
||||
- **TrustGate Harness:** PASS (36/36 + 27/27 + 17/17).
|
||||
- **Service:** healthy.
|
||||
- **Keine produktive Backfill-Ausführung** während Abschlussvalidation.
|
||||
|
||||
---
|
||||
|
||||
## 4. Wichtige Limitation — DB E2E
|
||||
|
||||
**Es wurde KEIN mutierender DB-E2E-Test erzwungen.** Über den verfügbaren Red-Queen-Control-Plane-Zugang war kein sicherer isolierter/read-only DB-Verifikationspfad vorhanden. Gemäß Fail-Closed-Prinzip wurde **§7D** angewendet. Persistence-Invariants wurden statisch + Harness verifiziert.
|
||||
|
||||
**Korrekte Aussage:** *"Canonical Runner/TrustGate path and persistence gating verified statically and via isolated harness; production DB mutation was intentionally not performed."*
|
||||
|
||||
---
|
||||
|
||||
## 5. Deferred Risks (offen, NICHT jetzt ändern)
|
||||
|
||||
- **#1 Legacy Scripts:** `run_backfill.py` / `run_phase11_pilot.py` existieren im Container, rufen `orch.run()` ohne TrustGate — über produktiven Wrapper unerreichbar, aber Code-Präsenz vergrößert Attack-Surface. → spätere Hardening-Mission.
|
||||
- **#2 Dataset-Version TOCTOU:** `get_or_create_dataset_version` ohne UNIQUE auf Hash → potenzielle doppelte Dataset-Version-Metadaten bei parallelen identischen Requests. → spätere Repair-/Hardening-Mission.
|
||||
- **Deferred Validation:** Vor echtem produktivem Backfill erforderlich: vertrauenswürdige Data Source, ausreichende Provenance, TrustGate TRUSTED, kontrollierter kleiner Pilot, DB Before/After, Qualitätsprüfung. **Der frühere Dukascopy-Pilot ist NICHT als trusted Dataset freigegeben.**
|
||||
|
||||
---
|
||||
|
||||
## 6. Closure Scope
|
||||
|
||||
- CONTROL-PLANE / RUNNER REMEDIATION: **CLOSED**
|
||||
- TRUSTGATE CANONICAL PATH: **VERIFIED**
|
||||
- PRODUCTION HISTORICAL DATA QUALITY: **NICHT automatisch freigegeben**
|
||||
- PRODUCTION BACKFILL: **NICHT freigegeben**
|
||||
- PHASE 14: **NICHT automatisch gestartet**
|
||||
|
||||
---
|
||||
|
||||
## 7. Offene Punkte (nach Closure)
|
||||
|
||||
- **A.** Trusted Historical Data Source / Provenance
|
||||
- **B.** Controlled Trusted Provider Pilot
|
||||
- **C.** Dataset-Version TOCTOU Hardening
|
||||
- **D.** Legacy Entry-Point Hardening
|
||||
- **E.** Production Backfill
|
||||
- **F.** Phase 14
|
||||
|
||||
---
|
||||
|
||||
## 8. Verweis auf autoritative Evidence
|
||||
|
||||
Vollständige Details (Evidence-SHAs, Mutation Accounting, Grenzen) siehe (Red-Queen-Workspace-Kontext):
|
||||
|
||||
- `/opt/data/rq_phase135_FORMAL_CLOSURE.md`
|
||||
- `/opt/data/rq_phase135_runner_trustgate_final/final-evidence.md`
|
||||
|
|
@ -1,193 +0,0 @@
|
|||
# Phase 13.6 — Hardening Closure (FORMAL CLOSURE)
|
||||
|
||||
**Status:** CLOSED / DEPLOYED / VERIFIED
|
||||
**Datum:** 01.09.2026
|
||||
**Autor:** Rain Ocampo (Hermes) / Red Queen
|
||||
**Freigabe:** Christian List (Owner)
|
||||
|
||||
---
|
||||
|
||||
## 1. Scope
|
||||
|
||||
Phase 13.6 umfasst die produktive Härtung des Historical-Service:
|
||||
|
||||
- **Korrigierter TOCTOU-Befund** (Dataset-Version)
|
||||
- **Dataset-Version-Repair** (repository.py)
|
||||
- **TrustGate Defense-in-Depth**
|
||||
- **Legacy Entry-Point Hardening**
|
||||
- **Schema-Entscheidung** (keine Migration nötig)
|
||||
|
||||
Phase 13.6 ist bereits erfolgreich produktiv deployed und verifiziert. Diese Mission persistiert ausschließlich die verifizierte Abschlussdokumentation in der autoritativen Git-Historie. **KEINE Trading-/Runtime-Änderung.**
|
||||
|
||||
---
|
||||
|
||||
## 2. Korrigierter TOCTOU-Befund
|
||||
|
||||
**Ursprünglicher Befund (Phase 13.5, Deferred Risk #2):** `get_or_create_dataset_version` ohne UNIQUE auf Hash → potenzielle doppelte Dataset-Version-Metadaten bei parallelen identischen Requests.
|
||||
|
||||
**Korrigierte Analyse:** `dataset_version_hash` hatte bereits UNIQUE-Schutz. Der tatsächliche Defekt war ein **SELECT→INSERT-Concurrency-Race**, das eine `UniqueViolation` werfen konnte (nicht doppelte Zeilen).
|
||||
|
||||
---
|
||||
|
||||
## 3. Dataset-Version-Repair (repository.py)
|
||||
|
||||
Repair in `app/persistence/repository.py` (Image-SHA `33cca6421a595c68a12bf6b25eb02b47edf868fb27501330e356de2c0c3a7bd6`):
|
||||
|
||||
- `pg_advisory_xact_lock(dataset_id)` — serialisiert konkurrierende Requests pro Dataset
|
||||
- `ON CONFLICT(dataset_version_hash) DO NOTHING RETURNING` — idempotenter Insert
|
||||
- kanonischer Re-SELECT nach Konflikt
|
||||
- deterministische kanonische ID
|
||||
- fail-closed bei ambigem/Fehlerzustand
|
||||
|
||||
**Concurrency-Validierung:**
|
||||
- Baseline Gold x10: 3 OK, 7 UniqueViolation, 1 row
|
||||
- Patched x10: 10/10 OK, gleiche kanonische ID, 1 row
|
||||
|
||||
---
|
||||
|
||||
## 4. TrustGate Defense-in-Depth
|
||||
|
||||
- Fehlender TrustGate auf produktionsfähigem Pfad: **FAIL CLOSED**
|
||||
- UNKNOWN / UNTRUSTED / CONDITIONAL (Default) / MISSING / EXCEPTION: **BLOCK**
|
||||
- Nur TRUSTED (oder CONDITIONAL mit expliziter Policy) persistierbar
|
||||
|
||||
---
|
||||
|
||||
## 5. Legacy Entry-Point Hardening
|
||||
|
||||
- `run_backfill.py`: HARD-FAIL (non-zero exit, kein Provider/Persistence-Continuation)
|
||||
- `run_phase11_pilot.py`: HARD-FAIL (non-zero exit, kein Provider/Persistence-Continuation)
|
||||
|
||||
---
|
||||
|
||||
## 6. Schema-Entscheidung
|
||||
|
||||
**SCHEMA MIGRATION = NOT REQUIRED.** Bestehendes `UNIQUE(dataset_version_hash)` bleibt erhalten.
|
||||
|
||||
---
|
||||
|
||||
## 7. Production DB Precheck
|
||||
|
||||
- UNIQUE existiert: **JA**
|
||||
- duplicates: **0**
|
||||
- NULL anomalies: **0**
|
||||
- FK anomalies: **0**
|
||||
- dataset_version: **336**
|
||||
|
||||
---
|
||||
|
||||
## 8. Candidate Image / Build Provenance
|
||||
|
||||
| Feld | Wert |
|
||||
|---|---|
|
||||
| Image ID | `sha256:21598a3597cfd53c4bc22c01cb7f7874259878ecd7608528ccd6327305ea07fc` |
|
||||
| Tag | `historical-v2-historical-service:candidate-phase13-6-hardening` |
|
||||
| Build Input SHA | `f45e4885…` (nur gekürzt in Evidence verfügbar) |
|
||||
| Archive SHA | `92a9982a212e1f7576305aa765f3c0163812a486f0afb64cb1dd58ba5b55f0c5` |
|
||||
| Pin | `rq-historical-pin-phase13-6-hardening` |
|
||||
| Status | **VERIFIED + ARCHIVED + PINNED = READY** |
|
||||
|
||||
---
|
||||
|
||||
## 9. Production Deployment
|
||||
|
||||
| Feld | Wert |
|
||||
|---|---|
|
||||
| Source | Phase-13.5 Remediation `sha256:f72b27e72fc6418edde5810090dad62538f2f85903424992691d79cfc69b8884` |
|
||||
| Target | Phase-13.6 `sha256:21598a3597cfd53c4bc22c01cb7f7874259878ecd7608528ccd6327305ea07fc` |
|
||||
| Real Recreate Count | **1** |
|
||||
| Exit Code | **0** |
|
||||
| Container (nach) | `d274568ae85173b408531315e7a51aa9fd54b0587931d8c662548736b154313b` |
|
||||
| running / RestartCount | `true` / `0` |
|
||||
| health / ready | `200` / `200` |
|
||||
|
||||
---
|
||||
|
||||
## 10. Target Manifest
|
||||
|
||||
- **58** Gold/unchanged matches
|
||||
- **5** autorisierte Divergenzen:
|
||||
1. `run_backfill_year.py` — bestehende Phase-13.5-Autorisierung (Runner `6cca3c55bb559d75be2f58e59c231003902f124a54ada25582970524599f0752`)
|
||||
2. `repository.py` — `33cca6421a595c68a12bf6b25eb02b47edf868fb27501330e356de2c0c3a7bd6`
|
||||
3. `backfill.py` — `fb93ee1f09b3708eae155b12df1b8a1103ea7c0a4a6aea70c476fd544d58ac0e`
|
||||
4. `run_backfill.py` — `79ee38fd51cf3f92550997cab2ffd6700468d4d7b3921ca97bbc7a38d6a8f383`
|
||||
5. `run_phase11_pilot.py` — `34e345a951a92eecfa8418ae078e30d9fef24c5ef2043dc0b194b064f84cf551`
|
||||
- unauthorized: **0**
|
||||
- missing: **0**
|
||||
- extra: **0**
|
||||
|
||||
---
|
||||
|
||||
## 11. Rollback Control Plane
|
||||
|
||||
- Historischer Gold-Anker `1f3ba210…` ist unverfügbar (Coolify-GC, kein SHA-verifiziertes docker-save-Archiv).
|
||||
- Bewiesen: `1f3ba210…` war ein **stale/wrong-target pre-gate** für den Remediation-Rollback (SOURCE-state/historische Referenz, NICHT Rollback-TARGET-Requirement).
|
||||
- **Kein Fake/Retag/Rebuild** von `1f3ba210…` durchgeführt.
|
||||
- Primary Rollback zielt/verifiziert jetzt exakt auf den Phase-13.5-Remediation-Target.
|
||||
- Direct no-op bleibt **FAIL CLOSED**.
|
||||
- Primary Rollback dry-run: **PASS**
|
||||
- Secondary Recovery Rollback dry-run: **PASS**
|
||||
|
||||
---
|
||||
|
||||
## 12. Production DB Integrity
|
||||
|
||||
- dataset_version: **336 → 336**
|
||||
- duplicates: **0 → 0**
|
||||
- NULL anomalies: **0 → 0**
|
||||
- FK anomalies: **0**
|
||||
- **Keine produktiven DB-Writes durch den Deployment.**
|
||||
|
||||
---
|
||||
|
||||
## 13. Mutation Accounting
|
||||
|
||||
| Mutation | Anzahl |
|
||||
|---|---|
|
||||
| Wrapper Target-Class Deploy | 1 |
|
||||
| Rollback Control-Plane Wrapper Patch | 1 |
|
||||
| Production Recreate | 1 |
|
||||
| Production DB Write | 0 |
|
||||
| Schema | 0 |
|
||||
| Backfill | 0 |
|
||||
| Provider | 0 |
|
||||
| Rollback | 0 |
|
||||
| Phase 14 | 0 |
|
||||
|
||||
---
|
||||
|
||||
## 14. Security Invariants
|
||||
|
||||
- TrustGate fail-closed auf produktionsfähigem Pfad: **PASS**
|
||||
- Legacy Entry-Points (`run_backfill.py`, `run_phase11_pilot.py`): **HARD-FAIL**
|
||||
- Kein Runner/Backfill aktiv
|
||||
- Keine produktiven DB-Writes
|
||||
|
||||
---
|
||||
|
||||
## 15. Deferred Work (NICHT durch Phase 13.6 autorisiert/gelöst)
|
||||
|
||||
Phase 13.6 autorisiert oder löst NICHT:
|
||||
|
||||
- **kein produktiver Historical Backfill** ausgeführt
|
||||
- **kein Trusted Historical Provider** ausgewählt
|
||||
- **kein Trusted Provider Pilot** ausgeführt
|
||||
- **keine Provider/Provenance-Acceptance** abgeschlossen
|
||||
- **kein Phase 14**
|
||||
- **keine Live-Capital/Trading-Aktivierung**
|
||||
|
||||
Der nächste große Workstream bleibt separat.
|
||||
|
||||
---
|
||||
|
||||
## 16. Final Closure Statement
|
||||
|
||||
> **PHASE 13.6 CLOSED — DEPLOYED — VERIFIED — DOCUMENTED — FAST-FORWARD PUSHED — REMOTE SOT PERSISTED — RUNTIME UNCHANGED**
|
||||
|
||||
Phase 13.6 ist formal geschlossen. Die autoritative Closure-Evidence wurde in Red-Queen-/Rain-Ocampo-Ausführungskontext erzeugt und verifiziert:
|
||||
|
||||
- `/opt/data/rq_phase136_repair/` (Primary Rollback Control-Plane Repair)
|
||||
- `/opt/data/rq_phase136_owner_verify/` (Owner Candidate Verification)
|
||||
- `/opt/data/rq_phase136_deployment/` (Controlled Deployment)
|
||||
|
||||
**Git-Baseline vor dieser Closure:** `51c03f7ade49db263f4908b087418ec92979e4e6` (`phase13.5: formally close runner trustgate remediation`).
|
||||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/ca9b3551-53f8-408b-8697-9a21fa3a24d8
|
||||
type: arch
|
||||
role: history
|
||||
representation: canonical
|
||||
state: historical
|
||||
knowledge_schema: 1
|
||||
type: Note
|
||||
tags: [trading, mt5, architecture, historical-v2, m12, m13, phase9, readiness-audit, phase2]
|
||||
created: 2026-08-22
|
||||
---
|
||||
|
|
|
|||
|
|
@ -1,17 +1,9 @@
|
|||
---
|
||||
id: object/ebe08190-5cd5-a649-d6ab-38cb8f50f8b4
|
||||
type: arch
|
||||
role: reference
|
||||
representation: canonical
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
derived_from: object/09fd796f-51b6-4104-bc69-5f9dc79db0d6
|
||||
source_system: forgejo
|
||||
source_path: ports-reference.md
|
||||
type: Trading-Modul-System
|
||||
_organized: true
|
||||
---
|
||||
|
||||
# Ports & Service-Referenz — Trading-System VPS
|
||||
# Ports & Service-Referenz — Modul-Trading-System VPS
|
||||
|
||||
> Stand: 20.08.2026 · VPS: `187.124.31.123` · SSH: Public-Key-Auth (nur)
|
||||
|
||||
|
|
|
|||
|
|
@ -1,16 +1,8 @@
|
|||
---
|
||||
knowledge_schema: 1
|
||||
id: object/31671ffa-fc39-ae9c-9449-e9322c9bb7be
|
||||
type: design
|
||||
role: design
|
||||
representation: canonical
|
||||
state: current
|
||||
derived_from: object/91c7786c-cffa-1197-0837-e55a87cdc161
|
||||
|
||||
type: Trading-Modul-System
|
||||
_organized: true
|
||||
---
|
||||
|
||||
|
||||
# trend_pullback_v1.1 — Spezifikation
|
||||
|
||||
> Status: **IMPLEMENTIERT + DEPLOYT** (`backtesting:0.1.2`, Modul-12), V1-Kompatibilitätsmodus bitgenau.
|
||||
|
|
|
|||
|
|
@ -1,173 +0,0 @@
|
|||
# Wave 1 Provenance — Production Deployment Closure (FORMAL CLOSURE)
|
||||
|
||||
**Status:** CLOSED / DEPLOYED / VERIFIED / DOCUMENTED
|
||||
**Datum:** 04.09.2026
|
||||
**Autor:** Red Queen (Rain Ocampo / Hermes)
|
||||
**Freigabe:** Christian List (Owner)
|
||||
|
||||
---
|
||||
|
||||
## 1. Scope
|
||||
|
||||
Diese Mission persistiert ausschließlich die verifizierte Abschlussdokumentation des **Wave1-Provenance-Production-Deployments** in der autoritativen Git-/SoT-Historie. **KEINE Trading-/Runtime-/DB-Änderung.**
|
||||
|
||||
Wave1 Provenance ist bereits erfolgreich produktiv deployed und unabhängig verifiziert. Diese Closure dokumentiert den finalen, verifizierten Zustand.
|
||||
|
||||
---
|
||||
|
||||
## 2. Deployment Chain (exakt)
|
||||
|
||||
| Schritt | Wert |
|
||||
|---|---|
|
||||
| Backup | **1** |
|
||||
| Wave1 Migration | **1** |
|
||||
| WaveD Migration | **0** (ALREADY PRESENT / VERIFIED / SKIPPED) |
|
||||
| Production Recreate | **1** |
|
||||
| Rollback | **0** |
|
||||
| Restore | **0** |
|
||||
| Provider | **0** |
|
||||
| Backfill | **0** |
|
||||
| Git während Deployment | **0** |
|
||||
| Phase14 | **0** |
|
||||
|
||||
---
|
||||
|
||||
## 3. Wave1 Identity
|
||||
|
||||
| Objekt | SHA / Wert |
|
||||
|---|---|
|
||||
| Wave1 Migration | `6983526da3ca51fd2ececffbef0ccbd66b048a3d11c228000011aaba3e1d6bc5` |
|
||||
| Candidate Image | `sha256:c70d47677c455963553280c7594d4005b3dcf1650ee99a49708997a624dee42f` |
|
||||
| Candidate Archive | `b3c72fb9ed78f0fb2e57292a90686fd9d49116953b23c976750ff4ffb8628c99` |
|
||||
| Application Manifest | `41b9032b694cdf958d7aefb06870fd181a284568447e06396c48cc915e19b06b` |
|
||||
| WaveD Historical Contract | `7aa4e2a10c98fa327e000a49f6a497efb5814c8808dba429dfebfc90c544a02b` |
|
||||
|
||||
**Wichtig:** Die WaveD-Migration (`7aa4e2a1…`) wurde in diesem Deployment **NICHT ausgeführt**. Der WaveD-Runtime-Schema-Zustand war bereits vor dem Deployment vorhanden und wurde **PRESERVED** (nicht verändert).
|
||||
|
||||
---
|
||||
|
||||
## 4. Production State (final, verifiziert)
|
||||
|
||||
| Feld | Wert |
|
||||
|---|---|
|
||||
| Container | `9eb3480e4395a549cb6e7c063cab7f812d90408bdf7101aaa7168f8f1221c7be` |
|
||||
| Image | `sha256:c70d47677c455963553280c7594d4005b3dcf1650ee99a49708997a624dee42f` |
|
||||
| running / RestartCount | `true` / `0` |
|
||||
| health / ready | `200` / `200` |
|
||||
| PostgreSQL | `16.13` |
|
||||
| database / schema | `historical` / `public` |
|
||||
|
||||
### Final Schema Fingerprint
|
||||
|
||||
| Tabelle | Spalten |
|
||||
|---|---|
|
||||
| historical_bar | 41 |
|
||||
| derived_bar | 35 |
|
||||
| data_ingestion_run | 29 |
|
||||
| dataset_version | 23 |
|
||||
| ingestion_provenance | PRESENT / 41 cols / 0 rows |
|
||||
|
||||
### Canonical Fields
|
||||
|
||||
| Feld | Typ |
|
||||
|---|---|
|
||||
| last_successful_chunk | `timestamp with time zone` (timestamptz) |
|
||||
| interrupt_reason | `text` / preserved |
|
||||
| technical_quality | `text` / nullable / default `'VALID'` |
|
||||
|
||||
### Data Invariants
|
||||
|
||||
| Check | Wert |
|
||||
|---|---|
|
||||
| dataset_version rows | 336 |
|
||||
| duplicate dataset_version_hash | 0 |
|
||||
| null identity | 0 |
|
||||
| FK orphans | 0 |
|
||||
| historical_bar rows | 80,680 |
|
||||
| derived_bar rows | 24,948 |
|
||||
| RUNNING ingestion | 0 |
|
||||
| Provider Calls | 0 |
|
||||
| Backfill | 0 |
|
||||
| Pilot | 0 |
|
||||
|
||||
---
|
||||
|
||||
## 5. Backup / Rollback
|
||||
|
||||
| Objekt | Wert |
|
||||
|---|---|
|
||||
| Backup Path | `/opt/historical-v2/backups/pre_wave1_20260904_164812/` |
|
||||
| Full Dump SHA | `0f9c9bbc32fe24fb2fbb622b69356794ef0278c5f71032870323a7262d94b1e4` |
|
||||
| Schema Dump SHA | `71294b98483ee76cec2096505293a4a6cfd9ddf2565defdf032f365fb188861e` |
|
||||
| Compose SHA | `3c0cc9da53260b01f977d051fe3cb67ddbe29f62b3e07f5279942cb795908a52` |
|
||||
| Primary App Rollback | `sha256:21598a3597cfd53c4bc22c01cb7f7874259878ecd7608528ccd6327305ea07fc` |
|
||||
| Recovery Gold | `sha256:b93b0fe141d08d29a09861f5b4728e7816cfcf3e0c3ea36897ea826a9491eab9` |
|
||||
|
||||
---
|
||||
|
||||
## 6. Independent Checker Result
|
||||
|
||||
**RED QUEEN Independent Post-Deploy Review: VERDICT A**
|
||||
|
||||
- Wave1 Production Deployment Verified
|
||||
- Runtime healthy
|
||||
- Wave1 schema exact
|
||||
- Existing WaveD contract preserved
|
||||
- Data invariants intact
|
||||
- Backup/Restore verified
|
||||
- Primary rollback verified
|
||||
- No Provider/Backfill activity
|
||||
|
||||
---
|
||||
|
||||
## 7. Mutation Accounting
|
||||
|
||||
| Mutation | Anzahl |
|
||||
|---|---|
|
||||
| Backup | 1 |
|
||||
| Wave1 Migration | 1 |
|
||||
| WaveD Migration | 0 |
|
||||
| Production Recreate | 1 |
|
||||
| Rollback | 0 |
|
||||
| Restore | 0 |
|
||||
| Provider | 0 |
|
||||
| Backfill | 0 |
|
||||
| Git während Deployment | 0 |
|
||||
| Phase14 | 0 |
|
||||
|
||||
---
|
||||
|
||||
## 8. Open Contract Debts (NICHT geschlossen)
|
||||
|
||||
Diese Debts bleiben **OPEN** und sichtbar. Sie invalidieren den aktuellen Production-Zustand **NICHT**:
|
||||
|
||||
- **A)** Historical V2 DDL provenance missing
|
||||
- **B)** `interrupt_reason` origin unknown
|
||||
- **C)** Historical WaveD migration artifact definiert `last_successful_chunk` als `text`, während der Owner-Canonical-Current-Production-Contract `timestamptz` ist.
|
||||
|
||||
---
|
||||
|
||||
## 9. Explicitly Excluded Work (NICHT autorisiert)
|
||||
|
||||
Diese Closure autorisiert oder löst NICHT:
|
||||
|
||||
- **kein Provider Pilot**
|
||||
- **kein Historical Backfill**
|
||||
- **kein Phase14**
|
||||
- **keine weitere Runtime-Mutation**
|
||||
- **kein weiterer Git-Commit / Git-Push**
|
||||
|
||||
Der nächste Workstream erfordert separate Owner-Autorisierung.
|
||||
|
||||
---
|
||||
|
||||
## 10. Final Closure Statement
|
||||
|
||||
> **WAVE1 PROVENANCE CLOSED — DEPLOYED — VERIFIED — DOCUMENTED — FAST-FORWARD PUSHED — REMOTE SOT PERSISTED — RUNTIME UNCHANGED**
|
||||
|
||||
Wave1 Provenance ist formal geschlossen. Die autoritative Closure-Evidence wurde in Red-Queen-/Rain-Ocampo-Ausführungskontext erzeugt und verifiziert:
|
||||
|
||||
- `/opt/data/rq_wave1_postdeploy_production_review_resume/` (Independent Post-Deploy Review, Verdict A)
|
||||
- `/opt/data/rq_wave1_closure_git_authority_recovery/` (Git-Authority-Recovery / diese Closure)
|
||||
|
||||
**Git-Baseline vor dieser Closure:** `36916a073d57d9d236c87a23892ad072e9bedd65` (`phase13.6: formally close hardening repair`).
|
||||
|
|
@ -1,10 +1,5 @@
|
|||
---
|
||||
id: object/8b362849-2a5e-4a07-42f0-557a58e1cf14
|
||||
type: arch
|
||||
role: index
|
||||
representation: standalone
|
||||
state: current
|
||||
knowledge_schema: 1
|
||||
type: Bereich
|
||||
title: Trading Systeme
|
||||
tags: [trading, mt5, home]
|
||||
created: 2026-08-20
|
||||
|
|
|
|||
|
|
@ -1,76 +0,0 @@
|
|||
# NOTION PROJECT COMMAND CENTER
|
||||
|
||||
> **Mission 006 — REAL MISSION 006 NOTION PROJECT COMMAND CENTER, MASTER TODO, CONTEXT RECOVERY & AGENT GOVERNANCE v1**
|
||||
> Status: **COMPLETE** (2026-08-25) · Fresh Checker: **PASS** (19/19, unabhängig)
|
||||
|
||||
## Zweck
|
||||
|
||||
Der **PROJECT COMMAND CENTER** ist der persistente operative Leitfaden für Christians gesamtes KI-/Agenten-Projekt. Ein neuer ChatGPT-Chat (ohne Vorkenntnis) muss via „Lies meinen aktuellen Projektstand aus dem Command Center" alles verstehen: Was, Warum, Teilprojekte, abgeschlossen, aktiv, nächster Schritt, danach, verschoben, Originale, Verifikationspflicht vor Mutation.
|
||||
|
||||
**Rollen-Trennung (verbindlich):**
|
||||
- **Notion Role A = SAFETY BRAIN** — Backup/Recovery (Index-DB `3c793a20-cfac-8059-ae2b-000bbfaa5a60`)
|
||||
- **Notion Role B = PROJECT COMMAND CENTER** — operativ/Kontext-Recovery (diese Struktur)
|
||||
|
||||
Notion ist **NICHT** die globale Source of Truth.
|
||||
|
||||
## Source-of-Truth-Modell
|
||||
|
||||
| System | Rolle |
|
||||
|---|---|
|
||||
| **FORGEJO** | Technische/versionierte Source of Truth |
|
||||
| **TOLARIA** | Vernetztes Second Brain |
|
||||
| **NOTION SAFETY BRAIN** | Backup/Recovery |
|
||||
| **NOTION PROJECT COMMAND CENTER** | Status/Roadmap/To-do/Handoff/Entscheidungen/Governance/Referenzen/Context Anchors |
|
||||
| **RUNTIME** | Wahrheit über laufende Systeme |
|
||||
|
||||
## Struktur (14 Unterseiten + MASTER TODO-DB)
|
||||
|
||||
Command Center Page: `3c793a20-cfac-8117-ba35-e9152628dc30`
|
||||
|
||||
| Seite | Inhalt |
|
||||
|---|---|
|
||||
| 00 — START HERE | Einstieg: System Purpose, aktueller Zustand, nächster Schritt |
|
||||
| 01 — MASTER ROADMAP | Strategisches Dreieck (A–D) |
|
||||
| 02 — CURRENT PROJECT STATE | Aktueller Projektzustand |
|
||||
| 03 — MASTER TODO | Link zur MASTER TODO-DB |
|
||||
| 04 — DECISION LOG | Entscheidungsprotokoll |
|
||||
| 05 — SYSTEM & AGENT CAPABILITIES | Capabilities vs. aktive Systeme |
|
||||
| 06 — SOURCE OF TRUTH MAP | SoT-Zuordnung |
|
||||
| 07 — AGENT GOVERNANCE | READ-BEFORE-WRITE, Write Classification |
|
||||
| 08 — PROMPT LIBRARY | Prompt-Templates (DATA, DO NOT EXECUTE) |
|
||||
| 09 — OPEN ISSUES & RISKS | Offene Punkte & Risiken |
|
||||
| 10 — MISSION HISTORY | Missionshistorie |
|
||||
| 11 — HANDOFF FOR NEW CHAT | Handoff für neuen Chat |
|
||||
| 12 — VERIFICATION REQUESTS | Verifikationsanfragen |
|
||||
| 13 — REFERENCES & CONTEXT ANCHORS | Referenzen & Context Anchors |
|
||||
|
||||
## MASTER TODO-DB
|
||||
|
||||
- **data_source:** `23d2b57b-a303-41d8-a0da-4f3aaa1f88b8`
|
||||
- **Rows:** 24 Tasks (12 DONE, 12 offen)
|
||||
- **Pflichtfelder (24):** TASK_ID, TITLE, PROJECT, ROADMAP_BLOCK, STATUS, PRIORITY, OBJECTIVE, WHY, CURRENT_STATE, WHAT_HAS_ALREADY_BEEN_DONE, EXPECTED_NEXT_STEP, SUCCESS_CRITERIA, DEPENDENCIES, BLOCKERS, SOURCE_SYSTEM, SOURCE_REFERENCE, SOURCE_VERSION, CONTEXT_ANCHORS, EXTERNAL_REFERENCES, VERIFICATION_REQUIRED, PROMPT_READY, OWNER, LAST_VERIFIED, NOTES
|
||||
- **Roadmap-Blöcke:** A (Foundation), B (Command Center), C (TOLARIA Second Brain), D (HERMES/RQ Autonomisierung)
|
||||
|
||||
## Recovery-Tests (4/4 PASS)
|
||||
|
||||
1. **Context Recovery** — neuer Chat versteht Gesamtprojekt, Stand, nächsten Schritt, Quellen, Verifikationspflicht
|
||||
2. **Todo Recovery** — Tasks selbsterklärend (WHY, CURRENT_STATE, NEXT, SOURCE, SUCCESS)
|
||||
3. **Agent Governance** — Rollen/SoT/Secrets/STOPP klar
|
||||
4. **Roadmap Recovery** — JETZT (Command Center) → DANACH (Tolaria) → DANACH (Hermes-Autonomie)
|
||||
|
||||
## Fresh Checker
|
||||
|
||||
- **Builder-Checker:** 26/26 PASS (`/opt/data/m006_checker_result.json`)
|
||||
- **Unabhängiger Fresh Checker:** 19/19 PASS (`/opt/data/m006_freshchecker_result.json`, Delegation `deleg_1a2520cf`)
|
||||
|
||||
## Sicherheit
|
||||
|
||||
- Keine Secrets in Notion/Repo/Chat — nur Name + EXISTS/LENGTH
|
||||
- Prompt Library: DATA / DO NOT EXECUTE (keine ausführbaren Prompts)
|
||||
- Keine Automation aktiv (statisch OFF dokumentiert)
|
||||
- READ-BEFORE-WRITE vor jeder Mutation
|
||||
|
||||
## Nächste Schritte (NICHT automatisch)
|
||||
|
||||
- **Tolaria v1** (Block C) — auf Christian warten
|
||||
- **Hermes/RQ-Autonomisierung** (Block D) — auf Christian warten
|
||||
|
|
@ -1,54 +0,0 @@
|
|||
# NOTION SAFETY BRAIN — Access & Architecture (Mission 003)
|
||||
|
||||
Datum: 2026-08-25 · Red Queen · Status: **DISCOVERY + ARCHITECTURE (READ-ONLY)**
|
||||
|
||||
> Notion als **externer Safety Brain / Disaster-Recovery-Wissensspeicher** für
|
||||
> **zwei getrennte Systeme**: Forgejo (technische Source of Truth) und Tolaria
|
||||
> (operatives Second Brain). **Notion ist KEINE neue Source of Truth.**
|
||||
|
||||
---
|
||||
|
||||
## 1. Architekturprinzip
|
||||
|
||||
```
|
||||
FORGEJO ────────────► NOTION SAFETY BRAIN
|
||||
TOLARIA ────────────► NOTION SAFETY BRAIN
|
||||
|
||||
NICHT (kein Rückfluss):
|
||||
NOTION ────────────► FORGEJO / TOLARIA
|
||||
```
|
||||
|
||||
- **ONE-WAY** Datenfluss: Quelle → Notion.
|
||||
- Ein Fehler / falsche Bearbeitung / Löschung in Notion darf **niemals** Forgejo oder Tolaria verändern.
|
||||
- **Kein** bidirektionaler Sync. **Kein** automatischer Rück-Sync.
|
||||
|
||||
## 2. Rollen
|
||||
|
||||
| System | Rolle |
|
||||
|---|---|
|
||||
| **FORGEJO** | technische/versionierte Source of Truth |
|
||||
| **TOLARIA** | operatives Second Brain / Knowledge System |
|
||||
| **NOTION** | externer Safety Brain / Knowledge Backup / Disaster-Recovery View |
|
||||
|
||||
> Notion ist ausdrücklich **KEINE Source of Truth** — sondern ein abgeleiteter,
|
||||
> externer Sicherungsspeicher.
|
||||
|
||||
## 3. Ausgangslage (EVIDENCE)
|
||||
- Christians Notion wurde **bewusst geleert** (vorher unbrauchbare Inhalte) → saubere Struktur von Grund auf planbar.
|
||||
- Rain und Alice besitzen bereits funktionierenden Notion-Zugriff.
|
||||
- **Red Queen hat aktuell KEINEN eigenen Notion-Zugriff** (kein `NOTION_API_KEY`, kein `ntn`, keine Config) — verifiziert via read-only Discovery.
|
||||
|
||||
## 4. Zugangskonzept — Red Queen (MINIMUM NECESSARY PRIVILEGE)
|
||||
- Red Queen erhält einen **eigenen, sauber getrennten, minimal privilegierten** Notion-Zugriff.
|
||||
- **Kein Credential-Transfer** von Rain/Alice. **Keine** fremden Tokens kopieren.
|
||||
- **Keine Owner-/Admin-Rechte** — nur der Scope, den der Safety-Brain-Write benötigt.
|
||||
- Empfohlener Weg (Implementierung erst nach Freigabe):
|
||||
- Eigene Notion-Integration anlegen (Read+Insert) mit dediziertem Token.
|
||||
- Token in `${HERMES_HOME}/.env` als `NOTION_API_KEY` (nie ausgeben).
|
||||
- Nur die nötigen Safety-Brain-Seiten/DBs mit der Integration teilen (Seiten-`...` → Connect to).
|
||||
- **Keine Secrets in den Artefakten**; Token nur ephemer nutzen.
|
||||
|
||||
## 5. Kein-Cloud-Write in dieser Mission
|
||||
- Diese Mission ist **READ-ONLY** (Discovery + Architecture).
|
||||
- Kein Anlegen von Notion-Seiten, keine Datenbank, kein Schreiben, kein Sync, kein Cron.
|
||||
- Produktive Daten / Credentials / Berechtigungen bleiben unverändert.
|
||||
|
|
@ -1,34 +0,0 @@
|
|||
# NOTION SAFETY BRAIN — BACKUP MANIFEST (Mission 003)
|
||||
|
||||
Datum: 2026-08-25 · Red Queen · Status: **DESIGN (nicht implementiert)**
|
||||
|
||||
> Ein Backup-Run erzeugt ein Manifest, das Lauf-Ergebnis und Verifikation abbildet.
|
||||
> **Ein Backup ist NICHT allein deshalb erfolgreich, weil ein API-Write HTTP 200 lieferte.**
|
||||
|
||||
## Manifest-Felder
|
||||
```
|
||||
BACKUP_RUN_ID
|
||||
TIMESTAMP
|
||||
FORGEJO_HEAD
|
||||
TOLARIA_HEAD
|
||||
FORGEJO_OBJECTS_EXPECTED
|
||||
FORGEJO_OBJECTS_BACKED_UP
|
||||
TOLARIA_OBJECTS_EXPECTED
|
||||
TOLARIA_OBJECTS_BACKED_UP
|
||||
SKIPPED_OBJECTS
|
||||
FAILED_OBJECTS
|
||||
SECRET_BLOCKED_OBJECTS
|
||||
HASH_STATUS
|
||||
VERIFY_STATUS
|
||||
RESULT
|
||||
```
|
||||
|
||||
- `HASH_STATUS` = `all_match` | `mismatch` (alle Source-Hashes vs. Notion-read-back).
|
||||
- `VERIFY_STATUS` = `verified` | `unverified`.
|
||||
- `RESULT` = `BACKUP VERIFIED` nur wenn Write + Read-back + Compare + Hash + Count + Logs ok.
|
||||
|
||||
## Registry-Zuordnung
|
||||
Manifest-Zeile wird in Notion-DB **05 Backup Registry** je RUN_ID abgelegt (siehe RESTORE_MODEL.md).
|
||||
|
||||
## Verify-Prinzip (WRITE ≠ SUCCESS)
|
||||
1. Write → 2. Read-back → 3. Compare → 4. Hash/Version-Check → 5. Count-Match → 6. Error/Skip-Log → **BACKUP VERIFIED**.
|
||||
|
|
@ -1,45 +0,0 @@
|
|||
# NOTION SAFETY BRAIN — BUILD PLAN (Mission 003)
|
||||
|
||||
Datum: 2026-08-25 · Red Queen · Status: **PLAN (nicht implementiert)**
|
||||
|
||||
> Konkretes, gestuftes Umsetzungsmodell. Jede Phase erst nach **ausdrücklicher Freigabe** durch Christian.
|
||||
|
||||
---
|
||||
|
||||
## Phase 0 — Access (Red Queen)
|
||||
- [ ] Eigene Notion-Integration anlegen (Read + Insert), minimal privilegiert.
|
||||
- [ ] `NOTION_API_KEY` in `${HERMES_HOME}/.env` (nie ausgeben, nur EXISTS/LENGTH).
|
||||
- [ ] Nur Safety-Brain-Seiten/DBs mit der Integration teilen.
|
||||
- [ ] Kein Credential-Transfer von Rain/Alice.
|
||||
|
||||
## Phase 1 — Struktur in Notion anlegen
|
||||
- [ ] Root-Seite **KI-SYSTEM — SAFETY BRAIN**.
|
||||
- [ ] Bereiche 00–10 (siehe NOTION_STRUCTURE.md).
|
||||
- [ ] Datenbank `05 Backup Registry` + `07 Knowledge Index`.
|
||||
|
||||
## Phase 2 — Backup-Engine (lokal)
|
||||
- [ ] Secret-Scan (SECRET_POLICY.md) fest einbauen.
|
||||
- [ ] Provenance-Frontmatter (PROVENANCE_MODEL.md) je Objekt.
|
||||
- [ ] Incremental / Hash-Change-Detection (SCALING_MODEL.md).
|
||||
- [ ] Manifest-Erzeugung (BACKUP_MANIFEST.md).
|
||||
|
||||
## Phase 3 — Verify & Registry
|
||||
- [ ] READ-BACK + COMPARE + Hash-Verify (WRITE ≠ SUCCESS).
|
||||
- [ ] Registry-Zeilen pflegen.
|
||||
|
||||
## Phase 4 — Restore-Modul
|
||||
- [ ] Staging → Validate → Human Review → Approval → Restore (RESTORE_MODEL.md).
|
||||
|
||||
## Phase 5 — Red Queen als Knowledge Backup Steward
|
||||
- Änderungen erkennen · klassifizieren · Secrets blocken · Delta · Backup schreiben ·
|
||||
zurücklesen · verifizieren · Manifest · Status · Fehler melden.
|
||||
|
||||
> **NOCH NICHT AUTOMATISIEREN.** Kein Cron / Daemon / Heartbeat in dieser Mission.
|
||||
> **A5 bleibt AUS.**
|
||||
|
||||
---
|
||||
|
||||
## Wichtig
|
||||
- Diese Mission = **Discovery + Architecture + Checker + Forgejo-Doku**, dann **STOPP**.
|
||||
- Kein Bau von Notion-Seiten, keine Datenbank, kein Upload, kein Sync, kein Cron.
|
||||
- Warte auf Christians ausdrückliche Freigabe vor Phase 1.
|
||||
|
|
@ -1,24 +0,0 @@
|
|||
# NOTION SAFETY BRAIN — CURRENT STATE (Mission 003)
|
||||
|
||||
Datum: 2026-08-25 · Red Queen · Status: **READ-ONLY Discovery-Stand**
|
||||
|
||||
## Notion
|
||||
- **WORKSPACE:** nicht verifizierbar ohne Zugriff (Red Queen hat noch kein Token) — UNKNOWN bis Phase 0.
|
||||
- **AUTH METHOD:** geplant = eigene Notion-Integration (OAuth-token-basiert), min. privilegiert.
|
||||
- **PERMISSIONS (Red Queen):** keine — Zugriff noch NICHT eingerichtet.
|
||||
- **CURRENT NOTION STATE:** bewusst geleert (Christian) → saubere Struktur planbar. Kein Inhalt von RQ verifizierbar (kein Zugang).
|
||||
|
||||
## Red Queen Zugriff
|
||||
- **TOKEN_EXISTS=NO** · `ntn` CLI: nicht installiert · keine Notion-Config/Credential-Datei unter RQ-Volume.
|
||||
|
||||
## Rain / Alice
|
||||
- Haben funktionierenden Notion-Zugriff (Missionsangabe). **Nicht herangezogen** für Credentials.
|
||||
- Diese Mission: **RAIN REFERENCE USED = NO**, **ALICE REFERENCE USED = NO** (kein Credential-Transfer nötig; Architekturweg dokumentiert).
|
||||
|
||||
## Quellen-Stand (EVIDENCE)
|
||||
- **Forgejo** `main` = `819fd8a` (Repo-Struktur lokal verifiziert).
|
||||
- **Tolaria** Vault = HEAD `69aecf2…`, 84 .md (API-Read verifiziert).
|
||||
|
||||
## Grundsatz (verbindlich)
|
||||
- Notion = externer Safety Brain, **KEINE Source of Truth**.
|
||||
- **One-Way** von Forgejo/Tolaria → Notion. Kein Rückfluss.
|
||||
|
|
@ -1,55 +0,0 @@
|
|||
# NOTION SAFETY BRAIN — DATA CLASSIFICATION (Mission 003)
|
||||
|
||||
Datum: 2026-08-25 · Red Queen · Status: **EVIDENCE-basiert (read-only)**
|
||||
|
||||
> Klassifikation, welche Inhalte aus Forgejo und Tolaria in den Notion Safety Brain
|
||||
> gesichert werden sollen. Kategorien:
|
||||
> **BACKUP_REQUIRED** · **BACKUP_RECOMMENDED** · **OPTIONAL** · **DO_NOT_BACKUP** · **SECRET/FORBIDDEN**
|
||||
|
||||
---
|
||||
|
||||
## 1. FORGEJO (technische Source of Truth)
|
||||
|
||||
EVIDENCE (lokaler Klon `/opt/data/forgejo/trading-system-docs`, main=`819fd8a`):
|
||||
|
||||
| Inhalt | Kategorie | Begründung |
|
||||
|---|---|---|
|
||||
| `red-queen-architecture/` (10 Contracts) | **BACKUP_REQUIRED** | Architektur-Entscheidungen, Safety/External-Review-Contracts, MISSION_STATE_MACHINE |
|
||||
| `tolaria/` (Discovery-Report, B1-Docs) | **BACKUP_REQUIRED** | Tolaria-Dokumentation, Baseline, Recovery-Runbook |
|
||||
| `a2/`..`a5/` (Design/README/Code) | **BACKUP_RECOMMENDED** | Red-Queen-Automatisierung, relevant für Recovery |
|
||||
| Modul-Doku (`modul-*.md`, `ports-reference`, `infrastructure-handbook`, `trend-pullback-spec`) | **BACKUP_RECOMMENDED** | System-/Modul-Dokumentation |
|
||||
| `backup_patches/` | **BACKUP_RECOMMENDED** | Wichtige Patches |
|
||||
| `notes/` (Hubs/Wikilinks) | **BACKUP_RECOMMENDED** | Verknüpfungs-Struktur |
|
||||
| `README.md` | OPTIONAL | Überblick (bereits in modul-Doku enthalten) |
|
||||
| `__pycache__/`, `.git/`, Binär-/Build-Artefakte | **DO_NOT_BACKUP** | Rebuildbar, keine Knowledge-Inhalte |
|
||||
| `.env`, `*.pem`, `*.key`, `*.token`, config mit Secrets | **SECRET/FORBIDDEN** | Nie nach Notion |
|
||||
|
||||
**Doppel-Zählung beachten:** Modul-Dateien existieren teils auch im Vault (Root + system-docs) → Deduplizierung nötig (siehe Provenance).
|
||||
|
||||
## 2. TOLARIA (operatives Second Brain, 84 .md via API)
|
||||
|
||||
EVIDENCE (Vault-API, `/app/vault`, HEAD `69aecf2…`):
|
||||
|
||||
| Bereich | Kategorie | Begründung |
|
||||
|---|---|---|
|
||||
| `notes/trading/second-brain/` (journal, t212-logs, Phase-Docs) | **BACKUP_REQUIRED** | Operative Knowledge-Logs, Verlauf |
|
||||
| `notes/trading/system-docs/*` (30) | **BACKUP_RECOMMENDED** | Modul-/System-Doku (teils identisch mit Forgejo → dedupe) |
|
||||
| `/app/vault/*.md` Root (README, infrastructure-handbook, modul-*) | **BACKUP_RECOMMENDED** | Duplikat-Struktur (19 Paare) |
|
||||
| `notes/` Hubs (ai-agents, alice-betrieb, projekte, start, inbox, trading, reference) | **BACKUP_RECOMMENDED** | Wikilink-/Hub-Struktur, Beziehungen |
|
||||
| **Frontmatter / Wikilinks / Provenance** | **BACKUP_REQUIRED** (als Metadaten) | Für Beziehungs-/Graph-Rebuild |
|
||||
| `.git/` (Objects) | **DO_NOT_BACKUP** via API | Binär, nicht via JSON-API vollständig ziehbar; separat technisch sichern |
|
||||
| secrets in Frontmatter/Inhalt (falls vorhanden) | **SECRET/FORBIDDEN** | Secret-Scan vor Upload |
|
||||
|
||||
## 3. Kategorientabelle (Provenance-Vorbereitung)
|
||||
|
||||
Jede Notion-Seite erhält im Frontmatter/Property die Quelle:
|
||||
`SOURCE_SYSTEM` (forgejo|tolaria), `SOURCE_PATH`, `SOURCE_HASH`, `SOURCE_VERSION` (git-HEAD / Vault-HEAD).
|
||||
|
||||
## 4. Zwei-Quellen-Prinzip
|
||||
- Forgejo und Tolaria enthalten **nicht zwingend** dieselben Informationen.
|
||||
- Notion sichert **beide getrennt und nachvollziehbar** (je `SOURCE_SYSTEM`).
|
||||
- **Kein blindes Mergen** der beiden Quellen in Notion.
|
||||
|
||||
## 5. DO_NOT_BACKUP + SECRET — Grundregeln
|
||||
- Secrets/Tokens/Passwörter/private Keys/.env/Credentials → **SECRET_BLOCKED**, nie hochladen.
|
||||
- Binär-/Build-Artefakte, `.git/objects`, große Dateien → **DO_NOT_BACKUP** (technical backup übernimmt das).
|
||||
|
|
@ -1,23 +0,0 @@
|
|||
# NOTION SAFETY BRAIN — NOTION STRUCTURE (Mission 003)
|
||||
|
||||
Datum: 2026-08-25 · Red Queen · Status: **DESIGN (Vorschlag, nicht angelegt)**
|
||||
|
||||
Arbeitstitel: **KI-SYSTEM — SAFETY BRAIN**
|
||||
|
||||
| Bereich | Inhalt | Typ |
|
||||
|---|---|---|
|
||||
| 00 — Dashboard | Registry-Übersicht, Status | Seite |
|
||||
| 01 — Forgejo Safety Backup | Gesicherte Forgejo-Dokumente | Seite(n) |
|
||||
| 02 — Tolaria Safety Backup | Gesicherte Tolaria-Notes | Seite(n) |
|
||||
| 03 — Architecture & ADR | Architektur-Entscheidungen | Seite |
|
||||
| 04 — System Recovery | Recovery-Runbooks, Restore-Fluss | Seite |
|
||||
| 05 — Backup Registry | Lauf-Manifeste | **Datenbank** |
|
||||
| 06 — Integrity Reports | Verify-Ergebnisse | Seite |
|
||||
| 07 — Knowledge Index | Index der Objekte | **Datenbank** |
|
||||
| 08 — Historical Snapshots | Historische Stande | Seite |
|
||||
| 09 — Incidents / Recovery Events | Vorfälle, Restore | Seite |
|
||||
| 10 — Safety Brain Documentation | Diese Docs, Build-Plan | Seite |
|
||||
|
||||
**Empfehlung:** Seiten für statische Bereiche + **Datenbanken** für Registry (05) und Index (07),
|
||||
da dort je Zeile Provenance/Status/Hash/Head als Properties abbildbar und filterbar sind.
|
||||
Notion API: Markdown für Seiten (agent-freundlich), Properties für DB-Zeilen (Relation/Select/Date).
|
||||
|
|
@ -1,159 +0,0 @@
|
|||
# RED QUEEN — NOTION SAFETY BRAIN · PILOT BACKUP & RESTORE-READINESS TEST
|
||||
|
||||
**Status:** ABGESCHLOSSEN (Pilot)
|
||||
**Run-ID:** `RQ-NOTION-PILOT-20260825-001`
|
||||
**Datum:** 2026-08-25
|
||||
**Writer:** Red Queen
|
||||
**Sprache:** Deutsch
|
||||
|
||||
> Diese Datei dokumentiert die Pilot-Architektur und die Ergebnisse des ersten
|
||||
> kontrollierten Backup-Pilots von Forgejo + Tolaria in das Notion Safety Brain.
|
||||
> **Keine Secrets.** Keine Credentials. Keine Quell-System-Mutation.
|
||||
|
||||
---
|
||||
|
||||
## 1. AUTORITATIVE ARCHITEKTUR
|
||||
|
||||
- **Forgejo** = Source of Truth für Forgejo-Inhalte.
|
||||
- **Tolaria** = Source of Truth für Tolaria-Inhalte.
|
||||
- **Notion** = SAFETY BRAIN / Sicherheitskopie / Knowledge Index / Restore-Hilfe.
|
||||
- **KEIN** bidirektionaler Sync. **KEINE** Rückschreibung Notion → Forgejo/Tolaria.
|
||||
|
||||
---
|
||||
|
||||
## 2. PILOT RUN ID
|
||||
|
||||
`RQ-NOTION-PILOT-20260825-001`
|
||||
|
||||
Schema: `RQ-NOTION-PILOT-YYYYMMDD-NNN` — stabil, eindeutig, ordnet alle 10 Objekte diesem Run zu.
|
||||
|
||||
---
|
||||
|
||||
## 3. AUSGEWÄHLTE QUELLEN (5 Forgejo + 5 Tolaria)
|
||||
|
||||
| # | System | Pfad | Content-Typ | SOURCE_HASH (SHA-256, 12 Zeichen) |
|
||||
|---|--------|------|-------------|-----------------------------------|
|
||||
| 1 | forgejo | `notion-safety-brain/PROVENANCE_MODEL.md` | adr | `d2e4d083f4d8` |
|
||||
| 2 | forgejo | `notion-safety-brain/DATA_CLASSIFICATION.md` | decision | `4538e4f14fbf` |
|
||||
| 3 | forgejo | `tolaria/BACKUP_ARCHITECTURE.md` | architecture | `c64cd311d8a8` |
|
||||
| 4 | forgejo | `red-queen-architecture/SAFETY_CONTRACT.md` | contract | `046da63c5731` |
|
||||
| 5 | forgejo | `infrastructure-handbook.md` | reference | `708ccc532585` |
|
||||
| 6 | tolaria | `notes/start.md` | note | `c2ffd4e071ec` |
|
||||
| 7 | tolaria | `notes/trading/trading.md` | note | `a09aef8409cb` |
|
||||
| 8 | tolaria | `notes/ai-agents/alice-betrieb.md` | note | `45444b29163c` |
|
||||
| 9 | tolaria | `notes/ai-agents/ai-agents.md` | hub | `691949bb64cd` |
|
||||
| 10 | tolaria | `notes/projects/projekte.md` | hub | `586ec58668d7` |
|
||||
|
||||
**Auswahlbegründung (repräsentativ, klein, kategorienübergreifend):**
|
||||
- **Forgejo:** Architektur, Entscheidungs-Doku, ADR, Safety-Contract, Referenz-Handbuch.
|
||||
- **Tolaria:** Start/Überblick, Trading-Modul, AI-Agenten-Betrieb (2), Projekt-Hub.
|
||||
|
||||
---
|
||||
|
||||
## 4. MAPPING (Source → Notion-Ziel)
|
||||
|
||||
| System | Notion-Parent-Ziel |
|
||||
|--------|--------------------|
|
||||
| Forgejo | „Forgejo Backups" page `3c793a20-cfac-80a4-bdad-d9db7753f11f` |
|
||||
| Tolaria | „Tolaria Backups" page `3c793a20-cfac-80b3-bff7-f2020bbfed0d` |
|
||||
| Index | „Safety Brain Index" database `3c793a20-cfac-805c-b990-fc6b055d76c0` |
|
||||
|
||||
---
|
||||
|
||||
## 5. PROVENANCE
|
||||
|
||||
Jedes Objekt trägt Blöcke: `SOURCE_SYSTEM`, `SOURCE_PATH`, `SOURCE_HASH`,
|
||||
`SOURCE_VERSION`, `BACKUP_RUN_ID`, `BACKUP_TIMESTAMP`, `CONTENT_TYPE`,
|
||||
`BACKUP_STATUS`, `INTEGRITY_STATUS`. Version: `FORGEJO_HEAD:<sha>` bzw. `TOLARIA:host-vault`.
|
||||
|
||||
---
|
||||
|
||||
## 6. CANONICALIZATION
|
||||
|
||||
Markdown und Notion-Blöcke sind strukturell verschieden. Raw-Byte-Hash der
|
||||
Notion-Darstellung ≠ Markdown-Dateihash. Getrennt:
|
||||
- **SOURCE_HASH** = SHA-256 des unveränderten Quellobjekts.
|
||||
- **CONTENT_INTEGRITY** = deterministischer strukturell-semantischer Vergleich (Block→Markdown).
|
||||
|
||||
Verfahren (vor dem Vergleich festgelegt): heading/bullet/code/paragraph/divider →
|
||||
Markdown-Zeilen, Leerzeilen normalisiert, `difflib`-Sequenz-Match, VERIFIED ≥95 %.
|
||||
**Ergebnis: 10/10 VERIFIED.**
|
||||
|
||||
---
|
||||
|
||||
## 7. WRITE-ERGEBNIS (Notion)
|
||||
|
||||
- **NOTION_WRITTEN = 10** (alle HTTP 200).
|
||||
- SAFETY_CONTRACT (>100 Blöcke) per 3 Chunks wegen Notions 100-Block-Limit.
|
||||
- Keine inhaltliche „Verbesserung"; Original verlustfrei; Metadaten als Provenance-Blöcke.
|
||||
|
||||
---
|
||||
|
||||
## 8. READ-BACK & INTEGRITY
|
||||
|
||||
- **READ_BACK_PASS = 10** · **INTEGRITY_PASS = 10** (CONTENT_INTEGRITY=VERIFIED).
|
||||
- Pagination über 100-Block-Limit korrekt.
|
||||
|
||||
---
|
||||
|
||||
## 9. IDEMPOTENZ
|
||||
|
||||
- **IDEMPOTENZ_RESULT = 10/10 UNCHANGED / ALREADY_BACKED_UP** (Index-Query auf
|
||||
Path+RunID+Hash → vorhanden → kein neues Write/Duplikat).
|
||||
|
||||
---
|
||||
|
||||
## 10. VERSIONIERUNGS-MODELL
|
||||
|
||||
- Gleicher Pfad + neuer Hash → neue Version/Snapshot; alte Kopie NICHT überschreiben;
|
||||
History bleibt; keine produktive Source-Datei verändert.
|
||||
|
||||
---
|
||||
|
||||
## 11. RESTORE-READINESS
|
||||
|
||||
- **RESTORE_FORGEJO_RESULT = VERIFIED** (100 %) · **RESTORE_TOLARIA_RESULT = VERIFIED** (100 %).
|
||||
- Rekonstruktion isoliert unter `/opt/data/rq_restore_staging/`. Kein produktiver Restore.
|
||||
|
||||
---
|
||||
|
||||
## 12. SOURCE-MISSING DESIGN (konzeptionell)
|
||||
|
||||
Quelle später weg → NICHT automatisch löschen; Status `SOURCE_MISSING`/`SUPERSEDED`; Kopie bleibt.
|
||||
|
||||
---
|
||||
|
||||
## 13. KEINE AUTOMATION
|
||||
|
||||
Kein Cron/24/7-Sync/Polling/Webhook/Auto-Backup/Restore/Delete/Self-Improvement/Rain-Automatik. A5 Kill-Switch OFF.
|
||||
|
||||
---
|
||||
|
||||
## 14. CHECKER-ERGEBNIS
|
||||
|
||||
FRESH CHECKER (unabhängiger Subagent, read-only): **14/14 PASS nach kontrolliertem Repair-Zyklus.**
|
||||
|
||||
- Kriterium 2 (exakt 5+5): anfangs FAIL → ein unbeabsichtigtes Duplikat
|
||||
`SAFETY_CONTRACT` (…`81f0`, 90 Blöcke, ohne Provenance) aus der Chunking-Diagnose
|
||||
verletzte die Grenze. Via A3-kontrolliertem Repair entfernt (HTTP 200, DELETE).
|
||||
Danach: **Forgejo = exakt 5**, Tolaria = exakt 5. **PASS.**
|
||||
- Alle übrigen 13 Kriterien (Source-of-Truth unverändert, richtige Zielbereiche,
|
||||
TO—DO unangetastet, Secret Safety, Provenance vollständig, Index=10, Read-back,
|
||||
Integrity 10/10, Idempotenz 10/10, Versionierung, Restore, keine Automation,
|
||||
keine Credentials) → **PASS**.
|
||||
- **Verdict: PASS.**
|
||||
|
||||
---
|
||||
|
||||
## 15. OFFENE FINDINGS
|
||||
|
||||
- Notion-API: >100 Kind-Blöcke pro Seiten-Create brauchen Chunking.
|
||||
- `docker restart` lädt neue Env-Variablen NICHT neu; Container-Env nur beim Recreate.
|
||||
- Memory (Red Queen) voll; Safety-Brain-IDs nur in Skripten.
|
||||
- Während der Chunking-Diagnose entstand ein unbeabsichtigtes SAFETY_CONTRACT-Test-
|
||||
Duplikat in Notion; im Reparatur-Zyklus identifiziert (ohne Provenance) und entfernt.
|
||||
Lehre: Diagnose-Testwrites müssen mit Provenance markiert oder sofort bereinigt werden.
|
||||
|
||||
---
|
||||
|
||||
*Ende Pilot-Dokumentation.*
|
||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Reference in a new issue