Compare commits

...

88 commits

Author SHA1 Message Date
41e68aeb21 wave1: persist canonical L0 schema and fingerprint tooling 2026-09-08 13:10:06 +00:00
e56567f7cf wave1: formally close provenance foundation production deployment 2026-09-04 20:02:48 +00:00
root
36916a073d phase13.6: formally close hardening repair 2026-09-01 10:20:24 +00:00
root
51c03f7ade phase13.5: formally close runner trustgate remediation 2026-08-31 23:00:47 +00:00
root
738af54c93 phase13.5: verify recreate post-gates against target class 2026-08-31 18:54:03 +00:00
root
35bbdd89e7 phase13.5: add bounded recreate startup readiness gate 2026-08-31 17:29:06 +00:00
root
b61acd3f08 phase13.5: fix remediation recreate Gate O runner allowance 2026-08-31 16:13:40 +00:00
root
d6394ef327 RAIN: Phase 13.5 Emergency Rollback CP Repair - target-image-based rollback pre-gates (current/recovery gold). SoT local, no push. 2026-08-31 12:54:03 +00:00
root
a7336c2585 RAIN: Phase 13.5 Remediation Recreate Gate Decoupling + fixed-ID rollback-to-current-gold-anchor (SoT) 2026-08-31 11:38:41 +00:00
root
41c82f14d3 RAIN Hotfix: Partial Archive Cleanup Semantics (PARTIAL statt UNKNOWN fuer image.tar+fehlende Metadaten, fail-closed) 2026-08-31 07:51:49 +00:00
root
561133f39b RAIN Hotfix: Remediation Archive Build-Input-Semantik (branch-spezifisch, recovery unveraendert) 2026-08-31 07:02:48 +00:00
root
75b594f225 Phase 13.5: verify-runner-remediation-image session-binding hotfix (load_remediation_source_state) 2026-08-31 05:57:34 +00:00
root
ce58a067ee Phase 13.5: Deploy verified partial-archive control-plane patch (cleanup-partial-remediation-archive + active remediation image state validation) 2026-08-30 20:39:13 +00:00
root
3e65c36cce Phase 13.5: Orphan Remediation Staging Cleanup Control-Plane Hotfix (cleanup-partial-remediation-staging) 2026-08-30 17:31:47 +00:00
root
f69774e34b Phase 13.5 Remediation Staging Consistency + Active Image ID Hotfix: remediation_staging_matches_expected entfernt fehlerhaftes | sort (SHA-sortiert) das gegen Pfad-sortiertes Staging-Manifest verglich -> Staging-Check FAIL CLOSED trotz identischem Inhalt. Fix: exp ohne | sort (remediation_expected_manifest erzeugt bereits Pfad-sortiert). Active-ID bereits state-basiert (remediation_session/full_image_id), 41e357a1 nur noch dokumentarisch. Verdict A. 2026-08-30 16:58:10 +00:00
root
c95fbe6b3c Phase 13.5 Remediation Build-Context Control-Plane Rotation: build-runner-remediation von /opt/historical-v2 auf immutable Gold-Source-Session 20260830_104127 + autorisierter Runner-Overlay 6cca3c55; Guards A/B/C/D/N auf Gold+Overlay; Guard I diagnostisch (No-op-Tag-Loss NON-BLOCKING); verify_runner_remediation_image 62+1 + repository.py Hard Gate; verify_archive/pin/archive aktive Build-Session-Identity; deterministischer REMEDIATION_BUILD_INPUT_SHA 2026-08-30 16:36:42 +00:00
root
8161208f1c Phase 13.5 Archive Metadata Variable Hotfix: gen_archive_meta nach write_runtime_config/write_app_manifest + SHA-Validierung (unbound variable Fix) 2026-08-30 14:22:14 +00:00
root
9685b616cf Phase 13.5 Recovery Archive Hotfix: Fix A (Reihenfolge-Bug nach docker image save) + cleanup-partial-recovery-archive Command 2026-08-30 11:34:11 +00:00
root
a9d7da14f5 Phase 13.5 Recovery Rebuild Guard Source Rotation: rebuild_guards Guard A/B/C auf Gold-Source-Session (verify-recovered-gold-build-source + verify_recovery_build_input), Host-SoT-Drift non-blocking Diagnostic, kein /opt/historical-v2 als Recovery-Build-Blocker 2026-08-30 11:10:07 +00:00
Rain Ocampo
d7db920116 Phase 13.5 Nested Verify Exit Hotfix: verify_deployed_gold_source + verify_recovered_gold_build_source exit 0 -> return 0 (nested-call Guard-Defekt) 2026-08-30 10:28:46 +00:00
Rain Ocampo
8f29924ec6 Phase 13.5 Recovery Control-Plane Rotation: real recover freigegeben, Build-Context-Rotation (kein /opt/historical-v2), Recovery-Identity-Rotation (CURRENT_RECOVERY_ID statt d2db), READY-Gate, wrapper-sha 2026-08-30 09:26:59 +00:00
root
563ed111b9 Phase 13.5: Deployed Gold Source Recovery Control Plane - feste Subcommands verify-deployed-gold-source / dry-run recover-gold-build-source / recover-gold-build-source (real blockiert) / verify-recovered-gold-build-source; Gold-Source fest an Container+Image+63-File-Manifest gebunden, kein latest, kein full_sot_backup, immutable Target, FAIL CLOSED 2026-08-30 08:46:32 +00:00
root
65c1d57fa7 Phase 13.5: Image Retention & Archive Control Plane - feste Subcommands archive/pin/verify-archive/verify-pin/verify-ready/dry-run restore (FAIL CLOSED, kein Overwrite, kein Coolify-Cleanup-Disable, kein coolify.managed-Spoof); Restore nur dry-run (real blockiert) 2026-08-30 06:05:56 +00:00
Rain Ocampo
8187a4bc1d Phase 13.5: Authorized Remediation Divergence Verify Fix - verify-runner-remediation-image nutzt gemeinsame authorized_gold_delta-Policy (exakt 1 autorisierte Runner-Divergenz, alles andere FAIL CLOSED); Candidate-vs-Host-SoT direkter Vergleich; verify_critical_files runner-Flag 2026-08-29 23:56:14 +00:00
Rain Ocampo
6194868100 Phase 13.5: Remediation Build/Deploy Control Plane - build-runner-remediation, verify-runner-remediation-image, recreate-runner-remediation-safe, verify-remediation-build-input; Full/Partial SoT-Backup-Klassifikation (Blocker A) 2026-08-29 23:01:57 +00:00
root
223fa33f50 Phase 13.5: Persistent Host-SoT Patch Promotion Channel - promote-runner-patch, rollback-runner-patch (SHA-bound, atomic, FAIL CLOSED) 2026-08-29 20:59:30 +00:00
root
6bcad03e18 Phase 13.5: Final Reproduction Control Plane - pin-current-candidate, verify-candidate-image, recreate-candidate-safe, rollback-to-recovery-gold (Safe Recreate + Rollback Prep) 2026-08-29 19:17:23 +00:00
root
9d2bfe3ae4 CP2A2: verify-recovery-gold-image PATH-NORMALIZATION-FIX (Image-Manifest app/-Präfix, Container-Pfad /app/app) + Critical-File-Pfadfix 2026-08-29 19:08:42 +00:00
root
c6046feaf2 CP2A2: rq-historical build-recovery-gold Kanal (fix Context/Dockerfile/Tag, 7 Guards, DIRECT-Tag no-latest, verify-recovery-gold-image, Host-Legacy-Guard) 2026-08-29 18:46:00 +00:00
root
efe13ec92d Phase 13.5: Recovery-Gold-Snapshot-Kanal (snapshot-recovery/verify/dry-run, 14 Guards, fail-closed) | Rain Ocampo 2026-08-29 2026-08-29 18:12:04 +00:00
root
c9b3dafd9f rq-historical: CONTROLLED NO-OP BUILD/RECREATE Kanal (build-historical, recreate-historical, verify-gold-reproduction) — dry-run Mutation Guard, .dockerignore Build-Pollution-Exclusion, fail-closed, kein echter Build/Recreate 2026-08-29 16:50:17 +00:00
Rain Ocampo
64240ea3b1 rq-historical: HOST-SOT-RECOVERY Kanal (backup-sot, recover-sot, verify-sot-recovery, restore-sot) — Gold-SHA-gebunden, fail-closed, kein Build 2026-08-29 15:56:57 +00:00
Rain Ocampo
9129e76a6d rq-historical: SOT-Discovery read-only Kanal (build-def, build-files, host/container-manifest, host/container-code, tree-diff) — redacted Secrets 2026-08-29 15:24:28 +00:00
root
6324c8aaac rq-historical: add dev/test/deploy channel (stage-check,test,deploy,backup,rollback,search-writes,healthcheck) 2026-08-29 10:34:59 +00:00
root
bbfd85e472 rq-historical: add READ-ONLY code/hashes/code-list (allowlist) for RQ Phase 13.5 2026-08-29 09:51:04 +00:00
root
a0c388ab18 CP2A2.2B: Fix worker UID/GID to 10000/10000 (hermes user in base image) - non-root hardening correction 2026-08-29 07:17:11 +00:00
root
7a4220d86d CP2A2.2B: L1 Red Queen Worker Foundation - separate minimal worker runtime (non-root, read-only rootfs, cap-drop ALL, no-new-privileges, internal-net, credential-zero, inert idle, NO autonomy/heartbeat/scheduler) 2026-08-29 07:16:28 +00:00
root
7b58d27fa0 CP2A1: Trusted Gate Evaluator (External Gate Enforcement Foundation) - non-productive, no autonomy, CP1-logic reuse, fail-closed, internal-net, no credentials 2026-08-28 13:24:58 +00:00
Red Queen
058c92e5d0 CP1.1: PRE_HERMES Control Plane SOT closure - 5-control scope (trading_execution removed), RED_QUEEN_TRADING_AUTHORITY=NEVER, reproducible deployment 2026-08-28 11:27:31 +00:00
Rain Ocampo
b6c9da3e12 CP1.1: Control Plane SoT closure - reader, tests, deploy script, ops contract 2026-08-28 09:18:04 +00:00
Rain Ocampo
dcc6d8a098 AUTH.4D: P10/P11 test suite + mutation/sensitivity tests
- test_delete_worker.py: 42 tests (T1-T42) against productive delete_* SoT code
  (T29 delete_request_id mismatch, T41 second-delete guard, T42 immutable guard)
- test_mutations.py: 15 mutations A-O, all detected (P11 sensitivity)

AUTH.4D P14. No deployment, no token injection, no key provisioning.
2026-08-28 02:30:19 +00:00
Rain Ocampo
9c2d6e365a AUTH.4D: DELETE-scoped executor + delete_request_id binding in SoT core
- delete_worker.py: DELETE-scoped worker loop (ApprovalStateStore, fail-closed)
- delete_tolaria_client.py: DELETE-only Tolaria client (fixed /delete endpoint)
- delete_entrypoint.py: DELETE runtime entrypoint (health/status + worker)
- delete_Dockerfile: DELETE executor image (no SAVE code, no credentials)
- delete_executor_core.py: minimal SoT patch — VALID_NONCE replay guard +
  VALID_DELETE_REQUEST binding (approval binds delete_request_id, fail-closed
  APPROVAL_MISMATCH). Closes P10/T29 security gap in productive SoT.

AUTH.4D P14. No deployment, no token injection, no key provisioning.
2026-08-28 02:28:48 +00:00
5f3adda73f AUTH.4C3: Dockerfile kopiert save_reconciliation.py ins Image (BUILD_REPRODUCIBLE_FROM_SOT) 2026-08-27 16:52:11 +00:00
dbc31ad1bb AUTH.4C3: OUTCOME_UNKNOWN read-only reconciliation contract — RECONCILED state, immutable-field guard, provenance recompute, fail-closed classification, 61 isolated tests (reconciliation 30, incident fixture 5, adversarial 14, sensitivity 12) 2026-08-27 15:57:06 +00:00
2e631fd5fa AUTH.4C2: relative vault path contract repair (OPTION A) — canonical relative path, absolute/traversal fail-closed, no approval_payload dual-semantics, 20 path-contract tests 2026-08-27 13:23:56 +00:00
2ba503df0e AUTH.4C1: SoT reconciliation — runtime wiring (entrypoint, worker, tolaria_client, forgejo_source_loader, Dockerfile) + Read-Back-Verifikation + SHA1-Korrektur 2026-08-27 13:23:43 +00:00
9f3ed82cfc AUTH.4C2: First Productive SAVE Canary (tolaria/auth4c2-canary.md) 2026-08-27 12:47:50 +00:00
373425fc0d AUTH.3E: Audit-Trail trennt REQUESTED/AUTHORIZED/EXECUTED (Fresh Checker Punkt 27)
Fresh Checker identifizierte Defekt: audit_trail() lieferte nur einen Job-Snapshot,
kein persistentes Audit-Event-Log. Design §18 verlangt Trennung der Phasen.

- job_store: audit_events-Tabelle + _record_audit/record_audit_event; REQUESTED bei create_job
- delete_executor_core: AUTHORIZED nach AUTH.3D-Validierung, EXECUTED nach Mutation
- save_executor_core: AUTHORIZED nach Provenance-Validierung, EXECUTED nach Mutation
- test_job_channel: +4 AuditTrailTests (REQUESTED/AUTHORIZED/EXECUTED)

REPAIR_CYCLES=1. Volle Regression 444 PASS (76+50+19+299).
2026-08-27 10:09:18 +00:00
10761f52b2 AUTH.3E: Executor Command Channel + Runtime Boundary Contract
- job_schema: geschlossene Job-Type-Allowlist (C5_SAVE_OBJECT/C5_DELETE_OBJECT), Pfad-/Längen-Validierung
- job_state_machine: deterministische States (CREATED/READY/CLAIMED/EXECUTING/SUCCEEDED/FAILED)
- job_claim: atomare Claim-/Lease-/Recovery-Logik (kein TOCTOU)
- job_store: getrennte SQLite-Inbox-DBs (c5a_save.db/c5a_delete.db), delegiert an job_claim
- save_executor_core: SAVE-only, Content-Rekonstruktion, Provenance-Validierung
- delete_executor_core: DELETE-only, AUTH.3D-Composition, TOCTOU-Defense (Re-Read nach Claim)
- test_job_channel: T1-T40 + adversarial (72 Tests)
- test_job_channel_adversarial: adversarial + Substitution + DB-Manipulation
- sensitivity_proof_auth3e: Mutationen A-L (12/12 Invarianten PRESENT)
- AUTH3B/3C/3D/3E_DESIGN: autoritative Security-Dokumentation (e25 Reconciliation)

COMMAND != AUTHORIZATION. Kein generischer Dispatcher. RQ credential-free.
Keine produktive Mutation. Keine echten Credentials.
2026-08-27 10:03:32 +00:00
9c8d239ae5 AUTH.3D: Human DELETE Approval Authenticity (Ed25519, verify-only)
- approval_payload: kanonischer, deterministischer Approval-Payload
- approval_signature: Ed25519 sign/verify (Christian=Private Key, Executor=Public Key only)
- approval_verifier: verify-only, alle Bindings fail-closed
- approval_state: Lifecycle CREATED->CONSUMED, Single-Use, Reservation, OUTCOME_UNKNOWN
- test_approval_auth3d: T1-T30 + adversarial (50 Tests)
- test_approval_helpers: synthetische Test-Keypairs
- sensitivity_proof: Mutationen A-J machen Tests ROT

Nur synthetische Test-Keypairs. Kein produktives Deployment.
DELETE_OPERATION_ACTIVATION bleibt BLOCKED bis AUTH.3D geprueft.
2026-08-27 09:36:21 +00:00
a11c1bbe53 AUTH.3A: C5 Caller Auth Integration (SAVE/DELETE Credential, fail-closed, Tests, Sensitivity, Gap-Doku)
- rq_c5c.py: TolariaClient save_token/delete_token DI, _require_token fail-closed,
  write()/delete() senden Bearer (SAVE/DELETE), read()/list() ohne Credential
- rq_c5_cli.py: _delete_executor liest nur DELETE-Credential (Least Privilege)
- test_c5c.py: write()-Tests injizieren synthetisches SAVE-Token
- test_c5_auth3a.py: isolierte AUTH.3A-Testsuite (19 Tests, Fake/Mock Tolaria)
- auth3a-sensitivity.sh: 8 Sensitivitaets-Mutationen (A-H) -> ROT
- AUTH3A_HUMAN_APPROVAL_AUTHENTICITY_GAP.md: Gap dokumentiert (OPEN, nicht repariert)

Keine echten Tokens. Keine ENV-Mutation. Kein Deployment. Keine produktive Auth-Aktivierung.
2026-08-27 08:42:26 +00:00
40bbc40a49 PRE_HERMES_SECURITY_GATE AUTH.1: Tolaria Write Auth Contract + isolierte Test-Suite
- Contract (ADR): Auth-Modell (SAVE/DELETE-Scope getrennt), Credential-Modell
  (RQ/Hermes NO SAVE/DELETE; C5C Writer SAVE only; DeleteExecutor DELETE only),
  FAIL-CLOSED-Regeln, Human-DELETE-Gate (2 unabhaengige Ebenen AUTH+APPROVAL),
  Logging/Secret-Regeln, Rotation/Revocation, Path-Safety-Contract (Defense-in-Depth).
- Isolierte Test-Suite (47 Tests): T1-T20 Auth-Matrix, Human-Approval-Composition A-G,
  Adversarial, Sensitivitaet (Mutationen -> ROT). KEIN Produktionscode geaendert.
- KEINE produktive Mutation, KEINE echten Tokens, nur synthetische Fixture-Werte.
2026-08-27 05:57:22 +00:00
9402a92cc3 C5G.1: Acceptance/Exit-Criteria test (17 tests, 12/12 adversarial sensitivity) 2026-08-27 04:12:20 +00:00
951868dda9 C5F: Remove controlled canary object (c5f-controlled-canary.md)
Human-gated cleanup of the C5F canary object/6edb6869-0dfd-4046-993a-a727a8cab029.
Authorized exclusively for this single canary (C5F PRODUCTIVE CANARY CLEANUP AUTHORIZATION).
2026-08-26 19:26:28 +00:00
3db1c71b68 C5: Human-Gated DELETE Execution Contract (Todo 7-13)
- DeleteExecutor (rq_c5_delete.py): Pre-Gates, Read-Back, idempotenter replay
- TolariaClient.delete() (rq_c5c.py): kontrollierter DELETE, keine Probes
- Approval-Store + Reason Codes RC_DELETE_APPROVAL_MISSING/MISMATCH (rq_c5a.py)
- CLI: c5-delete-approve/execute/replay/status (rq_c5_cli.py)
- C5E: recover()/replay() DELETE-Integration
- C5D: verify_integrity prueft secret_blocked_objects (FAIL CLOSED)
- Security: Path-Traversal-Block in _normalize_vault_path
- Drift nach DELETE -> FAIL CLOSED zurueck zu HUMAN_REVIEW_REQUIRED
- Tests: test_c5_delete (19), test_c5_delete_integration (22),
  test_c5_delete_fresh_checker (17) — alle gruen
- ADR: C5_DELETE_EXECUTION_ARCHITECTURE_DECISION.md (ACCEPTED)
2026-08-26 19:22:44 +00:00
363f27327b fix(tolaria): C5D source_provenance persistence contract (OPTION A)
Persistiert die Search-Source-Provenance commit-spezifisch im C5-State
(meta-KV-Key search_source_provenance:<workflow_commit_sha>), WRITE POINT
nach validiertem Source-Build. Adoption prueft ausschliesslich gegen die
persistierte source_provenance, nie gegen workflow_commit_sha oder
current_repo_head. FAIL CLOSED ohne persistierte Provenance. Generischer,
evidence-validierter Recovery-Pfad fuer extern abgeschlossene Builds.
14 neue Contract-Tests (A-N); volle Regression gruen.
2026-08-26 16:07:49 +00:00
76512ddd48 fix(tolaria): C5A reason-code closed-set accepts C5D build code
RC_SEARCH_SOURCE_BUILD_FAILURE (added in 5e41915, C5D search-source
pipeline) was missing from test_reason_codes_closed_set, so the C5A
suite falsely FAILed (18 vs 17). Pre-existing defect exposed by the
C5D adoption fresh-checker regression gate; not introduced by 747376e.
Harden the assertion to the real closed set (len 18).
2026-08-26 15:24:03 +00:00
747376ebb0 fix(tolaria): adopt externally completed search rebuild
C5F P5 Option A: C5DEngine.apply_commit() uebernimmt einen extern
bereits vollstaendig korrekt rebuildeten Suchzustand bei UPDATING_SEARCH
idempotent (ADOPT_ALREADY_AT_TARGET) ohne zweiten Rebuild.

- Neues evaluate_external_adoption(): read-only, exakte Set-Gleichheit
  (ids, paths, count, source_head), kein object_count-only shortcut
- apply_commit(): UPDATING_SEARCH -> Adoption-Check -> bei exaktem Match
  direkt VERIFYING_SEARCH ohne rebuild() -> verify -> APPLIED
- Neuer Report-Feld 'adoption' mit Status + rebuild_count
- Testsuite test_c5d_adoption.py (Faelle A-O, 15 Tests)

Kein zweiter Rebuild, kein manueller State-Set, kein Tolaria-/Forgejo-Write.
2026-08-26 15:18:27 +00:00
12635e8672 fix(tolaria): C5F Phase B — search-source indexability path-independent (valid C3 object_id suffices; LEGACY kept; out-of-scope dirs no longer auto-exclude indexable knowledge) 2026-08-26 13:05:13 +00:00
5e4191578a fix(tolaria): C5D search-source pipeline — build+verify source from current Tolaria before rebuild
- SearchSourceBuilder: deterministischer Vault->Source-Snapshot (read-only),
  atomar (temp->validate->fsync->replace), Secret-Scan fail-closed
- verify_integrity: exakte object_id/path Set-Equality (stale Source kann
  nie APPLIED), Canary implizit ueber erwartetes Objekt-Set
- apply_commit: Source-Build+Verification vor Rebuild; VERIFYING_SEARCH-Resume
  (kein Doppel-Rebuild) — C5E-Replay-Crash-Fall abgedeckt
- Fix: source_object_count ist keine 0-Fehlerbedingung (echter Defekt)
- SearchSourceBuildError + RC_SEARCH_SOURCE_BUILD_FAILURE (Human Gate)
- c4b: source_path env-konfigurierbar, indexed_object_ids/paths Read-Back
- Testsuite: 17 neue Tests (Test-Plan A-O + Realistic C4-Integration)
2026-08-26 12:12:45 +00:00
f451283276 fix(tolaria): allow propagate_commit resume from PROPAGATING_TOLARIA crash-window
C5F canary commit parked in PROPAGATING_TOLARIA with objects already written
to Tolaria (real write before crash). C5EEngine.replay() delegates the pending
case to propagate_commit, whose entry guard only accepted READY/RETRY_PENDING,
so the commit could never resume past the crash window and the ALREADY_AT_TARGET
idempotency (pre_write_drift_check) was never reached.

Minimal fix: accept PROPAGATING_TOLARIA as a resume entry state (idempotency
still determined per-object via pre_write_drift_check -> no double write; read-back
verify() remains the mandatory gate) and skip the READY->PROPAGATING_TOLARIA
transition on resume (no self-transition entry exists in _ALLOWED_TRANSITIONS).

Adds 2 regression tests covering the crash-window resume (already-at-target and
pending-create). Full C5A-E suite: 190 tests, 0 failures.
2026-08-26 10:48:10 +00:00
c5b3db153a fix(tolaria): handle null success response from vault save 2026-08-26 10:23:14 +00:00
238536710a fix(tolaria): handle missing vault object in C5C read path 2026-08-26 10:03:40 +00:00
63f957f4fb c5f: add controlled canary object (C5F CONTROLLED CANARY) 2026-08-26 08:11:42 +00:00
Rain Ocampo
3289098040 C5E: Fix replay crash-window (PROPAGATING_TOLARIA/READY + all objects propagated) via formal VERIFYING_TOLARIA path
FRESH CHECKER-VERDICT (deleg_3ea8ed5a): FAIL.
Defekt: rq_c5e.py Replay 'alle Objekte bereits propagated' rief
transition_commit(ST_UPDATING_SEARCH) aus PROPAGATING_TOLARIA/READY direkt auf,
was InvalidTransitionError warf (Transition nicht in _ALLOWED_TRANSITIONS).

Reparatur (invarianten-treu): statt den VERIFYING_TOLARIA-Schritt zu ueberspringen
(wuerde Read-Back/DRIFT-Check verletzen), wird der formale State-Pfad
READY->PROPAGATING->VERIFYING->UPDATING_SEARCH durchlaufen. Nutzt nur bereits
erlaubte Transitions; keine _ALLOWED_TRANSITIONS-Aenderung, kein C5A/C5C-Risiko.
Kein Tolaria-Doppel-Write (prop_calls=0), kein verfrühter Search (search_calls=0).

+ 2 Regressionstests (test_c5e.py): crash-window + ready-edge-case.
Volle Suite: C5A 25/0 + B/C/D/E 170/0 = 195 OK. Guarantees true.
2026-08-26 07:06:50 +00:00
22e1cd0d44 C5E: FAILURE/REPLAY/RECOVERY + OBSERVABILITY (fail-closed Library)
- rq_c5e.py: C5EEngine (Recovery-Entscheidung RESUME/RETRY/WAIT/HUMAN_REVIEW/ALREADY_APPLIED,
  deterministisches Replay, Partial-Commit-Recovery via object_progress), C5EReconciler
  (read-only, kein blindes Repair), observability(), health_contract() (HEALTHY/DEGRADED/BLOCKED),
  failure_evidence(). FAIL-CLOSED: Standard allow_writes=False, kein Polling/Daemon.
- rq_c5a.py: +4 Reason-Codes (SEARCH_UNAVAILABLE, NETWORK_TIMEOUT, MALFORMED_RESPONSE,
  INTEGRITY_FAILURE) -> REASON_CODES 13->17 (minimal, dokumentiert, regressionsgetestet).
- rq_c5_cli.py: +7 C5E-Befehle (recover, replay, reconcile, observability, health,
  evidence, guarantees), alle fail-closed.
- test_c5a.py: Assertions auf 17 Reason-Codes angehoben.
- test_c5e.py: 58 Tests (Restart/Retry/Replay/Partial/Ordering/Drift/Health/
  Observability/Persistence/Reconciliation/No-Doppel-Writes).

Regression: C5A 25/25, C5B 40/40, C5C 35/35, C5D 35/35, C5E 58/58. Alle gruen.
Keine produktive Aktivierung.
2026-08-26 06:54:10 +00:00
1404e78cb9 c5d: search integration + commit completion (UPDATING_SEARCH->VERIFYING_SEARCH->APPLIED) 2026-08-26 06:15:33 +00:00
8d7647b113 fix(tolaria): repair C5C retry state transitions and add retry tests 2026-08-26 05:57:55 +00:00
763f1ca9da feat(tolaria): add C5 Tolaria propagation and drift verification 2026-08-26 05:55:38 +00:00
ed374ac4ce fix(c5b): persist human-review object changes + read content_before from parent commit
- C5A objects table: object_id nullable + reason_code column so HUMAN_REVIEW/
  SECRET_DETECTED object changes (object_id=None) are persisted, not silently
  dropped (was: object_id TEXT NOT NULL, no reason_code field)
- C5B poll_once: content_before now read from parent_sha (state BEFORE the
  change) instead of sha, so MODIFIED changes classify as CONTENT_UPDATE
  instead of being misclassified (Checker-Befund)
- test_c5b: add test_modified_content_reads_parent regression test

C5B 40/40, C5A 25/25, real-repo dry run: 233 object changes (109 IN_SCOPE
with valid id, 124 HUMAN_REVIEW), idempotent.
2026-08-26 05:20:38 +00:00
091c1828c7 feat(tolaria): add C5 Forgejo polling and change detection 2026-08-26 05:13:33 +00:00
183afed1d6 feat(tolaria): add C5 sync state machine 2026-08-26 05:03:19 +00:00
68db35f55c docs(tolaria): add C5 sync architecture design 2026-08-26 04:31:56 +00:00
Rain Ocampo
4122bdd408 fix(tolaria-search): repair admin rebuild source path
server.py resolved SOURCE_JSON to ../c4a_evidence/index_source.json which does not exist in the deployed container. The C4B service ships index_source.json in its own directory. Fix resolves robustly relative to the service root.
2026-08-26 03:37:26 +00:00
29cbca9357 docs(tolaria): add C4 Search v1 production deployment plan (C4B) 2026-08-26 02:45:34 +00:00
aad56a26ea feat(tolaria): add C4B keyword and metadata search service 2026-08-25 20:28:37 +00:00
35446c03df feat(tolaria): C3I final individual decisions — reconcile modul-12 + migrate modul-18/19 root-only to schema v1
C3I FINAL INDIVIDUAL DECISIONS:
- modul-12: CONTROLLED ROOT<->CANONICAL RECONCILIATION + PAIRED MIGRATION.
  Canonical auf aktuellen Root-Stand (inkl. Phase 10e Intrabar/Gap Execution)
  gebracht, Root unveraendert, atomare Unit, derived_from=Root-ID.
- modul-18/19: ROOT-ONLY MIGRATION (kein Canonical-Partner).
- vps.md: NICHT migriert/löschen/archivieren, DEFERRED_STUB_CLEANUP dokumentiert.
Body byte-genau. IDs stabil. Knowledge schema v1. (Red Queen)
2026-08-25 19:27:07 +00:00
022c30e537 feat(tolaria): migrate C3H HR wave 1 — 16 safe root↔canonical pairs to schema v1
C3H HUMAN REVIEW WAVE 1: migriert 16 freigegebene HR-A PAIR_MIGRATION_SAFE Paare.
Pro Pair: Root (representation=source) + Canonical (representation=canonical,
derived_from=Root-ID) atomar. Body byte-genau unveraendert. IDs stabil aus
C3 Mapping Preview v2. modul-12/18/19/vps.md NICHT angefasst.
Knowledge schema v1. (Red Queen)
2026-08-25 19:19:02 +00:00
adbb96d04a feat(tolaria): migrate final auto-safe knowledge batch 3 to schema v1
C3F FINAL AUTO_SAFE BATCH 3: migriert alle 21 verbleibenden AUTO_SAFE
Knowledge Objects auf knowledge_schema v1 (18 standalone logs + 3
standalone-canonical). Body byte-genau, IDs stabil (0 Kollisionen).
Keine HR/DO_NOT_TOUCH berührt. C3 MIGRATION weiter IN_PROGRESS.
2026-08-25 19:06:44 +00:00
7ae7249c00 feat(tolaria): migrate auto-safe knowledge batch 2 to schema v1
15 knowledge objects (log, note, index, module, history) migrated to
C3 knowledge schema v1. Bodies unchanged, metadata preserved.
2026-08-25 18:48:25 +00:00
38049b1475 feat(tolaria): migrate auto-safe knowledge batch 1 to schema v1 2026-08-25 18:29:35 +00:00
c1453b65ce feat(tolaria): migrate ports-reference pair to knowledge schema v1 2026-08-25 18:03:57 +00:00
47f80bc34d docs(tolaria): canonicalize modul-09 system documentation 2026-08-25 16:56:37 +00:00
e8a9804f8e docs(tolaria): align four canonical system-docs with master 2026-08-25 16:34:44 +00:00
fd4ac59acd M006: Notion PROJECT COMMAND CENTER Doku (COMPLETE, Fresh Checker PASS) 2026-08-25 13:04:19 +00:00
Red Queen
36c804325b docs: RQ-NOTION-PILOT-20260825-001 Pilot-Backup-Doku (Notion Safety Brain) 2026-08-25 07:35:31 +00:00
9b89ed77bb notion-safety-brain: Mission 003 Discovery & Architecture docs (11 files)
Red Queen - REAL MISSION 003 NOTION SAFETY BRAIN DISCOVERY & ARCHITECTURE.
One-Way Safety-Brain-Design (Forgejo/Tolaria -> Notion, keine Source of Truth),
Provenance-Modell, Backup-Manifest, Verify (WRITE != SUCCESS), Restore Human Gate,
Secret-Policy, Scaling/Versions/Delete-Safety, RQ-Access (Minimum Privilege).
Fresh Checker PASS. READ-ONLY, keine Notion-Implementierung.
2026-08-25 05:49:25 +00:00
819fd8a6ff tolaria: Mission 002 B1 Foundation & Backup docs (10 files)
- BACKUP_ARCHITECTURE, STORAGE_MAP, RECOVERY_RUNBOOK
- TRANSFORMATION_BASELINE, DUPLICATE_RESOLUTION_PLAN
- CANONICAL_KNOWLEDGE_PRINCIPLE, SEARCH_ARCHITECTURE_PROPOSAL
- KNOWLEDGE_GRAPH_PRINCIPLE, FORGEJO_TOLARIA_SYNC_PRINCIPLE
- HOST_CONTAINER_DISCOVERY

Fresh Checker PASS (deleg_a19028bc). No productive vault mutation.
2026-08-25 05:21:29 +00:00
221 changed files with 44339 additions and 65 deletions

View file

@ -1,3 +1,11 @@
---
knowledge_schema: 1
id: object/f4e559f5-403c-f241-be50-0962d8174644
type: arch
role: reference
representation: source
state: current
---
# Infrastruktur & Betriebs-Handbuch — Trading-System VPS # Infrastruktur & Betriebs-Handbuch — Trading-System VPS
> Stand: 20.08.2026 · VPS: `187.124.31.123` > Stand: 20.08.2026 · VPS: `187.124.31.123`

View file

@ -1,3 +1,11 @@
---
knowledge_schema: 1
id: object/1761a726-55af-f97b-d509-84aa4577f102
type: arch
role: module
representation: source
state: current
---
# Modul-03-Market-Data — Betriebsdokumentation # Modul-03-Market-Data — Betriebsdokumentation
> Erstellt: 20.08.2026 (Rain Ocampo) · Status: ✅ In Betrieb (healthy) > Erstellt: 20.08.2026 (Rain Ocampo) · Status: ✅ In Betrieb (healthy)

View file

@ -1,3 +1,11 @@
---
knowledge_schema: 1
id: object/6fca3e3a-70b2-0927-d7fb-e650a8f0e7f8
type: arch
role: module
representation: source
state: current
---
# Modul-04-Market-Regime — Betriebsdokumentation # Modul-04-Market-Regime — Betriebsdokumentation
> Erstellt: 20.08.2026 (Rain Ocampo) · Status: ✅ Freigegeben (E2E bestanden) > Erstellt: 20.08.2026 (Rain Ocampo) · Status: ✅ Freigegeben (E2E bestanden)

View file

@ -1,3 +1,11 @@
---
knowledge_schema: 1
id: object/0aef122a-790b-b17a-9176-cc32df98c5a1
type: arch
role: module
representation: source
state: current
---
# Modul-05-Strategy-Engine — Betriebsdokumentation # Modul-05-Strategy-Engine — Betriebsdokumentation
> Erstellt: 20.08.2026 (Rain Ocampo) · Status: ✅ Freigegeben (E2E bestanden) > Erstellt: 20.08.2026 (Rain Ocampo) · Status: ✅ Freigegeben (E2E bestanden)

View file

@ -1,3 +1,11 @@
---
knowledge_schema: 1
id: object/2b4c4170-b5b2-8a2b-fdee-8ea5eff191b5
type: arch
role: module
representation: source
state: current
---
# Modul-06-Signal-Ranking — Betriebsdokumentation # Modul-06-Signal-Ranking — Betriebsdokumentation
> Erstellt: 20.08.2026 (Rain Ocampo) · Status: ✅ **Freigegeben** (E2E+Idempotenz+Reconnect+Doku grün) > Erstellt: 20.08.2026 (Rain Ocampo) · Status: ✅ **Freigegeben** (E2E+Idempotenz+Reconnect+Doku grün)

View file

@ -1,3 +1,11 @@
---
knowledge_schema: 1
id: object/645f8ff7-5a1a-bcf3-8dfe-0a8cfdf15dbc
type: arch
role: module
representation: source
state: current
---
# Modul-07: Risk-Manager # Modul-07: Risk-Manager
**Status:** ✅ **Freigegeben** (20.08.2026) **Status:** ✅ **Freigegeben** (20.08.2026)

View file

@ -1,3 +1,11 @@
---
knowledge_schema: 1
id: object/603e6b87-115a-e258-5bfb-7d94d6d78c97
type: arch
role: module
representation: source
state: current
---
# Modul-08: Portfolio-Manager # Modul-08: Portfolio-Manager
**Status: FREIGEGEBEN** (20.08.2026) **Status: FREIGEGEBEN** (20.08.2026)

View file

@ -1,3 +1,11 @@
---
knowledge_schema: 1
id: object/48bd264f-607b-15f1-5f73-3e922af9b19d
type: arch
role: module
representation: source
state: current
---
# Modul-09: Execution-Service # Modul-09: Execution-Service
**Status: FREIGEGEBEN** (20.08.2026) **Status: FREIGEGEBEN** (20.08.2026)

View file

@ -1,3 +1,11 @@
---
knowledge_schema: 1
id: object/0b393153-53e5-99fe-1098-e8bb4e0b6c7c
type: journal
role: module
representation: source
state: current
---
# Modul-10: Trade-Journal # Modul-10: Trade-Journal
**Status: FREIGEGEBEN** (20.08.2026) **Status: FREIGEGEBEN** (20.08.2026)

View file

@ -1,3 +1,11 @@
---
knowledge_schema: 1
id: object/31cf1506-0ab6-5f7a-cecc-0e3a7a2e6572
type: arch
role: module
representation: source
state: current
---
# Modul-11: Analytics # Modul-11: Analytics
**Status: FREIGEGEBEN** · Container `Modul-11-Analytics` · Image `analytics-service:0.1.0` **Status: FREIGEGEBEN** · Container `Modul-11-Analytics` · Image `analytics-service:0.1.0`

View file

@ -1,3 +1,11 @@
---
knowledge_schema: 1
id: object/302e9929-e186-c930-2406-ad4a8f17c6fd
type: arch
role: module
representation: source
state: current
---
# Modul-12: Backtesting # Modul-12: Backtesting
**Status: FREIGEGEBEN** · Container `Modul-12-Backtesting` · Image `backtesting:0.1.2` (V1 + V1.1 parallel) **Status: FREIGEGEBEN** · Container `Modul-12-Backtesting` · Image `backtesting:0.1.2` (V1 + V1.1 parallel)

View file

@ -1,3 +1,11 @@
---
knowledge_schema: 1
id: object/bde121b1-121f-590d-2b54-fedc13b02173
type: arch
role: module
representation: source
state: current
---
# Modul-13: Optimization # Modul-13: Optimization
**Status: FREIGEGEBEN ✅** · Container `Modul-13-Optimization` · Image `optimization:0.1.0` **Status: FREIGEGEBEN ✅** · Container `Modul-13-Optimization` · Image `optimization:0.1.0`

View file

@ -1,3 +1,11 @@
---
knowledge_schema: 1
id: object/0a0c9ca4-1cf4-aee1-3072-e1c3e8cfd29e
type: arch
role: module
representation: source
state: current
---
# Modul-14: Notification-Service # Modul-14: Notification-Service
**Status: FREIGEGEBEN ✅** · Container `Modul-14-Notification` · Image `notification:0.1.0` **Status: FREIGEGEBEN ✅** · Container `Modul-14-Notification` · Image `notification:0.1.0`

View file

@ -1,3 +1,11 @@
---
knowledge_schema: 1
id: object/1d6c2323-a31a-8b50-fcdd-3c9268175a69
type: design
role: design
representation: source
state: current
---
# Design-Vorschlag: Modul-15 → Modul-09 Anbindung (Safety-/Trading-Control) # Design-Vorschlag: Modul-15 → Modul-09 Anbindung (Safety-/Trading-Control)
**Status: NUR DESIGN-VORSCHLAG — keine Implementierung.** **Status: NUR DESIGN-VORSCHLAG — keine Implementierung.**

View file

@ -1,3 +1,11 @@
---
knowledge_schema: 1
id: object/a3cffdc1-41fd-ab84-c211-7b586f153e8d
type: arch
role: implementation
representation: source
state: current
---
# Modul-15 → Modul-09 Control-Anbindung — Implementierung & Live-E2E # Modul-15 → Modul-09 Control-Anbindung — Implementierung & Live-E2E
**Status: FREIGEGEBEN / PRODUKTIV VERIFIZIERT** (20.08.2026, Nutzer-Bestätigung). **Status: FREIGEGEBEN / PRODUKTIV VERIFIZIERT** (20.08.2026, Nutzer-Bestätigung).

View file

@ -1,3 +1,11 @@
---
knowledge_schema: 1
id: object/c2641bbf-0612-0737-86fd-622d899109ca
type: arch
role: module
representation: source
state: current
---
# Modul-15: Monitoring-Control # Modul-15: Monitoring-Control
**Status: FREIGEGEBEN** (20.08.2026) · Container `Modul-15-Monitoring-Control` · Image `monitoring-control:0.1.0` **Status: FREIGEGEBEN** (20.08.2026) · Container `Modul-15-Monitoring-Control` · Image `monitoring-control:0.1.0`

View file

@ -1,3 +1,11 @@
---
knowledge_schema: 1
id: object/2ad746ca-67f1-ce90-1304-4a1f4b052eb7
type: arch
role: module
representation: source
state: current
---
# Modul-18: Position-Manager # Modul-18: Position-Manager
**Status: FREIGEGEBEN (21.08.2026)** · Container `Modul-18-Position-Manager` · Port `55018` (nur intern/expose) **Status: FREIGEGEBEN (21.08.2026)** · Container `Modul-18-Position-Manager` · Port `55018` (nur intern/expose)

View file

@ -1,3 +1,11 @@
---
knowledge_schema: 1
id: object/3f834840-867d-9e9f-c7d2-8f1caf1550a4
type: arch
role: module
representation: source
state: current
---
# Modul-19: Broker-Reconciliation # Modul-19: Broker-Reconciliation
**Status: FREIGEGEBEN (21.08.2026)** · Container `Modul-19-Broker-Reconciliation` · Port `55019` (nur intern/expose) **Status: FREIGEGEBEN (21.08.2026)** · Container `Modul-19-Broker-Reconciliation` · Port `55019` (nur intern/expose)

View file

@ -1,11 +1,17 @@
--- ---
type: Agent id: object/fb050d49-41a1-a58b-3baf-e95a1765c7ed
type: arch
role: hub
representation: standalone
state: current
knowledge_schema: 1
title: AI-Agents Übersicht title: AI-Agents Übersicht
tags: [ai, agents, home] tags: [ai, agents, home]
created: 2026-08-20 created: 2026-08-20
_organized: true _organized: true
--- ---
# AI-Agents # AI-Agents
Multi-Agent-System von Christian List (Resident-Evil-Theme). Multi-Agent-System von Christian List (Resident-Evil-Theme).

View file

@ -1,5 +1,10 @@
--- ---
type: Agent id: object/6923ab6b-5132-55d7-4fd5-66890048e12b
type: arch
role: note
representation: standalone
state: current
knowledge_schema: 1
title: Alice Betrieb title: Alice Betrieb
tags: [ai, alice, openclaw, betrieb] tags: [ai, alice, openclaw, betrieb]
created: 2026-08-20 created: 2026-08-20

View file

@ -1,5 +1,10 @@
--- ---
type: Note id: object/7e4314d1-6c2c-df92-e40d-c3bc04d88182
type: arch
role: index
representation: standalone
state: current
knowledge_schema: 1
title: Inbox title: Inbox
tags: [inbox] tags: [inbox]
created: 2026-08-20 created: 2026-08-20

View file

@ -1,5 +1,10 @@
--- ---
type: Projekt id: object/2325c9b5-a668-9a04-3cf3-38be914b170c
type: arch
role: index
representation: standalone
state: current
knowledge_schema: 1
title: Projekte title: Projekte
tags: [projects, home] tags: [projects, home]
created: 2026-08-20 created: 2026-08-20

View file

@ -1,11 +1,17 @@
--- ---
type: Referenz id: object/9bd5b7db-bb0f-ae22-3a65-9410b77ad3de
type: arch
role: reference
representation: standalone
state: current
knowledge_schema: 1
title: VPS Infrastruktur title: VPS Infrastruktur
tags: [vps, infrastruktur, reference] tags: [vps, infrastruktur, reference]
created: 2026-08-20 created: 2026-08-20
_organized: true _organized: true
--- ---
# VPS-Infrastruktur # VPS-Infrastruktur
## Kern-Container ## Kern-Container

View file

@ -1,11 +1,17 @@
--- ---
type: Start id: object/4dd3d5ea-da35-5265-a501-a2108253464e
type: arch
role: index
representation: standalone
state: current
knowledge_schema: 1
title: Vault-Start title: Vault-Start
tags: [home, index] tags: [home, index]
created: 2026-08-20 created: 2026-08-20
_organized: true _organized: true
--- ---
# Vault-Start # Vault-Start
Willkommen im Tolaria-Second-Brain von Christian List. Willkommen im Tolaria-Second-Brain von Christian List.

View file

@ -1,8 +1,14 @@
--- ---
type: Note id: object/deea2ed6-cd6f-b006-2e4e-949abea60904
type: arch
role: history
representation: standalone
state: current
knowledge_schema: 1
_organized: true _organized: true
--- ---
# Phase 10c — Deterministische Slippage (DETERMINISTIC_FIXED) # Phase 10c — Deterministische Slippage (DETERMINISTIC_FIXED)
**Geändert von:** Rain Ocampo (Hermes) **Geändert von:** Rain Ocampo (Hermes)

View file

@ -1,5 +1,10 @@
--- ---
type: Notiz id: object/799b279e-9709-3c05-972b-3751d1dc8a8d
type: arch
role: note
representation: standalone
state: current
knowledge_schema: 1
_organized: true _organized: true
--- ---

View file

@ -1,10 +1,16 @@
--- ---
id: object/879703ed-5d7c-5446-397a-d2ec7b1a1d22
type: journal
role: journal
representation: standalone
state: current
knowledge_schema: 1
date: 2026-07-20 date: 2026-07-20
summary: T212 Tages-Recap summary: T212 Tages-Recap
type: Journal
_organized: true _organized: true
--- ---
# 📊 Tages-Recap — 20.07.2026 # 📊 Tages-Recap — 20.07.2026
**📊 T212 Tages-Recap — 20.07.2026** **📊 T212 Tages-Recap — 20.07.2026**

View file

@ -1,5 +1,10 @@
--- ---
type: Log id: object/99e784de-4e66-c9d3-7d3e-1a62afc91d90
type: log
role: log
representation: standalone
state: current
knowledge_schema: 1
_organized: true _organized: true
--- ---

View file

@ -1,5 +1,10 @@
--- ---
type: Log id: object/d334663d-ec48-f6e7-03a9-e9715571955f
type: log
role: log
representation: standalone
state: current
knowledge_schema: 1
_organized: true _organized: true
--- ---

View file

@ -1,5 +1,10 @@
--- ---
type: Log id: object/48e725ec-b9cb-43f1-f719-0e977fb11570
type: log
role: log
representation: standalone
state: current
knowledge_schema: 1
_organized: true _organized: true
--- ---

View file

@ -1,5 +1,10 @@
--- ---
type: Log id: object/b8d1e818-00ba-c4de-c12b-819b9bc92c48
type: log
role: log
representation: standalone
state: current
knowledge_schema: 1
_organized: true _organized: true
--- ---

View file

@ -1,5 +1,10 @@
--- ---
type: Log id: object/69dce38b-d44c-8361-ee76-128f4639c165
type: log
role: log
representation: standalone
state: current
knowledge_schema: 1
_organized: true _organized: true
--- ---

View file

@ -1,5 +1,10 @@
--- ---
type: Log id: object/10f3cd5a-aa77-9cfd-cfc0-f4ab7abd0906
type: log
role: log
representation: standalone
state: current
knowledge_schema: 1
_organized: true _organized: true
--- ---

View file

@ -1,5 +1,10 @@
--- ---
type: Log id: object/1748b435-164d-5a9b-674c-233b1a0a1cff
type: log
role: log
representation: standalone
state: current
knowledge_schema: 1
_organized: true _organized: true
--- ---

View file

@ -1,5 +1,10 @@
--- ---
type: Log id: object/03b4a75f-71b1-b33f-c960-c12e948560e1
type: log
role: log
representation: standalone
state: current
knowledge_schema: 1
_organized: true _organized: true
--- ---

View file

@ -1,5 +1,10 @@
--- ---
type: Log id: object/acf5b5da-0138-6bf8-b470-f80ba8ca7e03
type: log
role: log
representation: standalone
state: current
knowledge_schema: 1
_organized: true _organized: true
--- ---

View file

@ -1,5 +1,10 @@
--- ---
type: Log id: object/730534ed-b44a-46de-0082-268a476003c7
type: log
role: log
representation: standalone
state: current
knowledge_schema: 1
_organized: true _organized: true
--- ---

View file

@ -1,5 +1,10 @@
--- ---
type: Log id: object/1b4b6582-403a-d1d3-6f4c-a8f5ea868ddb
type: log
role: log
representation: standalone
state: current
knowledge_schema: 1
_organized: true _organized: true
--- ---

View file

@ -1,5 +1,10 @@
--- ---
type: Log id: object/866ae1ee-7fd7-4175-c6a7-5a5c0e663d8d
type: log
role: log
representation: standalone
state: current
knowledge_schema: 1
_organized: true _organized: true
--- ---

View file

@ -1,5 +1,10 @@
--- ---
type: Log id: object/6d67c1dc-044b-9e40-b519-b3dc57a85d3a
type: log
role: log
representation: standalone
state: current
knowledge_schema: 1
_organized: true _organized: true
--- ---

View file

@ -1,5 +1,10 @@
--- ---
type: Log id: object/286f4f60-ab5d-f872-ae6b-cb997c214dc5
type: log
role: log
representation: standalone
state: current
knowledge_schema: 1
_organized: true _organized: true
--- ---

View file

@ -1,5 +1,10 @@
--- ---
type: Log id: object/73fdf272-78c3-1c55-1198-84181898f9f0
type: log
role: log
representation: standalone
state: current
knowledge_schema: 1
_organized: true _organized: true
--- ---

View file

@ -1,5 +1,10 @@
--- ---
type: Log id: object/5a6cf599-8023-e788-1d6c-eb72bfa2ca58
type: log
role: log
representation: standalone
state: current
knowledge_schema: 1
_organized: true _organized: true
--- ---

View file

@ -1,5 +1,10 @@
--- ---
type: Log id: object/3fbb2b3c-921b-3db5-73f3-7de26261ef2f
type: log
role: log
representation: standalone
state: current
knowledge_schema: 1
_organized: true _organized: true
--- ---

View file

@ -1,5 +1,10 @@
--- ---
type: Log id: object/badaaaea-79fb-3c65-c1d3-f0028040204b
type: log
role: log
representation: standalone
state: current
knowledge_schema: 1
_organized: true _organized: true
--- ---

View file

@ -1,8 +1,14 @@
--- ---
type: Log id: object/43673951-0706-29e0-12bb-55b652456962
type: log
role: log
representation: standalone
state: current
knowledge_schema: 1
_organized: true _organized: true
--- ---
# T212 Logs — 14.08.2026 23:45 UTC # T212 Logs — 14.08.2026 23:45 UTC
## Offen (4) | Unrealisiert: +0.0% ## Offen (4) | Unrealisiert: +0.0%
- XRX_US_EQ | +0.0% | SL: 2.98€ - XRX_US_EQ | +0.0% | SL: 2.98€

View file

@ -1,5 +1,10 @@
--- ---
type: Log id: object/3b9de5f6-784d-a2dd-a9ea-d154450c3586
type: log
role: log
representation: standalone
state: current
knowledge_schema: 1
_organized: true _organized: true
--- ---

View file

@ -1,5 +1,10 @@
--- ---
type: Log id: object/5811e85a-191f-c7a3-8f2e-e3768ae15044
type: log
role: log
representation: standalone
state: current
knowledge_schema: 1
_organized: true _organized: true
--- ---

View file

@ -1,5 +1,10 @@
--- ---
type: Log id: object/3479a3df-f762-0cd1-bf71-2e29fff56cca
type: log
role: log
representation: standalone
state: current
knowledge_schema: 1
_organized: true _organized: true
--- ---

View file

@ -0,0 +1,105 @@
# D1/D2 — Canonical Schema Contract & Fingerprint Tooling (SoT)
**Status:** FORMALLY PERSISTED — authoritative origin/main
**Datum:** 2026-09-08
**Verdict:** D1+D2 FORMALLY PERSISTED TO AUTHORITATIVE SOT
---
## 1. Zweck
Dieses Verzeichnis materialisiert den **CURRENT VERIFIED PRODUCTION SCHEMA CONTRACT (L0)**
als kanonische, deterministisch fingerprintbare Autorität. Es ist das Ergebnis der
D1/D2-Remediation (Schema-Contract + deterministischer Fingerprint) und wurde durch den
unabhängigen D1+D2-Checker mit **Verdict A** bestätigt.
## 2. Canonical Fingerprint
```
b49f64337ffafc0e73d46e009aa802cd0172d59421b7df61d3eb198775c1986d
```
Dieser Fingerprint ist deterministisch (H1==H2==H3), sensitiv auf alle semantischen
Schema-Änderungen (type, nullable, default, PK, UNIQUE, FK, CHECK, index), und fail-closed.
## 3. DR-Reproducibility (ehrlich getrennt)
| Aspekt | Status |
|--------|--------|
| **CURRENT-STATE SCHEMA REPRODUCIBLE FROM L0** | **YES** — frische DB aus L0-SQL rekonstruiert, REBUILT_HASH == PRODUCTION_HASH == b49f6433…, Semantic-Diff TOTAL_MISMATCH=0 |
| **HISTORICAL EVOLUTION REPRODUCIBLE** | **NO** — historische DDL-Chronologie (Wave1→WaveD→Closure) bleibt nicht aus L0 rekonstruierbar |
Der L0-Contract beansprucht ausschliesslich **CURRENT PRODUCTION TRUTH**, nicht historische
Migrationschronologie. Historische Artefakte bleiben immutable.
## 4. Canonical Contract Semantics
| Tabelle | Spalten |
|---------|---------|
| historical_bar | 41 |
| derived_bar | 35 |
| data_ingestion_run | 29 |
| dataset_version | 23 |
| ingestion_provenance | 41 |
**Semantic Overrides (Owner-Canonical):**
- `last_successful_chunk` = **timestamp with time zone / timestamptz** (Production + Owner-Entscheidung)
- `interrupt_reason` = **text** · PRESERVED · origin unknown · do not remove
- `technical_quality` = **text** · nullable · default `'VALID'`
## 5. Historische Wahrheit (nicht rückwirkend ändern)
- **WaveD historisches Artefakt:** `last_successful_chunk = text` (immutable Evidence)
- **Owner/current canonical contract:** `last_successful_chunk = timestamptz`
- Historische Artefakte: Wave1 `6983526da3ca51fd2ececffbef0ccbd66b048a3d11c228000011aaba3e1d6bc5`,
WaveD `7aa4e2a10c98fa327e000a49f6a497efb5814c8808dba429dfebfc90c544a02b`,
Closure `e56567f7cfae68da70c11ece6f576e974488f343`**unverändert**.
## 6. Open Debts (nicht fälschlich geschlossen)
| Debt | Status |
|------|--------|
| **Debt A** — Current-state DR/schema reproducibility gap | **CLOSED** by D1/D2 |
| **Debt A** — Historical DDL evolution provenance gap | **REMAINS OPEN** |
| **Debt B** — interrupt_reason origin unknown | **OPEN** (non-blocking) |
| **Debt C** — WaveD text vs current Owner-canonical timestamptz | **OPEN** (non-blocking) |
## 7. SoT-Layout
```
d1d2-schema-contract/
├── SCHEMA_AUTHORITY.md # dieses Dokument
├── contract/
│ ├── historical_v2_schema.sql # L0-Schema-Contract (schema-only)
│ ├── historical_v2_schema.json # maschinenlesbares kanonisches Modell
│ ├── historical_v2_schema.sha256 # deterministischer Fingerprint
│ └── README.md # Contract-Dokumentation
└── tools/
├── canonical_schema.py # Kanonisierungs-Implementierung
├── capture_schema.py # Capture-Parser (read-only)
├── compare_schema.py # Semantic-Diff-CLI
├── fingerprint_schema.py # Fingerprint-CLI
└── gen_l0_sql.py # L0-SQL-Generator
```
**Nicht persistiert:** Maker-Evidence, Checker-Evidence, temporäre DBs, /tmp-Dateien,
raw Scratch Outputs, runtime-spezifische Dateien, Credentials/Secrets, Transport-Manifeste,
`gen_evidence.py` (enthält hardcodierte Maker-Pfade, nicht environment-neutral).
## 8. Tooling-Nutzung
```bash
# Fingerprint aus machine contract
python3 tools/fingerprint_schema.py contract/historical_v2_schema.json
# → b49f64337ffafc0e73d46e009aa802cd0172d59421b7df61d3eb198775c1986d
# Semantic-Diff (expected vs actual)
python3 tools/compare_schema.py contract/historical_v2_schema.json <actual_model.json>
# → TOTAL_MISMATCH=0 bei Übereinstimmung; Exit 1 bei Drift (fail-closed)
# L0-SQL generieren
python3 tools/gen_l0_sql.py contract/historical_v2_schema.json --out /tmp/l0.sql
```
Alle Tools sind environment-neutral (keine absoluten Pfade, keine DB-Credentials,
keine mutierenden Production-SQL-Pfade). Read-only.

View file

@ -0,0 +1,20 @@
# Historical V2 — Canonical Current Schema Contract (L0)
Diese Artefakte materialisieren den **CURRENT VERIFIED PRODUCTION SCHEMA CONTRACT**
als kanonische L0-Autorität (D1/D2 Remediation, forward-only).
- `historical_v2_schema.sql` — schema-only PostgreSQL-Contract (kein DML/Secrets/volatile Artefakte)
- `historical_v2_schema.json` — maschinenlesbares kanonisches Modell
- `historical_v2_schema.sha256` — deterministischer SHA-256-Fingerprint
- Kanonischer Fingerprint: `b49f64337ffafc0e73d46e009aa802cd0172d59421b7df61d3eb198775c1986d`
Dies ist KEINE historische Migrationskette. DO NOT INVENT HISTORY: der L0-Contract
beansprucht ausschliesslich CURRENT PRODUCTION TRUTH (§1).
Historische Artefakte (Wave1 `6983526da3ca51fd2ececffbef0ccbd66b048a3d11c228000011aaba3e1d6bc5`, WaveD `7aa4e2a10c98fa327e000a49f6a497efb5814c8808dba429dfebfc90c544a02b`) bleiben unverändert.
**Semantic Overrides (§4):**
- last_successful_chunk = timestamp with time zone (Production+Owner; WaveD-historical text bleibt immutable Evidence)
- interrupt_reason = text (preserved, origin unknown, do not remove)
- technical_quality = text nullable default 'VALID' (historical provenance debt OPEN)
**SoT:** origin/main = `e56567f7cfae68da70c11ece6f576e974488f343`

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1 @@
b49f64337ffafc0e73d46e009aa802cd0172d59421b7df61d3eb198775c1986d

View file

@ -0,0 +1,601 @@
-- ============================================================
-- CANONICAL CURRENT PRODUCTION SCHEMA CONTRACT (L0)
-- db: historical · schema: public
-- Auto-generated by gen_l0_sql.py from read-only capture
-- Modus: CURRENT VERIFIED PRODUCTION TRUTH — NICHT historische Migration
-- ============================================================
-- SEMANTIC OVERRIDES (Owner-Canonical)
-- last_successful_chunk = timestamp with time zone (Production + Owner decision)
-- interrupt_reason = text · PRESERVED CURRENT CONTRACT · origin unknown · do not remove
-- technical_quality = text · nullable · default 'VALID' · historical provenance debt OPEN
CREATE SEQUENCE IF NOT EXISTS "calendar_event_event_id_seq";
CREATE SEQUENCE IF NOT EXISTS "corporate_action_action_id_seq";
CREATE SEQUENCE IF NOT EXISTS "daily_quality_report_report_id_seq";
CREATE SEQUENCE IF NOT EXISTS "data_gap_gap_id_seq";
CREATE SEQUENCE IF NOT EXISTS "data_ingestion_run_ingestion_id_seq";
CREATE SEQUENCE IF NOT EXISTS "data_quality_issue_issue_id_seq";
CREATE SEQUENCE IF NOT EXISTS "dataset_dataset_id_seq";
CREATE SEQUENCE IF NOT EXISTS "dataset_version_dataset_version_id_seq";
CREATE SEQUENCE IF NOT EXISTS "derived_bar_derived_bar_id_seq";
CREATE SEQUENCE IF NOT EXISTS "historical_bar_bar_id_seq";
CREATE SEQUENCE IF NOT EXISTS "historical_tick_tick_id_seq";
CREATE SEQUENCE IF NOT EXISTS "ingestion_provenance_provenance_id_seq";
CREATE SEQUENCE IF NOT EXISTS "instrument_instrument_id_seq";
CREATE SEQUENCE IF NOT EXISTS "market_calendar_calendar_id_seq";
CREATE SEQUENCE IF NOT EXISTS "provider_provider_id_seq";
-- Table: calendar_event (9 cols)
CREATE TABLE IF NOT EXISTS "calendar_event" (
"event_id" bigint NOT NULL DEFAULT nextval('calendar_event_event_id_seq'::regclass),
"calendar_id" bigint NOT NULL,
"event_date" date NOT NULL,
"kind" text NOT NULL,
"reason" text,
"session_phase_override" text,
"source" text,
"notes" text,
"created_at" timestamp with time zone NOT NULL DEFAULT now()
);
-- Table: corporate_action (9 cols)
CREATE TABLE IF NOT EXISTS "corporate_action" (
"action_id" bigint NOT NULL DEFAULT nextval('corporate_action_action_id_seq'::regclass),
"instrument_id" bigint NOT NULL,
"action_type" text NOT NULL,
"ex_date" date NOT NULL,
"ratio_num" double precision,
"ratio_denom" double precision,
"dividend" double precision,
"currency" text,
"provider_id" bigint
);
-- Table: daily_quality_report (23 cols)
CREATE TABLE IF NOT EXISTS "daily_quality_report" (
"report_id" bigint NOT NULL DEFAULT nextval('daily_quality_report_report_id_seq'::regclass),
"instrument_id" bigint NOT NULL,
"provider_id" bigint NOT NULL,
"report_date" date NOT NULL,
"feed_type" text DEFAULT 'REFERENCE'::text,
"expected_m1" bigint,
"received_m1" bigint,
"eligible" bigint,
"excluded" bigint,
"conditional" bigint,
"stale_count" bigint,
"zero_volume" bigint,
"spread_median_price" double precision,
"spread_p95_price" double precision,
"spread_p99_price" double precision,
"gaps_count" bigint,
"coverage_ratio" double precision,
"session_anomalies" bigint,
"status" text NOT NULL,
"quality_model_version" text,
"session_model_version" text,
"calendar_version" text,
"generated_at" timestamp with time zone NOT NULL DEFAULT now()
);
-- Table: data_gap (9 cols)
CREATE TABLE IF NOT EXISTS "data_gap" (
"gap_id" bigint NOT NULL DEFAULT nextval('data_gap_gap_id_seq'::regclass),
"provider_id" bigint NOT NULL,
"instrument_id" bigint NOT NULL,
"timeframe" text NOT NULL,
"start_utc" timestamp with time zone NOT NULL,
"end_utc" timestamp with time zone NOT NULL,
"kind" text NOT NULL,
"backfilled" boolean NOT NULL DEFAULT false,
"filled_at" timestamp with time zone
);
-- Table: data_ingestion_run (29 cols)
CREATE TABLE IF NOT EXISTS "data_ingestion_run" (
"ingestion_id" bigint NOT NULL DEFAULT nextval('data_ingestion_run_ingestion_id_seq'::regclass),
"provider_id" bigint NOT NULL,
"instrument_id" bigint NOT NULL,
"timeframe" text NOT NULL,
"start_utc" timestamp with time zone,
"end_utc" timestamp with time zone,
"rows_inserted" bigint DEFAULT 0,
"rows_skipped" bigint DEFAULT 0,
"status" text NOT NULL DEFAULT 'RUNNING'::text,
"started_at" timestamp with time zone NOT NULL DEFAULT now(),
"finished_at" timestamp with time zone,
"checkpoint" jsonb,
"status_fsm" text DEFAULT 'PENDING'::text,
"rows_downloaded" bigint DEFAULT 0,
"rows_decoded" bigint DEFAULT 0,
"rows_valid" bigint DEFAULT 0,
"rows_suspect" bigint DEFAULT 0,
"rows_excluded" bigint DEFAULT 0,
"rows_derived" bigint DEFAULT 0,
"last_successful_chunk" timestamp with time zone,
"last_timestamp" timestamp with time zone,
"updated_at" timestamp with time zone DEFAULT now(),
"interrupt_reason" text,
"rows_existing" bigint DEFAULT 0,
"rows_deduplicated" bigint DEFAULT 0,
"rows_404" bigint DEFAULT 0,
"rows_503" bigint DEFAULT 0,
"rows_provider_unavailable" bigint DEFAULT 0,
"rows_failed" bigint DEFAULT 0
);
-- Table: data_quality_issue (9 cols)
CREATE TABLE IF NOT EXISTS "data_quality_issue" (
"issue_id" bigint NOT NULL DEFAULT nextval('data_quality_issue_issue_id_seq'::regclass),
"instrument_id" bigint NOT NULL,
"timeframe" text NOT NULL,
"ts_utc" timestamp with time zone,
"severity" text NOT NULL,
"issue_type" text NOT NULL,
"detail" jsonb,
"resolved" boolean NOT NULL DEFAULT false,
"created_at" timestamp with time zone NOT NULL DEFAULT now()
);
-- Table: dataset (6 cols)
CREATE TABLE IF NOT EXISTS "dataset" (
"dataset_id" bigint NOT NULL DEFAULT nextval('dataset_dataset_id_seq'::regclass),
"name" text NOT NULL,
"provider_id" bigint NOT NULL,
"feed_type" text NOT NULL,
"description" text,
"created_at" timestamp with time zone NOT NULL DEFAULT now()
);
-- Table: dataset_version (23 cols)
CREATE TABLE IF NOT EXISTS "dataset_version" (
"dataset_version_id" bigint NOT NULL DEFAULT nextval('dataset_version_dataset_version_id_seq'::regclass),
"dataset_id" bigint NOT NULL,
"version_label" text NOT NULL,
"provider_id" bigint NOT NULL,
"feed_type" text NOT NULL,
"instrument_id" bigint NOT NULL,
"timeframe" text NOT NULL,
"start_utc" timestamp with time zone NOT NULL,
"end_utc" timestamp with time zone NOT NULL,
"raw_source" text NOT NULL,
"normalization_version" text NOT NULL,
"aggregation_version" text,
"quality_version" text NOT NULL,
"dataset_version_hash" text NOT NULL,
"data_as_of" timestamp with time zone NOT NULL,
"notes" text,
"created_at" timestamp with time zone NOT NULL DEFAULT now(),
"quality_model_version" text,
"stale_model_version" text,
"session_model_version" text,
"calendar_version" text,
"eligibility_model_version" text,
"is_fixture" boolean NOT NULL DEFAULT false
);
-- Table: derived_bar (35 cols)
CREATE TABLE IF NOT EXISTS "derived_bar" (
"derived_bar_id" bigint NOT NULL DEFAULT nextval('derived_bar_derived_bar_id_seq'::regclass),
"provider_id" bigint NOT NULL,
"instrument_id" bigint NOT NULL,
"feed_type" text NOT NULL DEFAULT 'REFERENCE'::text,
"price_basis" text NOT NULL DEFAULT 'bid_ask'::text,
"timeframe" text NOT NULL,
"ts_utc" timestamp with time zone NOT NULL,
"source_timeframe" text NOT NULL,
"aggregation_version" text NOT NULL,
"bid_open" double precision,
"bid_high" double precision,
"bid_low" double precision,
"bid_close" double precision,
"ask_open" double precision,
"ask_high" double precision,
"ask_low" double precision,
"ask_close" double precision,
"mid_open" double precision,
"mid_high" double precision,
"mid_low" double precision,
"mid_close" double precision,
"volume" double precision,
"volume_type" text DEFAULT 'tick'::text,
"quality_status" text NOT NULL DEFAULT 'VALID'::text,
"quality_flags" _text,
"dataset_version_id" bigint NOT NULL,
"expected_bar_count" bigint,
"actual_bar_count" bigint,
"eligible_bar_count" bigint,
"excluded_bar_count" bigint,
"coverage_ratio" double precision,
"aggregation_quality" text,
"eligibility_model_version" text DEFAULT 'eligibility_v1'::text,
"session_model_version" text DEFAULT 'fx_session_v1'::text,
"calendar_version" text DEFAULT 'fx_cal_v1'::text
);
-- Table: historical_bar (41 cols)
CREATE TABLE IF NOT EXISTS "historical_bar" (
"bar_id" bigint NOT NULL DEFAULT nextval('historical_bar_bar_id_seq'::regclass),
"provider_id" bigint NOT NULL,
"instrument_id" bigint NOT NULL,
"feed_type" text NOT NULL DEFAULT 'REFERENCE'::text,
"price_basis" text NOT NULL DEFAULT 'bid_ask'::text,
"timeframe" text NOT NULL,
"ts_utc" timestamp with time zone NOT NULL,
"raw_or_derived" text NOT NULL DEFAULT 'raw'::text,
"source_timeframe" text,
"aggregation_version" text,
"bid_open" double precision,
"bid_high" double precision,
"bid_low" double precision,
"bid_close" double precision,
"ask_open" double precision,
"ask_high" double precision,
"ask_low" double precision,
"ask_close" double precision,
"mid_open" double precision,
"mid_high" double precision,
"mid_low" double precision,
"mid_close" double precision,
"volume" double precision,
"volume_type" text DEFAULT 'tick'::text,
"quality_status" text NOT NULL DEFAULT 'VALID'::text,
"quality_flags" _text,
"source_timestamp" timestamp with time zone,
"ingested_at" timestamp with time zone NOT NULL DEFAULT now(),
"dataset_version_id" bigint NOT NULL,
"technical_quality" text DEFAULT 'VALID'::text,
"market_quality" text,
"session_state" text,
"session_phase" text,
"eligibility" text DEFAULT 'ELIGIBLE'::text,
"exclusion_reason" text,
"quality_model_version" text DEFAULT 'quality_v1'::text,
"session_model_version" text,
"calendar_version" text,
"eligibility_model_version" text DEFAULT 'eligibility_v1'::text,
"effective_eligibility" text,
"stale_model_version" text
);
-- Table: historical_tick (10 cols)
CREATE TABLE IF NOT EXISTS "historical_tick" (
"tick_id" bigint NOT NULL DEFAULT nextval('historical_tick_tick_id_seq'::regclass),
"provider_id" bigint NOT NULL,
"instrument_id" bigint NOT NULL,
"feed_type" text NOT NULL DEFAULT 'REFERENCE'::text,
"ts_utc" timestamp with time zone NOT NULL,
"bid" double precision,
"ask" double precision,
"last" double precision,
"volume" double precision,
"dataset_version_id" bigint NOT NULL
);
-- Table: ingestion_provenance (41 cols)
CREATE TABLE IF NOT EXISTS "ingestion_provenance" (
"provenance_id" bigint NOT NULL,
"ingestion_run_id" bigint NOT NULL,
"provider_id" bigint NOT NULL,
"provider_name_snapshot" text NOT NULL,
"provider_environment_class" text NOT NULL,
"provider_dataset" text NOT NULL,
"provider_feed" text NOT NULL,
"provider_instrument_id" text NOT NULL,
"canonical_instrument_id" bigint NOT NULL,
"asset_class" text NOT NULL,
"request_start" timestamp with time zone NOT NULL,
"request_end" timestamp with time zone NOT NULL,
"actual_start" timestamp with time zone NOT NULL,
"actual_end" timestamp with time zone NOT NULL,
"request_timestamp" timestamp with time zone NOT NULL,
"response_timestamp" timestamp with time zone NOT NULL,
"http_status" integer NOT NULL,
"provider_metadata" jsonb,
"provider_version" text,
"raw_payload_sha256" text NOT NULL,
"raw_content_length" bigint NOT NULL,
"compression" text NOT NULL,
"content_type" text,
"parser_version" text NOT NULL,
"adapter_version" text NOT NULL,
"normalization_version" text NOT NULL,
"timezone_source" text NOT NULL,
"timezone_target" text NOT NULL,
"timezone_transform_version" text NOT NULL,
"price_type" text NOT NULL,
"granularity" text NOT NULL,
"correction_status" text NOT NULL DEFAULT 'ORIGINAL'::text,
"license_class" text NOT NULL DEFAULT 'UNKNOWN'::text,
"software_git_commit" text NOT NULL,
"software_image_id" text NOT NULL,
"software_build_input_sha" text,
"request_identity_hash" text NOT NULL,
"supersedes_provenance_id" bigint,
"raw_storage_ref" text,
"extra" jsonb,
"created_at" timestamp with time zone NOT NULL DEFAULT now()
);
-- Table: instrument (13 cols)
CREATE TABLE IF NOT EXISTS "instrument" (
"instrument_id" bigint NOT NULL DEFAULT nextval('instrument_instrument_id_seq'::regclass),
"symbol" text NOT NULL,
"asset_class" text NOT NULL,
"name" text,
"currency" text,
"exchange" text,
"mic" text,
"isin" text,
"tick_size" double precision,
"contract_size" double precision,
"min_deal_size" double precision,
"active" boolean NOT NULL DEFAULT true,
"created_at" timestamp with time zone NOT NULL DEFAULT now()
);
-- Table: market_calendar (13 cols)
CREATE TABLE IF NOT EXISTS "market_calendar" (
"calendar_id" bigint NOT NULL DEFAULT nextval('market_calendar_calendar_id_seq'::regclass),
"calendar_name" text NOT NULL,
"asset_class" text NOT NULL,
"venue_exchange" text,
"timezone" text NOT NULL,
"week_open_utc" text,
"week_close_utc" text,
"rollover_utc" text,
"session_open_utc" text,
"session_close_utc" text,
"notes" text,
"calendar_version" text NOT NULL,
"created_at" timestamp with time zone NOT NULL DEFAULT now()
);
-- Table: provider (6 cols)
CREATE TABLE IF NOT EXISTS "provider" (
"provider_id" bigint NOT NULL DEFAULT nextval('provider_provider_id_seq'::regclass),
"provider_key" text NOT NULL,
"name" text NOT NULL,
"data_type" text NOT NULL,
"pricing_plan" text,
"license_terms" text
);
-- Table: provider_instrument (6 cols)
CREATE TABLE IF NOT EXISTS "provider_instrument" (
"provider_id" bigint NOT NULL,
"instrument_id" bigint NOT NULL,
"provider_symbol" text NOT NULL,
"broker_epic" text,
"extra_metadata" jsonb,
"active" boolean NOT NULL DEFAULT true
);
ALTER TABLE "calendar_event" ADD CONSTRAINT "calendar_event_pkey" PRIMARY KEY ("event_id");
ALTER TABLE "corporate_action" ADD CONSTRAINT "corporate_action_pkey" PRIMARY KEY ("action_id");
ALTER TABLE "daily_quality_report" ADD CONSTRAINT "daily_quality_report_pkey" PRIMARY KEY ("report_id");
ALTER TABLE "data_gap" ADD CONSTRAINT "data_gap_pkey" PRIMARY KEY ("gap_id");
ALTER TABLE "data_ingestion_run" ADD CONSTRAINT "data_ingestion_run_pkey" PRIMARY KEY ("ingestion_id");
ALTER TABLE "data_quality_issue" ADD CONSTRAINT "data_quality_issue_pkey" PRIMARY KEY ("issue_id");
ALTER TABLE "dataset" ADD CONSTRAINT "dataset_pkey" PRIMARY KEY ("dataset_id");
ALTER TABLE "dataset_version" ADD CONSTRAINT "dataset_version_pkey" PRIMARY KEY ("dataset_version_id");
ALTER TABLE "derived_bar" ADD CONSTRAINT "derived_bar_pkey" PRIMARY KEY ("derived_bar_id");
ALTER TABLE "historical_bar" ADD CONSTRAINT "historical_bar_pkey" PRIMARY KEY ("bar_id");
ALTER TABLE "historical_tick" ADD CONSTRAINT "historical_tick_pkey" PRIMARY KEY ("tick_id");
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "ingestion_provenance_pkey" PRIMARY KEY ("provenance_id");
ALTER TABLE "instrument" ADD CONSTRAINT "instrument_pkey" PRIMARY KEY ("instrument_id");
ALTER TABLE "market_calendar" ADD CONSTRAINT "market_calendar_pkey" PRIMARY KEY ("calendar_id");
ALTER TABLE "provider" ADD CONSTRAINT "provider_pkey" PRIMARY KEY ("provider_id");
ALTER TABLE "provider_instrument" ADD CONSTRAINT "provider_instrument_pkey" PRIMARY KEY ("provider_id", "instrument_id");
ALTER TABLE "calendar_event" ADD CONSTRAINT "calendar_event_calendar_id_event_date_kind_key" UNIQUE ("calendar_id", "event_date", "kind");
ALTER TABLE "corporate_action" ADD CONSTRAINT "corporate_action_instrument_id_action_type_ex_date_provider_key" UNIQUE ("instrument_id", "action_type", "ex_date", "provider_id");
ALTER TABLE "daily_quality_report" ADD CONSTRAINT "daily_quality_report_instrument_id_provider_id_report_date__key" UNIQUE ("instrument_id", "provider_id", "report_date", "feed_type");
ALTER TABLE "data_gap" ADD CONSTRAINT "data_gap_unique" UNIQUE ("provider_id", "instrument_id", "timeframe", "start_utc", "end_utc", "kind");
ALTER TABLE "data_ingestion_run" ADD CONSTRAINT "data_ingestion_run_provider_id_instrument_id_timeframe_star_key" UNIQUE ("provider_id", "instrument_id", "timeframe", "started_at");
ALTER TABLE "dataset" ADD CONSTRAINT "dataset_name_key" UNIQUE ("name");
ALTER TABLE "dataset_version" ADD CONSTRAINT "dataset_version_dataset_version_hash_key" UNIQUE ("dataset_version_hash");
ALTER TABLE "derived_bar" ADD CONSTRAINT "derived_bar_provider_id_instrument_id_feed_type_price_basis_key" UNIQUE ("provider_id", "instrument_id", "feed_type", "price_basis", "timeframe", "ts_utc", "aggregation_version");
ALTER TABLE "historical_bar" ADD CONSTRAINT "historical_bar_provider_id_instrument_id_feed_type_price_ba_key" UNIQUE ("provider_id", "instrument_id", "feed_type", "price_basis", "timeframe", "ts_utc", "raw_or_derived");
ALTER TABLE "historical_tick" ADD CONSTRAINT "historical_tick_provider_id_instrument_id_feed_type_ts_utc__key" UNIQUE ("provider_id", "instrument_id", "feed_type", "ts_utc", "bid", "ask");
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "ingestion_provenance_ingestion_run_id_key" UNIQUE ("ingestion_run_id");
ALTER TABLE "instrument" ADD CONSTRAINT "instrument_symbol_key" UNIQUE ("symbol");
ALTER TABLE "market_calendar" ADD CONSTRAINT "market_calendar_calendar_name_key" UNIQUE ("calendar_name");
ALTER TABLE "provider" ADD CONSTRAINT "provider_provider_key_key" UNIQUE ("provider_key");
ALTER TABLE "provider_instrument" ADD CONSTRAINT "provider_instrument_provider_id_provider_symbol_key" UNIQUE ("provider_id", "provider_symbol");
ALTER TABLE "calendar_event" ADD CONSTRAINT "calendar_event_calendar_id_fkey" FOREIGN KEY ("calendar_id") REFERENCES "market_calendar" ("calendar_id");
ALTER TABLE "corporate_action" ADD CONSTRAINT "corporate_action_instrument_id_fkey" FOREIGN KEY ("instrument_id") REFERENCES "instrument" ("instrument_id");
ALTER TABLE "corporate_action" ADD CONSTRAINT "corporate_action_provider_id_fkey" FOREIGN KEY ("provider_id") REFERENCES "provider" ("provider_id");
ALTER TABLE "daily_quality_report" ADD CONSTRAINT "daily_quality_report_instrument_id_fkey" FOREIGN KEY ("instrument_id") REFERENCES "instrument" ("instrument_id");
ALTER TABLE "daily_quality_report" ADD CONSTRAINT "daily_quality_report_provider_id_fkey" FOREIGN KEY ("provider_id") REFERENCES "provider" ("provider_id");
ALTER TABLE "data_gap" ADD CONSTRAINT "data_gap_instrument_id_fkey" FOREIGN KEY ("instrument_id") REFERENCES "instrument" ("instrument_id");
ALTER TABLE "data_gap" ADD CONSTRAINT "data_gap_provider_id_fkey" FOREIGN KEY ("provider_id") REFERENCES "provider" ("provider_id");
ALTER TABLE "data_ingestion_run" ADD CONSTRAINT "data_ingestion_run_instrument_id_fkey" FOREIGN KEY ("instrument_id") REFERENCES "instrument" ("instrument_id");
ALTER TABLE "data_ingestion_run" ADD CONSTRAINT "data_ingestion_run_provider_id_fkey" FOREIGN KEY ("provider_id") REFERENCES "provider" ("provider_id");
ALTER TABLE "data_quality_issue" ADD CONSTRAINT "data_quality_issue_instrument_id_fkey" FOREIGN KEY ("instrument_id") REFERENCES "instrument" ("instrument_id");
ALTER TABLE "dataset" ADD CONSTRAINT "dataset_provider_id_fkey" FOREIGN KEY ("provider_id") REFERENCES "provider" ("provider_id");
ALTER TABLE "dataset_version" ADD CONSTRAINT "dataset_version_dataset_id_fkey" FOREIGN KEY ("dataset_id") REFERENCES "dataset" ("dataset_id");
ALTER TABLE "dataset_version" ADD CONSTRAINT "dataset_version_instrument_id_fkey" FOREIGN KEY ("instrument_id") REFERENCES "instrument" ("instrument_id");
ALTER TABLE "dataset_version" ADD CONSTRAINT "dataset_version_provider_id_fkey" FOREIGN KEY ("provider_id") REFERENCES "provider" ("provider_id");
ALTER TABLE "derived_bar" ADD CONSTRAINT "derived_bar_dataset_version_id_fkey" FOREIGN KEY ("dataset_version_id") REFERENCES "dataset_version" ("dataset_version_id");
ALTER TABLE "derived_bar" ADD CONSTRAINT "derived_bar_instrument_id_fkey" FOREIGN KEY ("instrument_id") REFERENCES "instrument" ("instrument_id");
ALTER TABLE "derived_bar" ADD CONSTRAINT "derived_bar_provider_id_fkey" FOREIGN KEY ("provider_id") REFERENCES "provider" ("provider_id");
ALTER TABLE "historical_bar" ADD CONSTRAINT "historical_bar_dataset_version_id_fkey" FOREIGN KEY ("dataset_version_id") REFERENCES "dataset_version" ("dataset_version_id");
ALTER TABLE "historical_bar" ADD CONSTRAINT "historical_bar_instrument_id_fkey" FOREIGN KEY ("instrument_id") REFERENCES "instrument" ("instrument_id");
ALTER TABLE "historical_bar" ADD CONSTRAINT "historical_bar_provider_id_fkey" FOREIGN KEY ("provider_id") REFERENCES "provider" ("provider_id");
ALTER TABLE "historical_tick" ADD CONSTRAINT "historical_tick_dataset_version_id_fkey" FOREIGN KEY ("dataset_version_id") REFERENCES "dataset_version" ("dataset_version_id");
ALTER TABLE "historical_tick" ADD CONSTRAINT "historical_tick_instrument_id_fkey" FOREIGN KEY ("instrument_id") REFERENCES "instrument" ("instrument_id");
ALTER TABLE "historical_tick" ADD CONSTRAINT "historical_tick_provider_id_fkey" FOREIGN KEY ("provider_id") REFERENCES "provider" ("provider_id");
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "ingestion_provenance_canonical_instrument_id_fkey" FOREIGN KEY ("canonical_instrument_id") REFERENCES "instrument" ("instrument_id");
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "ingestion_provenance_provider_id_fkey" FOREIGN KEY ("provider_id") REFERENCES "provider" ("provider_id");
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "ingestion_provenance_supersedes_fkey" FOREIGN KEY ("supersedes_provenance_id") REFERENCES "ingestion_provenance" ("provenance_id");
ALTER TABLE "provider_instrument" ADD CONSTRAINT "provider_instrument_instrument_id_fkey" FOREIGN KEY ("instrument_id") REFERENCES "instrument" ("instrument_id");
ALTER TABLE "provider_instrument" ADD CONSTRAINT "provider_instrument_provider_id_fkey" FOREIGN KEY ("provider_id") REFERENCES "provider" ("provider_id");
ALTER TABLE "calendar_event" ADD CONSTRAINT "2200_16679_1_not_null" CHECK (event_id IS NOT NULL);
ALTER TABLE "calendar_event" ADD CONSTRAINT "2200_16679_2_not_null" CHECK (calendar_id IS NOT NULL);
ALTER TABLE "calendar_event" ADD CONSTRAINT "2200_16679_3_not_null" CHECK (event_date IS NOT NULL);
ALTER TABLE "calendar_event" ADD CONSTRAINT "2200_16679_4_not_null" CHECK (kind IS NOT NULL);
ALTER TABLE "calendar_event" ADD CONSTRAINT "2200_16679_9_not_null" CHECK (created_at IS NOT NULL);
ALTER TABLE "corporate_action" ADD CONSTRAINT "2200_16567_1_not_null" CHECK (action_id IS NOT NULL);
ALTER TABLE "corporate_action" ADD CONSTRAINT "2200_16567_2_not_null" CHECK (instrument_id IS NOT NULL);
ALTER TABLE "corporate_action" ADD CONSTRAINT "2200_16567_3_not_null" CHECK (action_type IS NOT NULL);
ALTER TABLE "corporate_action" ADD CONSTRAINT "2200_16567_4_not_null" CHECK (ex_date IS NOT NULL);
ALTER TABLE "daily_quality_report" ADD CONSTRAINT "2200_16696_19_not_null" CHECK (status IS NOT NULL);
ALTER TABLE "daily_quality_report" ADD CONSTRAINT "2200_16696_1_not_null" CHECK (report_id IS NOT NULL);
ALTER TABLE "daily_quality_report" ADD CONSTRAINT "2200_16696_23_not_null" CHECK (generated_at IS NOT NULL);
ALTER TABLE "daily_quality_report" ADD CONSTRAINT "2200_16696_2_not_null" CHECK (instrument_id IS NOT NULL);
ALTER TABLE "daily_quality_report" ADD CONSTRAINT "2200_16696_3_not_null" CHECK (provider_id IS NOT NULL);
ALTER TABLE "daily_quality_report" ADD CONSTRAINT "2200_16696_4_not_null" CHECK (report_date IS NOT NULL);
ALTER TABLE "data_gap" ADD CONSTRAINT "2200_16613_1_not_null" CHECK (gap_id IS NOT NULL);
ALTER TABLE "data_gap" ADD CONSTRAINT "2200_16613_2_not_null" CHECK (provider_id IS NOT NULL);
ALTER TABLE "data_gap" ADD CONSTRAINT "2200_16613_3_not_null" CHECK (instrument_id IS NOT NULL);
ALTER TABLE "data_gap" ADD CONSTRAINT "2200_16613_4_not_null" CHECK (timeframe IS NOT NULL);
ALTER TABLE "data_gap" ADD CONSTRAINT "2200_16613_5_not_null" CHECK (start_utc IS NOT NULL);
ALTER TABLE "data_gap" ADD CONSTRAINT "2200_16613_6_not_null" CHECK (end_utc IS NOT NULL);
ALTER TABLE "data_gap" ADD CONSTRAINT "2200_16613_7_not_null" CHECK (kind IS NOT NULL);
ALTER TABLE "data_gap" ADD CONSTRAINT "2200_16613_8_not_null" CHECK (backfilled IS NOT NULL);
ALTER TABLE "data_ingestion_run" ADD CONSTRAINT "2200_16588_10_not_null" CHECK (started_at IS NOT NULL);
ALTER TABLE "data_ingestion_run" ADD CONSTRAINT "2200_16588_1_not_null" CHECK (ingestion_id IS NOT NULL);
ALTER TABLE "data_ingestion_run" ADD CONSTRAINT "2200_16588_2_not_null" CHECK (provider_id IS NOT NULL);
ALTER TABLE "data_ingestion_run" ADD CONSTRAINT "2200_16588_3_not_null" CHECK (instrument_id IS NOT NULL);
ALTER TABLE "data_ingestion_run" ADD CONSTRAINT "2200_16588_4_not_null" CHECK (timeframe IS NOT NULL);
ALTER TABLE "data_ingestion_run" ADD CONSTRAINT "2200_16588_9_not_null" CHECK (status IS NOT NULL);
ALTER TABLE "data_quality_issue" ADD CONSTRAINT "2200_16633_1_not_null" CHECK (issue_id IS NOT NULL);
ALTER TABLE "data_quality_issue" ADD CONSTRAINT "2200_16633_2_not_null" CHECK (instrument_id IS NOT NULL);
ALTER TABLE "data_quality_issue" ADD CONSTRAINT "2200_16633_3_not_null" CHECK (timeframe IS NOT NULL);
ALTER TABLE "data_quality_issue" ADD CONSTRAINT "2200_16633_5_not_null" CHECK (severity IS NOT NULL);
ALTER TABLE "data_quality_issue" ADD CONSTRAINT "2200_16633_6_not_null" CHECK (issue_type IS NOT NULL);
ALTER TABLE "data_quality_issue" ADD CONSTRAINT "2200_16633_8_not_null" CHECK (resolved IS NOT NULL);
ALTER TABLE "data_quality_issue" ADD CONSTRAINT "2200_16633_9_not_null" CHECK (created_at IS NOT NULL);
ALTER TABLE "dataset" ADD CONSTRAINT "2200_16430_1_not_null" CHECK (dataset_id IS NOT NULL);
ALTER TABLE "dataset" ADD CONSTRAINT "2200_16430_2_not_null" CHECK (name IS NOT NULL);
ALTER TABLE "dataset" ADD CONSTRAINT "2200_16430_3_not_null" CHECK (provider_id IS NOT NULL);
ALTER TABLE "dataset" ADD CONSTRAINT "2200_16430_4_not_null" CHECK (feed_type IS NOT NULL);
ALTER TABLE "dataset" ADD CONSTRAINT "2200_16430_6_not_null" CHECK (created_at IS NOT NULL);
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_10_not_null" CHECK (raw_source IS NOT NULL);
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_11_not_null" CHECK (normalization_version IS NOT NULL);
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_13_not_null" CHECK (quality_version IS NOT NULL);
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_14_not_null" CHECK (dataset_version_hash IS NOT NULL);
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_15_not_null" CHECK (data_as_of IS NOT NULL);
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_17_not_null" CHECK (created_at IS NOT NULL);
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_1_not_null" CHECK (dataset_version_id IS NOT NULL);
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_23_not_null" CHECK (is_fixture IS NOT NULL);
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_2_not_null" CHECK (dataset_id IS NOT NULL);
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_3_not_null" CHECK (version_label IS NOT NULL);
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_4_not_null" CHECK (provider_id IS NOT NULL);
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_5_not_null" CHECK (feed_type IS NOT NULL);
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_6_not_null" CHECK (instrument_id IS NOT NULL);
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_7_not_null" CHECK (timeframe IS NOT NULL);
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_8_not_null" CHECK (start_utc IS NOT NULL);
ALTER TABLE "dataset_version" ADD CONSTRAINT "2200_16447_9_not_null" CHECK (end_utc IS NOT NULL);
ALTER TABLE "derived_bar" ADD CONSTRAINT "2200_16508_1_not_null" CHECK (derived_bar_id IS NOT NULL);
ALTER TABLE "derived_bar" ADD CONSTRAINT "2200_16508_24_not_null" CHECK (quality_status IS NOT NULL);
ALTER TABLE "derived_bar" ADD CONSTRAINT "2200_16508_26_not_null" CHECK (dataset_version_id IS NOT NULL);
ALTER TABLE "derived_bar" ADD CONSTRAINT "2200_16508_2_not_null" CHECK (provider_id IS NOT NULL);
ALTER TABLE "derived_bar" ADD CONSTRAINT "2200_16508_3_not_null" CHECK (instrument_id IS NOT NULL);
ALTER TABLE "derived_bar" ADD CONSTRAINT "2200_16508_4_not_null" CHECK (feed_type IS NOT NULL);
ALTER TABLE "derived_bar" ADD CONSTRAINT "2200_16508_5_not_null" CHECK (price_basis IS NOT NULL);
ALTER TABLE "derived_bar" ADD CONSTRAINT "2200_16508_6_not_null" CHECK (timeframe IS NOT NULL);
ALTER TABLE "derived_bar" ADD CONSTRAINT "2200_16508_7_not_null" CHECK (ts_utc IS NOT NULL);
ALTER TABLE "derived_bar" ADD CONSTRAINT "2200_16508_8_not_null" CHECK (source_timeframe IS NOT NULL);
ALTER TABLE "derived_bar" ADD CONSTRAINT "2200_16508_9_not_null" CHECK (aggregation_version IS NOT NULL);
ALTER TABLE "historical_bar" ADD CONSTRAINT "2200_16474_1_not_null" CHECK (bar_id IS NOT NULL);
ALTER TABLE "historical_bar" ADD CONSTRAINT "2200_16474_25_not_null" CHECK (quality_status IS NOT NULL);
ALTER TABLE "historical_bar" ADD CONSTRAINT "2200_16474_28_not_null" CHECK (ingested_at IS NOT NULL);
ALTER TABLE "historical_bar" ADD CONSTRAINT "2200_16474_29_not_null" CHECK (dataset_version_id IS NOT NULL);
ALTER TABLE "historical_bar" ADD CONSTRAINT "2200_16474_2_not_null" CHECK (provider_id IS NOT NULL);
ALTER TABLE "historical_bar" ADD CONSTRAINT "2200_16474_3_not_null" CHECK (instrument_id IS NOT NULL);
ALTER TABLE "historical_bar" ADD CONSTRAINT "2200_16474_4_not_null" CHECK (feed_type IS NOT NULL);
ALTER TABLE "historical_bar" ADD CONSTRAINT "2200_16474_5_not_null" CHECK (price_basis IS NOT NULL);
ALTER TABLE "historical_bar" ADD CONSTRAINT "2200_16474_6_not_null" CHECK (timeframe IS NOT NULL);
ALTER TABLE "historical_bar" ADD CONSTRAINT "2200_16474_7_not_null" CHECK (ts_utc IS NOT NULL);
ALTER TABLE "historical_bar" ADD CONSTRAINT "2200_16474_8_not_null" CHECK (raw_or_derived IS NOT NULL);
ALTER TABLE "historical_tick" ADD CONSTRAINT "2200_16539_10_not_null" CHECK (dataset_version_id IS NOT NULL);
ALTER TABLE "historical_tick" ADD CONSTRAINT "2200_16539_1_not_null" CHECK (tick_id IS NOT NULL);
ALTER TABLE "historical_tick" ADD CONSTRAINT "2200_16539_2_not_null" CHECK (provider_id IS NOT NULL);
ALTER TABLE "historical_tick" ADD CONSTRAINT "2200_16539_3_not_null" CHECK (instrument_id IS NOT NULL);
ALTER TABLE "historical_tick" ADD CONSTRAINT "2200_16539_4_not_null" CHECK (feed_type IS NOT NULL);
ALTER TABLE "historical_tick" ADD CONSTRAINT "2200_16539_5_not_null" CHECK (ts_utc IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_10_not_null" CHECK (asset_class IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_11_not_null" CHECK (request_start IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_12_not_null" CHECK (request_end IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_13_not_null" CHECK (actual_start IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_14_not_null" CHECK (actual_end IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_15_not_null" CHECK (request_timestamp IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_16_not_null" CHECK (response_timestamp IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_17_not_null" CHECK (http_status IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_1_not_null" CHECK (provenance_id IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_20_not_null" CHECK (raw_payload_sha256 IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_21_not_null" CHECK (raw_content_length IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_22_not_null" CHECK (compression IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_24_not_null" CHECK (parser_version IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_25_not_null" CHECK (adapter_version IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_26_not_null" CHECK (normalization_version IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_27_not_null" CHECK (timezone_source IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_28_not_null" CHECK (timezone_target IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_29_not_null" CHECK (timezone_transform_version IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_2_not_null" CHECK (ingestion_run_id IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_30_not_null" CHECK (price_type IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_31_not_null" CHECK (granularity IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_32_not_null" CHECK (correction_status IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_33_not_null" CHECK (license_class IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_34_not_null" CHECK (software_git_commit IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_35_not_null" CHECK (software_image_id IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_37_not_null" CHECK (request_identity_hash IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_3_not_null" CHECK (provider_id IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_41_not_null" CHECK (created_at IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_4_not_null" CHECK (provider_name_snapshot IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_5_not_null" CHECK (provider_environment_class IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_6_not_null" CHECK (provider_dataset IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_7_not_null" CHECK (provider_feed IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_8_not_null" CHECK (provider_instrument_id IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "2200_16741_9_not_null" CHECK (canonical_instrument_id IS NOT NULL);
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "chk_prov_actual_range" CHECK (((actual_end >= actual_start)));
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "chk_prov_compression" CHECK (((compression = ANY (ARRAY['none'::text, 'lzma'::text, 'gzip'::text, 'zstd'::text, 'deflate'::text, 'unknown'::text]))));
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "chk_prov_content_length" CHECK (((raw_content_length >= 0)));
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "chk_prov_correction" CHECK (((correction_status = ANY (ARRAY['ORIGINAL'::text, 'CORRECTED'::text, 'SUPERSEDED'::text, 'UNKNOWN'::text]))));
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "chk_prov_env_class" CHECK (((provider_environment_class = ANY (ARRAY['PROD'::text, 'SANDBOX'::text, 'TEST'::text, 'UNKNOWN'::text]))));
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "chk_prov_http_status" CHECK ((((http_status >= 100) AND (http_status <= 599))));
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "chk_prov_license" CHECK (((license_class = ANY (ARRAY['UNKNOWN'::text, 'PRIVATE_INTERNAL'::text, 'INTERNAL_DERIVED_ALLOWED'::text, 'STORAGE_RESTRICTED'::text, 'REDISTRIBUTION_RESTRICTED'::text, 'EXPIRED'::text, 'REVOKED'::text]))));
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "chk_prov_price_type" CHECK (((price_type = ANY (ARRAY['bid'::text, 'ask'::text, 'mid'::text, 'bid_ask'::text, 'last'::text, 'unknown'::text]))));
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "chk_prov_sha256" CHECK (((raw_payload_sha256 ~ '^[0-9a-f]{64}$'::text)));
ALTER TABLE "ingestion_provenance" ADD CONSTRAINT "chk_prov_time_range" CHECK (((request_end >= request_start)));
ALTER TABLE "instrument" ADD CONSTRAINT "2200_16386_12_not_null" CHECK (active IS NOT NULL);
ALTER TABLE "instrument" ADD CONSTRAINT "2200_16386_13_not_null" CHECK (created_at IS NOT NULL);
ALTER TABLE "instrument" ADD CONSTRAINT "2200_16386_1_not_null" CHECK (instrument_id IS NOT NULL);
ALTER TABLE "instrument" ADD CONSTRAINT "2200_16386_2_not_null" CHECK (symbol IS NOT NULL);
ALTER TABLE "instrument" ADD CONSTRAINT "2200_16386_3_not_null" CHECK (asset_class IS NOT NULL);
ALTER TABLE "market_calendar" ADD CONSTRAINT "2200_16667_12_not_null" CHECK (calendar_version IS NOT NULL);
ALTER TABLE "market_calendar" ADD CONSTRAINT "2200_16667_13_not_null" CHECK (created_at IS NOT NULL);
ALTER TABLE "market_calendar" ADD CONSTRAINT "2200_16667_1_not_null" CHECK (calendar_id IS NOT NULL);
ALTER TABLE "market_calendar" ADD CONSTRAINT "2200_16667_2_not_null" CHECK (calendar_name IS NOT NULL);
ALTER TABLE "market_calendar" ADD CONSTRAINT "2200_16667_3_not_null" CHECK (asset_class IS NOT NULL);
ALTER TABLE "market_calendar" ADD CONSTRAINT "2200_16667_5_not_null" CHECK (timezone IS NOT NULL);
ALTER TABLE "provider" ADD CONSTRAINT "2200_16399_1_not_null" CHECK (provider_id IS NOT NULL);
ALTER TABLE "provider" ADD CONSTRAINT "2200_16399_2_not_null" CHECK (provider_key IS NOT NULL);
ALTER TABLE "provider" ADD CONSTRAINT "2200_16399_3_not_null" CHECK (name IS NOT NULL);
ALTER TABLE "provider" ADD CONSTRAINT "2200_16399_4_not_null" CHECK (data_type IS NOT NULL);
ALTER TABLE "provider_instrument" ADD CONSTRAINT "2200_16409_1_not_null" CHECK (provider_id IS NOT NULL);
ALTER TABLE "provider_instrument" ADD CONSTRAINT "2200_16409_2_not_null" CHECK (instrument_id IS NOT NULL);
ALTER TABLE "provider_instrument" ADD CONSTRAINT "2200_16409_3_not_null" CHECK (provider_symbol IS NOT NULL);
ALTER TABLE "provider_instrument" ADD CONSTRAINT "2200_16409_6_not_null" CHECK (active IS NOT NULL);
CREATE INDEX idx_bar_dsver ON public.historical_bar USING btree (dataset_version_id);
CREATE INDEX idx_bar_elig ON public.historical_bar USING btree (eligibility);
CREATE INDEX idx_bar_inst_tf_ts ON public.historical_bar USING btree (instrument_id, timeframe, ts_utc);
CREATE INDEX idx_bar_sess_ts ON public.historical_bar USING btree (session_state, instrument_id, ts_utc);
CREATE INDEX idx_bar_session ON public.historical_bar USING btree (session_state, session_phase);
CREATE INDEX idx_bar_techq ON public.historical_bar USING btree (technical_quality);
CREATE INDEX idx_derived_inst_tf_ts ON public.derived_bar USING btree (instrument_id, timeframe, ts_utc);
CREATE INDEX idx_historical_bar_market_quality ON public.historical_bar USING btree (instrument_id, market_quality);
CREATE INDEX idx_historical_bar_session_state ON public.historical_bar USING btree (instrument_id, session_state);
CREATE INDEX idx_prov_adapter_version ON public.ingestion_provenance USING btree (adapter_version);
CREATE INDEX idx_prov_instrument ON public.ingestion_provenance USING btree (canonical_instrument_id);
CREATE INDEX idx_prov_provider_feed ON public.ingestion_provenance USING btree (provider_id, provider_feed);
CREATE INDEX idx_prov_raw_sha ON public.ingestion_provenance USING btree (raw_payload_sha256);
CREATE INDEX idx_prov_request_identity ON public.ingestion_provenance USING btree (request_identity_hash);
CREATE INDEX idx_prov_time_range ON public.ingestion_provenance USING btree (request_start, request_end);
CREATE INDEX idx_tick_inst_ts ON public.historical_tick USING btree (instrument_id, ts_utc);

View file

@ -0,0 +1,174 @@
#!/usr/bin/env python3
"""
D2 canonical_schema.py
Deterministischer Schema-Kanonisierer + Fingerprint für das Historical V2 Subsystem.
Liefert EINE autoritative Kanonisierungsfunktion, die aus einem Captured-Schema-Modell
(JSON, wie von capture_schema.py erzeugt) eine deterministische semantische
Serialisierung baut und daraus den SHA-256-Fingerprint berechnet.
Kanonisierungsregeln (siehe Mission §7):
- Schemas/Tabellen alphabetisch
- Spalten nach ordinal_position
- Constraints deterministisch nach (type, name, content)
- Indexe deterministisch
- Whitespace/Identifier-Quoting normalisiert, aber keine semantischen Unterschiede weggewaschen
Fingerprint-Coverage (§8): Tabellenexistenz, Spaltenname, ordinal_position,
Datentyp, Nullability, Default, Primary Key, UNIQUE, FK, CHECK, Index-Definitionen.
"""
import hashlib
import json
def _norm_whitespace(s):
"""Normalisiere Whitespace; reduziere Wiederholungen, aber erhalte Wörter/Zeichen."""
if s is None:
return None
return " ".join(str(s).split())
def sorted_cols(columns):
"""Spalten einer Tabelle, sortiert nach ordinal_position."""
return sorted(columns, key=lambda c: (c.get("ordinal", 0), c.get("column", "")))
def canonical_columns(columns):
"""Spalten → deterministische Liste von Tuplen."""
out = []
for c in sorted_cols(columns):
out.append((
c["column"],
int(c.get("ordinal", 0)),
_norm_whitespace(c.get("type")),
_norm_whitespace(c.get("udt")),
bool(c.get("nullable")),
_norm_whitespace(c.get("default")),
))
return out
def canonical_constraints(constraints, kind):
"""Constraints eines Typs (PRIMARY KEY, UNIQUE) → sortierte Liste."""
out = []
for con in constraints:
cols = tuple(c for c in con.get("columns") or [])
out.append((
con.get("table", ""),
con.get("constraint", ""),
cols,
))
return sorted(out, key=lambda x: (x[0], x[1], x[2]))
def canonical_fks(fks):
"""FK → sortierte Liste inkl. ref_table und ref_columns."""
out = []
for fk in fks:
out.append((
fk.get("table", ""),
fk.get("constraint", ""),
tuple(fk.get("columns") or []),
fk.get("ref_table", ""),
tuple(fk.get("ref_columns") or []),
))
return sorted(out, key=lambda x: (x[0], x[1]))
def canonical_checks(checks):
"""CHECK-Constraints → sortierte Liste mit normierter Definition.
Auto-generierte PostgreSQL 'NOT NULL'-Checks (CONSTRAINT-Name endet auf
'_not_null', von PostgreSQL automatisch pro NOT-NULL-Spalte erzeugt) werden
AUSGESCHLOSSEN: sie sind redundant zu 'nullable' und ihre Definition
unterscheidet sich zwischen DBs (Production 'x IS NOT NULL' vs. Checker
'((x IS NOT NULL))') semantisches Rauschen. Zuverlässige Erkennung NUR
über den Namenssuffix '_not_null'. Explizit benannte User-Checks
(chk_* etc.) bleiben erhalten.
"""
out = []
for chk in checks:
name = chk.get("constraint", "")
if name.endswith("_not_null"):
continue
out.append((
chk.get("table", ""),
name,
_norm_whitespace(chk.get("definition")) or "",
))
return sorted(out, key=lambda x: (x[0], x[1], x[2]))
def canonical_indexes(indexes):
"""Index-Definitionen → sortierte Liste mit normierter Definition."""
out = []
for idx in indexes:
out.append((
idx.get("table", ""),
idx.get("index", ""),
_norm_whitespace(idx.get("definition")),
))
return sorted(out, key=lambda x: (x[0], x[1], x[2]))
def canonical_sequences(sequences):
"""Sequenzen → sortierte Liste (nur Existenz/Name)."""
return sorted(sequences or [])
def canonical_document(model):
"""Baue die vollständige deterministische semantische Serialisierung."""
tables = sorted(model.get("tables") or [])
cols_by_table = {}
for c in model.get("columns") or []:
cols_by_table.setdefault(c["table"], []).append(c)
pks = canonical_constraints(model.get("primary_keys") or [], "PRIMARY KEY")
uniques = canonical_constraints(model.get("unique_constraints") or [], "UNIQUE")
fks = canonical_fks(model.get("foreign_keys") or [])
checks = canonical_checks(model.get("check_constraints") or [])
indexes = canonical_indexes(model.get("indexes") or [])
sequences = canonical_sequences(model.get("sequences") or [])
doc = []
doc.append("schema=public")
doc.append("sequences=" + "|".join(sequences))
doc.append("tables=" + "|".join(tables))
for t in tables:
doc.append(f"table:{t}:cols=" + ";".join(
f"{name}|{ordp}|{typ}|{udt}|{'n' if nul else 'y'}|{dflt}"
for (name, ordp, typ, udt, nul, dflt) in canonical_columns(cols_by_table.get(t, []))
))
doc.append("pk=" + ";".join(
f"{t}|{n}|{','.join(c)}"
for (t, n, c) in pks
))
doc.append("unique=" + ";".join(
f"{t}|{n}|{','.join(c)}"
for (t, n, c) in uniques
))
doc.append("fk=" + ";".join(
f"{t}|{n}|{','.join(c)}->{rt}|{','.join(rc)}"
for (t, n, c, rt, rc) in fks
))
doc.append("check=" + ";".join(
f"{t}|{n}|{d}"
for (t, n, d) in checks
))
doc.append("index=" + ";".join(
f"{t}|{n}|{d}"
for (t, n, d) in indexes
))
return "\n".join(doc)
def fingerprint(model):
"""SHA-256 über die kanonische Serialisierung."""
doc = canonical_document(model)
return hashlib.sha256(doc.encode("utf-8")).hexdigest(), doc
def load_model(path):
"""Lade ein Captured-Schema-Modell (JSON)."""
with open(path) as f:
return json.load(f)

View file

@ -0,0 +1,95 @@
#!/usr/bin/env python3
"""
D2 capture_schema.py
Liest Schema-Metadaten (read-only) von einer DB und erzeugt das kanonische Modell-JSON.
Quellenmodus: Erwartet eine Datei mit den 7 JSON-Aggregaten aus dem Capture-SQL
(der read-only psql-Ausgabe), ODER ein bereits geparstes Modell-JSON.
Usage:
capture_schema.py --from-capture <capture.json> --out <model.json>
capture_schema.py --from-model <model.json> --out <model.json> # Normalisierung
Der DB-Zugriff selbst erfolgt ausschließlich über das read-only Capture-SQL
(BEGIN TRANSACTION READ ONLY). Dieses Tool verarbeitet nur die Ergebnisse.
"""
import json
import sys
import argparse
AGG_NAMES = ["columns", "primary_keys", "unique_constraints",
"foreign_keys", "check_constraints", "indexes", "tables",
"sequences"]
def strip_psql_noise(text):
"""Entferne psql-Kontrollzeilen (BEGIN/on/COMMIT/leer/--)."""
out = []
for line in text.splitlines():
s = line.strip()
if s and s not in ("BEGIN", "on", "COMMIT") and not s.startswith("--"):
out.append(s)
return out
def parse_raw_capture(text):
"""Extrahiere die N JSON-Aggregate aus der psql-Ausgabe.
psql splittet große JSON-Arrays über mehrere Zeilen; daher wird nicht
zeilenzählend, sondern per JSON-Streaming (raw_decode) geparst.
"""
data = "\n".join(strip_psql_noise(text))
decoder = json.JSONDecoder()
aggs = []
i = 0
n = len(data)
while i < n and len(aggs) < len(AGG_NAMES):
while i < n and data[i] in " \t\r\n":
i += 1
if i >= n:
break
if data[i] not in "[{":
# non-JSON noise (z.B. Fehlermeldung) -> nächste Position
i += 1
continue
try:
val, end = decoder.raw_decode(data, i)
except json.JSONDecodeError as e:
raise SystemExit(f"FEHLER: JSON-Parse bei {i}: {e}")
aggs.append(val if val is not None else [])
i = end
if len(aggs) != len(AGG_NAMES):
raise SystemExit(
f"FEHLER: erwartete {len(AGG_NAMES)} JSON-Aggregate, fand {len(aggs)}")
return dict(zip(AGG_NAMES, aggs))
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--from-capture", help="Rohe read-only psql-Capture-Datei (7 Aggregate)")
ap.add_argument("--from-model", help="Bereits geparstes Modell-JSON")
ap.add_argument("--out", required=True, help="Ziel-Modell-JSON")
args = ap.parse_args()
if args.from_capture:
with open(args.from_capture) as f:
model = parse_raw_capture(f.read())
elif args.from_model:
with open(args.from_model) as f:
model = json.load(f)
else:
raise SystemExit("FEHLER: --from-capture oder --from-model erforderlich")
# Konsistenz: columns/tables müssen übereinstimmen
col_tables = sorted({c["table"] for c in model["columns"]})
model["tables"] = sorted(model.get("tables") or col_tables)
with open(args.out, "w") as f:
json.dump(model, f, indent=2, sort_keys=True)
print(f"Modell geschrieben: {args.out}")
print(f"Tabellen: {len(model['tables'])} | Spalten: {len(model['columns'])}")
return 0
if __name__ == "__main__":
sys.exit(main())

View file

@ -0,0 +1,108 @@
#!/usr/bin/env python3
"""
D2 compare_schema.py
Vergleicht zwei Schema-Modelle (Soll = kanonischer L0-Contract, Ist = tatsächlich).
Semantische Diff-Dimensionen (§12):
fehlende/extras Tabellen, Spalten, Typ-Mismatches, Nullable-Mismatches,
Default-Mismatches, PK, UNIQUE, FK, CHECK, Index-Mismatches.
Fail-closed: exit-Nonzero bei beliebigem semantischem Mismatch.
Acceptance: alle Mismatch-Counts == 0.
"""
import sys
import json
import argparse
from canonical_schema import (canonical_columns, canonical_constraints,
canonical_fks, canonical_checks, canonical_indexes)
def diff_counts(actual, expected):
"""Liefert dict mit Mismatch-Counts. 0 = keine."""
counts = {
"missing_tables": 0, "extra_tables": 0,
"missing_columns": 0, "extra_columns": 0,
"type_mismatch": 0, "nullable_mismatch": 0, "default_mismatch": 0,
"pk_mismatch": 0, "unique_mismatch": 0, "fk_mismatch": 0,
"check_mismatch": 0, "index_mismatch": 0,
}
# Tabellen
a_tbl = set((actual.get("tables") or []))
e_tbl = set((expected.get("tables") or []))
counts["missing_tables"] = len(e_tbl - a_tbl)
counts["extra_tables"] = len(a_tbl - e_tbl)
# Spalten je Tabelle
a_cols = {}
for c in actual.get("columns") or []:
a_cols.setdefault(c["table"], []).append(c)
e_cols = {}
for c in expected.get("columns") or []:
e_cols.setdefault(c["table"], []).append(c)
for tbl in e_tbl:
ec = {c["column"]: c for c in e_cols.get(tbl, [])}
ac = {c["column"]: c for c in a_cols.get(tbl, [])}
for cname, cexp in ec.items():
if cname not in ac:
counts["missing_columns"] += 1
continue
cact = ac[cname]
if _n(cact.get("type")) != _n(cexp.get("type")):
counts["type_mismatch"] += 1
if bool(cact.get("nullable")) != bool(cexp.get("nullable")):
counts["nullable_mismatch"] += 1
if _n(cact.get("default")) != _n(cexp.get("default")):
counts["default_mismatch"] += 1
for cname in ac:
if cname not in ec:
counts["extra_columns"] += 1
# Constraints
if canonical_constraints(actual.get("primary_keys") or [], "PK") != \
canonical_constraints(expected.get("primary_keys") or [], "PK"):
counts["pk_mismatch"] += 1
if canonical_constraints(actual.get("unique_constraints") or [], "UNIQUE") != \
canonical_constraints(expected.get("unique_constraints") or [], "UNIQUE"):
counts["unique_mismatch"] += 1
if canonical_fks(actual.get("foreign_keys") or []) != \
canonical_fks(expected.get("foreign_keys") or []):
counts["fk_mismatch"] += 1
if canonical_checks(actual.get("check_constraints") or []) != \
canonical_checks(expected.get("check_constraints") or []):
counts["check_mismatch"] += 1
if canonical_indexes(actual.get("indexes") or []) != \
canonical_indexes(expected.get("indexes") or []):
counts["index_mismatch"] += 1
return counts
def _n(x):
return None if x is None else " ".join(str(x).split())
def main():
ap = argparse.ArgumentParser()
ap.add_argument("expected", help="Kanonisches L0-Contract-Modell")
ap.add_argument("actual", help="Tatsächlich gecaptures Modell")
args = ap.parse_args()
with open(args.expected) as f:
expected = json.load(f)
with open(args.actual) as f:
actual = json.load(f)
counts = diff_counts(actual, expected)
total = sum(counts.values())
print(json.dumps(counts, indent=2))
print(f"TOTAL_MISMATCH={total}")
if total == 0:
print("PASS: Semantic diff = 0")
return 0
print("FAIL: semantic drift detected", file=sys.stderr)
return 1 # FAIL CLOSED
if __name__ == "__main__":
sys.exit(main())

View file

@ -0,0 +1,47 @@
#!/usr/bin/env python3
"""
D2 fingerprint_schema.py
Deterministischer SHA-256 über ein Captured-Schema-Modell.
Usage:
fingerprint_schema.py <model.json>
schreibt nur den Hex-Digest auf stdout (fail-closed: exit 0 bei Erfolg)
fingerprint_schema.py <model.json> --expect <digest>
exit 0 wenn MATCH, sonst exit 1 (FAIL CLOSED)
Der Fingerprint deckt ab: Tabellenexistenz, Spaltenname, ordinal_position,
Datentyp, Nullability, Default, PK, UNIQUE, FK, CHECK, Index-Definitionen (§8).
"""
import sys
import argparse
from canonical_schema import fingerprint, load_model
def main():
ap = argparse.ArgumentParser()
ap.add_argument("model")
ap.add_argument("--expect", default=None, help="Erwarteter Digest (fail-closed Vergleich)")
ap.add_argument("--no-hash", action="store_true", help="Nur kanonische Serialisierung ausgeben")
args = ap.parse_args()
model = load_model(args.model)
digest, doc = fingerprint(model)
if args.no_hash:
print(doc)
return 0
if args.expect:
if digest == args.expect:
print(f"MATCH {digest}")
return 0
else:
print(f"MISMATCH expected={args.expect} actual={digest}", file=sys.stderr)
return 1 # FAIL CLOSED
print(digest)
return 0
if __name__ == "__main__":
sys.exit(main())

View file

@ -0,0 +1,136 @@
#!/usr/bin/env python3
"""
D2 gen_l0_sql.py
Erzeugt den kanonischen L0-SQL-Contract (schema-only) aus einem Captured-Schema-Modell.
Repräsentiert die aktuelle kanonische Production-Truth. Enthält KEINE:
data rows, credentials, volatile timestamps, container IDs, host paths, secrets.
Kein CREATE DATABASE, keine Grant/GRANT-relevanten Owner, keine Umgebungs-Sequenzen.
Semantische Overrides (§4) werden als Kommentar-Blöcke dokumentiert:
last_successful_chunk = timestamp with time zone (Owner-canonical)
interrupt_reason = text (preserved current contract)
technical_quality = text nullable default 'VALID'
"""
import json
import sys
import argparse
TYPE_MAP = {
'bigint': 'bigint', 'integer': 'integer', 'smallint': 'smallint',
'numeric': 'numeric', 'timestamp with time zone': 'timestamp with time zone',
'boolean': 'boolean', 'text': 'text', 'date': 'date', 'double precision': 'double precision',
'jsonb': 'jsonb', 'real': 'real', 'character varying': 'character varying',
}
def ident(s):
return '"' + str(s).replace('"', '""') + '"'
def col_def(c):
typ = TYPE_MAP.get(c.get('type'), c.get('udt', c.get('type', 'text')))
nullable = '' if not c.get('nullable') else ''
notnull = ' NOT NULL' if not c.get('nullable') else ''
dflt = c.get('default')
default_part = ''
if dflt:
default_part = ' DEFAULT ' + dflt
return f" {ident(c['column'])} {typ}{notnull}{default_part}"
def build_sql(model, dbname="historical"):
out = []
out.append("-- ============================================================")
out.append(f"-- CANONICAL CURRENT PRODUCTION SCHEMA CONTRACT (L0)")
out.append(f"-- db: {dbname} · schema: public")
out.append("-- Auto-generated by gen_l0_sql.py from read-only capture")
out.append("-- Modus: CURRENT VERIFIED PRODUCTION TRUTH — NICHT historische Migration")
out.append("-- ============================================================")
out.append("")
out.append("-- SEMANTIC OVERRIDES (Owner-Canonical)")
out.append("-- last_successful_chunk = timestamp with time zone (Production + Owner decision)")
out.append("-- interrupt_reason = text · PRESERVED CURRENT CONTRACT · origin unknown · do not remove")
out.append("-- technical_quality = text · nullable · default 'VALID' · historical provenance debt OPEN")
out.append("")
# Sequenzen (für nextval-Defaults)
for seq in sorted(model.get("sequences") or []):
out.append(f"CREATE SEQUENCE IF NOT EXISTS {ident(seq)};")
if model.get("sequences"):
out.append("")
tables = sorted(model.get("tables") or [])
cols_by_table = {}
for c in model.get("columns") or []:
cols_by_table.setdefault(c["table"], []).append(c)
pks = {}
for pk in model.get("primary_keys") or []:
cols = sorted(pk.get("columns") or [], key=lambda x: 0)
pks[pk["table"]] = [pk.get("columns") or []]
for t in tables:
cols = sorted(cols_by_table.get(t, []), key=lambda c: c.get("ordinal", 0))
out.append(f"-- Table: {t} ({len(cols)} cols)")
out.append(f"CREATE TABLE IF NOT EXISTS {ident(t)} (")
body_lines = [col_def(c) for c in cols]
out.append(",\n".join(body_lines))
out.append(");")
out.append("")
# PKs
for t in tables:
for pk in model.get("primary_keys") or []:
if pk["table"] == t:
colstr = ", ".join(ident(c) for c in pk.get("columns") or [])
out.append(f"ALTER TABLE {ident(t)} ADD CONSTRAINT {ident(pk['constraint'])} PRIMARY KEY ({colstr});")
out.append("")
# UNIQUE
for uq in sorted(model.get("unique_constraints") or [], key=lambda x: x["table"]):
colstr = ", ".join(ident(c) for c in uq.get("columns") or [])
out.append(f"ALTER TABLE {ident(uq['table'])} ADD CONSTRAINT {ident(uq['constraint'])} UNIQUE ({colstr});")
out.append("")
# FK
for fk in sorted(model.get("foreign_keys") or [], key=lambda x: x["table"]):
colstr = ", ".join(ident(c) for c in fk.get("columns") or [])
rcolstr = ", ".join(ident(c) for c in fk.get("ref_columns") or [])
out.append(f"ALTER TABLE {ident(fk['table'])} ADD CONSTRAINT {ident(fk['constraint'])} FOREIGN KEY ({colstr}) REFERENCES {ident(fk['ref_table'])} ({rcolstr});")
out.append("")
# CHECK - semantische Definition
for chk in sorted(model.get("check_constraints") or [], key=lambda x: (x["table"], x["constraint"])):
out.append(f"ALTER TABLE {ident(chk['table'])} ADD CONSTRAINT {ident(chk['constraint'])} CHECK ({chk.get('definition','')});")
out.append("")
# Indexe (non-PK/UNIQUE)
idx_set = set()
for pk in model.get("primary_keys") or []:
idx_set.add(pk["constraint"])
for uq in model.get("unique_constraints") or []:
idx_set.add(uq["constraint"])
for idx in sorted(model.get("indexes") or [], key=lambda x: x["index"]):
if idx["index"] in idx_set:
continue
out.append(f"{idx['definition']};")
out.append("")
return "\n".join(out)
def main():
ap = argparse.ArgumentParser()
ap.add_argument("model")
ap.add_argument("--out", required=True)
args = ap.parse_args()
with open(args.model) as f:
model = json.load(f)
sql = build_sql(model)
with open(args.out, "w") as f:
f.write(sql)
print(f"L0 SQL-Contract geschrieben: {args.out} ({len(sql)} B)")
return 0
if __name__ == "__main__":
sys.exit(main())

View file

@ -1,5 +1,10 @@
--- ---
type: Trading-Modul-System id: object/90957134-8e8b-021b-717c-e9c7ca6cd887
type: arch
role: history
representation: canonical
state: historical
knowledge_schema: 1
_organized: true _organized: true
--- ---

View file

@ -1,3 +1,11 @@
---
id: object/ac1df780-019b-bb11-cb91-4edaa49e1843
type: arch
role: history
representation: canonical
state: historical
knowledge_schema: 1
---
# Phase 8 — M12 Historical-V2 DatasetGate + Run-Audit # Phase 8 — M12 Historical-V2 DatasetGate + Run-Audit
Autor: Rain Ocampo (Hermes) | Datum: 2026-08-22 | Status: IMPLEMENTIERT + GETESTET Autor: Rain Ocampo (Hermes) | Datum: 2026-08-22 | Status: IMPLEMENTIERT + GETESTET

View file

@ -1,9 +1,17 @@
--- ---
type: Trading-Modul-System knowledge_schema: 1
id: object/5fcf4885-a361-f49f-2164-f4e10709da34
type: arch
role: reference
representation: canonical
state: current
derived_from: object/f4e559f5-403c-f241-be50-0962d8174644
_organized: true _organized: true
--- ---
# Infrastruktur & Betriebs-Handbuch — Modul-Trading-System VPS
# Infrastruktur & Betriebs-Handbuch — Trading-System VPS
> Stand: 20.08.2026 · VPS: `187.124.31.123` > Stand: 20.08.2026 · VPS: `187.124.31.123`

View file

@ -1,5 +1,10 @@
--- ---
type: Trading-Modul-System id: object/ebdc6b2b-4b30-c839-729b-c240ad433a60
type: arch
role: module
representation: canonical
state: current
knowledge_schema: 1
_organized: true _organized: true
--- ---

View file

@ -1,5 +1,10 @@
--- ---
type: Trading-Modul-System id: object/b909ccbc-f7cf-7f71-46ea-817f0b697d35
type: arch
role: module
representation: canonical
state: current
knowledge_schema: 1
_organized: true _organized: true
--- ---

View file

@ -1,14 +1,22 @@
--- ---
type: Trading-Modul-System knowledge_schema: 1
id: object/cf54abfc-4c8b-7477-1b4f-21e76ca7b055
type: arch
role: module
representation: canonical
state: current
derived_from: object/1761a726-55af-f97b-d509-84aa4577f102
_organized: true _organized: true
--- ---
# Modul-03-Market-Data — Betriebsdokumentation # Modul-03-Market-Data — Betriebsdokumentation
> Erstellt: 20.08.2026 (Rain Ocampo) · Status: ✅ In Betrieb (healthy) > Erstellt: 20.08.2026 (Rain Ocampo) · Status: ✅ In Betrieb (healthy)
## Zweck ## Zweck
Zentrale Marktdatenquelle des Modul-Trading-Systems. Pipeline: Zentrale Marktdatenquelle des Trading-Systems. Pipeline:
`Marktdaten → Validierung → Normalisierung → PostgreSQL → RabbitMQ-Event` `Marktdaten → Validierung → Normalisierung → PostgreSQL → RabbitMQ-Event`
Keine Strategie, keine Orders, keine KI — nur zuverlässige Marktdaten. Keine Strategie, keine Orders, keine KI — nur zuverlässige Marktdaten.

View file

@ -1,8 +1,16 @@
--- ---
type: Trading-Modul-System knowledge_schema: 1
id: object/36d6dc60-1965-ed65-a002-cdb477ccd638
type: arch
role: module
representation: canonical
state: current
derived_from: object/6fca3e3a-70b2-0927-d7fb-e650a8f0e7f8
_organized: true _organized: true
--- ---
# Modul-04-Market-Regime — Betriebsdokumentation # Modul-04-Market-Regime — Betriebsdokumentation
> Erstellt: 20.08.2026 (Rain Ocampo) · Status: ✅ Freigegeben (E2E bestanden) > Erstellt: 20.08.2026 (Rain Ocampo) · Status: ✅ Freigegeben (E2E bestanden)

View file

@ -1,8 +1,16 @@
--- ---
type: Trading-Modul-System knowledge_schema: 1
id: object/58f1bc40-5b12-87a0-e95f-5288b91ebc87
type: arch
role: module
representation: canonical
state: current
derived_from: object/0aef122a-790b-b17a-9176-cc32df98c5a1
_organized: true _organized: true
--- ---
# Modul-05-Strategy-Engine — Betriebsdokumentation # Modul-05-Strategy-Engine — Betriebsdokumentation
> Erstellt: 20.08.2026 (Rain Ocampo) · Status: ✅ Freigegeben (E2E bestanden) > Erstellt: 20.08.2026 (Rain Ocampo) · Status: ✅ Freigegeben (E2E bestanden)

View file

@ -1,8 +1,16 @@
--- ---
type: Trading-Modul-System knowledge_schema: 1
id: object/5c04acf8-baf8-0771-f36b-05bf5cf58138
type: arch
role: module
representation: canonical
state: current
derived_from: object/2b4c4170-b5b2-8a2b-fdee-8ea5eff191b5
_organized: true _organized: true
--- ---
# Modul-06-Signal-Ranking — Betriebsdokumentation # Modul-06-Signal-Ranking — Betriebsdokumentation
> Erstellt: 20.08.2026 (Rain Ocampo) · Status: ✅ **Freigegeben** (E2E+Idempotenz+Reconnect+Doku grün) > Erstellt: 20.08.2026 (Rain Ocampo) · Status: ✅ **Freigegeben** (E2E+Idempotenz+Reconnect+Doku grün)

View file

@ -1,8 +1,16 @@
--- ---
type: Trading-Modul-System knowledge_schema: 1
id: object/d87e2842-ed93-9f5b-e8ea-3b4de31f4bc6
type: arch
role: module
representation: canonical
state: current
derived_from: object/645f8ff7-5a1a-bcf3-8dfe-0a8cfdf15dbc
_organized: true _organized: true
--- ---
# Modul-07: Risk-Manager # Modul-07: Risk-Manager
**Status:** ✅ **Freigegeben** (20.08.2026) **Status:** ✅ **Freigegeben** (20.08.2026)

View file

@ -1,8 +1,16 @@
--- ---
type: Trading-Modul-System knowledge_schema: 1
id: object/7ff050c3-ebaa-d0ae-5aa4-6e9da67d4773
type: arch
role: module
representation: canonical
state: current
derived_from: object/603e6b87-115a-e258-5bfb-7d94d6d78c97
_organized: true _organized: true
--- ---
# Modul-08: Portfolio-Manager # Modul-08: Portfolio-Manager
**Status: FREIGEGEBEN** (20.08.2026) **Status: FREIGEGEBEN** (20.08.2026)

View file

@ -1,8 +1,16 @@
--- ---
type: Trading-Modul-System knowledge_schema: 1
id: object/ffa8db13-8aa6-d4a4-6756-69eabd0c8d77
type: arch
role: module
representation: canonical
state: current
derived_from: object/48bd264f-607b-15f1-5f73-3e922af9b19d
_organized: true _organized: true
--- ---
# Modul-09: Execution-Service # Modul-09: Execution-Service
**Status: FREIGEGEBEN** (20.08.2026) **Status: FREIGEGEBEN** (20.08.2026)
@ -50,10 +58,65 @@ RabbitMQPublisher (market.execution)
`submit_order()`, `get_order_status()` `submit_order()`, `get_order_status()`
- `PaperBrokerAdapter` (app/broker/paper.py): V1, simuliert realistischen - `PaperBrokerAdapter` (app/broker/paper.py): V1, simuliert realistischen
Lifecycle (SUBMITTED → FILLED, Partial Fill, Reject, Timeout) für Tests Lifecycle (SUBMITTED → FILLED, Partial Fill, Reject, Timeout) für Tests
- `IgDemoAdapter` (app/broker/ig_demo.py): IG-Markets-Demo-Broker (IG_DEMO-Modus),
echter externer Broker-Adapter. Liest/schreibt über IG REST API (demo-api.ig.com).
- `BrokerRegistry` (app/broker/registry.py): wählt Adapter anhand - `BrokerRegistry` (app/broker/registry.py): wählt Adapter anhand
`DEFAULT_BROKER` (V1: paper). Echte Broker-Adapter später austauschbar, `DEFAULT_BROKER` (V1: paper). Echte Broker-Adapter später austauschbar,
keine Brokerlogik im Core-Code. keine Brokerlogik im Core-Code.
## IG-Demo-Adapter (IG_DEMO-Modus, 21.08.2026)
**Status: FREIGEGEBEN / PRODUKTIV VERIFIZIERT** — erster echter externer Broker-Adapter.
- **Eigener Modus `IG_DEMO`** (nicht LIVE+Gate); PAPER/IG_DEMO/LIVE strikt getrennt.
- **Demo-Basis-URL erzwungen** (`demo-api.ig.com`), LIVE-Endpunkt (`api.ig.com`)
technisch blockiert, kein IG_DEMO→LIVE-Autowechsel.
- **Credentials ausschließlich VPS-Secret/ENV** (`.env.ig`, chmod 600/root-only);
nie in Code/Forgejo/Tolaria/DB/Logs.
- **Mengen instrumentabhängig** über `broker_mapping` (PostgreSQL); kein globales 1:1.
- **Epic-Mapping produktiv in PostgreSQL `broker_mapping`**; Forgejo nur Schema/Doku/Beispiele.
### Order-Aktions-Auswertung (CLOSE-Pfad-Fix, 21.08.2026)
`BrokerOrderRequest` trägt `action_type` (Default `OPEN`) + `broker_order_id`.
Der Service reicht beides durch; der Adapter wertet `action_type` aus:
| action_type | IG-Pfad |
|-------------|---------|
| `OPEN` / `INCREASE` | `POST /positions/otc` (create_position) |
| `CLOSE` / `REDUCE` | `POST /positions/otc` mit `_method:DELETE` (close_position) |
| unbekannt/ungültig | **FAIL-CLOSED** (`IG_UNSUPPORTED_ACTION`) |
**CLOSE-Direction korrekt umkehren:** CLOSE einer LONG/BUY-Position muss mit
**SELL** erfolgen (Gegenseite), nicht mit der Positionsrichtung. `_close_direction()`:
LONG→SELL, SHORT→BUY.
**broker_order_id/dealId sauber durchreichen:** `broker_order_id` wird vom Service
in die Order übernommen und an den Adapter durchgereicht; IG `dealId` wird als
`broker_order_id` persistiert.
**Unbekannte/ungültige Aktionen FAIL-CLOSED:** `_derive_action_type()` setzt
unbekannte Aktionen NICHT mehr auf OPEN zurück, sondern reicht sie durch →
Control blockt mit `UNKNOWN_ACTION_TYPE`, Adapter mit `IG_UNSUPPORTED_ACTION`.
### IG-Adapter-Tests (6/6 grün)
1. `test_unsupported_action_fail_closed` — unbekannte Aktion → FAIL-CLOSED
2. `test_close_direction_inverted` — CLOSE einer BUY-Position → SELL
3. `test_open_uses_create_position` — OPEN → create_position
4. `test_close_uses_close_position` — CLOSE → close_position mit `_method:DELETE`
5. `test_broker_order_id_passthrough` — broker_order_id/dealId durchgereicht
6. `test_fail_closed_without_credentials` — ohne Credentials → nicht konfiguriert
### Erster erfolgreicher IG-DEMO OPEN→CLOSE-E2E (21.08.2026)
- **OPEN** `IGE2E-US500-OPEN-1787292734` → dealId `DIAAAAYB46KMNAE`, size 1.0, BUY,
openLevel 7652.58 → **FILLED**
- **CLOSE-Bug gefunden + gefixt:** erster CLOSE sendete fälschlich als zweite OPEN
(action_type ignoriert) + falsche direction (BUY statt SELL)
- **CLOSE2** `IGE2E-US500-CLOSE2-1787293768` für `DIAAAAYB463S4AB` → **FILLED**
(filled_quantity 1, avg_fill_price 7652.19)
- **IG GET /positions: Anzahl 0** — alle Positionen geschlossen
- **Safety-Reset:** M09 zurück auf PAPER (`EXECUTION_MODE=PAPER`,
`TRADING_ENABLED=false`, `DEFAULT_BROKER=paper`)
## Idempotenz ## Idempotenz
- Unique-Index `uq_execution_order_src` auf `source_portfolio_decision_id` - Unique-Index `uq_execution_order_src` auf `source_portfolio_decision_id`
@ -89,10 +152,11 @@ sowie `REJECTED`, `CANCELLED`, `FAILED`.
## Tests ## Tests
- **21/21 Unit-Tests** (test_execution.py): gültige Order → PAPER FILLED, - **31/31 Unit-Tests** (test_execution.py + test_ig_demo.py + test_control.py):
Idempotenz, ungültige Quantity → REJECTED, Kill-Switch, fehlende gültige Order → PAPER FILLED, Idempotenz, ungültige Quantity → REJECTED,
Broker-Credentials → FAIL-CLOSED, Broker-Reject, Timeout → keine blinde Kill-Switch, fehlende Broker-Credentials → FAIL-CLOSED, Broker-Reject,
Doppelorder, Partial Fill, Reconnect-Backoff, parallele identische Events Timeout → keine blinde Doppelorder, Partial Fill, Reconnect-Backoff,
parallele identische Events, Control-Gate (M15), IG-Adapter (6/6)
- **E2E 6/6 Checks** (Kette 03→04→05→06→07→08→09): Execution-Order in DB, - **E2E 6/6 Checks** (Kette 03→04→05→06→07→08→09): Execution-Order in DB,
exakt 1 Paper-Order, Pflichtfelder, Idempotenz, ORDER_SUBMITTED+ORDER_FILLED, exakt 1 Paper-Order, Pflichtfelder, Idempotenz, ORDER_SUBMITTED+ORDER_FILLED,
gültiger Trade → PAPER FILLED gültiger Trade → PAPER FILLED

View file

@ -1,8 +1,16 @@
--- ---
type: Trading-Modul-System knowledge_schema: 1
id: object/e2beb080-49cc-0b05-64fb-291440fe9abb
type: journal
role: module
representation: canonical
state: current
derived_from: object/0b393153-53e5-99fe-1098-e8bb4e0b6c7c
_organized: true _organized: true
--- ---
# Modul-10: Trade-Journal # Modul-10: Trade-Journal
**Status: FREIGEGEBEN** (20.08.2026) **Status: FREIGEGEBEN** (20.08.2026)

View file

@ -1,8 +1,16 @@
--- ---
type: Trading-Modul-System knowledge_schema: 1
id: object/c86dba29-835c-6b6a-d4f9-ac7cec69b8c4
type: arch
role: module
representation: canonical
state: current
derived_from: object/31cf1506-0ab6-5f7a-cecc-0e3a7a2e6572
_organized: true _organized: true
--- ---
# Modul-11: Analytics # Modul-11: Analytics
**Status: FREIGEGEBEN** · Container `Modul-11-Analytics` · Image `analytics-service:0.1.0` **Status: FREIGEGEBEN** · Container `Modul-11-Analytics` · Image `analytics-service:0.1.0`

View file

@ -1,8 +1,14 @@
--- ---
type: Trading-Modul-System knowledge_schema: 1
id: object/47b8029c-5874-e73f-0279-d335785b359a
type: arch
role: module
representation: canonical
state: current
derived_from: object/302e9929-e186-c930-2406-ad4a8f17c6fd
_organized: true _organized: true
--- ---
# Modul-12: Backtesting # Modul-12: Backtesting
**Status: FREIGEGEBEN** · Container `Modul-12-Backtesting` · Image `backtesting:0.1.2` (V1 + V1.1 parallel) **Status: FREIGEGEBEN** · Container `Modul-12-Backtesting` · Image `backtesting:0.1.2` (V1 + V1.1 parallel)
@ -121,10 +127,62 @@ Jede Tabelle FK auf `backtest_run(run_id) ON DELETE CASCADE`.
- **Modul-13: Optimization** — Image `optimization:0.1.0`, E2E-verifiziert (12/12 Punkte), **FREIGEGEBEN** (20.08.2026). Details: `modul-13-optimization.md`. - **Modul-13: Optimization** — Image `optimization:0.1.0`, E2E-verifiziert (12/12 Punkte), **FREIGEGEBEN** (20.08.2026). Details: `modul-13-optimization.md`.
- **V1.1-Strategie**: `trend_pullback_v1.1` (Image `backtesting:0.1.2`) — Spezifikation und Parameter: `trend-pullback-v1.1-spec.md`. - **V1.1-Strategie**: `trend_pullback_v1.1` (Image `backtesting:0.1.2`) — Spezifikation und Parameter: `trend-pullback-v1.1-spec.md`.
---
## Phase 10e — Intrabar Execution + Gap Execution Realism (23.08.2026)
Versionierung der Gap-/Intrabar-Semantik (KEIN separates `gap_execution_version`):
- Träger: `intrabar_policy` + `intrabar_policy_version = "intrabar_policy_v1"` im
`ExecutionContext` (`shared/historical/execution_context.py`). Dadurch ist die neue
Semantik eindeutig versioniert und ändert den V2-`run_hash` bei Policy-Wechsel
(PESSIMISTIC ≠ OPTIMISTIC → anderer run_hash).
- Legacy-Pfad bleibt unangetastet (PESSIMISTIC-Default, identisches Verhalten).
Intrabar-Policy (nur V2/BID_ASK): `PESSIMISTIC` | `OPTIMISTIC` | `STOP_FIRST` |
`TARGET_FIRST` | `TICK_RESOLUTION` | `UNKNOWN`. **Fail-closed** §18: `UNKNOWN`/
`TICK_RESOLUTION`/`BOGUS` → `ValueError`/`NotImplementedError`, kein Auto-Default.
Gap-Semantik (deterministisch, kein Lookahead):
- LONG: Stop/Target/Exit auf **BID**; SHORT auf **ASK**.
- Gap-Stop → Fill zum Gap-Open (schlechter, NICHT auf Stop-Level geclampt).
- Target-Gap → Fill zum besseren Gap-Open (nicht geclampt auf Target).
- Intrabar-Target füllt auf `target` (nicht low/high); Intrabar-Ambiguity (Stop UND
Target in einer Bar) per Policy (PESSIMISTIC→Stop, OPTIMISTIC→Target).
- Slippage wird adversial auf den tatsächlichen Fill (Trigger ≠ Market ≠ Final Fill
bei Gap + Slippage); Commission auf finalem Fill (rate × qty × fill pro Seite).
- **Kein Doppelspread/Slippage/Commission** (Vier-Ebenen-Trennung 10b/10c/10d/10e).
Audit (18 Felder, forensisch rekonstruierbar): `reason`, `intrabar_policy`,
`trigger_price`, `market_execution_price`, `exit_fill_price`, `gap_execution`,
`gap_open_price`, `execution_model`, `price_basis`, `spread_model`, `slippage_model`,
`slippage_value`, `cost_model`, `entry_fee`, `exit_fee`, `gross_pnl`, `net_pnl`
(verwendete vorhandene Feldnamen, keine künstlichen).
Verifikation 23.08.2026:
- Unit-Suite `test_phase10e_intrabar_gap.py`: 40/40 grün (Intrabar, Gap, Slippage
nach Gap, Commission auf Final Fill, BID/ASK-Seiten, Doppelzählung, Audit-18,
pathologische Fälle, 4 numerische PnL-E2E-Beispiele).
- Fixture-E2E (`test_phase10e_fixture_e2e.py`): 7/7 grün (A LONG Stop-Gap, B SHORT
Stop-Gap, C Intrabar PESSIMISTIC, D Intrabar OPTIMISTIC run_hash-Differenz,
E LONG Target-Gap, F SHORT Target-Gap, ENV-RESET); numerische Kette
Trigger → Gap-Open → Slippage → Final Fill → Commission → Gross → Net nachgewiesen.
- Regression Lauf 1 + Lauf 2: vollständig grün (M12 169 inkl. 10e, Phase5 20,
Phase6 14, Phase7 AJ Legacy-Hash `8a5760ae…`, M13 Shared AJ, M13 Compat 5/5).
- Legacy Production Smoke 2×: A==B==VORHER (run_hash `bc6e2553…`, data_hash
`d76a7549…`, net_pnl 196.586062, 1× LONG target). DB-Wahrheit verifiziert
(backtest_run/backtest_trade konsistent). Legacy unverändert durch Phase 10e.
- Production Gates M12+M13: `HISTORICAL_DATA_SOURCE`/`ALLOW_FIXTURE_DATA` UNSET.
- Deployt: `core/engine.py`, `core/service.py`, `api/schemas.py` (Container-Hashes
aktualisiert); `shared/historical/execution_context.py` + `pricing.py` unverändert.
--- ---
## Geändert ## Geändert
``` ```
Geändert von: Rain Ocampo Geändert von: Rain Ocampo
Datum: 23.08.2026
Grund: Modul-12-Doku um Phase 10e (Intrabar Execution + Gap Realism) ergänzt — Versionierung, Intrabar-Policy, Gap-Semantik, Audit-18, Verifikation, Deploy, Smoke, Production Gates.
```
```
Geändert von: Rain Ocampo
Datum: 20.08.2026 Datum: 20.08.2026
Grund: Modul-12-Backtesting als FREIGEGEBEN markiert (deployt + E2E verifiziert: deterministisch, kein Lookahead, Stop/Target/Gap, LONG/SHORT, run_hash+data_hash). Grund: Modul-12-Backtesting als FREIGEGEBEN markiert (deployt + E2E verifiziert: deterministisch, kein Lookahead, Stop/Target/Gap, LONG/SHORT, run_hash+data_hash).
``` ```

View file

@ -1,8 +1,16 @@
--- ---
type: Trading-Modul-System knowledge_schema: 1
id: object/2c48489e-0fb9-bdbf-8000-6bbba26771fb
type: arch
role: module
representation: canonical
state: current
derived_from: object/bde121b1-121f-590d-2b54-fedc13b02173
_organized: true _organized: true
--- ---
# Modul-13: Optimization # Modul-13: Optimization
**Status: FREIGEGEBEN ✅** · Container `Modul-13-Optimization` · Image `optimization:0.1.0` **Status: FREIGEGEBEN ✅** · Container `Modul-13-Optimization` · Image `optimization:0.1.0`

View file

@ -1,8 +1,16 @@
--- ---
type: Trading-Modul-System knowledge_schema: 1
id: object/ea077314-9e29-6c7f-1c17-008fa8e720ae
type: arch
role: module
representation: canonical
state: current
derived_from: object/0a0c9ca4-1cf4-aee1-3072-e1c3e8cfd29e
_organized: true _organized: true
--- ---
# Modul-14: Notification-Service # Modul-14: Notification-Service
**Status: FREIGEGEBEN ✅** · Container `Modul-14-Notification` · Image `notification:0.1.0` **Status: FREIGEGEBEN ✅** · Container `Modul-14-Notification` · Image `notification:0.1.0`

View file

@ -1,8 +1,16 @@
--- ---
type: Trading-Modul-System knowledge_schema: 1
id: object/5f28e1fd-19f4-e5af-1c92-a17ec80b08f0
type: design
role: design
representation: canonical
state: current
derived_from: object/1d6c2323-a31a-8b50-fcdd-3c9268175a69
_organized: true _organized: true
--- ---
# Design-Vorschlag: Modul-15 → Modul-09 Anbindung (Safety-/Trading-Control) # Design-Vorschlag: Modul-15 → Modul-09 Anbindung (Safety-/Trading-Control)
**Status: NUR DESIGN-VORSCHLAG — keine Implementierung.** **Status: NUR DESIGN-VORSCHLAG — keine Implementierung.**

View file

@ -1,8 +1,16 @@
--- ---
type: Trading-Modul-System knowledge_schema: 1
id: object/42b89fc3-4572-b493-c05e-dfb85a639df2
type: arch
role: implementation
representation: canonical
state: current
derived_from: object/a3cffdc1-41fd-ab84-c211-7b586f153e8d
_organized: true _organized: true
--- ---
# Modul-15 → Modul-09 Control-Anbindung — Implementierung & Live-E2E # Modul-15 → Modul-09 Control-Anbindung — Implementierung & Live-E2E
**Status: FREIGEGEBEN / PRODUKTIV VERIFIZIERT** (20.08.2026, Nutzer-Bestätigung). **Status: FREIGEGEBEN / PRODUKTIV VERIFIZIERT** (20.08.2026, Nutzer-Bestätigung).

View file

@ -1,14 +1,22 @@
--- ---
type: Trading-Modul-System knowledge_schema: 1
id: object/a50e821b-9c56-fe60-7b46-a0b607ef9e00
type: arch
role: module
representation: canonical
state: current
derived_from: object/c2641bbf-0612-0737-86fd-622d899109ca
_organized: true _organized: true
--- ---
# Modul-15: Monitoring-Control # Modul-15: Monitoring-Control
**Status: FREIGEGEBEN** (20.08.2026) · Container `Modul-15-Monitoring-Control` · Image `monitoring-control:0.1.0` **Status: FREIGEGEBEN** (20.08.2026) · Container `Modul-15-Monitoring-Control` · Image `monitoring-control:0.1.0`
## Zweck ## Zweck
Zentrale **technische Überwachung und Sicherheits-/Kontrollebene** des Modul-Trading-Systems. Zentrale **technische Überwachung und Sicherheits-/Kontrollebene** des Trading-Systems.
Überwacht die Dienste M03M14 (Health/Readiness/Erreichbarkeit) plus RabbitMQ-Queues/Consumer Überwacht die Dienste M03M14 (Health/Readiness/Erreichbarkeit) plus RabbitMQ-Queues/Consumer
und PostgreSQL und leitet daraus deterministisch einen **globalen Trading-Status** und PostgreSQL und leitet daraus deterministisch einen **globalen Trading-Status**
(`ENABLED` / `PAUSED` / `HALTED` / `UNKNOWN`) ab. (`ENABLED` / `PAUSED` / `HALTED` / `UNKNOWN`) ab.

View file

@ -1,3 +1,11 @@
---
id: object/eb0ef2f8-1ed9-aaf1-1afb-2fdbdaa15c74
type: arch
role: module
representation: canonical
state: current
knowledge_schema: 1
---
# Modul-16: Paperclip (Agent-Orchestrierungsebene) # Modul-16: Paperclip (Agent-Orchestrierungsebene)
**Status: FREIGEGEBEN (21.08.2026)** · Container `Modul-16-Paperclip` · Image `trading-modules-modul16-paperclip:latest` **Status: FREIGEGEBEN (21.08.2026)** · Container `Modul-16-Paperclip` · Image `trading-modules-modul16-paperclip:latest`

View file

@ -1,3 +1,11 @@
---
id: object/3063a680-c35e-a62c-91b4-f382027930a5
type: arch
role: module
representation: canonical
state: current
knowledge_schema: 1
---
# Modul-17: Hermes-Agent (autonomer Research-/Analyse-Worker unter Paperclip) # Modul-17: Hermes-Agent (autonomer Research-/Analyse-Worker unter Paperclip)
**Status: FREIGEGEBEN (21.08.2026)** · Container `Modul-17-Hermes-Agent` · Image `trading-modules-modul17-hermes-agent:latest` **Status: FREIGEGEBEN (21.08.2026)** · Container `Modul-17-Hermes-Agent` · Image `trading-modules-modul17-hermes-agent:latest`

View file

@ -1,3 +1,11 @@
---
id: object/3b41fc80-9d60-e640-a19c-1107114d8495
type: arch
role: module
representation: canonical
state: current
knowledge_schema: 1
---
# Modul-26-Telegram-Gateway # Modul-26-Telegram-Gateway
**Status:** FREIGEGEBEN (2026-08-21) **Status:** FREIGEGEBEN (2026-08-21)

View file

@ -1,3 +1,11 @@
---
id: object/086b7673-c183-6ca2-52fc-6f394107c9cf
type: arch
role: history
representation: canonical
state: historical
knowledge_schema: 1
---
# Phase 10a — ExecutionContext-Dataclass + Versionierung + Tests (M12/M13) # Phase 10a — ExecutionContext-Dataclass + Versionierung + Tests (M12/M13)
> Autoren: Rain Ocampo (Hermes) | Datum: 2026-08-23 > Autoren: Rain Ocampo (Hermes) | Datum: 2026-08-23

View file

@ -1,5 +1,10 @@
--- ---
type: Note id: object/da874dfe-edd4-a6ba-1587-a145ae045ddc
type: arch
role: history
representation: canonical
state: historical
knowledge_schema: 1
_organized: true _organized: true
--- ---

View file

@ -1,5 +1,10 @@
--- ---
type: Note id: object/dd4f96c5-6cc8-28e4-c8d4-7069c949fa3f
type: arch
role: history
representation: canonical
state: historical
knowledge_schema: 1
_organized: true _organized: true
--- ---

View file

@ -0,0 +1,90 @@
# Phase 13.5 — Runner Remediation + Control Plane + TrustGate Canonical Path (FORMAL CLOSURE)
**Status:** FORMALLY CLOSED (Dokumentation/SoT abgeschlossen, Runtime unverändert)
**Datum:** 31.08.2026
**Autor:** Red Queen (Rain Ocampo / Hermes)
**Freigabe:** Christian List (Owner)
---
## 1. Finales Verdict
> **PHASE 13.5 CLOSED — RUNNER REMEDIATION + CONTROL PLANE + TRUSTGATE CANONICAL PATH VERIFIED**
Phase 13.5 ist formal geschlossen. Die autoritative Closure-Evidence wurde in Red Queens Ausführungs-/Workspace-Kontext erzeugt und dort verifiziert:
- **Closure Record:** `/opt/data/rq_phase135_FORMAL_CLOSURE.md` (Red-Queen-Workspace-Kontext)
- **Authoritative Runner/TrustGate Evidence:** `/opt/data/rq_phase135_runner_trustgate_final/final-evidence.md` (SHA `5538ecf57ce602afaddab58534ccda05ffcdf311ed6f840532d9bad8283ef097`, Red-Queen-Workspace-Kontext)
---
## 2. Runtime (unverändert, verifiziert 31.08.2026)
| Feld | Wert |
|---|---|
| Container | `513b5b814d8d245db3f4db57d169d24cc9a69def655a2d66de7ffe4460452c7f` |
| Image | `sha256:f72b27e72fc6418edde5810090dad62538f2f85903424992691d79cfc69b8884` |
| Running / RestartCount | `true` / `0` |
| health / ready | `200` / `200` |
| Wrapper SHA | `109a5258bc306084f6909b3903294870b356eb3a6b244ad02b3599d4d02e8e1d` |
| Commit | `738af54` (NO PUSH) |
| Runner SHA (deployt) | `6cca3c55bb559d75be2f58e59c231003902f124a54ada25582970524599f0752` |
| repository.py SHA | `8fff892f52a612f464b5be21c66ef7787a47f2d7d049f815ef87837553c2a228` |
---
## 3. Kernbefunde (verifiziert)
- **Runner → BackfillOrchestrator:** bewiesen (Thin CLI Facade, kein direkter insert_*).
- **TrustGate:** mandatory im produktiven Pfad.
- **UNKNOWN / UNTRUSTED / CONDITIONAL / MISSING / EXCEPTION:** FAIL CLOSED.
- **Legacy Direct-Write Bypass:** über produktiven Wrapper/Runner unerreichbar.
- **TrustGate Harness:** PASS (36/36 + 27/27 + 17/17).
- **Service:** healthy.
- **Keine produktive Backfill-Ausführung** während Abschlussvalidation.
---
## 4. Wichtige Limitation — DB E2E
**Es wurde KEIN mutierender DB-E2E-Test erzwungen.** Über den verfügbaren Red-Queen-Control-Plane-Zugang war kein sicherer isolierter/read-only DB-Verifikationspfad vorhanden. Gemäß Fail-Closed-Prinzip wurde **§7D** angewendet. Persistence-Invariants wurden statisch + Harness verifiziert.
**Korrekte Aussage:** *"Canonical Runner/TrustGate path and persistence gating verified statically and via isolated harness; production DB mutation was intentionally not performed."*
---
## 5. Deferred Risks (offen, NICHT jetzt ändern)
- **#1 Legacy Scripts:** `run_backfill.py` / `run_phase11_pilot.py` existieren im Container, rufen `orch.run()` ohne TrustGate — über produktiven Wrapper unerreichbar, aber Code-Präsenz vergrößert Attack-Surface. → spätere Hardening-Mission.
- **#2 Dataset-Version TOCTOU:** `get_or_create_dataset_version` ohne UNIQUE auf Hash → potenzielle doppelte Dataset-Version-Metadaten bei parallelen identischen Requests. → spätere Repair-/Hardening-Mission.
- **Deferred Validation:** Vor echtem produktivem Backfill erforderlich: vertrauenswürdige Data Source, ausreichende Provenance, TrustGate TRUSTED, kontrollierter kleiner Pilot, DB Before/After, Qualitätsprüfung. **Der frühere Dukascopy-Pilot ist NICHT als trusted Dataset freigegeben.**
---
## 6. Closure Scope
- CONTROL-PLANE / RUNNER REMEDIATION: **CLOSED**
- TRUSTGATE CANONICAL PATH: **VERIFIED**
- PRODUCTION HISTORICAL DATA QUALITY: **NICHT automatisch freigegeben**
- PRODUCTION BACKFILL: **NICHT freigegeben**
- PHASE 14: **NICHT automatisch gestartet**
---
## 7. Offene Punkte (nach Closure)
- **A.** Trusted Historical Data Source / Provenance
- **B.** Controlled Trusted Provider Pilot
- **C.** Dataset-Version TOCTOU Hardening
- **D.** Legacy Entry-Point Hardening
- **E.** Production Backfill
- **F.** Phase 14
---
## 8. Verweis auf autoritative Evidence
Vollständige Details (Evidence-SHAs, Mutation Accounting, Grenzen) siehe (Red-Queen-Workspace-Kontext):
- `/opt/data/rq_phase135_FORMAL_CLOSURE.md`
- `/opt/data/rq_phase135_runner_trustgate_final/final-evidence.md`

View file

@ -0,0 +1,193 @@
# Phase 13.6 — Hardening Closure (FORMAL CLOSURE)
**Status:** CLOSED / DEPLOYED / VERIFIED
**Datum:** 01.09.2026
**Autor:** Rain Ocampo (Hermes) / Red Queen
**Freigabe:** Christian List (Owner)
---
## 1. Scope
Phase 13.6 umfasst die produktive Härtung des Historical-Service:
- **Korrigierter TOCTOU-Befund** (Dataset-Version)
- **Dataset-Version-Repair** (repository.py)
- **TrustGate Defense-in-Depth**
- **Legacy Entry-Point Hardening**
- **Schema-Entscheidung** (keine Migration nötig)
Phase 13.6 ist bereits erfolgreich produktiv deployed und verifiziert. Diese Mission persistiert ausschließlich die verifizierte Abschlussdokumentation in der autoritativen Git-Historie. **KEINE Trading-/Runtime-Änderung.**
---
## 2. Korrigierter TOCTOU-Befund
**Ursprünglicher Befund (Phase 13.5, Deferred Risk #2):** `get_or_create_dataset_version` ohne UNIQUE auf Hash → potenzielle doppelte Dataset-Version-Metadaten bei parallelen identischen Requests.
**Korrigierte Analyse:** `dataset_version_hash` hatte bereits UNIQUE-Schutz. Der tatsächliche Defekt war ein **SELECT→INSERT-Concurrency-Race**, das eine `UniqueViolation` werfen konnte (nicht doppelte Zeilen).
---
## 3. Dataset-Version-Repair (repository.py)
Repair in `app/persistence/repository.py` (Image-SHA `33cca6421a595c68a12bf6b25eb02b47edf868fb27501330e356de2c0c3a7bd6`):
- `pg_advisory_xact_lock(dataset_id)` — serialisiert konkurrierende Requests pro Dataset
- `ON CONFLICT(dataset_version_hash) DO NOTHING RETURNING` — idempotenter Insert
- kanonischer Re-SELECT nach Konflikt
- deterministische kanonische ID
- fail-closed bei ambigem/Fehlerzustand
**Concurrency-Validierung:**
- Baseline Gold x10: 3 OK, 7 UniqueViolation, 1 row
- Patched x10: 10/10 OK, gleiche kanonische ID, 1 row
---
## 4. TrustGate Defense-in-Depth
- Fehlender TrustGate auf produktionsfähigem Pfad: **FAIL CLOSED**
- UNKNOWN / UNTRUSTED / CONDITIONAL (Default) / MISSING / EXCEPTION: **BLOCK**
- Nur TRUSTED (oder CONDITIONAL mit expliziter Policy) persistierbar
---
## 5. Legacy Entry-Point Hardening
- `run_backfill.py`: HARD-FAIL (non-zero exit, kein Provider/Persistence-Continuation)
- `run_phase11_pilot.py`: HARD-FAIL (non-zero exit, kein Provider/Persistence-Continuation)
---
## 6. Schema-Entscheidung
**SCHEMA MIGRATION = NOT REQUIRED.** Bestehendes `UNIQUE(dataset_version_hash)` bleibt erhalten.
---
## 7. Production DB Precheck
- UNIQUE existiert: **JA**
- duplicates: **0**
- NULL anomalies: **0**
- FK anomalies: **0**
- dataset_version: **336**
---
## 8. Candidate Image / Build Provenance
| Feld | Wert |
|---|---|
| Image ID | `sha256:21598a3597cfd53c4bc22c01cb7f7874259878ecd7608528ccd6327305ea07fc` |
| Tag | `historical-v2-historical-service:candidate-phase13-6-hardening` |
| Build Input SHA | `f45e4885…` (nur gekürzt in Evidence verfügbar) |
| Archive SHA | `92a9982a212e1f7576305aa765f3c0163812a486f0afb64cb1dd58ba5b55f0c5` |
| Pin | `rq-historical-pin-phase13-6-hardening` |
| Status | **VERIFIED + ARCHIVED + PINNED = READY** |
---
## 9. Production Deployment
| Feld | Wert |
|---|---|
| Source | Phase-13.5 Remediation `sha256:f72b27e72fc6418edde5810090dad62538f2f85903424992691d79cfc69b8884` |
| Target | Phase-13.6 `sha256:21598a3597cfd53c4bc22c01cb7f7874259878ecd7608528ccd6327305ea07fc` |
| Real Recreate Count | **1** |
| Exit Code | **0** |
| Container (nach) | `d274568ae85173b408531315e7a51aa9fd54b0587931d8c662548736b154313b` |
| running / RestartCount | `true` / `0` |
| health / ready | `200` / `200` |
---
## 10. Target Manifest
- **58** Gold/unchanged matches
- **5** autorisierte Divergenzen:
1. `run_backfill_year.py` — bestehende Phase-13.5-Autorisierung (Runner `6cca3c55bb559d75be2f58e59c231003902f124a54ada25582970524599f0752`)
2. `repository.py``33cca6421a595c68a12bf6b25eb02b47edf868fb27501330e356de2c0c3a7bd6`
3. `backfill.py``fb93ee1f09b3708eae155b12df1b8a1103ea7c0a4a6aea70c476fd544d58ac0e`
4. `run_backfill.py``79ee38fd51cf3f92550997cab2ffd6700468d4d7b3921ca97bbc7a38d6a8f383`
5. `run_phase11_pilot.py``34e345a951a92eecfa8418ae078e30d9fef24c5ef2043dc0b194b064f84cf551`
- unauthorized: **0**
- missing: **0**
- extra: **0**
---
## 11. Rollback Control Plane
- Historischer Gold-Anker `1f3ba210…` ist unverfügbar (Coolify-GC, kein SHA-verifiziertes docker-save-Archiv).
- Bewiesen: `1f3ba210…` war ein **stale/wrong-target pre-gate** für den Remediation-Rollback (SOURCE-state/historische Referenz, NICHT Rollback-TARGET-Requirement).
- **Kein Fake/Retag/Rebuild** von `1f3ba210…` durchgeführt.
- Primary Rollback zielt/verifiziert jetzt exakt auf den Phase-13.5-Remediation-Target.
- Direct no-op bleibt **FAIL CLOSED**.
- Primary Rollback dry-run: **PASS**
- Secondary Recovery Rollback dry-run: **PASS**
---
## 12. Production DB Integrity
- dataset_version: **336 → 336**
- duplicates: **0 → 0**
- NULL anomalies: **0 → 0**
- FK anomalies: **0**
- **Keine produktiven DB-Writes durch den Deployment.**
---
## 13. Mutation Accounting
| Mutation | Anzahl |
|---|---|
| Wrapper Target-Class Deploy | 1 |
| Rollback Control-Plane Wrapper Patch | 1 |
| Production Recreate | 1 |
| Production DB Write | 0 |
| Schema | 0 |
| Backfill | 0 |
| Provider | 0 |
| Rollback | 0 |
| Phase 14 | 0 |
---
## 14. Security Invariants
- TrustGate fail-closed auf produktionsfähigem Pfad: **PASS**
- Legacy Entry-Points (`run_backfill.py`, `run_phase11_pilot.py`): **HARD-FAIL**
- Kein Runner/Backfill aktiv
- Keine produktiven DB-Writes
---
## 15. Deferred Work (NICHT durch Phase 13.6 autorisiert/gelöst)
Phase 13.6 autorisiert oder löst NICHT:
- **kein produktiver Historical Backfill** ausgeführt
- **kein Trusted Historical Provider** ausgewählt
- **kein Trusted Provider Pilot** ausgeführt
- **keine Provider/Provenance-Acceptance** abgeschlossen
- **kein Phase 14**
- **keine Live-Capital/Trading-Aktivierung**
Der nächste große Workstream bleibt separat.
---
## 16. Final Closure Statement
> **PHASE 13.6 CLOSED — DEPLOYED — VERIFIED — DOCUMENTED — FAST-FORWARD PUSHED — REMOTE SOT PERSISTED — RUNTIME UNCHANGED**
Phase 13.6 ist formal geschlossen. Die autoritative Closure-Evidence wurde in Red-Queen-/Rain-Ocampo-Ausführungskontext erzeugt und verifiziert:
- `/opt/data/rq_phase136_repair/` (Primary Rollback Control-Plane Repair)
- `/opt/data/rq_phase136_owner_verify/` (Owner Candidate Verification)
- `/opt/data/rq_phase136_deployment/` (Controlled Deployment)
**Git-Baseline vor dieser Closure:** `51c03f7ade49db263f4908b087418ec92979e4e6` (`phase13.5: formally close runner trustgate remediation`).

View file

@ -1,5 +1,10 @@
--- ---
type: Note id: object/ca9b3551-53f8-408b-8697-9a21fa3a24d8
type: arch
role: history
representation: canonical
state: historical
knowledge_schema: 1
tags: [trading, mt5, architecture, historical-v2, m12, m13, phase9, readiness-audit, phase2] tags: [trading, mt5, architecture, historical-v2, m12, m13, phase9, readiness-audit, phase2]
created: 2026-08-22 created: 2026-08-22
--- ---

View file

@ -1,9 +1,17 @@
--- ---
type: Trading-Modul-System id: object/ebe08190-5cd5-a649-d6ab-38cb8f50f8b4
type: arch
role: reference
representation: canonical
state: current
knowledge_schema: 1
derived_from: object/09fd796f-51b6-4104-bc69-5f9dc79db0d6
source_system: forgejo
source_path: ports-reference.md
_organized: true _organized: true
--- ---
# Ports & Service-Referenz — Modul-Trading-System VPS # Ports & Service-Referenz — Trading-System VPS
> Stand: 20.08.2026 · VPS: `187.124.31.123` · SSH: Public-Key-Auth (nur) > Stand: 20.08.2026 · VPS: `187.124.31.123` · SSH: Public-Key-Auth (nur)

View file

@ -1,8 +1,16 @@
--- ---
type: Trading-Modul-System knowledge_schema: 1
id: object/31671ffa-fc39-ae9c-9449-e9322c9bb7be
type: design
role: design
representation: canonical
state: current
derived_from: object/91c7786c-cffa-1197-0837-e55a87cdc161
_organized: true _organized: true
--- ---
# trend_pullback_v1.1 — Spezifikation # trend_pullback_v1.1 — Spezifikation
> Status: **IMPLEMENTIERT + DEPLOYT** (`backtesting:0.1.2`, Modul-12), V1-Kompatibilitätsmodus bitgenau. > Status: **IMPLEMENTIERT + DEPLOYT** (`backtesting:0.1.2`, Modul-12), V1-Kompatibilitätsmodus bitgenau.

View file

@ -0,0 +1,173 @@
# Wave 1 Provenance — Production Deployment Closure (FORMAL CLOSURE)
**Status:** CLOSED / DEPLOYED / VERIFIED / DOCUMENTED
**Datum:** 04.09.2026
**Autor:** Red Queen (Rain Ocampo / Hermes)
**Freigabe:** Christian List (Owner)
---
## 1. Scope
Diese Mission persistiert ausschließlich die verifizierte Abschlussdokumentation des **Wave1-Provenance-Production-Deployments** in der autoritativen Git-/SoT-Historie. **KEINE Trading-/Runtime-/DB-Änderung.**
Wave1 Provenance ist bereits erfolgreich produktiv deployed und unabhängig verifiziert. Diese Closure dokumentiert den finalen, verifizierten Zustand.
---
## 2. Deployment Chain (exakt)
| Schritt | Wert |
|---|---|
| Backup | **1** |
| Wave1 Migration | **1** |
| WaveD Migration | **0** (ALREADY PRESENT / VERIFIED / SKIPPED) |
| Production Recreate | **1** |
| Rollback | **0** |
| Restore | **0** |
| Provider | **0** |
| Backfill | **0** |
| Git während Deployment | **0** |
| Phase14 | **0** |
---
## 3. Wave1 Identity
| Objekt | SHA / Wert |
|---|---|
| Wave1 Migration | `6983526da3ca51fd2ececffbef0ccbd66b048a3d11c228000011aaba3e1d6bc5` |
| Candidate Image | `sha256:c70d47677c455963553280c7594d4005b3dcf1650ee99a49708997a624dee42f` |
| Candidate Archive | `b3c72fb9ed78f0fb2e57292a90686fd9d49116953b23c976750ff4ffb8628c99` |
| Application Manifest | `41b9032b694cdf958d7aefb06870fd181a284568447e06396c48cc915e19b06b` |
| WaveD Historical Contract | `7aa4e2a10c98fa327e000a49f6a497efb5814c8808dba429dfebfc90c544a02b` |
**Wichtig:** Die WaveD-Migration (`7aa4e2a1…`) wurde in diesem Deployment **NICHT ausgeführt**. Der WaveD-Runtime-Schema-Zustand war bereits vor dem Deployment vorhanden und wurde **PRESERVED** (nicht verändert).
---
## 4. Production State (final, verifiziert)
| Feld | Wert |
|---|---|
| Container | `9eb3480e4395a549cb6e7c063cab7f812d90408bdf7101aaa7168f8f1221c7be` |
| Image | `sha256:c70d47677c455963553280c7594d4005b3dcf1650ee99a49708997a624dee42f` |
| running / RestartCount | `true` / `0` |
| health / ready | `200` / `200` |
| PostgreSQL | `16.13` |
| database / schema | `historical` / `public` |
### Final Schema Fingerprint
| Tabelle | Spalten |
|---|---|
| historical_bar | 41 |
| derived_bar | 35 |
| data_ingestion_run | 29 |
| dataset_version | 23 |
| ingestion_provenance | PRESENT / 41 cols / 0 rows |
### Canonical Fields
| Feld | Typ |
|---|---|
| last_successful_chunk | `timestamp with time zone` (timestamptz) |
| interrupt_reason | `text` / preserved |
| technical_quality | `text` / nullable / default `'VALID'` |
### Data Invariants
| Check | Wert |
|---|---|
| dataset_version rows | 336 |
| duplicate dataset_version_hash | 0 |
| null identity | 0 |
| FK orphans | 0 |
| historical_bar rows | 80,680 |
| derived_bar rows | 24,948 |
| RUNNING ingestion | 0 |
| Provider Calls | 0 |
| Backfill | 0 |
| Pilot | 0 |
---
## 5. Backup / Rollback
| Objekt | Wert |
|---|---|
| Backup Path | `/opt/historical-v2/backups/pre_wave1_20260904_164812/` |
| Full Dump SHA | `0f9c9bbc32fe24fb2fbb622b69356794ef0278c5f71032870323a7262d94b1e4` |
| Schema Dump SHA | `71294b98483ee76cec2096505293a4a6cfd9ddf2565defdf032f365fb188861e` |
| Compose SHA | `3c0cc9da53260b01f977d051fe3cb67ddbe29f62b3e07f5279942cb795908a52` |
| Primary App Rollback | `sha256:21598a3597cfd53c4bc22c01cb7f7874259878ecd7608528ccd6327305ea07fc` |
| Recovery Gold | `sha256:b93b0fe141d08d29a09861f5b4728e7816cfcf3e0c3ea36897ea826a9491eab9` |
---
## 6. Independent Checker Result
**RED QUEEN Independent Post-Deploy Review: VERDICT A**
- Wave1 Production Deployment Verified
- Runtime healthy
- Wave1 schema exact
- Existing WaveD contract preserved
- Data invariants intact
- Backup/Restore verified
- Primary rollback verified
- No Provider/Backfill activity
---
## 7. Mutation Accounting
| Mutation | Anzahl |
|---|---|
| Backup | 1 |
| Wave1 Migration | 1 |
| WaveD Migration | 0 |
| Production Recreate | 1 |
| Rollback | 0 |
| Restore | 0 |
| Provider | 0 |
| Backfill | 0 |
| Git während Deployment | 0 |
| Phase14 | 0 |
---
## 8. Open Contract Debts (NICHT geschlossen)
Diese Debts bleiben **OPEN** und sichtbar. Sie invalidieren den aktuellen Production-Zustand **NICHT**:
- **A)** Historical V2 DDL provenance missing
- **B)** `interrupt_reason` origin unknown
- **C)** Historical WaveD migration artifact definiert `last_successful_chunk` als `text`, während der Owner-Canonical-Current-Production-Contract `timestamptz` ist.
---
## 9. Explicitly Excluded Work (NICHT autorisiert)
Diese Closure autorisiert oder löst NICHT:
- **kein Provider Pilot**
- **kein Historical Backfill**
- **kein Phase14**
- **keine weitere Runtime-Mutation**
- **kein weiterer Git-Commit / Git-Push**
Der nächste Workstream erfordert separate Owner-Autorisierung.
---
## 10. Final Closure Statement
> **WAVE1 PROVENANCE CLOSED — DEPLOYED — VERIFIED — DOCUMENTED — FAST-FORWARD PUSHED — REMOTE SOT PERSISTED — RUNTIME UNCHANGED**
Wave1 Provenance ist formal geschlossen. Die autoritative Closure-Evidence wurde in Red-Queen-/Rain-Ocampo-Ausführungskontext erzeugt und verifiziert:
- `/opt/data/rq_wave1_postdeploy_production_review_resume/` (Independent Post-Deploy Review, Verdict A)
- `/opt/data/rq_wave1_closure_git_authority_recovery/` (Git-Authority-Recovery / diese Closure)
**Git-Baseline vor dieser Closure:** `36916a073d57d9d236c87a23892ad072e9bedd65` (`phase13.6: formally close hardening repair`).

View file

@ -1,5 +1,10 @@
--- ---
type: Bereich id: object/8b362849-2a5e-4a07-42f0-557a58e1cf14
type: arch
role: index
representation: standalone
state: current
knowledge_schema: 1
title: Trading Systeme title: Trading Systeme
tags: [trading, mt5, home] tags: [trading, mt5, home]
created: 2026-08-20 created: 2026-08-20

View file

@ -0,0 +1,76 @@
# NOTION PROJECT COMMAND CENTER
> **Mission 006 — REAL MISSION 006 NOTION PROJECT COMMAND CENTER, MASTER TODO, CONTEXT RECOVERY & AGENT GOVERNANCE v1**
> Status: **COMPLETE** (2026-08-25) · Fresh Checker: **PASS** (19/19, unabhängig)
## Zweck
Der **PROJECT COMMAND CENTER** ist der persistente operative Leitfaden für Christians gesamtes KI-/Agenten-Projekt. Ein neuer ChatGPT-Chat (ohne Vorkenntnis) muss via „Lies meinen aktuellen Projektstand aus dem Command Center" alles verstehen: Was, Warum, Teilprojekte, abgeschlossen, aktiv, nächster Schritt, danach, verschoben, Originale, Verifikationspflicht vor Mutation.
**Rollen-Trennung (verbindlich):**
- **Notion Role A = SAFETY BRAIN** — Backup/Recovery (Index-DB `3c793a20-cfac-8059-ae2b-000bbfaa5a60`)
- **Notion Role B = PROJECT COMMAND CENTER** — operativ/Kontext-Recovery (diese Struktur)
Notion ist **NICHT** die globale Source of Truth.
## Source-of-Truth-Modell
| System | Rolle |
|---|---|
| **FORGEJO** | Technische/versionierte Source of Truth |
| **TOLARIA** | Vernetztes Second Brain |
| **NOTION SAFETY BRAIN** | Backup/Recovery |
| **NOTION PROJECT COMMAND CENTER** | Status/Roadmap/To-do/Handoff/Entscheidungen/Governance/Referenzen/Context Anchors |
| **RUNTIME** | Wahrheit über laufende Systeme |
## Struktur (14 Unterseiten + MASTER TODO-DB)
Command Center Page: `3c793a20-cfac-8117-ba35-e9152628dc30`
| Seite | Inhalt |
|---|---|
| 00 — START HERE | Einstieg: System Purpose, aktueller Zustand, nächster Schritt |
| 01 — MASTER ROADMAP | Strategisches Dreieck (AD) |
| 02 — CURRENT PROJECT STATE | Aktueller Projektzustand |
| 03 — MASTER TODO | Link zur MASTER TODO-DB |
| 04 — DECISION LOG | Entscheidungsprotokoll |
| 05 — SYSTEM & AGENT CAPABILITIES | Capabilities vs. aktive Systeme |
| 06 — SOURCE OF TRUTH MAP | SoT-Zuordnung |
| 07 — AGENT GOVERNANCE | READ-BEFORE-WRITE, Write Classification |
| 08 — PROMPT LIBRARY | Prompt-Templates (DATA, DO NOT EXECUTE) |
| 09 — OPEN ISSUES & RISKS | Offene Punkte & Risiken |
| 10 — MISSION HISTORY | Missionshistorie |
| 11 — HANDOFF FOR NEW CHAT | Handoff für neuen Chat |
| 12 — VERIFICATION REQUESTS | Verifikationsanfragen |
| 13 — REFERENCES & CONTEXT ANCHORS | Referenzen & Context Anchors |
## MASTER TODO-DB
- **data_source:** `23d2b57b-a303-41d8-a0da-4f3aaa1f88b8`
- **Rows:** 24 Tasks (12 DONE, 12 offen)
- **Pflichtfelder (24):** TASK_ID, TITLE, PROJECT, ROADMAP_BLOCK, STATUS, PRIORITY, OBJECTIVE, WHY, CURRENT_STATE, WHAT_HAS_ALREADY_BEEN_DONE, EXPECTED_NEXT_STEP, SUCCESS_CRITERIA, DEPENDENCIES, BLOCKERS, SOURCE_SYSTEM, SOURCE_REFERENCE, SOURCE_VERSION, CONTEXT_ANCHORS, EXTERNAL_REFERENCES, VERIFICATION_REQUIRED, PROMPT_READY, OWNER, LAST_VERIFIED, NOTES
- **Roadmap-Blöcke:** A (Foundation), B (Command Center), C (TOLARIA Second Brain), D (HERMES/RQ Autonomisierung)
## Recovery-Tests (4/4 PASS)
1. **Context Recovery** — neuer Chat versteht Gesamtprojekt, Stand, nächsten Schritt, Quellen, Verifikationspflicht
2. **Todo Recovery** — Tasks selbsterklärend (WHY, CURRENT_STATE, NEXT, SOURCE, SUCCESS)
3. **Agent Governance** — Rollen/SoT/Secrets/STOPP klar
4. **Roadmap Recovery** — JETZT (Command Center) → DANACH (Tolaria) → DANACH (Hermes-Autonomie)
## Fresh Checker
- **Builder-Checker:** 26/26 PASS (`/opt/data/m006_checker_result.json`)
- **Unabhängiger Fresh Checker:** 19/19 PASS (`/opt/data/m006_freshchecker_result.json`, Delegation `deleg_1a2520cf`)
## Sicherheit
- Keine Secrets in Notion/Repo/Chat — nur Name + EXISTS/LENGTH
- Prompt Library: DATA / DO NOT EXECUTE (keine ausführbaren Prompts)
- Keine Automation aktiv (statisch OFF dokumentiert)
- READ-BEFORE-WRITE vor jeder Mutation
## Nächste Schritte (NICHT automatisch)
- **Tolaria v1** (Block C) — auf Christian warten
- **Hermes/RQ-Autonomisierung** (Block D) — auf Christian warten

View file

@ -0,0 +1,54 @@
# NOTION SAFETY BRAIN — Access & Architecture (Mission 003)
Datum: 2026-08-25 · Red Queen · Status: **DISCOVERY + ARCHITECTURE (READ-ONLY)**
> Notion als **externer Safety Brain / Disaster-Recovery-Wissensspeicher** für
> **zwei getrennte Systeme**: Forgejo (technische Source of Truth) und Tolaria
> (operatives Second Brain). **Notion ist KEINE neue Source of Truth.**
---
## 1. Architekturprinzip
```
FORGEJO ────────────► NOTION SAFETY BRAIN
TOLARIA ────────────► NOTION SAFETY BRAIN
NICHT (kein Rückfluss):
NOTION ────────────► FORGEJO / TOLARIA
```
- **ONE-WAY** Datenfluss: Quelle → Notion.
- Ein Fehler / falsche Bearbeitung / Löschung in Notion darf **niemals** Forgejo oder Tolaria verändern.
- **Kein** bidirektionaler Sync. **Kein** automatischer Rück-Sync.
## 2. Rollen
| System | Rolle |
|---|---|
| **FORGEJO** | technische/versionierte Source of Truth |
| **TOLARIA** | operatives Second Brain / Knowledge System |
| **NOTION** | externer Safety Brain / Knowledge Backup / Disaster-Recovery View |
> Notion ist ausdrücklich **KEINE Source of Truth** — sondern ein abgeleiteter,
> externer Sicherungsspeicher.
## 3. Ausgangslage (EVIDENCE)
- Christians Notion wurde **bewusst geleert** (vorher unbrauchbare Inhalte) → saubere Struktur von Grund auf planbar.
- Rain und Alice besitzen bereits funktionierenden Notion-Zugriff.
- **Red Queen hat aktuell KEINEN eigenen Notion-Zugriff** (kein `NOTION_API_KEY`, kein `ntn`, keine Config) — verifiziert via read-only Discovery.
## 4. Zugangskonzept — Red Queen (MINIMUM NECESSARY PRIVILEGE)
- Red Queen erhält einen **eigenen, sauber getrennten, minimal privilegierten** Notion-Zugriff.
- **Kein Credential-Transfer** von Rain/Alice. **Keine** fremden Tokens kopieren.
- **Keine Owner-/Admin-Rechte** — nur der Scope, den der Safety-Brain-Write benötigt.
- Empfohlener Weg (Implementierung erst nach Freigabe):
- Eigene Notion-Integration anlegen (Read+Insert) mit dediziertem Token.
- Token in `${HERMES_HOME}/.env` als `NOTION_API_KEY` (nie ausgeben).
- Nur die nötigen Safety-Brain-Seiten/DBs mit der Integration teilen (Seiten-`...` → Connect to).
- **Keine Secrets in den Artefakten**; Token nur ephemer nutzen.
## 5. Kein-Cloud-Write in dieser Mission
- Diese Mission ist **READ-ONLY** (Discovery + Architecture).
- Kein Anlegen von Notion-Seiten, keine Datenbank, kein Schreiben, kein Sync, kein Cron.
- Produktive Daten / Credentials / Berechtigungen bleiben unverändert.

View file

@ -0,0 +1,34 @@
# NOTION SAFETY BRAIN — BACKUP MANIFEST (Mission 003)
Datum: 2026-08-25 · Red Queen · Status: **DESIGN (nicht implementiert)**
> Ein Backup-Run erzeugt ein Manifest, das Lauf-Ergebnis und Verifikation abbildet.
> **Ein Backup ist NICHT allein deshalb erfolgreich, weil ein API-Write HTTP 200 lieferte.**
## Manifest-Felder
```
BACKUP_RUN_ID
TIMESTAMP
FORGEJO_HEAD
TOLARIA_HEAD
FORGEJO_OBJECTS_EXPECTED
FORGEJO_OBJECTS_BACKED_UP
TOLARIA_OBJECTS_EXPECTED
TOLARIA_OBJECTS_BACKED_UP
SKIPPED_OBJECTS
FAILED_OBJECTS
SECRET_BLOCKED_OBJECTS
HASH_STATUS
VERIFY_STATUS
RESULT
```
- `HASH_STATUS` = `all_match` | `mismatch` (alle Source-Hashes vs. Notion-read-back).
- `VERIFY_STATUS` = `verified` | `unverified`.
- `RESULT` = `BACKUP VERIFIED` nur wenn Write + Read-back + Compare + Hash + Count + Logs ok.
## Registry-Zuordnung
Manifest-Zeile wird in Notion-DB **05 Backup Registry** je RUN_ID abgelegt (siehe RESTORE_MODEL.md).
## Verify-Prinzip (WRITE ≠ SUCCESS)
1. Write → 2. Read-back → 3. Compare → 4. Hash/Version-Check → 5. Count-Match → 6. Error/Skip-Log → **BACKUP VERIFIED**.

View file

@ -0,0 +1,45 @@
# NOTION SAFETY BRAIN — BUILD PLAN (Mission 003)
Datum: 2026-08-25 · Red Queen · Status: **PLAN (nicht implementiert)**
> Konkretes, gestuftes Umsetzungsmodell. Jede Phase erst nach **ausdrücklicher Freigabe** durch Christian.
---
## Phase 0 — Access (Red Queen)
- [ ] Eigene Notion-Integration anlegen (Read + Insert), minimal privilegiert.
- [ ] `NOTION_API_KEY` in `${HERMES_HOME}/.env` (nie ausgeben, nur EXISTS/LENGTH).
- [ ] Nur Safety-Brain-Seiten/DBs mit der Integration teilen.
- [ ] Kein Credential-Transfer von Rain/Alice.
## Phase 1 — Struktur in Notion anlegen
- [ ] Root-Seite **KI-SYSTEM — SAFETY BRAIN**.
- [ ] Bereiche 0010 (siehe NOTION_STRUCTURE.md).
- [ ] Datenbank `05 Backup Registry` + `07 Knowledge Index`.
## Phase 2 — Backup-Engine (lokal)
- [ ] Secret-Scan (SECRET_POLICY.md) fest einbauen.
- [ ] Provenance-Frontmatter (PROVENANCE_MODEL.md) je Objekt.
- [ ] Incremental / Hash-Change-Detection (SCALING_MODEL.md).
- [ ] Manifest-Erzeugung (BACKUP_MANIFEST.md).
## Phase 3 — Verify & Registry
- [ ] READ-BACK + COMPARE + Hash-Verify (WRITE ≠ SUCCESS).
- [ ] Registry-Zeilen pflegen.
## Phase 4 — Restore-Modul
- [ ] Staging → Validate → Human Review → Approval → Restore (RESTORE_MODEL.md).
## Phase 5 — Red Queen als Knowledge Backup Steward
- Änderungen erkennen · klassifizieren · Secrets blocken · Delta · Backup schreiben ·
zurücklesen · verifizieren · Manifest · Status · Fehler melden.
> **NOCH NICHT AUTOMATISIEREN.** Kein Cron / Daemon / Heartbeat in dieser Mission.
> **A5 bleibt AUS.**
---
## Wichtig
- Diese Mission = **Discovery + Architecture + Checker + Forgejo-Doku**, dann **STOPP**.
- Kein Bau von Notion-Seiten, keine Datenbank, kein Upload, kein Sync, kein Cron.
- Warte auf Christians ausdrückliche Freigabe vor Phase 1.

View file

@ -0,0 +1,24 @@
# NOTION SAFETY BRAIN — CURRENT STATE (Mission 003)
Datum: 2026-08-25 · Red Queen · Status: **READ-ONLY Discovery-Stand**
## Notion
- **WORKSPACE:** nicht verifizierbar ohne Zugriff (Red Queen hat noch kein Token) — UNKNOWN bis Phase 0.
- **AUTH METHOD:** geplant = eigene Notion-Integration (OAuth-token-basiert), min. privilegiert.
- **PERMISSIONS (Red Queen):** keine — Zugriff noch NICHT eingerichtet.
- **CURRENT NOTION STATE:** bewusst geleert (Christian) → saubere Struktur planbar. Kein Inhalt von RQ verifizierbar (kein Zugang).
## Red Queen Zugriff
- **TOKEN_EXISTS=NO** · `ntn` CLI: nicht installiert · keine Notion-Config/Credential-Datei unter RQ-Volume.
## Rain / Alice
- Haben funktionierenden Notion-Zugriff (Missionsangabe). **Nicht herangezogen** für Credentials.
- Diese Mission: **RAIN REFERENCE USED = NO**, **ALICE REFERENCE USED = NO** (kein Credential-Transfer nötig; Architekturweg dokumentiert).
## Quellen-Stand (EVIDENCE)
- **Forgejo** `main` = `819fd8a` (Repo-Struktur lokal verifiziert).
- **Tolaria** Vault = HEAD `69aecf2…`, 84 .md (API-Read verifiziert).
## Grundsatz (verbindlich)
- Notion = externer Safety Brain, **KEINE Source of Truth**.
- **One-Way** von Forgejo/Tolaria → Notion. Kein Rückfluss.

View file

@ -0,0 +1,55 @@
# NOTION SAFETY BRAIN — DATA CLASSIFICATION (Mission 003)
Datum: 2026-08-25 · Red Queen · Status: **EVIDENCE-basiert (read-only)**
> Klassifikation, welche Inhalte aus Forgejo und Tolaria in den Notion Safety Brain
> gesichert werden sollen. Kategorien:
> **BACKUP_REQUIRED** · **BACKUP_RECOMMENDED** · **OPTIONAL** · **DO_NOT_BACKUP** · **SECRET/FORBIDDEN**
---
## 1. FORGEJO (technische Source of Truth)
EVIDENCE (lokaler Klon `/opt/data/forgejo/trading-system-docs`, main=`819fd8a`):
| Inhalt | Kategorie | Begründung |
|---|---|---|
| `red-queen-architecture/` (10 Contracts) | **BACKUP_REQUIRED** | Architektur-Entscheidungen, Safety/External-Review-Contracts, MISSION_STATE_MACHINE |
| `tolaria/` (Discovery-Report, B1-Docs) | **BACKUP_REQUIRED** | Tolaria-Dokumentation, Baseline, Recovery-Runbook |
| `a2/`..`a5/` (Design/README/Code) | **BACKUP_RECOMMENDED** | Red-Queen-Automatisierung, relevant für Recovery |
| Modul-Doku (`modul-*.md`, `ports-reference`, `infrastructure-handbook`, `trend-pullback-spec`) | **BACKUP_RECOMMENDED** | System-/Modul-Dokumentation |
| `backup_patches/` | **BACKUP_RECOMMENDED** | Wichtige Patches |
| `notes/` (Hubs/Wikilinks) | **BACKUP_RECOMMENDED** | Verknüpfungs-Struktur |
| `README.md` | OPTIONAL | Überblick (bereits in modul-Doku enthalten) |
| `__pycache__/`, `.git/`, Binär-/Build-Artefakte | **DO_NOT_BACKUP** | Rebuildbar, keine Knowledge-Inhalte |
| `.env`, `*.pem`, `*.key`, `*.token`, config mit Secrets | **SECRET/FORBIDDEN** | Nie nach Notion |
**Doppel-Zählung beachten:** Modul-Dateien existieren teils auch im Vault (Root + system-docs) → Deduplizierung nötig (siehe Provenance).
## 2. TOLARIA (operatives Second Brain, 84 .md via API)
EVIDENCE (Vault-API, `/app/vault`, HEAD `69aecf2…`):
| Bereich | Kategorie | Begründung |
|---|---|---|
| `notes/trading/second-brain/` (journal, t212-logs, Phase-Docs) | **BACKUP_REQUIRED** | Operative Knowledge-Logs, Verlauf |
| `notes/trading/system-docs/*` (30) | **BACKUP_RECOMMENDED** | Modul-/System-Doku (teils identisch mit Forgejo → dedupe) |
| `/app/vault/*.md` Root (README, infrastructure-handbook, modul-*) | **BACKUP_RECOMMENDED** | Duplikat-Struktur (19 Paare) |
| `notes/` Hubs (ai-agents, alice-betrieb, projekte, start, inbox, trading, reference) | **BACKUP_RECOMMENDED** | Wikilink-/Hub-Struktur, Beziehungen |
| **Frontmatter / Wikilinks / Provenance** | **BACKUP_REQUIRED** (als Metadaten) | Für Beziehungs-/Graph-Rebuild |
| `.git/` (Objects) | **DO_NOT_BACKUP** via API | Binär, nicht via JSON-API vollständig ziehbar; separat technisch sichern |
| secrets in Frontmatter/Inhalt (falls vorhanden) | **SECRET/FORBIDDEN** | Secret-Scan vor Upload |
## 3. Kategorientabelle (Provenance-Vorbereitung)
Jede Notion-Seite erhält im Frontmatter/Property die Quelle:
`SOURCE_SYSTEM` (forgejo|tolaria), `SOURCE_PATH`, `SOURCE_HASH`, `SOURCE_VERSION` (git-HEAD / Vault-HEAD).
## 4. Zwei-Quellen-Prinzip
- Forgejo und Tolaria enthalten **nicht zwingend** dieselben Informationen.
- Notion sichert **beide getrennt und nachvollziehbar** (je `SOURCE_SYSTEM`).
- **Kein blindes Mergen** der beiden Quellen in Notion.
## 5. DO_NOT_BACKUP + SECRET — Grundregeln
- Secrets/Tokens/Passwörter/private Keys/.env/Credentials → **SECRET_BLOCKED**, nie hochladen.
- Binär-/Build-Artefakte, `.git/objects`, große Dateien → **DO_NOT_BACKUP** (technical backup übernimmt das).

View file

@ -0,0 +1,23 @@
# NOTION SAFETY BRAIN — NOTION STRUCTURE (Mission 003)
Datum: 2026-08-25 · Red Queen · Status: **DESIGN (Vorschlag, nicht angelegt)**
Arbeitstitel: **KI-SYSTEM — SAFETY BRAIN**
| Bereich | Inhalt | Typ |
|---|---|---|
| 00 — Dashboard | Registry-Übersicht, Status | Seite |
| 01 — Forgejo Safety Backup | Gesicherte Forgejo-Dokumente | Seite(n) |
| 02 — Tolaria Safety Backup | Gesicherte Tolaria-Notes | Seite(n) |
| 03 — Architecture & ADR | Architektur-Entscheidungen | Seite |
| 04 — System Recovery | Recovery-Runbooks, Restore-Fluss | Seite |
| 05 — Backup Registry | Lauf-Manifeste | **Datenbank** |
| 06 — Integrity Reports | Verify-Ergebnisse | Seite |
| 07 — Knowledge Index | Index der Objekte | **Datenbank** |
| 08 — Historical Snapshots | Historische Stande | Seite |
| 09 — Incidents / Recovery Events | Vorfälle, Restore | Seite |
| 10 — Safety Brain Documentation | Diese Docs, Build-Plan | Seite |
**Empfehlung:** Seiten für statische Bereiche + **Datenbanken** für Registry (05) und Index (07),
da dort je Zeile Provenance/Status/Hash/Head als Properties abbildbar und filterbar sind.
Notion API: Markdown für Seiten (agent-freundlich), Properties für DB-Zeilen (Relation/Select/Date).

View file

@ -0,0 +1,159 @@
# RED QUEEN — NOTION SAFETY BRAIN · PILOT BACKUP & RESTORE-READINESS TEST
**Status:** ABGESCHLOSSEN (Pilot)
**Run-ID:** `RQ-NOTION-PILOT-20260825-001`
**Datum:** 2026-08-25
**Writer:** Red Queen
**Sprache:** Deutsch
> Diese Datei dokumentiert die Pilot-Architektur und die Ergebnisse des ersten
> kontrollierten Backup-Pilots von Forgejo + Tolaria in das Notion Safety Brain.
> **Keine Secrets.** Keine Credentials. Keine Quell-System-Mutation.
---
## 1. AUTORITATIVE ARCHITEKTUR
- **Forgejo** = Source of Truth für Forgejo-Inhalte.
- **Tolaria** = Source of Truth für Tolaria-Inhalte.
- **Notion** = SAFETY BRAIN / Sicherheitskopie / Knowledge Index / Restore-Hilfe.
- **KEIN** bidirektionaler Sync. **KEINE** Rückschreibung Notion → Forgejo/Tolaria.
---
## 2. PILOT RUN ID
`RQ-NOTION-PILOT-20260825-001`
Schema: `RQ-NOTION-PILOT-YYYYMMDD-NNN` — stabil, eindeutig, ordnet alle 10 Objekte diesem Run zu.
---
## 3. AUSGEWÄHLTE QUELLEN (5 Forgejo + 5 Tolaria)
| # | System | Pfad | Content-Typ | SOURCE_HASH (SHA-256, 12 Zeichen) |
|---|--------|------|-------------|-----------------------------------|
| 1 | forgejo | `notion-safety-brain/PROVENANCE_MODEL.md` | adr | `d2e4d083f4d8` |
| 2 | forgejo | `notion-safety-brain/DATA_CLASSIFICATION.md` | decision | `4538e4f14fbf` |
| 3 | forgejo | `tolaria/BACKUP_ARCHITECTURE.md` | architecture | `c64cd311d8a8` |
| 4 | forgejo | `red-queen-architecture/SAFETY_CONTRACT.md` | contract | `046da63c5731` |
| 5 | forgejo | `infrastructure-handbook.md` | reference | `708ccc532585` |
| 6 | tolaria | `notes/start.md` | note | `c2ffd4e071ec` |
| 7 | tolaria | `notes/trading/trading.md` | note | `a09aef8409cb` |
| 8 | tolaria | `notes/ai-agents/alice-betrieb.md` | note | `45444b29163c` |
| 9 | tolaria | `notes/ai-agents/ai-agents.md` | hub | `691949bb64cd` |
| 10 | tolaria | `notes/projects/projekte.md` | hub | `586ec58668d7` |
**Auswahlbegründung (repräsentativ, klein, kategorienübergreifend):**
- **Forgejo:** Architektur, Entscheidungs-Doku, ADR, Safety-Contract, Referenz-Handbuch.
- **Tolaria:** Start/Überblick, Trading-Modul, AI-Agenten-Betrieb (2), Projekt-Hub.
---
## 4. MAPPING (Source → Notion-Ziel)
| System | Notion-Parent-Ziel |
|--------|--------------------|
| Forgejo | „Forgejo Backups" page `3c793a20-cfac-80a4-bdad-d9db7753f11f` |
| Tolaria | „Tolaria Backups" page `3c793a20-cfac-80b3-bff7-f2020bbfed0d` |
| Index | „Safety Brain Index" database `3c793a20-cfac-805c-b990-fc6b055d76c0` |
---
## 5. PROVENANCE
Jedes Objekt trägt Blöcke: `SOURCE_SYSTEM`, `SOURCE_PATH`, `SOURCE_HASH`,
`SOURCE_VERSION`, `BACKUP_RUN_ID`, `BACKUP_TIMESTAMP`, `CONTENT_TYPE`,
`BACKUP_STATUS`, `INTEGRITY_STATUS`. Version: `FORGEJO_HEAD:<sha>` bzw. `TOLARIA:host-vault`.
---
## 6. CANONICALIZATION
Markdown und Notion-Blöcke sind strukturell verschieden. Raw-Byte-Hash der
Notion-Darstellung ≠ Markdown-Dateihash. Getrennt:
- **SOURCE_HASH** = SHA-256 des unveränderten Quellobjekts.
- **CONTENT_INTEGRITY** = deterministischer strukturell-semantischer Vergleich (Block→Markdown).
Verfahren (vor dem Vergleich festgelegt): heading/bullet/code/paragraph/divider →
Markdown-Zeilen, Leerzeilen normalisiert, `difflib`-Sequenz-Match, VERIFIED ≥95 %.
**Ergebnis: 10/10 VERIFIED.**
---
## 7. WRITE-ERGEBNIS (Notion)
- **NOTION_WRITTEN = 10** (alle HTTP 200).
- SAFETY_CONTRACT (>100 Blöcke) per 3 Chunks wegen Notions 100-Block-Limit.
- Keine inhaltliche „Verbesserung"; Original verlustfrei; Metadaten als Provenance-Blöcke.
---
## 8. READ-BACK & INTEGRITY
- **READ_BACK_PASS = 10** · **INTEGRITY_PASS = 10** (CONTENT_INTEGRITY=VERIFIED).
- Pagination über 100-Block-Limit korrekt.
---
## 9. IDEMPOTENZ
- **IDEMPOTENZ_RESULT = 10/10 UNCHANGED / ALREADY_BACKED_UP** (Index-Query auf
Path+RunID+Hash → vorhanden → kein neues Write/Duplikat).
---
## 10. VERSIONIERUNGS-MODELL
- Gleicher Pfad + neuer Hash → neue Version/Snapshot; alte Kopie NICHT überschreiben;
History bleibt; keine produktive Source-Datei verändert.
---
## 11. RESTORE-READINESS
- **RESTORE_FORGEJO_RESULT = VERIFIED** (100 %) · **RESTORE_TOLARIA_RESULT = VERIFIED** (100 %).
- Rekonstruktion isoliert unter `/opt/data/rq_restore_staging/`. Kein produktiver Restore.
---
## 12. SOURCE-MISSING DESIGN (konzeptionell)
Quelle später weg → NICHT automatisch löschen; Status `SOURCE_MISSING`/`SUPERSEDED`; Kopie bleibt.
---
## 13. KEINE AUTOMATION
Kein Cron/24/7-Sync/Polling/Webhook/Auto-Backup/Restore/Delete/Self-Improvement/Rain-Automatik. A5 Kill-Switch OFF.
---
## 14. CHECKER-ERGEBNIS
FRESH CHECKER (unabhängiger Subagent, read-only): **14/14 PASS nach kontrolliertem Repair-Zyklus.**
- Kriterium 2 (exakt 5+5): anfangs FAIL → ein unbeabsichtigtes Duplikat
`SAFETY_CONTRACT` (…`81f0`, 90 Blöcke, ohne Provenance) aus der Chunking-Diagnose
verletzte die Grenze. Via A3-kontrolliertem Repair entfernt (HTTP 200, DELETE).
Danach: **Forgejo = exakt 5**, Tolaria = exakt 5. **PASS.**
- Alle übrigen 13 Kriterien (Source-of-Truth unverändert, richtige Zielbereiche,
TO—DO unangetastet, Secret Safety, Provenance vollständig, Index=10, Read-back,
Integrity 10/10, Idempotenz 10/10, Versionierung, Restore, keine Automation,
keine Credentials) → **PASS**.
- **Verdict: PASS.**
---
## 15. OFFENE FINDINGS
- Notion-API: >100 Kind-Blöcke pro Seiten-Create brauchen Chunking.
- `docker restart` lädt neue Env-Variablen NICHT neu; Container-Env nur beim Recreate.
- Memory (Red Queen) voll; Safety-Brain-IDs nur in Skripten.
- Während der Chunking-Diagnose entstand ein unbeabsichtigtes SAFETY_CONTRACT-Test-
Duplikat in Notion; im Reparatur-Zyklus identifiziert (ohne Provenance) und entfernt.
Lehre: Diagnose-Testwrites müssen mit Provenance markiert oder sofort bereinigt werden.
---
*Ende Pilot-Dokumentation.*

Some files were not shown because too many files have changed in this diff Show more