Project Kanban

[Frontend] Visualisation du schéma des sinks dans l'éditeur de pipeline

[Core] Simplification des schémas des actions vers opérations vers sink

Connector - Add connector on demand

Core - Dynamic schema - Customer extensions in datasets

Streams - Per-file status tracking and retry

UI - Data source - Discovery running

When updating noesis ID, we also need to update lineage targetting it

UI - Administration Panel for Users

UI - Data sync activity when running

UI - Global refresh toolbar

UI - Data Source - Capabilities - We should see which datasets can be imported/ingested with which tool

Target schema auto-creation

Flink - Review incremental_join problems

Core - Sink schema management

Core - Python Transform Infrastructure (user-defined transformations with lake access)

Delete propagation

Reporting - Run event data model and Postgres storage

Reporting - API endpoints GET /runs/{id}/report and /errors

Flink - Study impact of upgrading to latest Flink version (1.20.3 → 2.x)

E2E - Tests avances de dependances cross-datasource (multi-source vers une cible)

E2E Bug — streaming_join scenario failing

UI - Data Lineage - Data Lineage should have a better display, currently it's only statistics

Architecture Pilot/Workers (ADR-010) — capacités, scheduling et gouvernance de concurrence

[Archi] Supprimer stepContext du workflow multi-step — remplacer par lookup en base

Infra - Étude multi-tenant: architecture, isolation, impact sur le code existant

Infra - Étude comparative des cloud providers pour Noesis SaaS (focus Europe/France)

Core + UI - Pipeline schema validation: conformité des data models entrée/sortie par opération

Core + UI - Pipeline pause/resume/restart transparent (masquer la complexité Flink)

UI - Pipeline live monitoring: real-time DAG visualization with per-operation metrics

Core - Pipeline bootstrap from lake when Kafka retention is expired (historical data access)

Core - Kafka Source Connector (streaming data source with JSON support)

UI - Demo review session: real-time feedback (NOE-23 + NOE-25)

Manage dataset dependencies to allow correct import orchestration

[Frontend] Construire IHM de dashboard/pilotage/suivi des dépendances et des déblocages

Batch Import Tool — Stratégies de regroupement configurables (ADR-011)

Plan d'ingestion — Sub-ingestions parallèles (batchId/subBatchId, SUB_BATCH_END)

Core — Stratégie fixed-size + time-window

Core — Stratégie tree (regroupement par sous-arbre hiérarchique — cas XPDM)

Core — Stratégie custom (script Python via infra transform)

Core — API CRUD BatchStrategy + état accumulation

3DX — Finaliser le connecteur XPDM batch (PackageBuilder + XML format NOE-232)

3DX Simulator — Support XPDM batch import (PR #3 en review)

E2E — Scénarios batch import (partial + complémentaire + timeout + erreurs)

Frontend — Configuration batch strategy sur pipeline + vue progression packages

Core — Modèle domain BatchStrategy + ToolModePackage + migrations

Core — Câbler le mode PACKAGE dans ImportMessageHandler + import resolver

Core — BatchAccumulator dans l'import resolver (Redis state + triggers BATCH_END/timeout)

UI - Pipeline - It should be possible to have a global view of the pipeline with also its impact (like sink -> data source tools and source with data sync info)

UI - Pipeline - It should be possible to see schema info on operation (in and out)

Core — Renommer import-resolver en import-orchestrator