dp-integration

Data platform integration surfaces

Data platform integration surfaces An integration topology with two focal components: MinIO, the storage hub everything writes into, and Trino, the federation engine that answers queries across it. Four sources attach on the left — a relational database over JDBC, an SFTP drop, a mailbox pulled over IMAP, and a mainframe file export. A dashed bypass shows Trino federating the relational database directly without going through ingest. Inside the DATA PLATFORM zone an Apache Airflow trigger bar drops dashed schedules into NiFi, MinIO and JupyterLab; MinIO feeds Trino, which serves four consumers on the right — statistics desktops over ODBC, notebook users over HTTPS, BI and reporting over JDBC, and an API gateway over REST. Two footer bars, identity and observability, sit below the zone and connect only to the zone's bottom edge. Source systems Consumer surfaces Data platform FEDERATE JDBC SFTP IMAP FILE S3 S3 ICEBERG ODBC JDBC REST HTTPS AUTH TELEMETRY Relational database PostgreSQL · SQL Server SFTP drop nightly partner files Mailbox returns inbox, attachments Mainframe export fixed-width batch Apache Airflow scheduler — triggers the row below, emits no labelled wires Apache NiFi ingest tool MinIO storage hub — focal JupyterLab notebook surface Trino federation engine — the platform's public query face, second focal component Statistics desktops SPSS · SAS · Stata Notebook users researchers, ad hoc work BI and reporting Superset · Power BI API gateway public indicator service Identity Keycloak · Active Directory — LDAP, SSO, group RBAC for every surface above Observability Prometheus · Grafana · Loki — metrics, dashboards and logs for the whole zone