dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
Feldera is a streaming SQL engine powered by
the DBSP incremental computation engine. It automatically incrementalizes
every SQL query without watermarks, scans, or MERGE. When input data
changes, only affected output rows are recomputed.
Important
This adapter deploys continuous pipelines, not ad-hoc queries.
Feldera supports two modes of query execution:
- Continuous pipelines compile SQL into an incremental dataflow that runs indefinitely, processing every input change as it arrives in near real-time.
- Ad-hoc queries are one-shot batch queries executed by DataFusion against the state of materialized tables and views. They exist primarily for development and debugging.
When you run dbt run, this adapter assembles your models into a Feldera
pipeline program, compiles it, and starts a continuously running pipeline.
The pipeline keeps processing input changes and updating outputs until it is
explicitly stopped. This differs from typical batch-oriented dbt adapters where dbt run executes
a query once, processes a batch of data and exits.
- Automatic incremental view maintenance (IVM) — Feldera's DBSP engine incrementalizes any SQL query out of the box. No manual merge logic or watermark tuning required.
- Continuous pipeline deployment —
dbt runcompiles and starts a long-running Feldera pipeline; it does not execute one-shot queries. - Connector integration — attach Kafka, Delta Lake, S3, and HTTP connectors directly to models via configuration.
- Easy setup — pure Python adapter with no ODBC driver needed.
pip install dbt-felderaor with uv:
uv add dbt-felderaRequires Python 3.10+ and dbt-core ~1.9.
Add a Feldera target to your profiles.yml:
my_project:
target: dev
outputs:
dev:
type: feldera
host: "http://localhost:8080"
api_key: "apikey:..." # optional — for authenticated instances
database: "default"
schema: "my_pipeline" # maps to the Feldera pipeline name
compilation_profile: dev # dev | unoptimized | optimized
workers: 4
timeout: 300Feldera uses different terminology than traditional databases. Here's how dbt concepts map to Feldera. Every materialization contributes SQL to a continuously running pipeline — nothing is executed as a one-shot batch query.
| dbt concept | Feldera concept | Description |
|---|---|---|
database |
(unused) | Set to any string (e.g. "default") |
schema |
Pipeline name | Each dbt schema maps to one Feldera pipeline (a continuously running SQL program) |
table materialization |
Input table | External data source (Kafka, HTTP, S3) |
view materialization |
View | SQL view inside the continuous pipeline (all views are incrementally maintained) |
view + stored: true |
Materialized view | Queryable via ad-hoc queries |
seed |
Table + HTTP push | Schema registered, data pushed via HTTP ingress |
| Option | Default | Description |
|---|---|---|
host |
http://localhost:8080 |
Feldera API base URL |
api_key |
(none) | API key for authenticated Feldera instances |
schema |
(required) | Pipeline name in Feldera |
compilation_profile |
dev |
SQL compilation profile: dev (fast compile), unoptimized, or optimized (best runtime performance) |
workers |
4 |
Number of pipeline worker threads |
timeout |
300 |
Max wait (seconds) for pipeline compilation + startup |
Creates a CREATE VIEW in the pipeline. Use for intermediate transformations
that don't need to be queried directly or connected to an output.
-- models/orders_enriched.sql
{{ config(materialized='view') }}
SELECT o.id, o.total, c.name AS customer_name
FROM {{ ref('orders') }} o
JOIN {{ ref('customers') }} c ON o.customer_id = c.idSet stored: true to promote to a CREATE MATERIALIZED VIEW — a view backed
by persistent storage, enabling ad-hoc queries:
-- models/sales_summary.sql
{{ config(
materialized='view',
stored=true,
connectors=[{'transport': {'name': 'my_delta_connector'}}]
) }}
SELECT
region,
product_category,
SUM(amount) AS total_sales,
COUNT(*) AS order_count
FROM {{ ref('orders') }}
GROUP BY region, product_categoryNote
Every view in Feldera is automatically incrementally maintained by the DBSP
engine. When inputs change, only affected output rows are recomputed — no
watermarks, merge logic, or special configuration required. The
stored flag controls only whether the view's state is
queryable (via ad-hoc queries); it does not change how the view is
computed.
On --full-refresh, the pipeline is stopped, all stored state (including
connector offsets) is cleared, and the pipeline is redeployed from scratch.
Creates a CREATE TABLE — an input source for external data ingress. The model
SQL defines the column schema, not a SELECT query. Attach connectors for
Kafka, S3, HTTP, or other input sources.
-- models/raw_events.sql
{{ config(
materialized='table',
connectors=[{
'transport': {
'name': 'kafka_in',
'config': {
'bootstrap.servers': 'redpanda:29092',
'topics': ['events']
}
},
'format': {'name': 'json'}
}]
) }}
event_id BIGINT NOT NULL,
event_type VARCHAR NOT NULL,
payload VARCHAR,
created_at TIMESTAMP NOT NULLImportant
dbt-feldera does not support the incremental materialization because
all views in Feldera are natively maintained incrementally by the DBSP
engine.
Use materialized='view' with stored=true instead:
{{ config(materialized='view', stored=true) }}Deploys an entire Feldera pipeline as one dbt model. The model SQL is the
complete pipeline program — containing CREATE TABLE and CREATE VIEW
statements. Useful for complex multi-table, multi-view pipelines managed as a
single unit.
-- models/my_pipeline.sql
{{ config(materialized='streaming_pipeline') }}
CREATE TABLE orders (
id BIGINT NOT NULL,
customer_id BIGINT NOT NULL,
amount DECIMAL(10, 2) NOT NULL
);
CREATE TABLE customers (
id BIGINT NOT NULL,
name VARCHAR NOT NULL
);
CREATE MATERIALIZED VIEW enriched_orders AS
SELECT o.id, o.amount, c.name AS customer_name
FROM orders o
JOIN customers c ON o.customer_id = c.id;Seeds register a CREATE TABLE and push row data via Feldera's HTTP ingress
API after the pipeline is deployed. Use for small reference datasets (CSVs).
dbt seed # push seed data
dbt seed --full-refresh # stop, clear storage, redeploy, then push| Materialization | Feldera SQL | Best for |
|---|---|---|
view |
CREATE VIEW |
Incrementally maintained intermediate transforms |
view + stored: true |
CREATE MATERIALIZED VIEW |
Queryable outputs |
table |
CREATE TABLE |
External input sources (Kafka, S3, HTTP) |
streaming_pipeline |
Full program | Multi-table/view pipelines as a single unit |
seed |
CREATE TABLE + data push |
Small reference datasets (HTTP ingress; any connector can also be attached) |
- Feldera documentation — platform docs, SQL reference, connectors
- Pipelines (continuous queries) — how Feldera compiles SQL into an incremental dataflow
- Ad-hoc queries — one-shot DataFusion queries for debugging materialized state
- Materialized tables and views — prerequisite for ad-hoc query access
- dbt documentation — general dbt usage and concepts
See CONTRIBUTING.md for development setup, testing, and project layout.
Apache-2.0 — see LICENSE for details.