Skip to content

feat(filtering): fine-grained table, column, row, and scope filtering#169

Merged
jasdeepkhalsa merged 2 commits intomasterfrom
feature/164-fine-grained-filtering
Apr 3, 2026
Merged

feat(filtering): fine-grained table, column, row, and scope filtering#169
jasdeepkhalsa merged 2 commits intomasterfrom
feature/164-fine-grained-filtering

Conversation

@jasdeepkhalsa
Copy link
Copy Markdown
Member

Summary

Implements #164 — fine-grained control over what enters the diff comparison across four dimensions.

What's New

Table filtering

  • tables (+ --tables CLI flag) — include list: only diff tables matching these names/globs
  • tablesToIgnore (+ --ignore-tables CLI flag) — exclude list: skip tables entirely (now supports glob patterns)
  • tablesDataToIgnore — exclude from data diff only; schema is still compared
  • Priority: include list narrows first → exclude list narrows further

Column filtering

  • fieldsToIgnore — per-table column exclusion. Keys now support glob patterns (wp_*, audit_?), so one rule can cover multiple tables.

Row filtering

  • rowsToIgnore — per-table list of { column, pattern } rules. Rows where column matches the regex pattern are skipped from the data diff.

Scope override

  • tableScope — override diff scope per table to schema, data, or all.

All list values support glob patterns: * (any chars), ? (single char).

Implementation

  • New TableFilter utility class (src/Params/TableFilter.php) — centralises all filtering logic, stateless, glob-aware, fully unit tested.
  • Six new params added to DefaultParams and parsed from config/CLI.
  • DBSchema and DBData updated to use glob-aware TableFilter::filterTables().
  • TableSchema, LocalTableData, DistTableData, ArrayDiff updated to use TableFilter::getFieldsToIgnore() (now glob-aware for keys).
  • StreamingMergeDiff, LocalTableData, TableData updated to apply rowsToIgnore filtering.

Tests

  • 40+ new unit tests in tests/Unit/TableFilterTest.php covering all filtering paths: include/exclude combination, glob matching, field resolution, row rule matching, regex edge cases.
  • Fixed StreamingMergeDiffTest — added ParamsFactory setup so row-filter code path is safe to call in tests.
  • All 563 tests pass (946 assertions).

Configuration example

# Only diff these tables (globs supported)
tables:
  - users
  - orders
  - wp_*

tablesToIgnore:
  - cache_*

tablesDataToIgnore:
  - audit_log

fieldsToIgnore:
  users:
    - updated_at
  wp_*:
    - ID

rowsToIgnore:
  wp_options:
    - { column: option_name, pattern: "_transient_.*" }

tableScope:
  audit_log: schema

CLI

dbdiff server1.db1:server2.db2 --tables=users,orders,wp_* --ignore-tables=cache_*

…#164)

Add fine-grained control over what enters the diff comparison:

- **Table include list** (`tables` / `--tables`): only diff matching tables
- **Table exclude list** (`tablesToIgnore` / `--ignore-tables`): skip tables entirely (enhanced with glob support)
- **Data-only exclude** (`tablesDataToIgnore`): skip data diff while keeping schema diff
- **Column exclusion** (`fieldsToIgnore`): per-table column skipping (keys now support glob patterns)
- **Row filtering** (`rowsToIgnore`): skip rows matching a column-value regex, per table
- **Scope override** (`tableScope`): override diff scope to `schema`, `data`, or `all` per table

All list values support glob patterns (`*`, `?`) for flexible matching.

New `TableFilter` utility class centralises all filtering logic — stateless,
glob-aware, tested in isolation.

Closes #164
@github-actions github-actions bot added the php Pull requests that update php code label Apr 3, 2026
@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud bot commented Apr 3, 2026

@jasdeepkhalsa jasdeepkhalsa merged commit 19d6517 into master Apr 3, 2026
130 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

php Pull requests that update php code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant