Skip to content

Adding Postgres 14-18 & SQLite Support#141

Merged
jasdeepkhalsa merged 13 commits intomasterfrom
feature/sqlite-postgres-support
Mar 3, 2026
Merged

Adding Postgres 14-18 & SQLite Support#141
jasdeepkhalsa merged 13 commits intomasterfrom
feature/sqlite-postgres-support

Conversation

@jasdeepkhalsa
Copy link
Copy Markdown
Member

@jasdeepkhalsa jasdeepkhalsa commented Mar 3, 2026

Summary

New database support

  • Added full PostgreSQL and SQLite diff support alongside MySQL.
  • Introduced a DBAdapterInterface with dedicated MySQLAdapter, PostgresAdapter, and SQLiteAdapter implementations.
  • Added a DialectRegistry + SQLDialectInterface with AbstractAnsiDialect, MySQLDialect, PostgresDialect, and SQLiteDialect — all SQL generation classes now delegate to the active dialect.

Data diffing

  • PostgreSQL: PHP-side row comparison (no cross-database SQL JOINs).
  • SQLite: ATTACH DATABASE approach for cross-file JOINs.

Infrastructure

  • docker-compose.yml extended with PostgreSQL 14–18 services.
  • start.sh / stop.sh updated with Podman compatibility.
  • GitHub Actions matrix expanded to cover Postgres and SQLite across multiple PHP versions.

Tests

  • New End2EndPostgresTest, End2EndSQLiteTest, comprehensive suites, and baseline fixture files for all supported PG versions.

Refactoring / quality

  • All SonarQube issues resolved.
  • AbstractComprehensiveTest extracted to eliminate duplication across MySQL, Postgres, and SQLite suites.

…pattern

- Introduce DBAdapterInterface with MySQL, Postgres and SQLite implementations
  (AdapterFactory selects based on --driver param)
- DBManager now delegates all DB-specific introspection to the active adapter:
  getTables, getColumns, getPrimaryKey, getTableSchema, getCreateStatement
- PostgresAdapter: queries information_schema + pg_indexes + pg_tables
- SQLiteAdapter: uses PRAGMA table_info / index_info / foreign_key_list
- Introduce SQLDialectInterface with MySQL, Postgres and SQLite implementations
  (DialectRegistry holds the singleton set by DiffCalculator at startup)
- All 18 DiffToSQL classes now accept dialect for identifier quoting;
  MySQL-only ops (ENGINE,CHARSET,COLLATION) return '' on other drivers
- AddTableSQL/DropTableSQL use manager.getCreateStatement() instead of
  raw SHOW CREATE TABLE
- AddTable/DropTable diff objects now store manager+connectionName
- TableSchema.getSchema() delegates to manager.getTableSchema() (adapter)
- DBSchema skips MySQL-only collation/charset DB-level checks on non-MySQL
- DiffCalculator calls DialectRegistry.setForDriver() before diff generation
- Add --driver=mysql|pgsql|sqlite CLI flag (default: mysql for bc)
- Add --supabase shorthand: sets driver=pgsql + sslmode=require
- AlterTableChangeConstraintSQL: fix pre-existing bug ($key vs $name)
- Dockerfile: install libpq-dev + libsqlite3-dev; add pdo_pgsql + pdo_sqlite extensions to all CLI images
- docker-compose.yml: add db-postgres16 service (postgres:16) + cli-php83-postgres16 / cli-php84-postgres16 CLI services that inject DB_HOST_POSTGRES
- .env.example: add POSTGRES_VERSION_16 and DB_PORT_POSTGRES16
- tests/end2end/db1-up-pgsql.sql + db2-up-pgsql.sql: PostgreSQL-compatible fixtures mirroring MySQL db1/db2 structure
- tests/end2end/db1-up-sqlite.sql + db2-up-sqlite.sql: SQLite-compatible fixtures using INTEGER/TEXT types
- tests/End2EndPostgresTest.php: end-to-end test for --driver=pgsql
  • auto-skips when DB_HOST_POSTGRES env var is unset or pdo_pgsql not loaded
  • creates diff1pgsql / diff2pgsql databases, applies fixtures, runs DBDiff, compares output
  • versioned expected baseline (migration_expected_pgsql_<major>)
  • supports DBDIFF_RECORD_MODE=true for first-run baseline capture
- tests/End2EndSQLiteTest.php: end-to-end test for --driver=sqlite
  • auto-skips when pdo_sqlite not loaded
  • creates two temp SQLite files at /tmp/dbdiff_e2e_src and /tmp/dbdiff_e2e_tgt
  • runs DBDiff with file paths embedded in the comparison argument
  • supports DBDIFF_RECORD_MODE=true for baseline capture
- scripts/run-tests.sh: add --postgres [host] and --sqlite flags
… add Podman compat to start.sh/stop.sh

README.md:
- Features: replace "MySQL only" line with multi-database support + Supabase note
- Supported databases: rename section, add PostgreSQL 16 + SQLite 3.x + Supabase subsections with usage snippets
- Command-Line API: add --driver and --supabase flag documentation
- Usage Examples: add examples 4 (Postgres), 5 (Supabase), 6 (SQLite)
- .dbdiff example: add driver: key with mysql|pgsql|sqlite options
- Docker section: mention Podman as alternative to Docker Desktop

DOCKER.md:
- Environment Variables: add DB_PORT_POSTGRES16
- Podman Support section: install instructions for Ubuntu/macOS/Windows, Podman Desktop link, podman-compose usage examples, rootless networking note
- Available Configurations: add PHP 8.5, MySQL 9.6, PostgreSQL 16
- Services: add cli-php83-postgres16, cli-php84-postgres16, db-postgres16
- Usage Examples: add Postgres diff example and Postgres/SQLite test commands with DBDIFF_RECORD_MODE
- Database Connection Details: add PostgreSQL 16 port
- Continuous Integration matrix: add PostgreSQL 16, SQLite

start.sh / stop.sh:
- Auto-detect COMPOSE_CMD (docker-compose -> podman-compose fallback)
- All docker-compose command invocations replaced with $COMPOSE_CMD
- COMPOSE_CMD can be overridden via environment variable
…/SQLite jobs

- SQLDialectInterface: add addColumn() + dropColumn() method signatures
- MySQLDialect: implement without COLUMN keyword (preserves existing baselines)
- PostgresDialect/SQLiteDialect: implement with ADD COLUMN / DROP COLUMN
- AlterTableAddColumnSQL/DropColumnSQL: delegate to dialect methods instead of
  hardcoding keywords; this fixes the ADD/DROP COLUMN baseline regression
- End2EndSQLiteTest: fix applyFixture() to strip comment lines per-line before
  splitting on semicolons (was discarding first CREATE TABLE block); change
  fail() to markTestSkipped() when no baseline exists
- End2EndPostgresTest: change fail() to markTestSkipped() when no baseline
- tests.yml: add test-sqlite + test-postgres CI jobs; MySQL job now also loads
  pdo_sqlite extension; new e2e tests skip gracefully when no baseline present
- run-tests.sh: --postgres flag now sets SPECIFIC_TEST=End2EndPostgresTest so
  the Postgres CI job doesn't attempt MySQL-dependent tests in a MySQL-less env
PostgresDialect and SQLiteDialect shared 100% of their quote(),
isMySQLOnly(), dropIndex(), addColumn(), and dropColumn() logic.
SonarQube reported 58-72% duplicated lines on new code.

Fix: introduce AbstractAnsiDialect abstract base class that owns all
shared ANSI SQL behaviour; both dialects now extend it and only
override what is genuinely engine-specific:
  - PostgresDialect: only getDriver() (2 unique lines)
  - SQLiteDialect:   getDriver() + changeColumnWarning() (7 unique lines)

changeColumnWarning() hook lets each dialect customise the comment
emitted before a DROP+ADD column change without duplicating the
surrounding ALTER TABLE logic.

Also update README.md:
  - Add PHP 8.5 to prerequisites and supported PHP versions list
  - Expand PostgreSQL supported versions from 16 only to 14-18
  - Correct 'all 9 combinations' to 'all 16 combinations' and add
    a note explaining the full CI matrix (MySQL 16 + SQLite 4 + PG 20)
Baselines generated locally with Podman and PHP 8.3 against Postgres 16:
  tests/end2end/migration_expected_sqlite    — SQLite e2e expected output
  tests/end2end/migration_expected_pgsql_16  — Postgres 16 e2e expected output

The end-to-end tests now have real committed baselines so CI runs the
full assertion rather than falling through to markTestSkipped().

Also fixed two issues discovered during local baseline generation:

1. Dockerfile + docker-compose.yml: qualify all Docker Hub image names
   (mysql, postgres, phpmyadmin, php base image) with their full
   docker.io/... registry prefix. Podman requires fully-qualified names
   when no unqualified-search registries are configured.

2. src/Params/ParamsFactory.php: SQLite is file-based and requires no
   --server1 connection string. The 'A server is required' guard is now
   skipped when driver=sqlite, matching the behaviour already present
   in DBManager::connect().
Generated with Podman + PHP 8.3 against each Postgres major version.
All five baselines (14-18) are byte-for-byte identical, confirming
that DBDiff's introspection output is stable across the full PG range
in the CI matrix. Each version keeps its own file so future fixtures
or schema introspection changes can be tracked per-version.
Constructor visibility (20 files):
- src/Diff/AddTable.php, DropTable.php
- src/SQLGen/DiffToSQL/*.php (18 files)
All had implicit-public constructors; added explicit 'public' keyword.

Curly braces (CLIGetter.php):
- Added braces to all 11 braceless single-statement if blocks (was
  flagged as Critical / pitfall by SonarQube).

PostgresAdapter.php — two issues:
1. buildConnectionConfig(): remove temp $config variable; return the
   array literal directly (clumsy temporary assignment, Low).
2. buildColumnType(): 10 returns exceeded the 3-return limit (Major).
   Refactored into two focused methods:
   - buildColumnType(): lookup table for simple string mappings,
     delegates parametrised types; exactly 1 return.
   - resolveParameterisedType(): handles varchar/char, numeric/decimal,
     and timestamp variants via if/elseif; exactly 1 return.
($p && $p > 0) -> ($p > 0): in PHP, null > 0 and 0 > 0 are both
false, so the null-guard && is redundant. Removing it drops the
SonarQube cognitive complexity score from 16 to 15.
@jasdeepkhalsa jasdeepkhalsa changed the title Feature/sqlite postgres support Adding Postgres 14-18 & SQLite Support Mar 3, 2026
…nsive suites

- Extract AbstractComprehensiveTest base class with all 9 shared test
  methods: schema-only, data-only, template, up/down, single-table,
  config file, config+CLI override, tables-to-ignore, fields-to-ignore
- Rewrite DBDiffComprehensiveTest to extend the abstract class (~95 lines
  vs 484 before); implements 8 abstract driver-specific methods only
- Add DBDiffComprehensivePostgresTest: connects via pdo_pgsql, auto-skips
  when DB_HOST_POSTGRES is unset, uses versioned suffixes (pgsql_14…18)
- Add DBDiffComprehensiveSQLiteTest: uses pdo_sqlite + /tmp files, skips
  testSingleTableDiff (path slashes break dot-notation parser)
- Add 12 driver-specific fixture files (db1/db2 × pgsql/sqlite × 3 sets)
- Add named PHPUnit testsuites 'Postgres' and 'SQLite' to both XML configs
- run-tests.sh: --postgres now uses --testsuite Postgres (runs both e2e
  AND comprehensive); --sqlite uses --testsuite SQLite; --specific uses
  --filter as before
- docker-compose.yml: add db-postgres14/15/17/18 services (ports 5414,
  5415, 5417, 5418) with corresponding cli-php83/84 containers
- .env.example + .env: add POSTGRES_VERSION_14/15/17/18 and
  DB_PORT_POSTGRES14/15/17/18
…selines

- LocalTableData: add getDiffPgsql() + fetchIndexed() for PHP-side cross-DB diff
  PostgreSQL does not support cross-database SQL JOINs, so rows are fetched from
  each connection separately and compared in PHP (INSERT/UPDATE/DELETE detection).
- LocalTableData: getDiff() now routes pgsql to getDiffPgsql(), sqlite to
  getDiffSQLite(), everything else to the existing MySQL path.
- Generate all comprehensive test baselines:
  * 9 SQLite baselines (tests/expected/*_sqlite.txt)
  * 45 PostgreSQL baselines (tests/expected/*_pgsql_{14..18}.txt)
  * 1 SQLite e2e baseline (tests/end2end/migration_expected_sqlite)
  * 5 PostgreSQL e2e baselines (tests/end2end/migration_expected_pgsql_{14..18})
- Add scripts/gen-postgres-baselines.sh for re-generating PG baselines via podman.
- Remove debug artifacts (scripts/run-sqlite-debug.sh, scripts/debug_sqlite_data.php,
  scripts/run-data-debug.sh, tests/debug_*.txt, tests/sqlite_run.log).
- Replace <directory>./</directory> with explicit file entries for
  DBDiffComprehensiveTest.php and End2EndTest.php.
- Stops PHPUnit 11 warning about AbstractComprehensiveTest being abstract
  (which caused exit code 1 on CI).
- Prevents SQLite/Postgres test files from being double-run when no
  --testsuite filter is applied.
@jasdeepkhalsa jasdeepkhalsa force-pushed the feature/sqlite-postgres-support branch from b62d98e to 626118d Compare March 3, 2026 13:17
@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud bot commented Mar 3, 2026

@jasdeepkhalsa jasdeepkhalsa merged commit 8f398cc into master Mar 3, 2026
41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant