-
Notifications
You must be signed in to change notification settings - Fork 55
Open
Labels
Description
The following inserts into bigquery will fail:
{"x": "y"}
{"x": true}
googleapi: Error 400: Invalid schema update. Field x has changed type from STRING to BOOLEAN, invalid
This is because when the table is first created, column x is a string. When the second JSON is inserted, x is a bool. We want to try and coerce data to match the column type if possible, otherwise use a null value. This will prevent inserts from failing outright.
BigQuery lists conversion rules here which may be helpful: https://cloud.google.com/bigquery/docs/reference/standard-sql/conversion_rules
- Create new configuration option big BigQuery:
strict_types bool. If it is set to true, then do not do any type conversion. If it is false, then do some sort of type conversion (either in SQL or on the go side) when inserting data.