Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Enable `schema_locked` on the watched table with the [`ALTER TABLE`]({% link {{
ALTER TABLE watched_table SET (schema_locked = true);
~~~

CockroachDB attempts to automatically unset `schema_locked` before performing a schema change and reapply it when done. However, certain schema changes (such as `ALTER TABLE ... SET LOCALITY`) cannot automatically unset it. For these cases, you must manually unlock the table with `schema_locked = false`, complete the schema change, and then lock the table again with `schema_locked = true`. The changefeed will run as normal while `schema_locked` is unset, but it will not benefit from the performance optimization.
CockroachDB automatically unsets `schema_locked` before performing metadata-only operations (such as renaming columns, constraints, or tables) and reapplies it when done. However, you must explicitly unset `schema_locked` for operations that require backfilling or data restructuring. For the full list of operations, refer to [Table parameters]({% link {{ page.version.version }}/with-storage-parameter.md %}#storage-parameter-schema-locked). For these cases, you must manually unlock the table with `schema_locked = false`, complete the schema change, and then lock the table again with `schema_locked = true`. The changefeed will run normally while `schema_locked` is unset, but it will not benefit from the performance optimization.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you must explicitly unset schema_locked for operations that require backfilling or data restructuring.

This is not quite accurate. For example, here's a demo in v25.3:

[email protected]:26257/defaultdb> create table t (a int primary key, b int) with (schema_locked = true);
CREATE TABLE

[email protected]:26257/defaultdb> insert into t values (1,1);
INSERT 0 1

[email protected]:26257/defaultdb> alter table t add column c int not null default 42;
ALTER TABLE

[email protected]:26257/defaultdb> alter table t alter primary key using columns (c);
ALTER TABLE

[email protected]:26257/defaultdb> show create table t;
  table_name |              create_statement
-------------+---------------------------------------------
  t          | CREATE TABLE public.t (
             |     a INT8 NOT NULL,
             |     b INT8 NULL,
             |     c INT8 NOT NULL DEFAULT 42:::INT8,
             |     CONSTRAINT t_pkey PRIMARY KEY (c ASC),
             |     UNIQUE INDEX t_a_key (a ASC)
             | ) WITH (schema_locked = true);
(1 row)

[email protected]:26257/defaultdb> alter table t drop column a;
NOTICE: dropping index "t_a_key" which depends on column "a"
ALTER TABLE

[email protected]:26257/defaultdb> alter table t add constraint ck CHECK (b > 0);
ALTER TABLE

[email protected]:26257/defaultdb> alter table t drop constraint ck ;
ALTER TABLE

As you can see, all of those backfilling changes did have the automatic unset/reapply behavior for schema_locked. In general, we expect the cases where explicit unsetting is required to be the vast minority, especially as we continue enhancing the declarative schema changer with support for more statements. The statements that require this change from version to version as we add more support to the declarative schema changer, so we decided that it was not worth explicitly enumerating the statements, since it would be hard to maintain that list accurately over time, and for the most part, users shouldn't have to think about it.

But I see in the PR description that a (DROP/ADD CONSTRAINT) schema change that you tried did not have the automatic behavior. Could you share that specific example? It's possible that we missed a corner case relating to constraints. (Also can you double check that you were testing in v25.3 or later?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey yes, it's this one: https://www.cockroachlabs.com/docs/v26.1/alter-table#drop-and-add-a-primary-key-constraint

I ran it in demo on v25.4.0-alpha.2-dev. The schema change throws this error:

[email protected]:26257/demoapp/movr> ALTER TABLE promo_codes
                                ->   DROP CONSTRAINT promo_codes_pkey,
                                ->   ADD CONSTRAINT promo_codes_pkey PRIMARY KEY (code, creation_time);
ERROR: this schema change is disallowed because table "promo_codes" is locked and this operation cannot automatically unlock the table
SQLSTATE: 57000
DETAIL: To unlock the table, execute `ALTER TABLE promo_codes SET (schema_locked = false);`
After the schema change completes, we recommend setting it back to true with `ALTER TABLE promo_codes SET (schema_locked = true);`.
HINT: Locking the table improves changefeed performance; see https://www.cockroachlabs.com/docs/v25.4/changefeed-best-practices.html#lock-the-schema-on-changefeed-watched-tables

Happy to just close this PR if you think it isn't necessary. However, I do feel like the description of schema_locked as "Indicates that a schema change is not currently ongoing on this table." is less clear than the error text itself, which says "this schema change is disallowed ... and this operation cannot automatically unlock the table".

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd be fine with updating the description of schema_locked to something more clear. The previous description, which was "disallow schema changes on the table," was not accurate, so let's just not go back to that. The text I added in my PR is at least accurate, but maybe too hard to understand. Let's keep iterating on something that works.

As for the rest of this PR, I do think we should stay away from trying to enumerate all the cases that require manual unsetting. As evidenced by the first draft of the PR, getting that list right is difficult, and since most schema changes should not require manually unsetting, I don't think having a list adds too much value.

Finally, thanks for sharing the DROP/ADD PK CONSTRAINT example. That's definitely something that we should aim to support in the declarative schema changer so that it doesn't require manually changing schema_locked. I will start working on that now -- I'd like to get that into 26.1 and we already have all the pieces.


{% include_cached copy-clipboard.html %}
~~~ sql
Expand Down
2 changes: 1 addition & 1 deletion src/current/_includes/v25.3/misc/session-vars.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
| <a id="copy-transaction-quality-of-service"></a> `copy_transaction_quality_of_service` | The default quality of service for [`COPY`]({% link {{ page.version.version }}/copy.md %}) statements in the current session. The supported options are `regular`, `critical`, and `background`. See [Set quality of service level]({% link {{ page.version.version }}/admission-control.md %}#copy-qos). | `background` | Yes | Yes |
| <a id="cost-scans-with-default-col-size"></a> `cost_scans_with_default_col_size` | Whether to prevent the optimizer from considering column size when costing plans. | `false` | Yes | Yes |
| <a id="crdb-version"></a> `crdb_version` | The version of CockroachDB. | <code>CockroachDB OSS <i>version</i></code> | No | Yes |
| <span class="version-tag">New in v25.3:</span> <a id="create_table_with_schema_locked"></a> `create_table_with_schema_locked` | When enabled, ensures that all tables created during a [session]({% link {{ page.version.version }}/show-sessions.md %}) enable the [`schema_locked` storage parameter]({% link {{ page.version.version }}/with-storage-parameter.md %}#storage-parameter-schema-locked), which improves [changefeed]({% link {{ page.version.version }}/change-data-capture-overview.md %}) performance by indicating that a schema change is not currently ongoing. | `off` | Yes | Yes |
| <span class="version-tag">New in v25.3:</span> <a id="create_table_with_schema_locked"></a> `create_table_with_schema_locked` | When enabled, tables created during a [session]({% link {{ page.version.version }}/show-sessions.md %}) have the [`schema_locked` storage parameter]({% link {{ page.version.version }}/with-storage-parameter.md %}#storage-parameter-schema-locked) enabled by default. This prevents schema changes on the table and improves [changefeed]({% link {{ page.version.version }}/change-data-capture-overview.md %}) performance by signaling that the table schema is stable. For details on which operations require explicitly unsetting `schema_locked`, refer to [Table parameters]({% link {{ page.version.version }}/with-storage-parameter.md %}#storage-parameter-schema-locked). | `off` | Yes | Yes |
| <a id="database"></a> `database` | The [current database]({% link {{ page.version.version }}/sql-name-resolution.md %}#current-database). | Database in connection string, or empty if not specified. | Yes | Yes |
| <a id="datestyle"></a> `datestyle` | The input string format for [`DATE`]({% link {{ page.version.version }}/date.md %}) and [`TIMESTAMP`]({% link {{ page.version.version }}/timestamp.md %}) values. Accepted values include `ISO,MDY`, `ISO,DMY`, and `ISO,YMD`. | The value set by the `sql.defaults.datestyle` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}) (`ISO,MDY`, by default). | Yes | Yes |
| <a id="default-int-size"></a> `default_int_size` | The size, in bytes, of an [`INT`]({% link {{ page.version.version }}/int.md %}) type. | `8` | Yes | Yes |
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
| Parameter name | Description | Data type | Default value |
|----------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------|---------------|
| `exclude_data_from_backup` | Exclude the data in this table from any future backups. | Boolean | `false` |
| <a name="storage-parameter-schema-locked"></a> `schema_locked` | Indicates that a [schema change]({% link {{ page.version.version }}/online-schema-changes.md %}) is not currently ongoing on this table. CockroachDB automatically unsets this parameter before performing a schema change and reapplies it when done. Enabling `schema_locked` can help [improve performance of changefeeds]({% link {{ page.version.version }}/create-changefeed.md %}#disallow-schema-changes-on-tables-to-improve-changefeed-performance) running on this table. | Boolean | `false` |
| <a name="storage-parameter-schema-locked"></a> `schema_locked` | Prevents [schema changes]({% link {{ page.version.version }}/online-schema-changes.md %}) on this table. Enabling `schema_locked` can help [improve performance of changefeeds]({% link {{ page.version.version }}/create-changefeed.md %}#disallow-schema-changes-on-tables-to-improve-changefeed-performance) running on this table by signaling that the table schema is stable.<br/><br/>CockroachDB automatically unsets this parameter before performing metadata-only operations (such as renaming columns, constraints, or tables) and reapplies it when done. However, you must explicitly unset `schema_locked` for operations that require backfilling or data restructuring, including:<ul><li>[<code>SET LOCALITY</code>]({% link {{ page.version.version }}/alter-table.md %}#set-locality)</li><li>[<code>ADD COLUMN</code>]({% link {{ page.version.version }}/alter-table.md %}#add-column)</li><li>[<code>DROP COLUMN</code>]({% link {{ page.version.version }}/alter-table.md %}#drop-column)</li><li>[<code>ALTER PRIMARY KEY</code>]({% link {{ page.version.version }}/alter-table.md %}#alter-primary-key)</li><li>[<code>ADD CONSTRAINT</code>]({% link {{ page.version.version }}/alter-table.md %}#add-constraint) (including foreign keys)</li><li>[<code>DROP CONSTRAINT</code>]({% link {{ page.version.version }}/alter-table.md %}#drop-constraint)</li><li>[<code>CREATE INDEX</code>]({% link {{ page.version.version }}/create-index.md %}) / [<code>DROP INDEX</code>]({% link {{ page.version.version }}/drop-index.md %})</li><li>[<code>SET (ttl_expire_after = ...)</code>]({% link {{ page.version.version }}/alter-table.md %}#set-storage-parameter)</li><li>[<code>TRUNCATE</code>]({% link {{ page.version.version }}/truncate.md %})</li></ul>The changefeed will run normally while <code>schema_locked</code> is unset, but will not benefit from the performance optimization. | Boolean | `false` |
| `sql_stats_automatic_collection_enabled` | Enable [automatic statistics collection]({% link {{ page.version.version }}/cost-based-optimizer.md %}#enable-and-disable-automatic-statistics-collection-for-tables) for this table. | Boolean | `true` |
| `sql_stats_automatic_collection_min_stale_rows` | Minimum number of stale rows in this table that will trigger a statistics refresh. | Integer | 500 |
| `sql_stats_automatic_collection_fraction_stale_rows` | Fraction of stale rows in this table that will trigger a statistics refresh. | Float | 0.2 |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Enable `schema_locked` on the watched table with the [`ALTER TABLE`]({% link {{
ALTER TABLE watched_table SET (schema_locked = true);
~~~

CockroachDB attempts to automatically unset `schema_locked` before performing a schema change and reapply it when done. However, certain schema changes (such as `ALTER TABLE ... SET LOCALITY`) cannot automatically unset it. For these cases, you must manually unlock the table with `schema_locked = false`, complete the schema change, and then lock the table again with `schema_locked = true`. The changefeed will run as normal while `schema_locked` is unset, but it will not benefit from the performance optimization.
CockroachDB automatically unsets `schema_locked` before performing metadata-only operations (such as renaming columns, constraints, or tables) and reapplies it when done. However, you must explicitly unset `schema_locked` for operations that require backfilling or data restructuring. For the full list of operations, refer to [Table parameters]({% link {{ page.version.version }}/with-storage-parameter.md %}#storage-parameter-schema-locked). For these cases, you must manually unlock the table with `schema_locked = false`, complete the schema change, and then lock the table again with `schema_locked = true`. The changefeed will run normally while `schema_locked` is unset, but it will not benefit from the performance optimization.

{% include_cached copy-clipboard.html %}
~~~ sql
Expand Down
2 changes: 1 addition & 1 deletion src/current/_includes/v25.4/misc/session-vars.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
| <a id="copy-transaction-quality-of-service"></a> `copy_transaction_quality_of_service` | The default quality of service for [`COPY`]({% link {{ page.version.version }}/copy.md %}) statements in the current session. The supported options are `regular`, `critical`, and `background`. See [Set quality of service level]({% link {{ page.version.version }}/admission-control.md %}#copy-qos). | `background` | Yes | Yes |
| <a id="cost-scans-with-default-col-size"></a> `cost_scans_with_default_col_size` | Whether to prevent the optimizer from considering column size when costing plans. | `false` | Yes | Yes |
| <a id="crdb-version"></a> `crdb_version` | The version of CockroachDB. | <code>CockroachDB OSS <i>version</i></code> | No | Yes |
| <a id="create_table_with_schema_locked"></a> `create_table_with_schema_locked` | When enabled, ensures that all tables created during a [session]({% link {{ page.version.version }}/show-sessions.md %}) enable the [`schema_locked` storage parameter]({% link {{ page.version.version }}/with-storage-parameter.md %}#storage-parameter-schema-locked), which improves [changefeed]({% link {{ page.version.version }}/change-data-capture-overview.md %}) performance by indicating that a schema change is not currently ongoing. | `off` | Yes | Yes |
| <a id="create_table_with_schema_locked"></a> `create_table_with_schema_locked` | When enabled, tables created during a [session]({% link {{ page.version.version }}/show-sessions.md %}) have the [`schema_locked` storage parameter]({% link {{ page.version.version }}/with-storage-parameter.md %}#storage-parameter-schema-locked) enabled by default. This prevents schema changes on the table and improves [changefeed]({% link {{ page.version.version }}/change-data-capture-overview.md %}) performance by signaling that the table schema is stable. For details on which operations require explicitly unsetting `schema_locked`, refer to [Table parameters]({% link {{ page.version.version }}/with-storage-parameter.md %}#storage-parameter-schema-locked). | `off` | Yes | Yes |
| <a id="database"></a> `database` | The [current database]({% link {{ page.version.version }}/sql-name-resolution.md %}#current-database). | Database in connection string, or empty if not specified. | Yes | Yes |
| <a id="datestyle"></a> `datestyle` | The input string format for [`DATE`]({% link {{ page.version.version }}/date.md %}) and [`TIMESTAMP`]({% link {{ page.version.version }}/timestamp.md %}) values. Accepted values include `ISO,MDY`, `ISO,DMY`, and `ISO,YMD`. | The value set by the `sql.defaults.datestyle` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}) (`ISO,MDY`, by default). | Yes | Yes |
| <a id="default-int-size"></a> `default_int_size` | The size, in bytes, of an [`INT`]({% link {{ page.version.version }}/int.md %}) type. | `8` | Yes | Yes |
Expand Down
Loading
Loading