sql/storage: consider if `TEXT COLUMNS`, `IGNORE COLUMNS` belongs on subsources about materialize HOT 5 OPEN

Oh shoot--it should be removed. Let me see if I push anything else today that I can get a CI run in with the removal.

from materialize.

Spent a good bit of time thinking about this with the upcoming work of turning Kafka sources' outputs into TABLEs.

In the case of Kafka, the options on the source affect the RelationDesc of its single output, but it would be really obnoxious to envision a world in which all of the source options (envelopes, encoding, etc) move into CREATE TABLE.

Using this same logic,then, options such as TEXT COLUMNS and IGNORE COLUMNS are the PG and MySQL version of encoding and more rightly belong on the source itself.

One way that I'm thinking about this is that a source should have a "catalog" of outputs available for subsources/ingestion exports/read-only tables to "pull from" and the ingestion export simply needs to be told what the RelationDesc of that output collection is. The ingestion export should not be able to arrive with its own desires about the shape of the output––that gets into the realm where subsources become a kind of second-class materialized view.

This is also "great news," because the MySQL source bakes in the TEXT COLUMNS, IGNORE COLUMNS values into the definition of the tables it stores. If we wanted to move these encoding options onto the ingestion exports themselves, we might want to consider storing the native representation of the MySQL tables. This would be a lot of annoying work with very little yield that I think everyone will be happy to side step.

from materialize.

@benesch mentions on Slack that this is work we will likely want to do eventually. My question and conclusion here was one with more immediacy––i.e. I don't plan to do this work and wouldn't urge someone else to begin doing it immediately.

from materialize.

@sploiselle: We still have a TODO referencing this issue; how shall we proceed with it? (The "detect closed issues" step in CI complains.)

from materialize.

Reopening since there is still an open TODO regarding this issue: https://buildkite.com/materialize/nightly/builds/7664#018f504b-cbec-4f26-8755-7d73d4b303c2

from materialize.