Comments (2)
@javrasya Do you mean when the endSnapshotId
is no longer an ancestor of the current snapshot, it should be skipped? For example, in the following test, the second change log should only contain row(1, "a", "INSERT", 0, snap1.snapshotId())
?
@TestTemplate
public void testQueryWithRollback() {
createTable();
sql("INSERT INTO %s VALUES (1, 'a')", tableName);
Table table = validationCatalog.loadTable(tableIdent);
Snapshot snap1 = table.currentSnapshot();
long rightAfterSnap1 = waitUntilAfter(snap1.timestampMillis());
sql("INSERT INTO %s VALUES (2, 'b')", tableName);
table.refresh();
Snapshot snap2 = table.currentSnapshot();
long rightAfterSnap2 = waitUntilAfter(snap2.timestampMillis());
sql("CALL %s.system.rollback_to_snapshot('%s', %d)", catalogName, tableIdent, snap1.snapshotId());
table.refresh();
Snapshot snap4 = table.currentSnapshot();
assertThat(snap4).isEqualTo(snap1);
sql("INSERT OVERWRITE %s VALUES (-2, 'a')", tableName);
table.refresh();
Snapshot snap3 = table.currentSnapshot();
long rightAfterSnap3 = waitUntilAfter(snap3.timestampMillis());
assertEquals(
"Should have expected changed rows up to snapshot 3",
ImmutableList.of(
row(1, "a", "INSERT", 0, snap1.snapshotId()),
row(1, "a", "DELETE", 1, snap3.snapshotId()),
row(-2, "a", "INSERT", 1, snap3.snapshotId())),
changelogRecords(null, rightAfterSnap3));
assertEquals(
"Should have expected changed rows up to snapshot 2",
ImmutableList.of(
row(1, "a", "INSERT", 0, snap1.snapshotId()),
row(2, "b", "INSERT", 1, snap2.snapshotId())),
changelogRecords(null, rightAfterSnap2));
from iceberg.
Exactly @manuzhang . It feels like it should filter that out and this is a bug. Wdyt?
from iceberg.
Related Issues (20)
- Flaky Test `TestIcebergSourceFailover > testBoundedWithSavepoint` HOT 1
- partiallyClusteredDistribution returns duplicate rows from GROUP BY HOT 1
- check-ordering enablement for flink config HOT 5
- remove orphan file question HOT 4
- unable to show views HOT 2
- Run RevAPI without Gradle HOT 3
- Introduce a parameter to control whether the flink writer is linked with the previous operator HOT 10
- Iceberg TTL setting HOT 3
- Javadoc issues HOT 8
- Add view support for Hadoop catalog
- Copy iceberg table from hdfs to GCS and register table to BLMS HOT 1
- Iceberg Roadmap is 404 HOT 6
- Add PrePlanTable and PlanTable Endpoints to open api spec
- Variant Data Type Support
- Iceberg REST Spec Extensions for Access Decision Exchange
- Support for loading different hive-metastore versions at Runtime
- How iceberg implements materialized views HOT 1
- Support Securable Objects in Iceberg REST Catalog
- Broken links in Spark Writes documentation
- Questions on some requirement in view spec HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from iceberg.