Comments (8)
I'll do some investigation later
from orc.
Thank you for reporting, @PengleiShi . Could you make a PR for that?
I'am willing to try.
from orc.
I reviewed SchemaEvolution.java and determined that it was a bug.
I think maybe we can initialize a map to store fileTypeId
to readTypeId
via readerFileTypes
.
Map<Integer, Integer> typeIdsMap = new HashMap<>();
for (int i = 0; i < this.readerFileTypes.length; i++) {
this.typeIdsMap.put(readerFileTypes[i].getId(), i);
}
public boolean isPPDSafeConversion(final int colId) {
if (hasConversion()) {
Integer readTypeId = typeIdsMap.get(colId);
return readTypeId != null &&
ppdSafeConversion[readTypeId];
}
// when there is no schema evolution PPD is safe
return true;
}
Welcome @PengleiShi for making this pr. 🍻
from orc.
Thank you for reporting, @PengleiShi . Could you make a PR for that?
from orc.
cc @pgaref , @williamhyun , @guiyanakuang
from orc.
@guiyanakuang Thanks, I'll do it. And should i open a new issue in Jira?
from orc.
Yes, we need the PR prefix to be associated with Jira.
from orc.
This is resolved via #1055
from orc.
Related Issues (20)
- ORC-1696: Fix ClassCastException when reading avro decimal type in bechmark HOT 1
- ORC-1696: Fix ClassCastException when reading avro decimal type in bechmark HOT 1
- ORC-1696: Fix ClassCastException when reading avro decimal type in bechmark HOT 1
- ORC-1699: Fix SparkBenchmark in Parquet format according to SPARK-40918 HOT 1
- orc-tools unknown subcommand "Merge" HOT 3
- Release ORC 2.0.1 HOT 1
- Buffer size too small. size HOT 5
- [CPP] Flushing without writing after registering a timeseries will cause a core dump.
- ORC-1721: Upgrade `aircompressor` to 0.27 HOT 1
- ORC-1721: Upgrade `aircompressor` to 0.27 HOT 1
- Release ORC 1.9.4 HOT 1
- [Java] Different semantic of lengths for CHAR(n) with C++ HOT 1
- ORC-1540: Remove MacOS 11 from GitHub Action CI and docs HOT 1
- ORC-1540: Remove MacOS 11 from GitHub Action CI and docs HOT 1
- ORC-1540: Remove MacOS 11 from GitHub Action CI and docs HOT 1
- ORC-1619: Add `MacOS 14` to GitHub Action HOT 1
- ORC-1738: [C++] Fix wrong Int128 maximum value HOT 1
- ORC-1738: [C++] Fix wrong Int128 maximum value HOT 1
- ORC-1738: [C++] Fix wrong Int128 maximum value HOT 1
- How does hive.exec.orc.default.buffer.size affect the file size? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from orc.