Tests failure on 1.7.3 about orc HOT 7 CLOSED

Thank you for reporting, @ArchangeGabriel . We don't have Arch Linux in our CI so far.

cc @wzhou-code , @wgtmac , @stiga-huang because this sounds like a regression.

In the worst case, we need to revert #1065 .

from orc.

Sorry if there has been a misunderstanding about how #1065 in involved here, I meant that we backport this patch on top of current release else tests fails, but we already patched for that since 1.6.7.

The error is unrelated and new, it did not occur in 1.7.1. I’m not sure I have tried rebuilding 1.7.1 recently though, I should check whether it does still build fine or if the regression comes from a dependency change.

from orc.

No problem. We are looking forward to seeing your test result. Let me know when you have a new result.

from orc.

Same failures on 1.7.1 (+TestDecompression.testLzoLong that was fixed since). So there has been a dependency related change. Since they are not a lot of dependencies involved (I don’t know for this particular tests, but you might be able to tell me), given what changed on our side in this lapse of time, it seems it could be either protobuf (that went from 3.17.3 to 3.19.4) or gcc (11.1 to 11.2). Unfortunately trying older version of those is quite involved, so I’ll not be able to pinpoint any of those if they are the culprit.

from orc.

According to the context, the root cause is unclear to me either.

The error is unrelated and new, it did not occur in 1.7.1.

Same failures on 1.7.1

Back to the basic, it's hard to consider something as a bug if we cannot reproduce it. In addition, we don't know what you have in your fork or what you have in your Arch Linux build and test system yet. We have four releases like the following. Could you confirm that which tag is the last working tag for you?

• https://github.com/apache/orc/releases/tag/v1.7.0
• https://github.com/apache/orc/releases/tag/v1.7.1
• https://github.com/apache/orc/releases/tag/v1.7.2
• https://github.com/apache/orc/releases/tag/v1.7.3

from orc.

We built 1.7.1 when it was released. At this time the only test failure was TestDecompression.testLzoLong. But rebuilding it again today has the same failure as reported above for 1.7.3. So the same version of orc is showing different failures depending on the dependency versions in the build environment.

I’ve tested back to 1.7.0, and all releases show the same failures now. So it’s definitively not a regression in orc itself, but something that broke due to a dependency update in our environment (… or something else, see below). You can see the version used for each dependency by clicking on the package names in the dependency list there: https://archlinux.org/packages/community/x86_64/apache-orc/

Our flags are as follow:

CFLAGS="-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions \
        -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security \
        -fstack-clash-protection -fcf-protection"
CXXFLAGS="$CFLAGS -Wp,-D_GLIBCXX_ASSERTIONS"
LDFLAGS="-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now"
LTOFLAGS="-flto=auto"

Which by the way makes me realize we changed something else recently, has we enabled LTO distro-wide. So I’ve decided to try disabling that and… bingo. At least for all the failures in the first test suite, they disappear if I disable LTO.

The only one remaining in this case, is:

[ RUN      ] TestFileScan.testErrorHandling
/build/apache-orc/src/orc-1.7.3/tools/test/TestFileScan.cc:209: Failure
Expected: (std::string::npos) != (error.find(error_msg)), actual: 18446744073709551615 vs 18446744073709551615
[  FAILED  ] TestFileScan.testErrorHandling (31 ms)

So they are two different issues here. This last one, maybe I should open a separated ticket for it. And tests failures in PredicateLeaf as well as BloomFilter, that are LTO related.

from orc.

Thank you for the details, @ArchangeGabriel . Given the analysis, I'm closing this issue Tests failure on 1.7.3.
As you mentioned here, we had better restart with a new narrow-downed specific GitHub issue for the specific failure later.

from orc.