Make sure Colt McAnlis properly reads issues before closing them about etc2comp HOT 16 CLOSED

Hi Reduz,

I encourage you to investigate and explore developing this requested feature on your own, for your specific use case and constraints. If you find that it does produce a significant performance improvement that the community could benefit from, please feel free to send a pull request (along with supporting data), and our team can take a look at it.

Moving forward, I'm going to close this issue for the following reasons:

Exact Issue resolved
The precise ask in this issue is for me to stop closing requests without reading them. I have read, evaluated, and responded to all your requests within the best of my abilities, given the information provided in them. I'll meet with my team for a review of these threads to see if there's anything we can learn to better serve our community in the future.

We cannot reproduce your results
I grabbed a 1CPU 500mb machine and ran the same test, with the same texture as listed above, the encode time in debug mode was 713ms. Granted, it's slower than my MBP, but still significantly faster than the 2 second encode time you quoted. Unless you provide some specific details on how you're doing encoding, I cannot reproduce your results.

Lack of evidence to support claims that this feature would improve performance
Your feature request is being made on the grounds that it will produce significant performance gains. There has been ample opportunity to provide details to support this claim however no legitimate profiling, tracing, or benchmarking has been provided to support that claim.

Please understand that our team must be very diligent in choosing what to spend their time on, and without correct evidence, we can't conclude this is a problem worth fixing.

Generally unproductive discussion
Github is a great and positive community that we're happy to support and be a part of. I've asked a handful of individuals to review your comments, and it's been noted that language and tone of your replies could be considered a violation of Google's anti-harassment policy which extend into online communities.

Please consider this your warning that future language along this line will not be tolerated. We encourage discussion between developers, but we maintain a zero tolerance policy on harassment at both developer events and on developer content.

from etc2comp.

I'm sorry you felt offended. I hope you understand that closing an issue three times without even bothering to understand what it is about could very well be considered offensive. You did not ask a question to clear up a possible misunderstanding even once (save for being ok closing the issue). All you did was to get defensive instead of even attempting a little to understand the discussion.

That's pretty far from the "Be excellent to each other." and could also be considered "verbal language that reinforces social structures of domination", as you are closing it against my will and without even bothering to understand what the problem is about (and much less the request). In fact, I believe my attitude fits more in line with the "Speak up if you see or hear something." item. As you are the Google employee, not me, I should be the one expecting you to follow your own company guidelines.

I still maintain that you did not get the point of the argument, not even answered once with a constructive attitude. All you did in every answer was defend against technical assertions I never made. As you mention that more developers are in your team, I would be very glad to discuss this issue with someone else that might be able to understand it without getting defensive.

from etc2comp.

Github allows users to close issues they opened themselves, so there's usually no need for authority to do so unless the user forgets or the conversation escalates. Just a hint for triage of future issues, understanding this can prevent misunderstandings that end up irritating for both sides.

from etc2comp.

optionally compress in ETC1 WHILE KEEPING SUPPORT FOR TRANSPARENCY

Not all ETC2 features are backwards compatible with ETC1. Transparency, specifically is not one of them. Please read the ETC1 specification. As noted there, transparency is not directly supported in the etc1 format and requires hackish client-side fixes to support (your own shaders).

As such, your request of to improve compression speed by back porting ETC2 features to ETC1 is not something supported by the hardware, it has nothing to do with etc2comp. If you've found a specific resource which shows otherwise, please highlight it and use it in your request.

While etc2comp supports exporting etc1 textures, it is not intended to be an etc1 compressor, and becoming one is not currently in the scope of this project. There are many other software packages which can export high quality etc1 textures, and I would suggest looking into those.

NEED OPTION FOR FASTER INITIAL COMPRESSION SPEED

When released last year, etc2comp was a leader in ETC2 texture compression speed compared to other codecs. If you would like, you can read Building a blazing fast etc2 compressor to understand more of the optimizations we've made, and what's made etc2comp so fast.

If you're finding a specific test case that it's failing on, please clearly highlight it in a new issue (with included data) and we can address it.

If you've found an optimization for performance, please send a pull request and we can take a look at it. The community would always welcome improved performance.

Given that transparency in ETC1 isn't directly GPU supported, may I close your request?

from etc2comp.

Hi Colt,

No, sorry, I believe we are still in a misunderstanding here. I honestly don't know how the internals of ETC2 works, nor much less about existing compression techniques, but I will try to make my case more technically.

Problem

The specific problem I have in Godot Engine is that users import textures in batches, and the turnaround time to test your code on Android is really quick. (Godot installs a small executable on Android and reads all files via USB cable). As a result, the long time required to import ETC2 textures (up to 10 seconds each large texture on a not very beefy machine, on low quality etc2comp settings) can be problematic.

Users don't mind if the quality of 3D textures is lower, and they can raise it later when they have more time, or only in the textures they care about the most. What they do mind is long compression times. In comparison, libquish takes less than a second on a large texture, even though I understand S3TC is a much simpler format.

So, the main need of our project is to have means of importing and using the textures quick, and later bother about adjusting the quality. If they want to spend five minutes on the highest quality setting of etc2comp is fine, as long as it's done later.

How to achieve this?

My proposal is based on the little I can understand of this document

It specifically says two things:

The first:

RGB ETC2 is a format for compressing RGB data. It is a superset of the older ETC1 format. This means that an older ETC1 texture can be decoded using an ETC2-compliant decoder. The main difference is that the newer version contains three new modes; the ‘T-mode’ and the ‘H-mode’ which are good for sharp chrominance blocks and the ‘Planar’ mode which is good for smooth blocks

The second:

RGBA ETC2 encodes RGBA 8-bit data. The RGB part is encoded exactly the same way as RGB ETC2. The alpha part is encoded separately.

With these two bits of information, I can imagine that it is possible to do a minor modification to ETC2Comp, where the color (RGB) information is compressed using only ETC1 features (for speed, no extra modes), and the alpha is encoded normally in the ETC2 alpha format.

Would this feature be possible? Something like "don't use extra ETC2 modes". It would help us enormously, as our main concern is compression speed, not quality (which can be increased later) while yet preserving ETC2 features such as RGBA support.

from etc2comp.

So, the main need of our project is to have means of importing and using the textures quick, and later bother about adjusting the quality

As described in our documentation you can use the -effort flag to trade off compression quality for encode speed. 0 is faster, with worse quality, 100 is slower, with best quality.

Something like "don't use extra ETC2 modes"

As described in Building a blazing fast ETC2 compressor, the lower the effort, the less time is spent searching through the permutations of combinations of blocks to find the best fit.

In many cases, this means avoiding ETC2 formats, and just going with ETC1 block types.

The order in which blocks are fitted, and tested has been decided through analysis of a large corpus of images, where we determine the block archetype, and sort potential block formats by the probability of a high quality match. As described this is why etc2comp out performs other encoders like MALI, which appear to be using a brute force method (aka, test every potential block combination).

As such, what you are asking for is already supported in etc2comp. It is done intelligently using the block pre-matching system, so that no command line flags need to be used to ignore un-needed block types.

Running your tests on -effort 0 will remove all searching through the block chain for matching, thus eliminating any best fit matches. It's as fast as etc2comp can possibly run, at the lowest possible quality, and would satisfy the needs of your particular use case.

Given that what you're asking for is already in etc2comp, may I close this request?

from etc2comp.

Given that what you're asking for is already in etc2comp, may I close this request?

I'm sorry but I don't think it is. I tried making a simple test of importing a 1024x1024 image at effort 0 in etc2comp and taking the time, repeatedly. The average result is:

ETC1 2.005 secs
ETC2 RGB8 - 3.952

So your claim:

As such, what you are asking for is already supported in etc2comp. It is done intelligently using the block pre-matching system, so that no command line flags need to be used to ignore un-needed block types.

Is not valid, and my request stands. We need a way to compress less quality and make etc2comp take less time. The suggested approach, which is using only ETC1 blocks in a ETC2 format (as in, with Alpha channel added) is clearly a way to solve this problem.

I suggest, why not allowing to use effort -1, and in such cases, the library uses only ETC1 blocks to compress ETC2_RGBA format?

from etc2comp.

I cannot reproduce your results.

I grabbed a fresh build of etc2comp, built it in release mode. I tested this Dice Texture with ./etctool 3.png -effort 0 -format RGBA8 -jobs 8 -output ./3.ktx on my macbook pro.
Debug mode yielded 216ms, and release mode yielded 104ms

I also tested it against the MALI tool, (downloaded last month), with the command parameters -s fast -c etc2 -f RGBA8 -e nonperceptual -ktx and got a result of 402ms.

These numbers are an order of magnitude faster than your test and suggest that etc2comp is still performing properly, and that there's an issue with your build / testing environment.

In order to help us better resolve the problem with your testing environment:

• Please test the provided texture above with the command line arguments in your build environment and report the results.
• Please encode your texture in MALI, and provide the timing information from that encoder as well.
• If you would like us to more accurately repo your use case, please provide your command line arguments, example textures, build configuration, and machine.

from etc2comp.

Sorry for using etc2comp on a shitty computer not worthy of it. I hope you didn't get offended, but please understand I live in South America and decent hardware is really expensive here and we can't afford to purchase the Boeing jet engines you run at Google for work.

That aside, i don't think absolute performance numbers are a problem. What I meant is that compressing ETC1 RGB8 is FASTER than ETC2 RGB8. Given ETC2 is backwards compatible to ETC1, it means you lied about etc2comp doing the best effort possible for encoding in low quality.

So, could you please add a mode in etc2comp where ETC2_RGBA8 is compressed using ETC1 blocks only, please? I don't want to have to fork this awesome library to add such function.

from etc2comp.

Implementation for reference.

from etc2comp.

Aside from not getting the technical aspect agressively, he became even funnier when he started taking about anti-harassment policy and topped it off with "consider this a warning".

I posted a link to the commit in the hopes that he'll understand what the idea is.

from etc2comp.

@tagcup For now will just fork it and do the changes needed, and eventually do a PR, as I feel words go to a bottomless pit otherwise.

from etc2comp.

Oh and let's not forget that it "will not be tolerated in the future". Jeez, what a personality.

from etc2comp.

@reduz Yep, that sounds like the best option to me, we should be able to cherry-pick necessary stuff.

from etc2comp.

I mean, this discussion is ridiculous, I say from the first closed post "Etc2comp is slow on low end systems (tested on AMD A8, mentioned in #20) for my specific use case, takes X amount of time. I would appreciate if certain option to speed up compression in low quality is added".

Answers from @mainroach are all unrelated to this, first one is in #20 "other compressors are slower". I'm happy to hear that, but has nothing to do with the request. Then "Use ETC1 if you need it", which misses the point as alpha support is clearly specified as a need in #21.

When I try to get more technical and request that T,H and Planar modes can be optionally turned off in formats such as RGBA8, as this speeds up compression in lower end systems, @mainroach answers "etc2comp already does that". To that, I answer with a simply benchmark showing that discarding those modes compresses twice as fast".

To this, @mainroach replies that he can't reproduce my results, and that Etc2comp is faster on his system. I honestly had no idea how to answer this, as it totally and completely missed the point. He did not even compare the compression with these options turned off. He simply compared etc2comp to mali compressor.. which is totally unrelated. So I got upset.

It missed the point entirely because my benchmark was about relative compression speed with a feature turned off. @mainroach simply asserted that it works faster on his computer and can't reproduce my compression times. Of course he can't I evidently have a much slower computer than him (AMD A8, as mentioned in #20). Even if I had his same computer, his answer misses the point.

My benchmark simply showed that compression can be sped up by disabling planar, T and H modes, countering the argument about "etc2comp already doing what I mentioned". It was totally unrelated to absolute compression speeds.

As mentioned before, would be happy to make a PR, but would like anyone from the etc2comp team undersand what it is about.

from etc2comp.

Alpha does take time to compress, maybe that's why you were getting

ETC1 2.005 secs
ETC2 RGB8 - 3.952

?

from etc2comp.