Comments (10)
RyanTheOptimist
commented on July 26, 2024
If I'm following you correctly, the you're hitting a stack trace from this
code:
QUIC_BUG_IF(quic_bug_12477_2, destruction_indicator_ != 123456789)
<< "QuicSpdySession use after free. " << destruction_indicator_
<< QuicStackTrace();
This indicates a use-after-free and is a major problem. Do you understand
how this is happening? (I'm not sure why the stack trace is taking so long
to generate, but the real problem, I think, is the use-after-free that
you're hitting)
Cheers,
Ryan
…On Sun, Dec 4, 2022 at 3:09 AM Marten Richter ***@***.***> wrote:
I had the following problem, my clients stalled the event loop when
closing their session for more than 10 seconds.
One or two clients (do not really know, should be one thread) for
webtransport (but it does not matter), called my equivalent of:
https://github.com/google/quiche/blob/30eeff5e759f83b1a93418b86c726710eda04f25/quiche/quic/tools/quic_client_base.h#L353
in their destructor.
This ended up in:
https://github.com/google/quiche/blob/30eeff5e759f83b1a93418b86c726710eda04f25/quiche/quic/core/http/quic_spdy_session.cc#L479
,
where
QuicStackTrace() caused the 10-second stall, I used the supplied default
implementation.
Putting debug code into the stack trace showed, that
https://github.com/google/quiche/blob/30eeff5e759f83b1a93418b86c726710eda04f25/quiche/common/platform/default/quiche_platform_impl/quiche_stack_trace_impl.cc#L35
took a very long time to progress.
Maybe a second per stack frame. It can be a locking issue since it has
some synchronization built in. ( I am not sure=)
(Only on mac and linux, windows is probably fine)
Anyway, I use now a dummy stack trace on my platform.
But I think, this issue should be addressed, since it can hit hard the
performance of an application, that uses it (and can take a day or more to
find).
Either
QuicStackTrace should only be called, if an error occurs, so put an if
around:
https://github.com/google/quiche/blob/30eeff5e759f83b1a93418b86c726710eda04f25/quiche/quic/core/http/quic_spdy_session.cc#L479
and probably similar places.
Or something needs to be done to prevent the slow performance of the
current stack trace. I can not rule out, that this is a problem of absl, I
am using lts_2022_06_23 .
—
Reply to this email directly, view it on GitHub
<#29>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AEVHVST44X4NIN566C4YGPDWLR3XFANCNFSM6AAAAAASTJLLBM>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
from quiche.
martenrichter
commented on July 26, 2024
Yes, this is the line am hitting.
But, the if clause is not hit, destruction_indicator_ == 123456789, so not use-after-free
So it was not printed, but the stack trace was of course executed anyway.
I assume that the idea is, that the compiler optimizes it away in this case.
However, in my case, it does not happen and the stack is "printed" to void anyway.
from quiche.
RyanTheOptimist
commented on July 26, 2024
Hm. Are you saying that even when destruction_indicator_ has the correct
value of 123456789, then you're *still* seeing the stack trace? That's
*really* surprising and should not be happening. Are you seeing this
behavior from other QUIC_BUG_IF()s, or just this one?
…On Mon, Dec 5, 2022 at 6:52 AM Marten Richter ***@***.***> wrote:
Yes, this is the line am hitting.
But, the if clause is not hit, destruction_indicator_ == 123456789, so not
use-after-free
So it was not printed, but the stack trace was of course executed anyway.
I assume that the idea is, that the compiler optimizes it away in this
case.
However, in my case, it does not happen and the stack is "printed" to void
anyway.
—
Reply to this email directly, view it on GitHub
<#29 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AEVHVSXKNAYNHRBHNS3JQ4LWLX6UPANCNFSM6AAAAAASTJLLBM>
.
You are receiving this because you commented.Message ID:
***@***.***>
from quiche.
martenrichter
commented on July 26, 2024
Well, I am not seeing it. But if I put a print to the stack trace I see that it is executed.
And if I overwrite the stack implementation my problems go away.
I am currently not using the default logging code, maybe I should switch to the default code? My current code was written before default dummies were provided.
from quiche.
RyanTheOptimist
commented on July 26, 2024
If you're not using the default logging code, I wonder if there's a bug
there which causes the stack trace to be executed when it wouldn't be
normally. I'd definitely give that a shot and see what happens.
…On Mon, Dec 5, 2022 at 7:36 AM Marten Richter ***@***.***> wrote:
Well, I am not seeing it. But if I put a print to the stack trace I see
that it is executed.
And if I overwrite the stack implementation my problems go away.
I am currently not using the default logging code, maybe I should switch
to the default code? My current code was written before default dummies
were provided.
—
Reply to this email directly, view it on GitHub
<#29 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AEVHVSUXQ6C6IA73EEOIWE3WLYDYLANCNFSM6AAAAAASTJLLBM>
.
You are receiving this because you commented.Message ID:
***@***.***>
from quiche.
martenrichter
commented on July 26, 2024
Ok, I have checked it.
It is the logging implementation.
So I will replace it with the default.
Then it is my fault. I wonder, if this is the reason, some users said the performance of the node addon is not as good as they expected.
So thank you!
But I am still puzzled that the stack trace takes so long, the lambda functions, (or it may be a deadlock).
from quiche.
RyanTheOptimist
commented on July 26, 2024
Glad to hear that explains the Stack Trace firing when it shouldn't have. Totally agree that the stack track performance is really surprising. In particular, the default implementation https://github.com/google/quiche/blob/30eeff5e759f83b1a93418b86c726710eda04f25/quiche/common/platform/default/quiche_platform_impl/quiche_stack_trace_impl.cc simply delegates to the ABSL stack track utilities, which should be pretty solid.
@vasilvv to you have any thoughts about this?
from quiche.
vasilvv
commented on July 26, 2024
I suspect that what happens here is, in canonical implementation of QUIC_BUG_IF, the stack trace is never called if the condition is not met. We probably just need to fix our logging impls.
from quiche.
martenrichter
commented on July 26, 2024
The quiche default logging impl is fine, mine was faulty, now replaced by the default.
But that the stack trace takes a long time may be a problem.
from quiche.
martenrichter
commented on July 26, 2024
Anyway, I close this..., was my faulty logging impl.
from quiche.
Related Issues (20)
- Masque Server segmentation fault
- optimize SerializedPacket copy/move HOT 2
- @platforms//os:linux is not a valid configuration key HOT 1
- Commit about ECN break UDP socket platform impl HOT 2
- how to build on window? HOT 3
- Does it support the Windows platform HOT 1
- How to access the spdsession object from a WebTransportSession HOT 2
- Toy Client not validating path or using Server Preferred Address
- time-wait handler should take precedence over `OnFailedToDispatchPacket` HOT 3
- how to enable QUIC_LOG HOT 1
- SyscallWrapper no matching problem when compile quiche on android
- [masque_client] Failed to enable receiving of self v4 ip on macOS HOT 1
- Undefined behavior sanitizer (ubsan) => Load of value <value>, which is not a valid value for type 'QuicErrorCode'. HOT 1
- Undefined behavior sanitizer (ubsan) => Null pointer passed as argument 2, which is declared to never be null. HOT 5
- IETF Connection Close Frame - Possible narrowing of error code coming from reason phrase HOT 1
- Allow sending NEW_TOKEN frames before TLS NewSessionTickets HOT 1
- QuicBatchWriterBuffer with custom allocated buffer HOT 2
- I have a question about compute min_rtt in bbrV2
- QuicIdleNetworkDetector::StopDetection error or duplicated?
- Can chrome send NEW_CONNECTION_ID frame to server after h3 handshake ? HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from quiche.