Comments (4)
drisspg
commented on July 18, 2024
1
this is weird, is this only happening under some compile context?
from pytorch.
jbschlosser
commented on July 18, 2024
1
@drisspg yes this test is for torch.compile() behavior with NJT + SDPA in a way that emulates what FIRST is doing
from pytorch.
drisspg
commented on July 18, 2024
Hmm @danthe3rd do you know if when max_seq_len > sum(seq_len) is it possible to iterate into bad memory? I think the max_seq_len sets a max iteration bound and but there is still checks to ensure that current token indexes are valid right?
from pytorch.
danthe3rd
commented on July 18, 2024
Probably you can remove that check. It does not make sense to have max_seq_len > sum(seq_len) tho, as it's always bounded by the sum, but it should be supported by the kernel I guess?
This is code I wrote some time ago so I don't have the context from the top of my head.
from pytorch.
Related Issues (20)
- Ineffective Buffer Size Adjustment in Multi-Node NCCL Communication
- torch.compile slows down paged flash attention HOT 1
- Increase (4x+) in build times HOT 4
- DISABLED test_fallback_to_eager_if_recompiling_too_many_times (__main__.CudaGraphTreeTests) HOT 1
- [dynamo] map in dynamo doesn't return an iterable HOT 3
- [dtensor] write aten.split_tensor using op strategy
- DISABLED test_cpp_extension_recommends_custom_ops_inline_inbuilt_nn_modules (__main__.InlineInbuiltNNModulesMiscTests) HOT 1
- Migrate gradcheck with check_batched_grad=True to use the new torch.vmap
- [Inductor] Add an API to register external callable candidates for inductor's Matmul/Conv tuning choices
- API To Make Custom Post Grad Passes Hashable HOT 2
- [inline-inbuilt-nn-modules] Unlimited recompiles because the nn.Module is going out of scope HOT 1
- [CUDA] Remove footgun related to non-blocking copies HOT 5
- Training on M1 MBP: Placeholder storage has not been allocated on MPS device HOT 3
- An additional dimension appears in the return when using torch.compile on torch.nn.LSTM models
- `with torch.device()` modifies the `as_tensor` intended behavior
- xpu: huggingface generation pipelines are missing xpu aten ops causing perf impact
- torch2.2.0+cuda12.1 image cant run on cuda12.2 HOT 1
- [rocm] F.embedding reports invalid configuration argument HOT 2
- Generalizing cross product to hodge star operator
- RuntimeError in _multi_tensor_adamw: Output Shape Mismatch During torch.foreach_lerp Operation HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pytorch.