When used in the FlashAttention-2 layer the GEMM FP32 kernel yields wrong results by o

It's because of this: <div class="Box-

Bug in GEMM FP32 baseline kernel about snitch_cluster HOT 2 CLOSED

viv-eth commented on August 11, 2024

Bug in GEMM FP32 baseline kernel

from snitch_cluster.

Comments (2)

colluca commented on August 11, 2024

With what parameters do you observe these errors?

from snitch_cluster.

fischeti commented on August 11, 2024

It's because of this:

snitch_cluster/sw/blas/gemm/src/gemm.h

Lines 1381 to 1387 in d325300

 // Transpose of A unsupported 

 if (transa) return -1; 

 if (transb) { 

 // Transpose of B supported only in FP64 

 if (prec != FP64) return -1; 

 ldb = frac_k; 

 }

Matrix B needs to be transposed for SIMD kernels. Now it just immediately returns for all FP32 gemm kernels. It would be nice to have proper C assertions there instead of return -1. I tried that once. I tried that once, but ran into some issues with newlib.

from snitch_cluster.

Related Issues (20)

make bin/snitch_cluster.vlt error HOT 2
Batchnorm and Maxpool layers unverified HOT 1
GEMM kernel stalls due to dimension mismatches HOT 1
Cluster wrapper generation fails for cluster with `xssr` fully disabled HOT 7
trace: `fseq_fpu_yield` identical to `fseq_yield`
fcvt.s.d instruction not working with SSR HOT 2
Docker container image failing after make command HOT 1
Building the hardware use command "make bin/snitch_cluster.vlt" has error HOT 1
Need login for nonfree part HOT 1
Add +define+COMMON_CELLS_ASSERTS_OFF HOT 3
Add documentation for trace performance metrics HOT 1
Fix legacy warnings in `make docs` target
Check parameters in HW config file against schema
Snitch paper link in "Custom Instructions" doc section broken
`iis-setup.sh` fails on Pisoc machines HOT 1
Add datagen and IPC-based verification tutorials to the documentation
Encountered an 'Illegal Instruction' error during the simulation HOT 1
Banshee simulator error: Unsupported instruction HOT 1
Update AXI dependency to new release when available

Bug in GEMM FP32 baseline kernel about snitch_cluster HOT 2 CLOSED

Comments (2)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent

	// Transpose of A unsupported
	if (transa) return -1;
	if (transb) {
	// Transpose of B supported only in FP64
	if (prec != FP64) return -1;
	ldb = frac_k;
	}