wuch15 / fastformer Goto Github PK

The dataset and the dataloading code is not proposed.

The 'all_head_size' in the FastSelfAttention module is always equal to the 'hidden_size' in the specified config, it is not necessary to name it individually by the calculation.

您好，请问文中对比的方法Poolingformer，您是从哪里获得的源码呢？我们也想与Poolingformer进行对比，但是没有找到该文章的源码。

Hello,

First of all, thank you very much for your work. I'm currently trying to reproduce your paper and I didn't find the proper Amazon dataset that you used in the paper. All of them are above 1M rows but you said to have used 41K rows for your train, val and test set. Did you sample 41K randomly from the original dataset or did you do something else ?

Thank you very much.
Mathias

I used VPN to get the ag_news dataset, but it occured the ValueError: check_hostname requires server_hostname, how to solve it?

由于注意力计算方式的不同 fastattention似乎不能用于序列预测任务？

The first line of Fastformer.ipynb is from datasets import load_dataset. But there is no datasets.py in the repository.
Can you please share where I can find the datasets module/ script?

能否发一个TF2.x的版本呢？还有有相应的模型权重文件吗？

What should I do?

There's a error when i try to load the ag_news dataset. It said that Couldn't reach https://raw.githubusercontent.com/huggingface/datasets/1.12.1/datasets/ag_news/ag_news.py. But I can open the url in my chrome.
If you load the dataset successfully, could u tell me how to solve the problem?
ConnectionError: Couldn't reach https://raw.githubusercontent.com/huggingface/datasets/1.12.1/datasets/ag_news/ag_news.py

I couldn't find code to run Fastformer on the news recommendation task on MIND dataset in the paper. Would be great if you can help with the same!