Comments (15)
I don't understand your question. It is "path" (e.g. train path or query path), not "1".
In market.py , fpaths = sorted(glob(osp.join(self.images_dir, path, '.jpg'))) but in cuhk.py it is fpaths = sorted(glob(osp.join(self.images_dir, 1, '.png'))) , I am confused about the datasets structure, can you make a list just like it ? Thank you. CUHK/ ├── xxxx/ ├────-------/xxx/ ├────-------/xxx/ ├────-------/xx/ ├── xxxx/ ├────-------/xxx/ ├────-------/xxx/ ├────-------/xxx/
For any dataset, the directory name should be the same with the dataset name. So for cuhk03_np_detected, you need to have a directory named cuhk03_np_detected under the data-dir, containing the three sub folders. That is,
data-dir/
├── market/
├────-------/bounding_box_test/
├────-------/bounding_box_train/
├────-------/query/
├── cuhk03_np_detected/
├────-------/bounding_box_test/
├────-------/bounding_box_train/
├────-------/query/
├── cuhk03_np_labeled/
├────-------/bounding_box_test/
├────-------/bounding_box_train/
├────-------/query/
from qaconv.
python main.py --dataset market --testset cuhk03_np_detected --data-dir ./reid/datasets/ --exp-dir ./Exp so this code is about training, I change the batchsize = 2, it is also out of cuda memory
This is not likely to be the problem in training. The code in main.py also performs testing. I guess it is raised in testing. Please reduce the values of the three parameters: --test_fea_batch, --test_gal_batch, --test_prob_batch.
from qaconv.
It is now merged into main.py.
For some public datasets, use the definitions in QAConv/reid/datasets. Or define your own dataset in this folder.
from qaconv.
It is now merged into main.py.
For some public datasets, use the definitions in QAConv/reid/datasets. Or define your own dataset in this folder.
Thank you for your patient answer, I will try it
from qaconv.
It is now merged into main.py.
For some public datasets, use the definitions in QAConv/reid/datasets. Or define your own dataset in this folder.
Hi, I'm confused about the cuhk.py about fpaths = sorted(glob(osp.join(self.images_dir, 1, '*.png'))) what's the osp.join(xx, 1, *.png) mean, "1"? What the function of cuhk.py ? Thank you.
the cuhk-np dataset I had download just like this:
cuhk03-np/
├── detected/
├────-------/bounding_box_test/
├────-------/bounding_box_train/
├────-------/query/
├── labeled/
├────-------/bounding_box_test/
├────-------/bounding_box_train/
├────-------/query/
from qaconv.
I don't understand your question. It is "path" (e.g. train path or query path), not "1".
from qaconv.
I don't understand your question. It is "path" (e.g. train path or query path), not "1".
In market.py , fpaths = sorted(glob(osp.join(self.images_dir, path, '.jpg'))) but in cuhk.py it is fpaths = sorted(glob(osp.join(self.images_dir, 1, '.png'))) , I am confused about the datasets structure, can you make a list just like it ? Thank you.
CUHK/
├── xxxx/
├────-------/xxx/
├────-------/xxx/
├────-------/xx/
├── xxxx/
├────-------/xxx/
├────-------/xxx/
├────-------/xxx/
from qaconv.
python main.py --dataset market --testset cuhk03_np_detected --data-dir ./reid/datasets/ --exp-dir ./Exp
RuntimeError: CUDA out of memory. Tried to allocate 9.00 GiB (GPU 0; 11.77 GiB total capacity; 198.24 MiB already allocated; 7.10 GiB free; 2.38 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
batchsize=16
from qaconv.
I don't understand your question. It is "path" (e.g. train path or query path), not "1".
In market.py , fpaths = sorted(glob(osp.join(self.images_dir, path, '.jpg'))) but in cuhk.py it is fpaths = sorted(glob(osp.join(self.images_dir, 1, '.png'))) , I am confused about the datasets structure, can you make a list just like it ? Thank you. CUHK/ ├── xxxx/ ├────-------/xxx/ ├────-------/xxx/ ├────-------/xx/ ├── xxxx/ ├────-------/xxx/ ├────-------/xxx/ ├────-------/xxx/
But I'm still confused how you get "1" but not "path" in the cuhk.py code.
from qaconv.
python main.py --dataset market --testset cuhk03_np_detected --data-dir ./reid/datasets/ --exp-dir ./Exp
RuntimeError: CUDA out of memory. Tried to allocate 9.00 GiB (GPU 0; 11.77 GiB total capacity; 198.24 MiB already allocated; 7.10 GiB free; 2.38 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF batchsize=16
I use V100 with 32GB of GPU memory. I'm not sure if 12GB memory is enough to run the training. By the way, do you encounter this during training or testing? During testing you can reduce some parameter values as described in the parameter help info in main.py.
from qaconv.
python main.py --dataset market --testset cuhk03_np_detected --data-dir ./reid/datasets/ --exp-dir ./Exp
so this code is about training, I change the batchsize = 2, it is also out of cuda memory
from qaconv.
python main.py --dataset market --testset cuhk03_np_detected --data-dir ./reid/datasets/ --exp-dir ./Exp so this code is about training, I change the batchsize = 2, it is also out of cuda memory
This is not likely to be the problem in training. The code in main.py also performs testing. I guess it is raised in testing. Please reduce the values of the three parameters: --test_fea_batch, --test_gal_batch, --test_prob_batch.
--batch-size=64 , (--test_fea_batch, --test_gal_batch, --test_prob_batch) =128 It works for me(3060 12G gpu)! Thank you so much!Result====
Mean AP: 18.1%
CMC Scores:
top 1: 18.9%
top 5: 33.9%
top 10: 44.1%
top 20: 54.1%
cuhk_np_detected : rank1 = 18.9 , mAP=18.1
Finishing training at epoch 9 , loss = 7.183, acc = 81.85%
from qaconv.
I don't understand your question. It is "path" (e.g. train path or query path), not "1".
In market.py , fpaths = sorted(glob(osp.join(self.images_dir, path, '.jpg'))) but in cuhk.py it is fpaths = sorted(glob(osp.join(self.images_dir, 1, '.png'))) , I am confused about the datasets structure, can you make a list just like it ? Thank you. CUHK/ ├── xxxx/ ├────-------/xxx/ ├────-------/xxx/ ├────-------/xx/ ├── xxxx/ ├────-------/xxx/ ├────-------/xxx/ ├────-------/xxx/
But I'm still confused how you get "1" but not "path" in the cuhk.py code.
I check the code, It is my fault, I guess maybe when I'm sleeping on the table during lunch break and accidentally pressed the keyboard, so it change to 1
from qaconv.
I don't understand your question. It is "path" (e.g. train path or query path), not "1".
In market.py , fpaths = sorted(glob(osp.join(self.images_dir, path, '.jpg'))) but in cuhk.py it is fpaths = sorted(glob(osp.join(self.images_dir, 1, '.png'))) , I am confused about the datasets structure, can you make a list just like it ? Thank you. CUHK/ ├── xxxx/ ├────-------/xxx/ ├────-------/xxx/ ├────-------/xx/ ├── xxxx/ ├────-------/xxx/ ├────-------/xxx/ ├────-------/xxx/
For any dataset, the directory name should be the same with the dataset name. So for cuhk03_np_detected, you need to have a directory named cuhk03_np_detected under the data-dir, containing the three sub folders. That is,
data-dir/ ├── market/ ├────-------/bounding_box_test/ ├────-------/bounding_box_train/ ├────-------/query/ ├── cuhk03_np_detected/ ├────-------/bounding_box_test/ ├────-------/bounding_box_train/ ├────-------/query/ ├── cuhk03_np_labeled/ ├────-------/bounding_box_test/ ├────-------/bounding_box_train/ ├────-------/query/
....
Finished epoch 7 at lr=[5e-05, 0.0005, 0.0005]. Loss: 7.771. Acc: 80.07%. Training time: 1217 seconds.
The learning converges at epoch 7.
Evaluate the learned model:
MSMT dataset loaded
subset | # ids | # images
train | 1041 | 32621
query | 3060 | 11659
gallery | 3060 | 82161
Compute similarity ... Traceback (most recent call last):
File "main.py", line 394, in
main(parser.parse_args())
File "main.py", line 266, in main
args.tau, args.sigma, args.K, args.alpha)
File "/home/luotao/QAConv/reid/evaluators.py", line 186, in evaluate
prob_fea, _ = extract_features(self.model, query_loader)
File "/home/luotao/QAConv/reid/evaluators.py", line 42, in extract_features
for i, (imgs, fnames, pids, _) in enumerate(data_loader):
File "/home/luotao/anaconda3/envs/QAConv/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 521, in next
data = self._next_data()
File "/home/luotao/anaconda3/envs/QAConv/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 1203, in _next_data
return self._process_data(data)
File "/home/luotao/anaconda3/envs/QAConv/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 1229, in _process_data
data.reraise()
File "/home/luotao/anaconda3/envs/QAConv/lib/python3.6/site-packages/torch/_utils.py", line 434, in reraise
raise exception
FileNotFoundError: Caught FileNotFoundError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/luotao/anaconda3/envs/QAConv/lib/python3.6/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
data = fetcher.fetch(index)
File "/home/luotao/anaconda3/envs/QAConv/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/luotao/anaconda3/envs/QAConv/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 49, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/luotao/QAConv/reid/utils/data/preprocessor.py", line 20, in getitem
return self._get_single_item(indices)
File "/home/luotao/QAConv/reid/utils/data/preprocessor.py", line 27, in _get_single_item
img = Image.open(fpath).convert('RGB')
File "/home/luotao/anaconda3/envs/QAConv/lib/python3.6/site-packages/PIL/Image.py", line 2975, in open
fp = builtins.open(filename, "rb")
FileNotFoundError: [Errno 2] No such file or directory: './reid/datasets/msmt/test/0000/0000_000_01_0303morning_0015_0.jpg'
So, what the structure of the msmt17 dataset ?This dataset I download is:
data-dir/
├── msmt/
├────-------/bounding_box_test/
├────-------/bounding_box_train/
├────-------/query/
├────-------/list_gallery.txt
├────-------/list_query.txt
├────-------/list_train.txt
├────-------/list_val.txt
I rename bounding_box_test to test ,rename bounding_box_train to train.
data-dir/
├── msmt/
├────-------/test/
├────-------/train/
├────-------/query/
├────-------/list_gallery.txt
├────-------/list_query.txt
├────-------/list_train.txt
├────-------/list_val.txt
from qaconv.
I don't understand your question. It is "path" (e.g. train path or query path), not "1".
In market.py , fpaths = sorted(glob(osp.join(self.images_dir, path, '.jpg'))) but in cuhk.py it is fpaths = sorted(glob(osp.join(self.images_dir, 1, '.png'))) , I am confused about the datasets structure, can you make a list just like it ? Thank you. CUHK/ ├── xxxx/ ├────-------/xxx/ ├────-------/xxx/ ├────-------/xx/ ├── xxxx/ ├────-------/xxx/ ├────-------/xxx/ ├────-------/xxx/
For any dataset, the directory name should be the same with the dataset name. So for cuhk03_np_detected, you need to have a directory named cuhk03_np_detected under the data-dir, containing the three sub folders. That is,
data-dir/ ├── market/ ├────-------/bounding_box_test/ ├────-------/bounding_box_train/ ├────-------/query/ ├── cuhk03_np_detected/ ├────-------/bounding_box_test/ ├────-------/bounding_box_train/ ├────-------/query/ ├── cuhk03_np_labeled/ ├────-------/bounding_box_test/ ├────-------/bounding_box_train/ ├────-------/query/....
Finished epoch 7 at lr=[5e-05, 0.0005, 0.0005]. Loss: 7.771. Acc: 80.07%. Training time: 1217 seconds.
The learning converges at epoch 7.
Evaluate the learned model:
MSMT dataset loaded
subset | # ids | # imagestrain | 1041 | 32621 query | 3060 | 11659 gallery | 3060 | 82161 Compute similarity ... Traceback (most recent call last): File "main.py", line 394, in main(parser.parse_args()) File "main.py", line 266, in main args.tau, args.sigma, args.K, args.alpha) File "/home/luotao/QAConv/reid/evaluators.py", line 186, in evaluate prob_fea, _ = extract_features(self.model, query_loader) File "/home/luotao/QAConv/reid/evaluators.py", line 42, in extract_features for i, (imgs, fnames, pids, _) in enumerate(data_loader): File "/home/luotao/anaconda3/envs/QAConv/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 521, in next data = self._next_data() File "/home/luotao/anaconda3/envs/QAConv/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 1203, in _next_data return self._process_data(data) File "/home/luotao/anaconda3/envs/QAConv/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 1229, in _process_data data.reraise() File "/home/luotao/anaconda3/envs/QAConv/lib/python3.6/site-packages/torch/_utils.py", line 434, in reraise raise exception FileNotFoundError: Caught FileNotFoundError in DataLoader worker process 0. Original Traceback (most recent call last): File "/home/luotao/anaconda3/envs/QAConv/lib/python3.6/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop data = fetcher.fetch(index) File "/home/luotao/anaconda3/envs/QAConv/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/luotao/anaconda3/envs/QAConv/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 49, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/luotao/QAConv/reid/utils/data/preprocessor.py", line 20, in getitem return self._get_single_item(indices) File "/home/luotao/QAConv/reid/utils/data/preprocessor.py", line 27, in _get_single_item img = Image.open(fpath).convert('RGB') File "/home/luotao/anaconda3/envs/QAConv/lib/python3.6/site-packages/PIL/Image.py", line 2975, in open fp = builtins.open(filename, "rb") FileNotFoundError: [Errno 2] No such file or directory: './reid/datasets/msmt/test/0000/0000_000_01_0303morning_0015_0.jpg'
So, what the structure of the msmt17 dataset ?This dataset I download is:
data-dir/ ├── msmt/ ├────-------/bounding_box_test/ ├────-------/bounding_box_train/ ├────-------/query/ ├────-------/list_gallery.txt ├────-------/list_query.txt ├────-------/list_train.txt ├────-------/list_val.txt I rename bounding_box_test to test ,rename bounding_box_train to train. data-dir/ ├── msmt/ ├────-------/test/ ├────-------/train/ ├────-------/query/ ├────-------/list_gallery.txt ├────-------/list_query.txt ├────-------/list_train.txt ├────-------/list_val.txt
OK, I see, the dataset we need is MSMT17_V1, NOT MSMT17_V2
data-dir/
├── MSMT17_V1/
├────-------/test/
├────-------/train/
├────-------/list_gallery.txt
├────-------/list_query.txt
├────-------/list_train.txt
├────-------/list_val.txt
from qaconv.
Related Issues (20)
- Question about backbone HOT 4
- Question about grad clipping in Graph Sampling Based Deep Metric Learning for Generalizable Person Re-Identification HOT 2
- some questions HOT 1
- self.model.eval() HOT 2
- 这个代码可以跑多个训练集混合训练的实验吗? HOT 1
- Out of memory,--test_fea_batch --test_gal_batch --test_prob_batch all had seted to 128 HOT 16
- Issues about evaluators.py HOT 2
- Unable to use ClassMemoryLoss to train the model HOT 4
- Error about main.py HOT 1
- The Graph Sampling work 相关问题 HOT 2
- Graph Sampler HOT 3
- 复现的小问题 HOT 1
- graph sampling的疑问 HOT 4
- 关于s=1 HOT 2
- where is the "main_gs.py" HOT 1
- Can't find qaconv_loss HOT 5
- OSNet-IBN-GS代码 HOT 3
- 多源域训练/目标域测试 HOT 1
- 训练结果为0 HOT 8
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from qaconv.