meliketoy / dreamchallenge Goto Github PK

View Code? Open in Web Editor NEW

1.0 1.0 0.0 4.47 MB

Repository for 2016-2017 Dream Challenge

Shell 1.65% Python 40.81% Lua 56.67% Makefile 0.87%

dreamchallenge's People

Contributors

Stargazers

Watchers

dreamchallenge's Issues

Model with Torch compatibility with PreprocessedData

/PreprocessedData 를 읽고 Torch로 구현된 모델과 호환되는 디렉토리 정리 python 스크립트 구현
preprocessing.py에 추가

compatibility with metadata/* in training/test phase

The format of image_crosswalk.tsv in training/testing phase is different with pilot dataset.

in pilot dataset

`/metadata/images_crosswalk.tsv'

[patient_id	exam_idx	image_idx	view	laterality	cancer]

/metadata/exams_metadata.t

[subjectId	examIndex	daysSincePreviousExam	cancerL	cancerR	invL	invR	age	implantEver	implantNow	bcHistory	yearsSincePreviousBc	previousBcLaterality	reduxHistory	reduxLaterality	hrt	antiestrogen	firstDegreeWithBc	firstDegreeWithBc50	bmi	race]

in training/testing phase

/metadata/exams_metadata.tsv

[patient_id	exam_idx	image_idx	view	laterality]

--> no cancer column

Challenge Pilot Data Wiki

The column cancer is no longer available in the images crosswalk files of the Challenge. It was placed initially for conveniency but has been now removed because it is redundant with the covariates cancerL and cancerR from the exams metadata table.

ROI extraction model check

fastRCNN, fasterRCNN 코드 확인
preprocessing.py와 합치기
추출된 ROI를 현재 /preprocessedData에 추가 디렉토리 구조로 떨구기
cpu-level multiprocessing으로

Mass/Cal extractor performance

Elapsed Time : Mass << Cal

Create Container for Ubuntu

드첼에 예제로 올라온 Docker는 모두 CentOS기반.

향후 유지 보수 및 환경 셋팅을 위해
ubuntu 환경의 컨테이터 만들기.

우선 training phase를 제외한 preprocessing쪽을 만들 예정 (/pre-processing/Dockerfile)

Training memory issue

계속해서 메모리 증가.

synapse Docker Push problem

dm-*-example 외의 docker container가 synapse docker repository에 업로드(push)가 안되는 상황.

주최측에서 확인 중인 것으로 파악됨.

해당 문제에 대해 지속적인 확인이 필요.

ROI extraction multiple GPU issue

setDeviceId(1) --> occur error: can't get gpu status

(p_id, exam_idx, img_idx, v, l, fname, cancer) = row

Preprocessing Performance Issue (slow in extracting image from dicom)

2016-11-18 00:34:05,073 DEBUG  [preprocessing.py:96] convert dicom to cv2
2016-11-18 00:34:06,968 DEBUG  [preprocessing.py:103] resize
2016-11-18 00:34:06,970 DEBUG  [preprocessing.py:103] flip
2016-11-18 00:34:06,971 DEBUG  [preprocessing.py:111] write image
2016-11-18 00:34:06,980 INFO   [preprocessing.py:60] Preprocessing... 65313 --> preprocessing/dreamCh/pilot/65313
2016-11-18 00:34:07,057 DEBUG  [preprocessing.py:96] convert dicom to cv2
2016-11-18 00:34:09,792 DEBUG  [preprocessing.py:103] resize
2016-11-18 00:34:09,794 DEBUG  [preprocessing.py:111] write image
2016-11-18 00:34:09,866 DEBUG  [preprocessing.py:96] convert dicom to cv2
2016-11-18 00:34:12,463 DEBUG  [preprocessing.py:103] resize
2016-11-18 00:34:12,466 DEBUG  [preprocessing.py:103] flip
2016-11-18 00:34:12,467 DEBUG  [preprocessing.py:111] write image
2016-11-18 00:34:12,477 INFO   [preprocessing.py:60] Preprocessing... 98 --> preprocessing/dreamCh/pilot/98
2016-11-18 00:34:12,520 DEBUG  [preprocessing.py:96] convert dicom to cv2
2016-11-18 00:34:14,408 DEBUG  [preprocessing.py:103] resize
2016-11-18 00:34:14,410 DEBUG  [preprocessing.py:111] write image
2016-11-18 00:34:14,452 DEBUG  [preprocessing.py:96] convert dicom to cv2
2016-11-18 00:34:16,441 DEBUG  [preprocessing.py:103] resize
2016-11-18 00:34:16,443 DEBUG  [preprocessing.py:103] flip
2016-11-18 00:34:16,444 DEBUG  [preprocessing.py:111] write image
2016-11-18 00:34:16,455 INFO   [preprocessing.py:60] Preprocessing... 43184 --> preprocessing/dreamCh/pilot/43184
2016-11-18 00:34:16,527 DEBUG  [preprocessing.py:96] convert dicom to cv2
2016-11-18 00:34:19,461 DEBUG  [preprocessing.py:103] resize
2016-11-18 00:34:19,464 DEBUG  [preprocessing.py:111] write image
2016-11-18 00:34:19,545 DEBUG  [preprocessing.py:96] convert dicom to cv2
2016-11-18 00:34:22,257 DEBUG  [preprocessing.py:103] resize

Solution Candidate

MultiProcessing
num of proc = num of core - 1
No writing tmp image

compatibility with InBreast, KUMC, etc Dataset

Duplicated Images in Pilot

Check duplicated images (same p_id, exam_idx, v, l) , not img_idx

img_idx가 크면 나중에 찍은건지 확인
img_idx에 따라 이미지가 얼마나 차이나는지 확인