Git Product home page Git Product logo

dl_bio_example's Introduction

dl_bio_example

Download the dataset on: https://www.kaggle.com/prasunroy/natural-images Unzip and remove the data folder (for some reason the data are saved two times).

Files:

docker/docker_build.sh

Script to build a Docker-Image based on a Dockerfile in the same folder. See Section "Docker" for further information.

docker/Dockerfile

A file containing all the dependencies and information on the folder structure, etc. of the project. This file must be named "Dockerfile"!

config.py

Contains parameters for launching the run_training.py

docker_run.sh

Script to launch a Docker-Container based on the image that is generated by the docker_build.sh script. See Section "Docker" for further information.

run_training.py

The starting point of the project. Execute this file to launch the training process.

train_interfaces.py

A supplementary file for the training process.

Docker

Upon execution of the docker_build.sh script, a Docker-Image will be created. This image will be based on the Dockerfile. Henceforth, you will always have to rebuild the image when your dependencies change or are updated (like for example the DLBio-repository) and you wish to use these updates. Execute the docker_run.sh script to launch a Docker-Container in your terminal. This container will be based on the Docker-image that was created by the docker_build.sh. You can run Files the same way as you would in any other terminal.

More on docker with vs-code: https://moodle.uni-luebeck.de/mod/forum/discuss.php?d=33479

Debugging:

The Debugger has two options. Select "Docker: Python - General" to start debugging the project from its main file. Select "Docker: Python - Current" to start debugging the currently in the editor selected file. More configurations can be added or present configurations can be changed in the ./vscode/launch.json and ./vscode/tasks.json.

Visdom

Visdom eases to display data remotely. For general infos see their git: visdom.

Here it is used to display the feature representation of a trained model. The represantation is saved in exe_save_embeddings.py and plotted in plot_embeddings.py.

Visdom can also be used for visualizing images and plots. Start the visdom server from the command line with visdom and add something to it for example like this:

import visdom
from datasets.ds_mnist import get_dataloader

dl = get_dataloader(False, batch_size=16)	
image = dl.dataset.data[0]

vis = visdom.Visdom()
vis.image(image)

Installation

You can install the stable version via pip but I would recommend installing it from source to get the current features (See https://github.com/fossasia/visdom#setup).

For plotting the embeddings this has to be done anyway at the moment (march 21). Because the vis.embeddings uses this package for the t-sne algorithm (https://github.com/lvdmaaten/bhtsne) you will probably have to install this as well. For this, clone the repo into visdom/py/visdom/extra_deps and follow the installation steps here: https://github.com/lvdmaaten/bhtsne#installation

Tensorboard

Tensorboard can be used to display training experiments. Infos at https://www.tensorflow.org/tensorboard and here for using it in pytorch. Tensor behaves similarly to a jupyter notebook.

The information which will be used by tensorboard can be stored during or after an experiment, see 'exe_log_tb.py' or tensorboard_test.py. For run_training this can be activated with --log_tb. By default the data will be stored in runs/_debug and can be visualized by 'tensorboard --logdir=runs' if tensorboard is installed.

dl_bio_example's People

Contributors

dadidange avatar pgruening avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.