aminzabardast / mastersthesis Goto Github PK

• execute around 10 examples from FlyingThings3D
• execute a couple of examples from KITTIE 2015
• create a PDF report
• think about the future path and structure

• A callback to render a batch of test images when each epoch ends.

After DisparityNet (FlowNet2.0, #34) is completed. Networks "Hyper-Parameters" should be optimized.

Make multi-stage learning Coherent and Object Oriented

It is more logical to save a network in a file (like model_init) then load and train for certain epochs.
This allows for having only one train code.

Some generic 3D meshes will slide and rotate in front of the camera and rendering the result into a dataset. This will be a test for SlidingOrgans3D.

Some things to look for:

• Disparities should be between 0 and 50.
• They should increment gradually.
• No sudden breaks in disparities.
• Lights should be static but rendered in the scene.

Bring Flownet Specialized functions to this repository.

• Correlation Function.
• Warping function.
• Correction on DisparityNet SD

Adding tensorboard support to keras model.
Check if it is possible to access the tensorboard itself from the server without downloading log fiels.

• Divide the data into Test and Validation.
• Re pack the data.
  Upload to Ehsan's Server.
• Upload to Ayber's Server.
• Create Data Generator.

This can be tested in Aybar's server.

• DisparityNet S
• DisparityNet C
• DisparityNet CS
• DisparityNet SD
• DisparityNet CS2 (Parallel)

Write a report for each here in the post

An initial and simple auto encoder structure for FlyingThings3D should be devised.

Create a simple piece of code to convert PFM files to PNG.

• The output should be color-coded, like a rainbow.
• the least and most should be detected automatically.

Warp left image to right image using TensorFlow and save it.
The outcome of this task is applicable in #34.

Implement a Keras Callback to adapt Learning Rate.

A good starting point will be TMI Dataset

• Crop rectified images into a 512x512 shape.
• Create a code to convert the disparity into a mesh.

Current Feed Forward Code Outputs the result as a PFM file. There should be

• a PFM output,
• a color-coded PNG output.
• a Print out of metrics such as Bad1.0, Bad2.0, and Bad4.0

FlyingThings3D has been downloaded to Thesis-Server.

Create an automated file listing:

• It should be object-oriented.
• It should mention "Specifically for Flying Things"
• It should be out of the main network folder.

Implement an implementation of Keras Data Generator

Example Here
Implementation should be detailed.

Implement DisparityNet modeled after FlowNet2.0

Normal measures of accuracy (metrics) in Keras core are not designed to depict the accuracy of matching disparity maps.

A Disparity Error measure should be found and implemented to work with Keras.

• Search for datasets.
• Email the researchers.

Gather a list of necessary organs.
Compare the organs shape and texture in Human, Cow and Lamb.
Create a report.

• Check this plugin and try to render into disparities.
• Create simple objects and check to see if the disparity is rendered correctly.

Try Initial Learning with FlyingThings3D and Transfer to SlidingTissues3D

• Import Human Anatomy Model that is purchased - hopefully in #52 - and import it correctly into Blender.
• Create render tests - with disparity - from the model to evaluate realism.
• Test to see if it is possible to use the Textures, Normal Maps etc on other models.

The initial network structure developed in issue #3, is (probably) not sufficient/good enough architecture to solve the issue at hand.
A new design should be derived and implemented.

What is the preferred loss?

Images from OpenCAS dataset TMI are rectified so they are the prime candidate for real test data.
Calculate the minimum and maximum disparities in examples.

Check and try to create and save mesh with and without disparity maps.

Create a CSV Logger to keep track of loss and metrics values.
Must be tested in case of None Termination or Raised Errors that is known to occur.

Right now, there are 3 directories for 3 types of outputs, csvs, logs, and models.
They should be combined.

Note: Back up the current progress especially models first.

There is a need to create a more flexible code, one that can resume training for some certain epochs.
For now, a separate code to resume the initial training stage suffices.

Look for meet texture. Perferably with Normals etc.

Create a feedforward code to input stereo images and outputs the disparities.

Adapt the code to use multiple GPUs.
Although the initial structure of the neural network implemented in issue #3 is proven to be too shallow, this adaption is merely a test so the old structure would be used.

Note: Current GPU numbers are 2 so the network should be designed for two GPUs.

Investigate why some Server trained network does not work in the laptop?

• Check the validity of Ground Truth (GT). GT is given in an unusual format.

• Check buying procedure. Maybe Ehsan's help is required.
• A realistic Human Anatomy model should be purchased. There are some examples in #45. Also ask for Mina's advise. Maybe it is better to wait for Erkan Hoca's response.

Testing the Initial Trained Network (#13) and Prediction Code (#32).

• Test more images from FlyingThings3D

Attach the result into the task.