remicres / otbtf_tutorials_resources Goto Github PK

Hello.

I've tried running the tutorial on a simple cnn and I got stucked with the Sampling part.

I'm using the docker image for the otb and here are the list of the modules installed. But unfortunately, there's no LabelImageSampleSelection.

Am I missing something?

Any help would be appreciated.

Thank you.

Hi,

I was wanting to see if I could get some assistance with completing the step 5.4 generating the model, where tricks.py is used to generate models used further in the tutorial "Tokyo Dataset - Patch-Based Classification". I see there are 2 other issues that have been raised on this subject, but they seem to be fixed by a naming convention CreateSaveModel

I am using the docker image mdl4eo/otbtf2.4:cpu, trying to run create_model1.py, I have run the PYTHONPATH command
export PYTHONPATH=$PYTHONPATH:/work/otb/otb/Modules/Remote/otbtf/python/
But every time I still get the error
ModuleNotFoundError: No module named 'tricks'

I am not that familiar with using docker images or PYTHONPATH, so am not sure where I am to run the PYTHONPATH command?

Am I supposed to run the PYTHONPATH command from my local system? Or from the docker container?

I can echo $PYTHONPATH, with the correct response from my local system. I am currently running the
python3 create_model1.py "model1"
from the directory with the correct python file.

How does the tricks.py file in the docker image get accessed by my local system Python?

Happy to give further information if needed?

sorry to interrupt，but what should I do to run it on pycharm. The model tricks I don't know how to install it. I REALLY need your help. PLEASE contact me at Emile [email protected]. THANK YOU VERY MUCH!!!!

Hello!

I've ran through the chapter 5 and trained a CNN from Tokyo dataset. Unfortunately, the result output is a green image like here:

I've checked through all the steps and still the result is the same.
Have you ever ran into it? Where should I pay attention to?

There is a normalized image on the output's background.

I'm using the docker image to check the first example "create_model1.py" and found the function CreateSavedModel() called in line 59 is defined as create_savedmodel() in tricks.py.
Cheers

Hello,
I try to modify the create_model1.py source code to traine the model model using patches extracted from Radar image with 2 polarities. My normalisation is made using two images (vv and vh polarities). I run PatchesExtraction to extract patches of size 16x16. I then so expect to have as input a set of data of 16x16x2.
I have 7 classes in my model identified by numbers from 0 to 6. And my shapfile content 2 fields (id and class) both integer.
This is the folowing code I have.

from tricks import *
import sys
import os

nclasses=7

def myModel(x):
  
  # input patches: 16x16x2
  conv1 = tf.layers.conv2d(inputs=x, filters=16, kernel_size=[5,5], padding="valid", 
                           activation=tf.nn.relu) # out size: 12x12x16
  pool1 = tf.layers.max_pooling2d(inputs=conv1, pool_size=[2, 2], strides=2) # out: 6x6x16
  conv2 = tf.layers.conv2d(inputs=pool1, filters=16, kernel_size=[3,3], padding="valid", 
                           activation=tf.nn.relu) # out size: 4x4x16
  pool2 = tf.layers.max_pooling2d(inputs=conv2, pool_size=[2, 2], strides=2) # out: 2x2x16
  conv3 = tf.layers.conv2d(inputs=pool2, filters=32, kernel_size=[2,2], padding="valid",
                           activation=tf.nn.relu) # out size: 1x1x32
  
  # Features
  features = tf.reshape(conv3, shape=[-1, 32], name="features")
  
  # Neurons for classes
  estimated = tf.layers.dense(inputs=features, units=nclasses, activation=None)
  estimated_label = tf.argmax(estimated, 1, name="prediction")

  return estimated, estimated_label
 
""" Main """
if len(sys.argv) != 2:
  print("Usage : <output directory for SavedModel>")
  sys.exit(1)

# Create the TensorFlow graph
with tf.Graph().as_default():
  
  # Placeholders
  x = tf.placeholder(tf.float32, [None, None, None, 2], name="x")
  y = tf.placeholder(tf.int32  , [None, None, None, 1], name="y")
  lr = tf.placeholder_with_default(tf.constant(0.0002, dtype=tf.float32, shape=[]),
                                   shape=[], name="lr")
  
  # Output
  y_estimated, y_label = myModel(x)
  
  # Loss function
  cost = tf.losses.sparse_softmax_cross_entropy(labels=tf.reshape(y, [-1, 1]), 
                                                logits=tf.reshape(y_estimated, [-1, nclasses]))
  
  # Optimizer
  optimizer = tf.train.AdamOptimizer(learning_rate=lr, name="optimizer").minimize(cost)
  
  # Initializer, saver, session
  init = tf.global_variables_initializer()
  saver = tf.train.Saver( max_to_keep=20 )
  sess = tf.Session()
  sess.run(init)

  # Create a SavedModel
  CreateSavedModel(sess, ["x:0", "y:0"], ["features:0", "prediction:0"], sys.argv[1])

When I run the script, the model is created and saved. But when I run the training command, it sames like it don't detect all the classes.
The kapa index is 0 or 1 and the confusion matrix is empty or is a 1x1 matrix.

Can you please help me and direct me about what to do.
Thanks in advance for your avaibility.

I relate to this some screenshot to better understand the behavior.

I'm using the docker image to check the first example "create_model1.py" and found the following Error while running the script.

ModuleNotFoundError: No module named 'tricks'

I install tricks module by running

docker run -i -t -u root -v $(pwd):/data mdl4eo/otbtf1.7:gpu /bin/sh
pip install tricks
python /data/scripts/model1.py /data/results/SavedModel_cnn

It still fail with the same error.
Can you please give me any solution about this?
Thanks in advance!

I have run this model (create_model1.py). But, I still don't know how to know the accuracy. The result is ok.

otbuser@fd3098e1f203:/home/mega/Documents/CNN$ otbcli_TensorflowModelServe \

-source1.il Sentinel-2_B4328_10m.tif
-source1.rfieldx 16
-source1.rfieldy 16
-source1.placeholder "x"
-model.dir "data/results/SavedModel_cnn"
-output.names "prediction"
-out "classif_model1.tif?&box=4000:4000:1000:1000"
2023-09-01 02:03:52 (INFO) TensorflowModelServe: Default RAM limit for OTB is 256 MB
2023-09-01 02:03:52 (INFO) TensorflowModelServe: GDAL maximum cache size is 186 MB
2023-09-01 02:03:52 (INFO) TensorflowModelServe: OTB will use at most 2 threads
2023-09-01 02:03:52.713603: I tensorflow/cc/saved_model/reader.cc:43] Reading SavedModel from: data/results/SavedModel_cnn
2023-09-01 02:03:52.717680: I tensorflow/cc/saved_model/reader.cc:78] Reading meta graph with tags { serve }
2023-09-01 02:03:52.717747: I tensorflow/cc/saved_model/reader.cc:119] Reading SavedModel debug info (if present) from: data/results/SavedModel_cnn
2023-09-01 02:03:52.717995: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: SSE3 SSE4.1 SSE4.2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-09-01 02:03:52.770161: I tensorflow/cc/saved_model/loader.cc:228] Restoring SavedModel bundle.
2023-09-01 02:03:52.815464: I tensorflow/core/profiler/lib/profiler_session.cc:110] Profiler session initializing.
2023-09-01 02:03:52.815551: I tensorflow/core/profiler/lib/profiler_session.cc:125] Profiler session started.
2023-09-01 02:03:52.817447: I tensorflow/core/profiler/lib/profiler_session.cc:67] Profiler session collecting data.
2023-09-01 02:03:52.817514: W tensorflow/core/profiler/convert/xplane_to_step_stats.cc:74] GPU trace was not collected.
2023-09-01 02:03:52.817712: I tensorflow/core/profiler/lib/profiler_session.cc:143] Profiler session tear down.
2023-09-01 02:03:52.818297: I tensorflow/cc/saved_model/loader.cc:301] SavedModel load for tags { serve }; Status: success: OK. Took 104713 microseconds.
2023-09-01 02:03:52 (INFO): Loading metadata from official product
2023-09-01 02:03:52 (INFO) TensorflowModelServe: Source info :
2023-09-01 02:03:52 (INFO) TensorflowModelServe: Receptive field : [16, 16]
2023-09-01 02:03:52 (INFO) TensorflowModelServe: Placeholder name : x
2023-09-01 02:03:52 (INFO) TensorflowModelServe: Output spacing ratio: 1
2023-09-01 02:03:52 (INFO) TensorflowModelServe: Output field of expression: [1, 1]
2023-09-01 02:03:52 (INFO) TensorflowModelServe: Force tiling with squared tiles of [16, 16]
2023-09-01 02:03:52 (INFO): Writing user defined region [4000, 4999]x[4000, 5000]
2023-09-01 02:03:52 (INFO): Estimated memory for full processing: 74.0692MB (avail.: 256 MB), optimal image partitioning: 1 blocks
2023-09-01 02:03:52 (INFO): File classif_model1.tif will be written in 1 blocks of 1000x1000 pixels
Writing classif_model1.tif?&box=4000:4000:1000:1000...: 100% [**************************************************] (1m 02s)

Thank you.