mdietrichstein / tensorflow-open_nsfw Goto Github PK

View Code? Open in Web Editor NEW

424.0 424.0 137.0 21.13 MB

Tensorflow Implementation of Yahoo's Open NSFW Model

License: Other

Python 100.00%

caffe deep-learning deep-neural-networks tensorflow tensorflow-models

tensorflow-open_nsfw's People

Contributors

Stargazers

Watchers

Forkers

dim25 roszcz alsan sherzz wqw547243068 technikempire kaijung q64545 yosemite1998 penalizedem wuzhuoran begemotv2718 dannycxh hdmjdp 0xp0 cncoder cekisakurek shubhampachori12110095 xieqihui tiyanproduct claytonbrown wenbotse jxlin dragonogre yifenzhong1920 yplam adgelbfish punit121 xianfengju freelogic aquawill salikin ecrows yunhai0920 a3w4e5r xunyixiangchao shivazi simfeng erkapilmehta wikipedia2008 berli widemeadows shining1995 ligo doutong ibraabukaff wuyijian jone666 duoergun0729 vuongducdai hschueh niwangyuan hien david30907d zvrr k1ic xiexiaoshinick fullp0wer eformat hanlen520 deepai-org peerwaya alexwangmac m0r13 sugior mogoweb yingmuying qyue aureliuspatiens victor8733 dreadlord1984 allensmile srivshobhit aryanshar lxknvlk petercang chenjinya zhujiawen12345 alexandremcosta jamshi hrnaudz mathstao shuimukang hevensun jarlped farmani 1163710217 josantadata sandeep5193 abbasidaniyal lucagiovagnoli zhaoyunxi shitianshiwa nuarknoir brainbicycle hehekyo sanjingshou11 yyj8209 zhp510730568 yshadows

tensorflow-open_nsfw's Issues

write a python client

Hi
@mdietrichstein @delta9

and thank you for this repository.
could you please do me a favor & tell how can i write a client for this model using python
i use tensorflow web and inception tutorial every thing is ok when i serve other model but for this one i don't know what should i do

can you tell me how should i change this inception code :

from __future__ import print_function
from grpc.beta import implementations
import tensorflow as tf
import numpy as np

from tensorflow_serving.apis import predict_pb2
from tensorflow_serving.apis import prediction_service_pb2
import base64
import json
from tensorflow.python.saved_model.signature_constants import PREDICT_INPUTS
tf.app.flags.DEFINE_string('server', 'localhost:9000',
                           'PredictionService host:port')
tf.app.flags.DEFINE_string('image', '', 'path to image in JPEG format')
FLAGS = tf.app.flags.FLAGS
def main(_):
  host, port = FLAGS.server.split(':')
  channel = implementations.insecure_channel(host, int(port))
  stub = prediction_service_pb2.beta_create_PredictionService_stub(channel)

  with open(FLAGS.image, "rb") as f:
    data = f.read()
    data = base64.urlsafe_b64encode(data)
    request = predict_pb2.PredictRequest()
    request.model_spec.name = 'test'
    request.model_spec.signature_name = 'serving_default'
    data = tf.contrib.util.make_tensor_proto(data,shape=[1])
    req=request.inputs['input'].CopyFrom(data)
    result = stub.Predict(request, 10.0)  # 10 secs timeout
    print(result)
if __name__ == '__main__':
  tf.app.run()

and i get this error :


Traceback (most recent call last):
  File "/home/serving/tensorflow_serving/example/porndetect_client.py", line 64, in <module>
    tf.app.run()
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 48, in run
    _sys.exit(main(_sys.argv[:1] + flags_passthrough))
  File "/home/serving/tensorflow_serving/example/porndetect_client.py", line 57, in main
    result = stub.Predict(request, 10.0)  # 10 secs timeout
  File "/usr/local/lib/python2.7/dist-packages/grpc/beta/_client_adaptations.py", line 309, in __call__
    self._request_serializer, self._response_deserializer)
  File "/usr/local/lib/python2.7/dist-packages/grpc/beta/_client_adaptations.py", line 195, in _blocking_unary_unary
    raise _abortion_error(rpc_error_call)
grpc.framework.interfaces.face.face.AbortionError: AbortionError(code=StatusCode.INVALID_ARGUMENT, details="input tensor alias not found in signature: input. Inputs expected to be in the set {inputs}.")

Special Thanks *

Check NSFW in video files

Hello.
I've used your libraries to detect NSFW content in videos.
It works correctly, but the problem is that I can't open the video frame and check it directly, as it tells me they are not compatible format. I need to open a frame, save it to disk in jpeg format, then open it and check.
Is there any way to load the frame and check it directly?

I use OPENCV libraries to open the video file.

I then put a piece of the code

while(True): 
  
	# reading from frame 
	ret,frame = cam.read() 

	if ret:
		if currentframe % salto_frames == 0:
			frames_analizados += 1
			cv2.imwrite('temp.jpg', frame)	
			image = fn_load_image('temp.jpg')
			
			predictions = \
				sess.run(model.predictions,
						feed_dict={model.input: image})

			nsf = predictions[0][0]
			nsfw = predictions[0][1]

I think it's possible to make

ret, frame = cam.read()
image = frame

Any idea??
Thank you for this great library.

when I convert with Caffe to TensorFlow method， I got wrong score 0.95353949

And I have found the difference is between model.py. Can you show me your convert methods in detail?
thanks

Error when testing with tensorflow 1.14.0

#python3.7 classify_nsfw.py -m data/open_nsfw-weights.npy ~/ww-empire-sub-ehader.jpg

/usr/local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/usr/local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/usr/local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/usr/local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/usr/local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/usr/local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
/usr/local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/usr/local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/usr/local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/usr/local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/usr/local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/usr/local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
WARNING:tensorflow:From classify_nsfw.py:41: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

2019-09-16 17:00:48.749803: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-09-16 17:00:48.764193: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3600000000 Hz
2019-09-16 17:00:48.764684: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55aaeb126190 executing computations on platform Host. Devices:
2019-09-16 17:00:48.764706: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): <undefined>, <undefined>
Traceback (most recent call last):
  File "classify_nsfw.py", line 69, in <module>
    main(sys.argv)
  File "classify_nsfw.py", line 44, in main
    model.build(weights_path=args.model_weights, input_type=input_type)
  File "/data/projects/tensorflow-open_nsfw/tensorflow-open_nsfw/model.py", line 30, in build
    self.weights = np.load(weights_path, encoding="latin1").item()
  File "/usr/local/lib64/python3.7/site-packages/numpy/lib/npyio.py", line 453, in load
    pickle_kwargs=pickle_kwargs)
  File "/usr/local/lib64/python3.7/site-packages/numpy/lib/format.py", line 722, in read_array
    raise ValueError("Object arrays cannot be loaded when "
ValueError: Object arrays cannot be loaded when allow_pickle=False

Tensorflow image loader image would cause different result

I randomly tested several images, the difference is between .10 to .20.

In fact, I tested the code one by one, and found the resize method might be the problem which cause this.

I also used opencv instead of PIL to do resize, the final result is similar to tensorflow resize. Moreover, I compared resize result between PIL and opencv, they are quite different, for example, the max difference value in one image is about 25, and the RMSD is about 3.

Last, I read some articles which point out that adding noise to a image might cause totally different result even though human being cannot find the difference between these two images.

PS: thanks to this repository which helps me to save time, otherwise I might need to spend lots of time to convert caffe to tensorflow. :)

dtype for the input placeholder

I exported the model serving with export_savedmodel.py and then wrote a Java program for deploying the model on end devices. The main part of the code is as follows:

SavedModelBundle savedModelBundle = SavedModelBundle.load( ... );
Tensor result = savedModelBundle.session().runner()
                .feed("input", normalizedImageTensor) // error here
                .fetch("predictions")
                .run().get(0);

normalizedImageTensor is the input image loaded and transformed as in __tf_jpeg_process and then casted to a tensor.

The error pertaining to the commented line is

Exception in thread "main" java.lang.IllegalArgumentException: Expects arg[0] to be string but float is provided
	at org.tensorflow.Session.run(Native Method)
	at org.tensorflow.Session.access$100(Session.java:48)
	at org.tensorflow.Session$Runner.runHelper(Session.java:298)
	at org.tensorflow.Session$Runner.run(Session.java:248)
	at com.paralleldots.lava.api.main(api.java:59)

This is saying that normalizedImageTensor is of type string.

What then is the dtype for the input placeholder in your actual graph? What kind of input is expected?

exported tflite always has nsfw range between 0.6 to 0.7

I have been able to export it using tf 1.15. Using it in Firebase ML kit sample, score is always between 0.6-0.7, no matter what is fed into the network.

I have also used one shared in #19 but same results.

Can you share your tflite if possible for double-check?

improve the model

any idea how can i improve the model and tune it with my dataset

Converting to tflite

Been struggling to convert the model to tflite.
I've used export_savedmodel.py to freeze it and then try to following to convert:
toco \ --graph_def_file=frozen_open_nsfw.pb --output_file=optimized_graph.tflite \ --output_format=TFLITE \ --input_arrays=input \ --output_arrays=predictions \

This results the following errors:
2018-08-16 01:54:39.677658: I tensorflow/contrib/lite/toco/import_tensorflow.cc:1096] Converting unsupported operation: TensorArrayV3 2018-08-16 01:54:39.677802: I tensorflow/contrib/lite/toco/import_tensorflow.cc:1096] Converting unsupported operation: TensorArrayScatterV3 2018-08-16 01:54:39.677834: I tensorflow/contrib/lite/toco/import_tensorflow.cc:1096] Converting unsupported operation: TensorArrayV3 2018-08-16 01:54:39.677847: I tensorflow/contrib/lite/toco/import_tensorflow.cc:1096] Converting unsupported operation: Enter

Any ideas?

Thanks a lot

ERROR: Variable conv_1/kernel already exists, disallowed - When using with FLASK and GUNICORN

This model works perfectly fine with Flask and its internal development server. But as soon as I use it with a production server like Gunicorn, it seems to throw a Tensorflow-error complaining about the use of a certain variable.

Here's the full stack. Thanks.

10:01:46 PM web.1 |  2018-07-06 22:01:46.254103: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
10:01:51 PM web.1 |  [2018-07-06 22:01:51,235] ERROR in app: Exception on / [POST]
10:01:51 PM web.1 |  Traceback (most recent call last):
10:01:51 PM web.1 |    File "/Users/emilwidlund/.local/share/virtualenvs/glotter-picasso-aMAwiIpj/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app
10:01:51 PM web.1 |      response = self.full_dispatch_request()
10:01:51 PM web.1 |    File "/Users/emilwidlund/.local/share/virtualenvs/glotter-picasso-aMAwiIpj/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request
10:01:51 PM web.1 |      rv = self.handle_user_exception(e)
10:01:51 PM web.1 |    File "/Users/emilwidlund/.local/share/virtualenvs/glotter-picasso-aMAwiIpj/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception
10:01:51 PM web.1 |      reraise(exc_type, exc_value, tb)
10:01:51 PM web.1 |    File "/Users/emilwidlund/.local/share/virtualenvs/glotter-picasso-aMAwiIpj/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
10:01:51 PM web.1 |      raise value
10:01:51 PM web.1 |    File "/Users/emilwidlund/.local/share/virtualenvs/glotter-picasso-aMAwiIpj/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request
10:01:51 PM web.1 |      rv = self.dispatch_request()
10:01:51 PM web.1 |    File "/Users/emilwidlund/.local/share/virtualenvs/glotter-picasso-aMAwiIpj/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request
10:01:51 PM web.1 |      return self.view_functions[rule.endpoint](**req.view_args)
10:01:51 PM web.1 |    File "/Users/emilwidlund/Documents/glotter/glotter-picasso/main.py", line 27, in process_image
10:01:51 PM web.1 |      prediction = classify(temp.getvalue(), 'open_nsfw/data/open_nsfw-weights.npy')
10:01:51 PM web.1 |    File "/Users/emilwidlund/Documents/glotter/glotter-picasso/open_nsfw/classify_nsfw.py", line 41, in classify
10:01:51 PM web.1 |      model.build(weights_path=model_weights, input_type=input_type)
10:01:51 PM web.1 |    File "/Users/emilwidlund/Documents/glotter/glotter-picasso/open_nsfw/model.py", line 50, in build
10:01:51 PM web.1 |      kernel_size=7, stride=2, padding='valid')
10:01:51 PM web.1 |    File "/Users/emilwidlund/Documents/glotter/glotter-picasso/open_nsfw/model.py", line 158, in __conv2d
10:01:51 PM web.1 |      self.__get_weights(name, "biases"), dtype=tf.float32))
10:01:51 PM web.1 |    File "/Users/emilwidlund/.local/share/virtualenvs/glotter-picasso-aMAwiIpj/lib/python3.6/site-packages/tensorflow/python/layers/convolutional.py", line 621, in conv2d
10:01:51 PM web.1 |      return layer.apply(inputs)
10:01:51 PM web.1 |    File "/Users/emilwidlund/.local/share/virtualenvs/glotter-picasso-aMAwiIpj/lib/python3.6/site-packages/tensorflow/python/layers/base.py", line 828, in apply
10:01:51 PM web.1 |      return self.__call__(inputs, *args, **kwargs)
10:01:51 PM web.1 |    File "/Users/emilwidlund/.local/share/virtualenvs/glotter-picasso-aMAwiIpj/lib/python3.6/site-packages/tensorflow/python/layers/base.py", line 699, in __call__
10:01:51 PM web.1 |      self.build(input_shapes)
10:01:51 PM web.1 |    File "/Users/emilwidlund/.local/share/virtualenvs/glotter-picasso-aMAwiIpj/lib/python3.6/site-packages/tensorflow/python/layers/convolutional.py", line 144, in build
10:01:51 PM web.1 |      dtype=self.dtype)
10:01:51 PM web.1 |    File "/Users/emilwidlund/.local/share/virtualenvs/glotter-picasso-aMAwiIpj/lib/python3.6/site-packages/tensorflow/python/layers/base.py", line 546, in add_variable
10:01:51 PM web.1 |      partitioner=partitioner)
10:01:51 PM web.1 |    File "/Users/emilwidlund/.local/share/virtualenvs/glotter-picasso-aMAwiIpj/lib/python3.6/site-packages/tensorflow/python/training/checkpointable.py", line 436, in _add_variable_with_custom_getter
10:01:51 PM web.1 |      **kwargs_for_getter)
10:01:51 PM web.1 |    File "/Users/emilwidlund/.local/share/virtualenvs/glotter-picasso-aMAwiIpj/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 1317, in get_variable
10:01:51 PM web.1 |      constraint=constraint)
10:01:51 PM web.1 |    File "/Users/emilwidlund/.local/share/virtualenvs/glotter-picasso-aMAwiIpj/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 1079, in get_variable
10:01:51 PM web.1 |      constraint=constraint)
10:01:51 PM web.1 |    File "/Users/emilwidlund/.local/share/virtualenvs/glotter-picasso-aMAwiIpj/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 425, in get_variable
10:01:51 PM web.1 |      constraint=constraint)
10:01:51 PM web.1 |    File "/Users/emilwidlund/.local/share/virtualenvs/glotter-picasso-aMAwiIpj/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 394, in _true_getter
10:01:51 PM web.1 |      use_resource=use_resource, constraint=constraint)
10:01:51 PM web.1 |    File "/Users/emilwidlund/.local/share/virtualenvs/glotter-picasso-aMAwiIpj/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 733, in _get_single_variable
10:01:51 PM web.1 |      name, "".join(traceback.format_list(tb))))
10:01:51 PM web.1 |  ValueError: Variable conv_1/kernel already exists, disallowed. Did you mean to set reuse=True or reuse=tf.AUTO_REUSE in VarScope? Originally defined at:
10:01:51 PM web.1 |    File "/Users/emilwidlund/Documents/glotter/glotter-picasso/open_nsfw/model.py", line 158, in __conv2d
10:01:51 PM web.1 |      self.__get_weights(name, "biases"), dtype=tf.float32))
10:01:51 PM web.1 |    File "/Users/emilwidlund/Documents/glotter/glotter-picasso/open_nsfw/model.py", line 50, in build
10:01:51 PM web.1 |      kernel_size=7, stride=2, padding='valid')
10:01:51 PM web.1 |    File "/Users/emilwidlund/Documents/glotter/glotter-picasso/open_nsfw/classify_nsfw.py", line 41, in classify
10:01:51 PM web.1 |      model.build(weights_path=model_weights, input_type=input_type)

dataset production

hi,I am sorry to bother you, but i want to kown how do you make a data set? nsfw set maybe easy to classify,but the types of sfw images are too complex and huge, you can't include all the categories of sfw image.

Error during conversion of tflite

when i run export_tflite.py i got this error, How to fix this i use tensorflow 1.12

RuntimeError: TOCO failed see console for info.
b'/bin/sh: toco_from_protos: command not found\n'
None
'

feed a batch of opencv images

I'll try to read a number of images with opencv and feed the batch of them to the model. but with both yahoo_loader and tensorflow_loader I got errors. is there a way to use opencv images instead of the tensorflow/yahoo loader?

Number of images used in training

Hi @mdietrichstein, as I totally understand that due to the nature of the data, you cannot release the info about the dataset used for training but I just want to know that how many NSFW images and SFW images were used in training, just the number, no additional details. This will really help to complete my project. Kindly respond on the same. Thank you!

add png support

It is said that only jpg images are supported. We definitely need to add png and other formats support.

ValueError: Didn't find op for builtin opcode 'CONV_2D' version '5' Registration failed.

Using export_tflite.py file I was able to convert the model into .tflite and interpret the model results using Python.

To Optimize the size of tflite file I added converter.optimizations = [tf.lite.Optimize.OPTIMIZE_FOR_SIZE] line to the export_tflite.py file, Now my tflite file is ~6MB, but when I try to load this file using tf.lite.Interpreter(model_path='nsfw_compresed.tflite') for Interpretation I get the following error

ValueError: Didn't find op for builtin opcode 'CONV_2D' version '5'
Registration failed.

Please help me hear to resolve the issue
Tensorflow version == '1.14.0'

How can I train this model useing my own images?

I just want to add some photos to the training set

export_model.py generates a protobuf unreadable in tensorflow

I run export_model.py and I've got a protobuf saved, but when I've tried to import the model with Tensorflow I got an "Invalid GraphDef" error.

Help exporting to saved model

I am new to tensorflow and I need help exporting this model as a savedmodel to use it in tensorflowJS.

I used this command:

python export_savedmodel.py

and it returns this:

usage: export_savedmodel.py [-h] -i {tensor,base64_jpeg} [-v EXPORT_VERSION]
-m MODEL_WEIGHTS
target
export_savedmodel.py: error: too few arguments

but I have no clue what to fill the arguments with.

can we find and blur area with nudity?

a segment fault happend when I translate this code to C++

Hi @mdietrichstein ,

I try to translate your code to C++ ,and a segment fault happend when call tf_session->Run(input_tname,output_tname,output_node,&output_tensor). I used caffe2tensoflow to convert caffe model to tensorflow pbfile would you mind having a look what's wrong in my code as i'm newer in tensorlow, thank you so much:

`#include
#include
#include
#include
#include
#include
#include
#include
#include<unistd.h>
#include<stdlib.h>

#include "tensorflow/cc/ops/const_op.h"
#include "tensorflow/cc/ops/image_ops.h"
#include "tensorflow/cc/ops/standard_ops.h"
#include "tensorflow/core/framework/graph.pb.h"
#include "tensorflow/core/framework/tensor.h"
#include "tensorflow/core/graph/default_device.h"
#include "tensorflow/core/graph/graph_def_builder.h"
#include "tensorflow/core/lib/core/errors.h"
#include "tensorflow/core/lib/core/stringpiece.h"
#include "tensorflow/core/lib/core/threadpool.h"
#include "tensorflow/core/lib/io/path.h"
#include "tensorflow/core/lib/strings/stringprintf.h"
#include "tensorflow/core/platform/init_main.h"
#include "tensorflow/core/platform/logging.h"
#include "tensorflow/core/platform/types.h"
#include "tensorflow/core/public/session.h"
#include "tensorflow/core/util/command_line_flags.h"

#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/core/core.hpp"

using tensorflow::Flag;
using tensorflow::Tensor;
using tensorflow::Status;
using tensorflow::string;
using tensorflow::int32;

// Given an image file name, read in the data,resize it to the yahoo requested size, and then scale the values as desired.
tensorflow::Status YahooImageLoader(string file_name, cv::Mat &img)
{
auto root = tensorflow::Scope::NewRootScope();
using namespace ::tensorflow::ops; // NOLINT(build/namespaces)

cv::Mat src= cv::imread(file_name );
if(!src.data)
{
    std::cerr << "Could not open or find the image at " << file_name << std::endl;
    return tensorflow::errors::NotFound("Failed to load compute graph at '", file_name, "'");
}
std::cout<<"file_name:"<<file_name<<std::endl;
int widthOrigin = src.cols;
int heightOrigin = src.rows;
//here hard code as Yahoo open_nsfw trained model using this size
cv::resize(src, src, cv::Size(224,224),0,0);
src.convertTo(src, CV_32FC3);
cv::Mat cv_data_mean(224, 224, CV_32FC3, cv::Scalar(104.00698793, 116.66876762, 122.67891434));
src -= cv_data_mean;
// std::vector<cv::Mat> mat_vec;
//split(src, mat_vec);
img = src;
std::cout<<"finished resize image:"<<file_name<<std::endl;
return Status::OK();

}
// Reads a model graph definition from disk, and creates a session object you can use to run it.
tensorflow::Status TF_LoadGraph(const std::string& graph_file_name, tensorflow::Session* session)
{
tensorflow::GraphDef graph_def;
tensorflow::Status load_graph_status =
tensorflow::ReadBinaryProto(tensorflow::Env::Default(), graph_file_name, &graph_def);
if (!load_graph_status.ok())
{
return tensorflow::errors::NotFound("Failed to load compute graph at '", graph_file_name, "'");
}
tensorflow::Status status = NewSession(tensorflow::SessionOptions(), &session);
if (!status.ok())
{
std::cerr << status.ToString() << std::endl;
//return 1;
} else
{
std::cout << "Session created successfully" << std::endl;
}
//debug
for (int i = 0; i < graph_def.node_size(); ++i)
{
auto node = graph_def.node(i);
//std::cout<< " ***name:" << node.name() << ";op:" << node.op() << std::endl;
}

return session->Create(graph_def);

}
bool TF_init(const std::string& graph_file_name, tensorflow::Session* tf_session)
{
int argc = 0;
tensorflow::port::InitMain(NULL, &argc, NULL);

tensorflow::Status status = TF_LoadGraph(graph_file_name, tf_session);
if (!status.ok()) {
    std::cerr << "TF_LoadGraph ERROR: " << status.error_message() << std::endl;
    return false;
}
return true;

}

int main(int argc, char* argv[])
{

//string graph_file_name = "./nsfw_model/resnet_50_1by2_nsfw.pb";
tensorflow::Session *tf_session;
 
std::string graph_file_name(argv[1]) ;
bool  status = TF_init(graph_file_name, tf_session);
if (!status)
{
    std::cerr << "TF_init ERROR" << std::endl;
    return -1;
}

cv::Mat cvIm ;
std::string image_path(argv[2]);
if ( !YahooImageLoader(image_path, cvIm).ok() )
{
    LOG(ERROR) << "Load image";
    return -1;
}
tensorflow::TensorShape image_shape = tensorflow::TensorShape{1, cvIm.rows, cvIm.cols, cvIm.channels()};
std::cout << "Input TensorShape ["
          << image_shape.dim_size(0) << ", "
          << image_shape.dim_size(1) << ", "
          << image_shape.dim_size(2) << ", "
          << image_shape.dim_size(3) << "]" << std::endl;

tensorflow::Tensor input = tensorflow::Tensor(tensorflow::DT_FLOAT, image_shape) ;
// copy data from OpenCv to TensorFlow Tensor
//std::copy_n((char*) image_float_data, image_shape.num_elements() * sizeof(float),
//               const_cast<char*>(input.tensor_data().data()));
std::vector<cv::Mat> mat_vec;
split(cvIm, mat_vec);
char* dst_data =const_cast<char*> (input.tensor_data().data());
float* src_data;
for (int k = 0; k < mat_vec.size(); ++k)
{
     for (int y = 0; y<mat_vec[k].rows; y++)
     {
         src_data = mat_vec[k].ptr<float>(y);
         memcpy(dst_data, src_data, sizeof(float)*mat_vec[k].cols);
         dst_data += mat_vec[k].cols*sizeof(float);
     }
}

string input_layer = "data:0";
string output_layer = "prob:0";

std::vector<std::pair<string, Tensor> > input_tname;
std::pair<string,Tensor> input0(input_layer, input);
input_tname.push_back(input0);

std::vector<string> output_tname;
std::vector<Tensor> output_tensor;


output_tname.push_back(output_layer);

std::vector<string> output_node;

std::cout<<"start classify image:"<<image_path<<std::endl;

tensorflow::Status run_status = tf_session->Run(input_tname,output_tname,output_node,&output_tensor);  

if (!run_status.ok())
{
    LOG(ERROR) << "Running model failed";
    return -1;
}
  // here deal the output which included the scores of sfw and nsfw
  tf_session->Close();
  return 0;

}
`

Need help to run this model on android

I am trying to run this model on the Android device.i think I have made some mistake here can you please review these value and let me know I these values are correct or not

Error : E/TensorFlowInferenceInterface: Failed to run TensorFlow inference with inputs:[input], outputs:[predictions]

Please reply Thank you.

        private const val INPUT_WIDTH = 224
        private const val INPUT_HEIGHT = 224
        private const val IMAGE_MEAN = 117
        private const val IMAGE_STD = 1f
        private const val INPUT_NAME = "input"
        private const val OUTPUT_NAME = "predictions"

Tensorflow Serving Issue

Hi,

Thanks so much for this repository, I think you're doing a great service by saving everybody a lot of time!

Did you successfully deploy the model on Tensorflow Serving? I've exported it using your tool but I keep getting an error message when trying to test it:

{ code: 3, details: 'Invalid character found in base64.\n\t [[Node: map/while/DecodeBase64 = DecodeBase64[_output_shapes=[[]], _device="/job:localhost/replica:0/task:0/cpu:0"](map/while/TensorArrayReadV3)]]', metadata: Metadata { _internal_repr: {} } }

My client code is here: https://github.com/delta9/tensorflow-serving-node-client

I've setup a test server reachable on: 192.99.0.117:9000

I assume it's an issue with the client (it works fine on Google Cloud ML-Engine) but I've sort of hit a road block here and would really appreciate it, if you have any advice.

Many thanks,

Melvin

Prediction via Cloud ML not working

I'm trying to run predictions on the model via Google Cloud ML. I'm however getting this error when I execute my request:

{"error":"Prediction failed: Error during model execution: AbortionError(code=StatusCode.INVALID_ARGUMENT, details=\"NodeDef mentions attr 'identical_element_shapes' not in Op<name=TensorArrayV3; signature=size:int32 -> handle:resource, flow:float; attr=dtype:type; attr=element_shape:shape,default=<unknown>; attr=dynamic_size:bool,default=false; attr=clear_after_read:bool,default=true; attr=tensor_array_name:string,default=\"\"; is_stateful=true>; NodeDef: map/TensorArray = TensorArrayV3[_output_shapes=[[2], []], clear_after_read=true, dtype=DT_STRING, dynamic_size=false, element_shape=<unknown>, identical_element_shapes=true, tensor_array_name=\"\", _device=\"/job:localhost/replica:0/task:0/cpu:0\"](map/strided_slice)\n\t [[Node: map/TensorArray = TensorArrayV3[_output_shapes=[[2], []], clear_after_read=true, dtype=DT_STRING, dynamic_size=false, element_shape=<unknown>, identical_element_shapes=true, tensor_array_name=\"\", _device=\"/job:localhost/replica:0/task:0/cpu:0\"](map/strided_slice)]]\")"}

I've exported the saved model via the tool and it deployed successfully in the Google Cloud Console.

Here is the relevant part of my code:

        # ...
       temp_base64 = base64.b64encode(temp.getvalue())

        model_name = 'projects/{}/models/{}'.format('glotter-80ac0', 'NSFW')

        ml = discovery.build('ml', 'v1')

        requestDict = {
            "instances": [{
                "b64": temp_base64.decode('utf-8')
            }]
        }

        mlRequest = ml.projects().predict(name=model_name, body=requestDict)

        try:
            response = mlRequest.execute()
            return jsonify(response)
        except errors.HttpError:
            print('There was an error creating the model. Check the details:')
            print(errors.HttpError._get_reason())