Comments (8)
Alright, I will update the version on my side and give it a try. Thanks for your support, maybe the last update fixed it 👍🏻
from tfgo.
Hi @jriegner , thank you for this report!
You're right, there's a leak. And I guess I found where the leak is 😄
LoadModel
returns a *tfgo.Model
that contains a tensorflow a *tf.SavedModel
. This tf.SavedModel
has a *tf.Session
field.
When the model is deleted, the session is not automatically closed (it's the old tf.Session
of Python - if you are familiar with TensorFlow < 2), and thus I guess the leak is in the missing invocation of session.Close().
I guess we have 2 options:
- Expose the session and let the user invoke the
Close()
mehtod (I don't like it) - Create a finalizer that automatically invokes the session.Close() when the
tfgo.Model
is garbage collected.
I kinda like more the second approach.
Right now I'm travelling so I don't know when I'll be able to work on it (I'm setting up the development environment right now while I'm on a train lol) - if you want to implement the second option, I'll be more then happy to review and merge it
from tfgo.
Update: I guess I made it b536202
Give it a try and let me know if it works
from tfgo.
Hey, thanks for your fast reply!
I ran my local test app with the updated code and memleak-bpfcc
still complained about leaking memory. I verified that the finalizer ran and also explicitly closed the session on every reload. Same result.
I will check with our production setup too and will come back then.
Edit:
I monitored our service with the latest version of tfgo
for a while and can see that the memory increases on each reload.
from tfgo.
If it's not the leak related to the missing session close, I have no other clue 🤔
For sure there was a leak caused by the missing session.close(), which now should be fixed (I forgot to add the same line in the tg.ImportModel
and I fixed it yesterday).
But I have no idea of what could cause this issue on tfgo side - maybe it's a leak in the TensorFlow C library
from tfgo.
Update: I guess I made it b536202
Give it a try and let me know if it works
I try it, but is not work!
from tfgo.
Hi @LoveVsLike - I guess the problem is inside the TensorFlow bindings. As you can see, in tfgo we just open the session and using a finalizer we close the session when the model is collected. So, the problem in the TensorFlow code, I guess.
Perhaps you can try to open an issue on https://github.com/tensorflow/tensorflow and link this thread there. Maybe, someone from the TensorFlow team can help us.
Anyway, since tfgo is still using TensorFlow 2.9.1 I can try to update my fork to 2.14. Maybe the leak is already fixed and we don't know it (but I'm not confident, since it has been years and this leak is still present version after version...).
I'll update the fork and I'll let you know.
from tfgo.
Hi @LoveVsLike - I guess the problem is inside the TensorFlow bindings. As you can see, in tfgo we just open the session and using a finalizer we close the session when the model is collected. So, the problem in the TensorFlow code, I guess.
Perhaps you can try to open an issue on https://github.com/tensorflow/tensorflow and link this thread there. Maybe, someone from the TensorFlow team can help us.
Anyway, since tfgo is still using TensorFlow 2.9.1 I can try to update my fork to 2.14. Maybe the leak is already fixed and we don't know it (but I'm not confident, since it has been years and this leak is still present version after version...).
I'll update the fork and I'll let you know.
Hi, I update TF to last version and tfgo, But is not work, Can you put a issue to tf?
from tfgo.
Related Issues (20)
- loading model HOT 2
- cc1.exe: sorry, unimplemented: 64-bit mode not compiled in HOT 1
- Error: W tensorflow/core/framework/op_kernel.cc:1745] OP_REQUIRES failed at lookup_table_op.cc:929 : FAILED_PRECONDITION: Table not initialized. HOT 1
- SparseTensor support HOT 2
- Failed to go get tfgo HOT 3
- tensor example support HOT 6
- libtensorflow.so: .dynsym local symbol at index 3 (>= sh_info of 3) HOT 2
- Matching between output tensor name and prediction HOT 7
- Thread safety HOT 2
- Panic vs Error HOT 2
- Reading jpg image from disk and passing as input to a pretrained model loaded from disk HOT 4
- Error building "Getting Started" example: rebuild TensorFlow with the appropriate compiler flags HOT 5
- Can I use tfgo for tensorflow 2.11.x? HOT 2
- Convert opencv's `mat` to tfgo's `tensor` HOT 7
- Dark Souls level frustration with compilation project with this library HOT 1
- Discrepancy between python and go outputs HOT 2
- tfgo Object detection
- Choose output layer of model . HOT 1
- Panic when loading a keras LSTM Model HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tfgo.