Comments (4)
Operators are meant to work in parallel, (that's the purpose). But EF core raises an exception when a query is already being run on the same DbContext
in a concurrent thread (this is for this very purpose I had to develop the method InvokeInDedicatedThreadAsync
).
Just as an extra information, in the current implementation, the fact the dbContext
is not disposed it is absolutely normal. It is not up to an ETL operator to decide that it must be trashed as it has been created in a much higher level context.
Of course, if you issue a different DbContext
at each use, everything that is mentioned above is irrelevant, but in most of situations a DbContext
is directly given to the ProcessRunner
. So what you suggest will be another way to work: if a IDbContextFactory
it will be used instead.
About proper release of entities, I will make a quick win: execute a dbContext.ChangeTracker.Clear()
at every batch unless the option KeepChangeTracker
is activated. This will permit entities to be released from memory if they are not used anywhere else.
from etl.net.
Hello,
It is a very good suggestion indeed, even if the implementation you bring here has a flaw (in case of several operators working against EF core, InvokeInDedicatedThreadAsync
ensures no call is done in parallel since it is not supported).
I will work on a amendment on this regard. Thanks a lot for this! 😃
from etl.net.
Great to hear!
I eagerly removed the dbcontext instance from the thread lock as I assumed the node instance was threadsafe, and the dbcontext instance was local to the batch anyway.
My thinking that parallel queries are supported, just not within a single dbcontext. Similar to how the .net request pipeline doesn't enforce singleton access to a dbcontext.
Thanks for picking this up. I'm enjoying learning about your framework.
from etl.net.
a new version 2.1.6-beta is available. I didn't test it too well to speak franckly. You can give it a try. I should work with IDbContextFactory out of the box. If you want to specify the type of the DbContext, you must call .WithContextType<your-dbContext-type>()
on the options builder.
Let me know if it suits you.
from etl.net.
Related Issues (20)
- Question: calling stored procedure which returns values HOT 5
- Getting two database streams from the same database connection in parallel HOT 5
- Getting in-memory list from the database stream HOT 13
- EF Core Spatial Integration HOT 4
- StreamProcessRunner is holding file in memory when I need it not to do that HOT 1
- Problem when generating SQL for .SqlServerSave when using ODBC and OleDB connections HOT 1
- SFTP connections not working without trailing slashes in folder paths HOT 1
- Running a stored procedure only once after all records in a CSV file have been saved to the database HOT 2
- Buffer operator
- How to compare two streams?
- .SqlServerSave() support on MS SQL Server versions prior to 2005 HOT 2
- MySQL PostgreSQL SQLite support HOT 1
- Specify timeout for .CrossApplySqlServerQuery and .ToSqlCommand HOT 4
- How can i move data between two SQL server db's HOT 2
- Raising an error and short-circuiting the execution of remaining operators HOT 1
- How to select from raw sql using EF Core? HOT 1
- .ToTextFileValue fails when output is bigger than 2GB HOT 2
- SqlServerSave fails because column names are not enclosed in square brackets HOT 1
- Bug--When I save an entity with a primary key of guid to the database using the EfCoreSave method, an error occurs, prompting me to "Cannot insert value NULL into column 'Id', table 'SimpleTutorial. dbo. Author'; columns do not allow null values. UPDATE fails." When I change the insertion mode to SaveMode.EntityFrameworkCore Successfully save HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from etl.net.