Comments (6)
The implication is correct. You can certainly try to use APSW as heavily concurrently as you want and there is no way that you will cause problems with APSW or SQLite (there is even a test for this). APSW does not limit the inherent concurrency SQLite provides.
But APSW does protect from problems. Consider calling execute in one thread and close in another on the same cursor. There is no control over what order the GIL is released and reacquired so there is a race between sqlite3_step and sqlite3_finalize calls on the same sqlite3_stmt. The underlying APSW cursor object tracks whether it is in a call, and refuses to attempt this, giving a ThreadingViolationError. The same tracking exists on the Connection for the same reason.
So the quoted text is true - you cannot use the same cursor object in multiple threads concurrently to execute statements - because the call tracking will refuse it. The text can be updated to be simpler - tracked in #427.
from apsw.
Right, but isn't the documentation actively misleading if SQLite happens to be compiled (or put into) multi-threaded or single-threaded mode? Currently, the documentation makes it sound as if it is always safe to concurrently use multiple cursors and connections.
from apsw.
It is always safe to concurrently use cursors and connections! Doing so will never result in deadlocks, memory corruption, operation on closed handles etc in APSW. But there is a list of details:
- If you try to do something that would have resulted in deadlocks, memory corruption, operation on closed handles etc then it will be detected and refused with ThreadingViolationError
- You are unlikely to get useful results - executing two different queries concurrently on the same cursor means one wins and so the other thread will see the results to the other query
- APSW refuses to initialize unless SQLite was compiled threadsafe
- APSW relies on the GIL to protect APSW's data structures, and SQLite's mutexes to protect SQLite's data structures.
- APSW releases the GIL around every sqlite3 call allowing for maximum concurrency in SQLite. (There are a small number of calls where it doesn't happen because the calls are trivial and SQLite doesn't do mutexes in them.)
- If you use sqlite3_config to put SQLite into single-thread mode (aka turn off SQLite mutexes) and then do concurrency you no longer have protection of SQLite's data structures
- The text was originally written during the days of pysqlite3 which made virtually no attempts to get concurrency right, hence being so emphatic that APSW does
from apsw.
"APSW refuses to initialize unless SQLite was compiled threadsafe" is the critical information I was looking for, thanks!
It would be great to add this to the multithreading doc page, together with the note that "If you use sqlite3_config to put SQLite into single-thread mode (aka turn off SQLite mutexes) and then do concurrency you no longer have protection of SQLite's data structures"
from apsw.
It does not refuse sqlite3_config for single thread mode. Someone deliberately turning off SQLite mutexes and then using concurrency presumably knows what they are doing and keep the pieces of whatever they break :)
from apsw.
I did a doc update. Didn't see your edit about using sqlite3_config but I don't think it is worth mentioning because it takes some determination to do that and then expect concurrency to somehow work! You can also pass flags to sqlite3_open to disable mutexes with the same consequences, which APSW also doesn't prevent. Anybody doing this presumably knows exactly what they are doing.
from apsw.
Related Issues (20)
- URIFilename doesn't detect underlying database has been closed
- Allow multiple auth/profile etc
- Make statement cache recycle bin be part of per Connection StatementCache and not separate, also check hash algorithm
- VFS Null filename opens HOT 10
- Unknown apsw attribute 'SQLiteValues' HOT 3
- Cursor.executemany should be annotated as accepting Iterable[Bindings] instead of Sequence[Bindings] HOT 1
- Implement Unicode TR-29 and TR-14 HOT 1
- error: static declaration of ‘PyLong_AsInt’ in Python 3.13+ HOT 3
- Switch to cosign for signing
- Incorrect values in `colUsed` HOT 2
- Update to Unicode 16.0 when released September 2024 HOT 1
- connection.set_row_trace - excepts; Connection.setrowtrace works HOT 4
- `connection.setrowtrace` stops working when adapters are registered HOT 5
- Datetimes into sqlite via APSW HOT 1
- Revise logging levels
- query execution row and time limits HOT 1
- Support subtypes in functions
- Don't cache pragmas
- con.pragma can't accept blobs HOT 1
- Update Connection.pragma with schema parameter
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from apsw.