Comments (8)
FiloSottile
commented on June 12, 2024
Here's the sequence of events: a few SQLITE_BUSY, then a few SQLITE_DONE, then a SQLITE_LOCKED that deadlocks on pthread_cond_wait. All this happens on the same goroutine, but with a new Pool.Get for each query (as you can see from the PRAGMA query bound to Get).
2018/04/08 23:23:28 sqlite3_step(PRAGMA foreign_keys = ON;) = SQLITE_DONE(not an error)
2018/04/08 23:23:28 sqlite3_step(INSERT INTO Users (id, handle, name, bio, first_seen) VALUES (?, ?, ?, ?, ?);) = SQLITE_DONE(not an error)
2018/04/08 23:23:28 sqlite3_step(PRAGMA foreign_keys = ON;) = SQLITE_DONE(not an error)
2018/04/08 23:23:28 sqlite3_step(INSERT INTO Tweets (id, created, user, message) VALUES (?, ?, ?, ?)) = SQLITE_CONSTRAINT
2018/04/08 23:23:51 sqlite3_step(PRAGMA foreign_keys = ON;) = SQLITE_DONE(not an error)
2018/04/08 23:23:51 sqlite3_step(INSERT INTO Messages (json, account) VALUES (?, json_array(?))) = SQLITE_BUSY
2018/04/08 23:23:53 sqlite3_step(PRAGMA foreign_keys = ON;) = SQLITE_DONE(not an error)
2018/04/08 23:23:53 sqlite3_step(INSERT INTO Messages (json, account) VALUES (?, json_array(?))) = SQLITE_BUSY
2018/04/08 23:23:55 sqlite3_step(PRAGMA foreign_keys = ON;) = SQLITE_DONE(not an error)
2018/04/08 23:23:55 sqlite3_step(INSERT INTO Messages (json, account) VALUES (?, json_array(?))) = SQLITE_BUSY
2018/04/08 23:24:59 sqlite3_step(PRAGMA foreign_keys = ON;) = SQLITE_DONE(not an error)
2018/04/08 23:24:59 sqlite3_step(INSERT INTO Messages (json, account) VALUES (?, json_array(?))) = SQLITE_DONE(not an error)
2018/04/08 23:24:59 sqlite3_step(PRAGMA foreign_keys = ON;) = SQLITE_DONE(not an error)
2018/04/08 23:24:59 sqlite3_step(INSERT INTO Tweets (id, created, user, message) VALUES (?, ?, ?, ?)) = SQLITE_DONE(not an error)
2018/04/08 23:24:59 sqlite3_step(PRAGMA foreign_keys = ON;) = SQLITE_DONE(not an error)
2018/04/08 23:24:59 sqlite3_step(INSERT INTO Users (id, handle, name, bio, first_seen) VALUES (?, ?, ?, ?, ?);) = SQLITE_DONE(not an error)
2018/04/08 23:24:59 sqlite3_step(PRAGMA foreign_keys = ON;) = SQLITE_DONE(not an error)
2018/04/08 23:24:59 sqlite3_step(UPDATE Messages SET account = json_insert(account, '$[' || json_array_length(account) || ']', ?) WHERE id = ?;) = SQLITE_LOCKED
sqlite3_unlock_notify
pthread_mutex_lock
pthread_cond_wait
I can't reproduce it with a pool size of 1, everything keeps working fine after the SQLITE_BUSY.
from sqlite.
crawshaw
commented on June 12, 2024
Ouch, thanks. I'll take a look at this tomorrow.
One possibility is that I'm being too imprecise on error matching. Right now the code uses wait_for_unlock_notify if it sees SQLITE_LOCKED. There may be some condition where competing with another process returns that error instead of SQLITE_BUSY, in which case we go into a wait that we don't escape from.
That would be easy to fix, look for the extended error code SQLITE_LOCKED_SHAREDCACHE. But I'd like to reproduce it before checking that in.
from sqlite.
FiloSottile
commented on June 12, 2024
It might be relevant that this is a WAL database.
from sqlite.
crawshaw
commented on June 12, 2024
I had some trouble replicating this when I tried last week (but forgot to update the issue). What OS are you using?
from sqlite.
FiloSottile
commented on June 12, 2024
This is on macOS.
from sqlite.
ksshannon
commented on June 12, 2024
I ran into a similar issue on Ubuntu 16.04, with both wal and delete journal modes.
from sqlite.
fasterthanlime
commented on June 12, 2024
Saw this too with a pool of size 2, also having a hard time coming up with a reproducible test case, but I'll attempt to post one as soon as I figure out the conditions.
edit: here's my stacktrace:
goroutine 25851 [syscall, 1 minutes]:
github.com/itchio/butler/vendor/crawshaw.io/sqlite._Cfunc_wait_for_unlock_notify(0x743a808, 0x7439780, 0x0)
_cgo_gotypes.go:1458 +0x54
github.com/itchio/butler/vendor/crawshaw.io/sqlite.(*Stmt).Step.func2(0x743a808, 0x7439780, 0x9)
C:/msys64/home/amos/Dev/go/src/github.com/itchio/butler/vendor/crawshaw.io/sqlite/sqlite.go:510 +0x96
github.com/itchio/butler/vendor/crawshaw.io/sqlite.(*Stmt).Step(0xc042a0d040, 0x180, 0xc042cef0f7, 0x7)
C:/msys64/home/amos/Dev/go/src/github.com/itchio/butler/vendor/crawshaw.io/sqlite/sqlite.go:510 +0x140
github.com/itchio/butler/vendor/github.com/itchio/hades/sqliteutil2.exec(0xc042a0d040, 0x0, 0xc042cd6000, 0x180, 0x180, 0x0, 0x10000c042e6b300)
C:/msys64/home/amos/Dev/go/src/github.com/itchio/butler/vendor/github.com/itchio/hades/sqliteutil2/exec.go:131 +0x3d8
github.com/itchio/butler/vendor/github.com/itchio/hades/sqliteutil2.Exec(0xc0420c8550, 0xc042973500, 0x327, 0x0, 0xc042cd6000, 0x180, 0x180, 0xb7a260, 0xc042bb8b20)
C:/msys64/home/amos/Dev/go/src/github.com/itchio/butler/vendor/github.com/itchio/hades/sqliteutil2/exec.go:77 +0xc1
github.com/itchio/butler/vendor/github.com/itchio/hades.(*Context).ExecRaw(0xc0420c85f0, 0xc0420c8550, 0xc042973500, 0x327, 0x0, 0xc042cd6000, 0x180, 0x180, 0x1, 0xc0421a2e80)
C:/msys64/home/amos/Dev/go/src/github.com/itchio/butler/vendor/github.com/itchio/hades/exec.go:41 +0xc2
github.com/itchio/butler/vendor/github.com/itchio/hades.(*Context).Exec(0xc0420c85f0, 0xc0420c8550, 0xc0421a2e80, 0x0, 0xc042e6b2e0, 0xd812a0)
C:/msys64/home/amos/Dev/go/src/github.com/itchio/butler/vendor/github.com/itchio/hades/exec.go:19 +0xcb
github.com/itchio/butler/vendor/github.com/itchio/hades.(*Context).deletePagedByPK(0xc0420c85f0, 0xc0420c8550, 0xc04245a3b0, 0xd, 0xc04245a3d8, 0x2, 0xc042cd4000, 0x180, 0x200, 0xd81260, ...)
C:/msys64/home/amos/Dev/go/src/github.com/itchio/butler/vendor/github.com/itchio/hades/paged.go:63 +0x1f9
github.com/itchio/butler/vendor/github.com/itchio/hades.(*Context).SaveNoTransaction.func3(0xbd0340, 0xc04215c5a0, 0x16, 0xc0428fcaf0, 0xb64ca0, 0xc04215c618, 0x197, 0xc0428fcb40, 0xbc3501, 0xd87d40, ...)
C:/msys64/home/amos/Dev/go/src/github.com/itchio/butler/vendor/github.com/itchio/hades/save.go:235 +0x1037
github.com/itchio/butler/vendor/github.com/itchio/hades.(*Context).SaveNoTransaction.func2(0xb798e0, 0xc042e420c0, 0x82, 0x0, 0xbd0340, 0xc04215c5a0, 0x16, 0xc0428fcaf0, 0x413400, 0xc042e4f548, ...)
C:/msys64/home/amos/Dev/go/src/github.com/itchio/butler/vendor/github.com/itchio/hades/save.go:140 +0x98a
github.com/itchio/butler/vendor/github.com/itchio/hades.(*Context).SaveNoTransaction.func3(0xb798e0, 0xc042e420c0, 0x82, 0x0, 0xbd0340, 0xc04215c5a0, 0x16, 0xc0428fcaf0, 0x0, 0x0, ...)
(snip)
This is with a pool of size 2, default options. Every other query (on the other conn presumably?) returns SQLITE_LOCKED directly.
from sqlite.
fasterthanlime
commented on June 12, 2024
Merging #25 closed this issue, but @FiloSottile may want to check if the original issue cannot still be reproduced.
from sqlite.
Related Issues (20)
- Further simplification of sqlitex.Pool.Get is not correct HOT 4
- Code generation? HOT 1
- doubled quoted key names as argument to json_object cause Prepare to fail HOT 1
- Multiple sqlitex.Pools giving off by 1 values HOT 2
- sqlitex.ExecScript fails to run scripts that include statements that cannot be executed within a transaction HOT 2
- API to retrieve column blob into slice of arbitrary length HOT 2
- BindBytes stores empty blobs as NULL HOT 2
- Pool.Close leaves behind WAL files HOT 8
- The state of the project. HOT 8
- No way to write blob in custom function
- Compile Time error HOT 5
- Compilation warning about memcpy with gcc 11.2.0 HOT 8
- sqlite.Stmt.Finalize should panic if called on a Stmt returned from Conn.Prepare HOT 5
- Running PRAGMA statments in sqlitex.OpenInit script
- sqlite3 math extension HOT 4
- Unable to build on Windows
- Build failure when using gcc 12. HOT 1
- ExecScript and Save panic on corrupt database
- sqlitex.ExecTransient and sqlitex.Exec not executing statements
- Support sqlite3_db_cacheflush HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sqlite.