Comments (8)
dinosaure
commented on July 3, 2024
1
This access is only on the PACK file directly and does not try to extract git object to a loose file then (so no write). So, I think, in a read-only context, it should continue to work.
from datakit.
talex5
commented on July 3, 2024
I see a similar thing:
bridge_1 | datakit-bridge-local-git: internal error, uncaught exception:
bridge_1 | (Sys_error
bridge_1 | "/data/repos/datakit/.git/tmp/9237df211d380111786b9da499078f3dac425c536cf0write: Read-only file system")
The compose file explicitly requests a read-only mount, so this must have worked before. I guess some change in Irmin means it now wants to write to the store for some reason.
@samoht : any idea what might cause this?
from datakit.
samoht
commented on July 3, 2024
I don't remember any related change which could have caused this. Can you add a bit more debug messages to understand what file it is trying to write? Maybe it is trying to write .git/HEAD?
Also I am not sure why this repo is mounted read-only.
from datakit.
Thegaram
commented on July 3, 2024
@samoht the directory the bridge is trying to create is visible in the error above (/data/repos/datakit/.git/tmp). I tried again with debug logs, here's the output:
Starting selfci_datakit_1 ... done
Recreating selfci_bridge_1 ...
Recreating selfci_bridge_1 ... done
Attaching to selfci_datakit_1, selfci_ci_1, selfci_bridge_1
datakit_1 | Starting datakit %%VERSION%% ...
ci_1 | 2018-03-09 11:55.24 INF [datakit-ci] Connecting to DataKit server on tcp:datakit:5640
datakit_1 | datakit: [INFO] accepted a new connection on tcp://0.0.0.0:5640
datakit_1 | datakit: [INFO] Using protocol TwoThousandU msize 8215
bridge_1 | 2018-03-09 11:55.25 INF [bridge-local-git] Connecting to DataKit server on tcp:datakit:5640
bridge_1 | 2018-03-09 11:55.25 INF [bridge-local-git] Monitoring local repository "/data/repos/datakit"
datakit_1 | datakit: [INFO] accepted a new connection on tcp://0.0.0.0:5640
datakit_1 | datakit: [INFO] Using protocol TwoThousandU msize 8215
ci_1 | 2018-03-09 11:55.24 INF [datakit-ci] Starting monitor loop
bridge_1 | 2018-03-09 11:55.25 DBG [git.unix] Reading /data/repos/datakit/.git/HEAD
bridge_1 | 2018-03-09 11:55.25 DBG [datakit.9p] create_dir ["branch"; "github-metadata"]
bridge_1 | 2018-03-09 11:55.25 DBG [irmin.watch] [1: 0k/0g|0]: start listening to /data/repos/datakit/.git/refs
bridge_1 | 2018-03-09 11:55.25 INF [irw-inotify] Inotify mode
bridge_1 | 2018-03-09 11:55.25 DBG [irw-inotify] start_watch /data/repos/datakit/.git/refs
bridge_1 | 2018-03-09 11:55.25 DBG [irw-polling] files=[(heads/master,
bridge_1 | e11593446767529c8512fac05c2ffe43);
bridge_1 | (remotes/origin/HEAD,
bridge_1 | 73a00957034783b7b5c8294c54cd3e12)] new_files=
bridge_1 | [(heads/master,
bridge_1 | e11593446767529c8512fac05c2ffe43);
bridge_1 | (remotes/origin/HEAD,
bridge_1 | 73a00957034783b7b5c8294c54cd3e12)]
bridge_1 | 2018-03-09 11:55.25 DBG [irmin-watcher] Start watchdog for /data/repos/datakit/.git/refs
bridge_1 | 2018-03-09 11:55.25 DBG [irmin.watch] watch [1: 0k/0g|1]: id=0
bridge_1 | 2018-03-09 11:55.25 DBG [git.unix] Reading /data/repos/datakit/.git/packed-refs
bridge_1 | 2018-03-09 11:55.25 DBG [bridge-local-git.sync] Notification for "master"
bridge_1 | 2018-03-09 11:55.25 DBG [git.unix] Reading /data/repos/datakit/.git/refs/heads/master
bridge_1 | 2018-03-09 11:55.25 DBG [git.fs] read 580fe5576d87b8377afeb45f9b078c6adf58b4fe
bridge_1 | 2018-03-09 11:55.25 DBG [git.fs] read: cache miss!
bridge_1 | 2018-03-09 11:55.25 DBG [git.fs-loose] read 580fe5576d87b8377afeb45f9b078c6adf58b4fe
bridge_1 | 2018-03-09 11:55.25 DBG [git.unix] Reading /data/repos/datakit/.git/objects/58/0fe5576d87b8377afeb45f9b078c6adf58b4fe
bridge_1 | 2018-03-09 11:55.25 DBG [git.fs-packed] list /data/repos/datakit/.git
bridge_1 | 2018-03-09 11:55.25 DBG [git.fs-packed] read_in_pack(read) 8ed1414ac9c7af604b59d6ca6fb73f819c1906ae:580fe5576d87b8377afeb45f9b078c6adf58b4fe
bridge_1 | 2018-03-09 11:55.25 DBG [git.fs-packed] read_pack_index 8ed1414ac9c7af604b59d6ca6fb73f819c1906ae
bridge_1 | 2018-03-09 11:55.25 DBG [git.unix] Reading /data/repos/datakit/.git/objects/pack/pack-8ed1414ac9c7af604b59d6ca6fb73f819c1906ae.idx
bridge_1 | 2018-03-09 11:55.25 DBG [git.pack-index] create: entering fanout table (ofs=8)
bridge_1 | 2018-03-09 11:55.25 DBG [git.pack-index] create: n_hashs:10883
bridge_1 | 2018-03-09 11:55.25 DBG [git.pack-index] create: entering hash listing (ofs=1032)
bridge_1 | 2018-03-09 11:55.25 DBG [git.pack-index] create: entering crc checksums (ofs=218692)
bridge_1 | 2018-03-09 11:55.25 DBG [git.pack-index] create: entering packfile offsets (ofs=262224)
bridge_1 | 2018-03-09 11:55.25 DBG [git.pack-index] create: entering large packfile offsets (ofs=305756)
bridge_1 | 2018-03-09 11:55.25 DBG [git.pack-index] find_offset: 580fe5576d87b8377afeb45f9b078c6adf58b4fe
bridge_1 | 2018-03-09 11:55.25 DBG [git.pack-index] get_hash_idx: 580fe5576d87b8377afeb45f9b078c6adf58b4fe
bridge_1 | 2018-03-09 11:55.25 DBG [git.pack-index] find_offset: found:3206580
bridge_1 | 2018-03-09 11:55.25 DBG [git.unix] Reading /data/repos/datakit/.git/objects/pack/pack-8ed1414ac9c7af604b59d6ca6fb73f819c1906ae.pack
bridge_1 | 2018-03-09 11:55.25 DBG [git.pack-raw] read: 580fe5576d87b8377afeb45f9b078c6adf58b4fe version=2 count=10883
bridge_1 | 2018-03-09 11:55.25 DBG [git.pack-index] find_offset: 580fe5576d87b8377afeb45f9b078c6adf58b4fe
bridge_1 | 2018-03-09 11:55.25 DBG [git.pack-index] find_offset: cache hit!
bridge_1 | 2018-03-09 11:55.25 DBG [git.pack-raw] read: offset=3206580
bridge_1 | 2018-03-09 11:55.25 DBG [git.packed-value] input: kind:1 size:815 (more=true)
bridge_1 | 2018-03-09 11:55.25 DBG [git.packed-value] to_value
bridge_1 | 2018-03-09 11:55.25 DBG [git.unix] mkdir /data/repos/datakit/.git/tmp
bridge_1 | datakit-bridge-local-git: internal error, uncaught exception:
bridge_1 | (Failure
bridge_1 | "Unix.Unix_error(Unix.EROFS, \"mkdir\", \"/data/repos/datakit/.git/tmp\")")
bridge_1 |
selfci_bridge_1 exited with code 1
from datakit.
Thegaram
commented on July 3, 2024
Removing :ro from the bridge's compose specification helps, here are the related log entries:
...
bridge_1 | 2018-03-09 12:37.43 DBG [git.unix] Reading /data/repos/datakit/.git/objects/pack/pack-8ed1414ac9c7af604b59d6ca6fb73f819c1906ae.pack
bridge_1 | 2018-03-09 12:37.43 DBG [git.pack-raw] read: 580fe5576d87b8377afeb45f9b078c6adf58b4fe version=2 count=10883
bridge_1 | 2018-03-09 12:37.43 DBG [git.pack-index] find_offset: 580fe5576d87b8377afeb45f9b078c6adf58b4fe
bridge_1 | 2018-03-09 12:37.43 DBG [git.pack-index] find_offset: cache hit!
bridge_1 | 2018-03-09 12:37.43 DBG [git.pack-raw] read: offset=3206580
bridge_1 | 2018-03-09 12:37.43 DBG [git.packed-value] input: kind:1 size:815 (more=true)
bridge_1 | 2018-03-09 12:37.43 DBG [git.packed-value] to_value
bridge_1 | 2018-03-09 12:37.43 DBG [git.unix] mkdir /data/repos/datakit/.git/tmp
bridge_1 | 2018-03-09 12:37.43 DBG [git.unix] mkdir /data/repos/datakit/.git/objects/58
bridge_1 | 2018-03-09 12:37.43 DBG [git.unix] Writing /data/repos/datakit/.git/objects/58/0fe5576d87b8377afeb45f9b078c6adf58b4fe (/data/repos/datakit/.git/tmp/0fe5576d87b8377afeb45f9b078c6adf58b4fe2b7e53write)
bridge_1 | 2018-03-09 12:37.43 DBG [git.fs] write -> 580fe5576d87b8377afeb45f9b078c6adf58b4fe
bridge_1 | 2018-03-09 12:37.43 DBG [git.packed-value] unpack-raw[0]: 580fe5576d87b8377afeb45f9b078c6adf58b4fe
bridge_1 | 2018-03-09 12:37.43 DBG [bridge-local-git.sync] Update for "master"
bridge_1 | 2018-03-09 12:37.43 INF [bridge-local-git.sync] Copy state to DataKit
...
It seems to be making a backup for /data/repos/datakit/.git/objects/58/0fe5576d87b8377afeb45f9b078c6adf58b4fe.
from datakit.
samoht
commented on July 3, 2024
I think I understand: 580fe5576d87b8377afeb45f9b078c6adf58b4fe is initially stored in the pack file, and ocaml-git unpacks it when it reads it. I guess :ro is only working when there is no pack files (for instance by running git unpack-objects manually before) so I think it's best to remove :ro to make it work reliably.
The next version of ocaml-git is much better at reading pack files and it will fix this issue /cc @dinosaure
from datakit.
dinosaure
commented on July 3, 2024
On my side, I just cloned moby/datakit and try to get this git object (580fe5576d87b8377afeb45f9b078c6adf58b4fe) and all seems to work:
The problem seems (again) about extra-field on commit (gpgsign) - so related to mirage/ocaml-git#276 and fixed on master.
from datakit.
samoht
commented on July 3, 2024
@dinosaure Does it work if you have a read-only filesystem (which is the issue here)? The current version of ocaml-git will unpack objects whenever it resolves delta in the pack file (to speed up the next reads).
from datakit.
Related Issues (20)
- Invalid use of `${ROOT}` HOT 2
- check version
- Datakit using irmin store not writing to repository HOT 1
- Minor release with s/per_file/per_module/ ? HOT 3
- "git commit" seems to break the transaction interface HOT 4
- Unable to run CI example (permission denied) HOT 7
- Unable to run DataKit Local-Git bridge HOT 2
- Unix.ENAMETOOLONG when using datakit-bridge-local-git HOT 3
- DataKitCI: configure docker run HOT 5
- CI: Rebuilding a job doesn't cancel its childen jobs HOT 5
- Can't create new live log on branch HOT 13
- Assert_failure ci/src/cI_monitored_pool.ml:44:2 HOT 5
- CI: local-bridge does not rebuild on new commit HOT 3
- CI: local state repo HOT 2
- CI: unable to use Git.run HOT 2
- Datakit-CI is rebuilding PR on title edit HOT 3
- Irmin.merge_head: conflict, retrying (223684) (100% CPU) HOT 10
- Irmin 2.0.0
- datakit-client is not compatible with result 1.5 on OCaml 4.08+
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from datakit.