Comments (4)
I have identified the cause: there is a regression in the way we compare the contents of reference files. For some reasons, we do not trim the contents anymore, so <hash>\n
-- which is created by git commit
-- is in conflict with <hash>
-- as understood by Datakit/Irmin: thus the transaction can never be resolved and enter in a loop.
I am trying to understand where does the regression come from exactly and will push a fix shortly. Thanks a lot for the nice reproduction case, it helped a lot to track that down!
from datakit.
This used to work with the very old version 6f04213 but seems broken with 97b3d23
Could you let me know what you think, @samoht @talex5?
from datakit.
I've written a test script which demonstrates the problem. The script assumes that
- your
$GOPATH
is$HOME/go
and you have a checkout ofgithub.com/moby/datakit
there. Modify the top of the script to change this assumption - you run
go get github.com/docker/go-p9p
The script is:
cat <<EOT > test-datakit.sh
#!/bin/sh
DB=${DB:-/Applications/Docker.app/Contents/MacOS/com.docker.db}
DATAKIT=${DATAKIT:-${HOME}/go/src/github.com/moby/datakit/api/go-datakit}
if [ ! -e "${DB}" ]; then
echo "Datakit binary not found at ${DB}: please set DB environment variable"
exit 1
fi
if [ ! -e "${DATAKIT}" ]; then
echo "Datakit source tree not found at "${DATAKIT}". Note it has to be on the GOPATH".
exit 1
fi
echo Using datakit from: "${DB}"
echo Using test cases from source tree: "${DATAKIT}"
database=$(mktemp -d)
echo Created temporary database dir "${database}"
DB_PID=0
TEST_PID=0
clean_up () {
rm -rf "${database}"
kill "${DB_PID}" 2> /dev/null
kill "${TEST_PID}" 2> /dev/null
}
trap clean_up EXIT
pushd "${database}"
git init .
echo foo > bar
git add bar
git commit bar -m test
popd
"${DB}" --git "${database}" --url file:///var/tmp/foo&
DB_PID=$!
sleep 1 # easier than performing fd passing in a shell script
cd "${DATAKIT}"
go test -run TestTransaction 2>/dev/null >/dev/null &
TEST_PID=$!
retries=5
while [ "${retries}" -ne 0 ]; do
if kill -0 "${TEST_PID}" 2> /dev/null; then
echo Test is still running
retries=$(( retries - 1 ))
sleep 1
else
echo Test has finished
echo Success
exit 0
fi
done
echo Test has timed out
echo Failure
exit 1
EOT
If you have a Docker for Mac install with a working (old) datakit, the test passes like so:
$ ./test-datakit.sh
Using datakit from: /Applications/Docker.app/Contents/MacOS/com.docker.db
Using test cases from source tree: /Users/djs/go/src/github.com/moby/datakit/api/go-datakit
Created temporary database dir /var/folders/5q/xqlk9dzn5jz7_hn2j31brc0r0000gn/T/tmp.M04AzKw9
/var/folders/5q/xqlk9dzn5jz7_hn2j31brc0r0000gn/T/tmp.M04AzKw9 ~
Initialized empty Git repository in /private/var/folders/5q/xqlk9dzn5jz7_hn2j31brc0r0000gn/T/tmp.M04AzKw9/.git/
[master (root-commit) af2828f] test
1 file changed, 1 insertion(+)
create mode 100644 bar
~
Starting com.docker.db 0.7.0-34-g6f04213 ...
Test is still running
com.docker.db: [INFO] accepted a new connection on file:///var/tmp/foo
com.docker.db: [INFO] Using protocol TwoThousand msize 16384
Test has finished
Success
If you have the latest master datakit then it fails like so:
DB=~/go/src/github.com/docker/pinata/mac/build/Docker.app/Contents/MacOS/com.docker.db ./test-datakit.sh
Using datakit from: /Users/djs/go/src/github.com/docker/pinata/mac/build/Docker.app/Contents/MacOS/com.docker.db
Using test cases from source tree: /Users/djs/go/src/github.com/moby/datakit/api/go-datakit
Created temporary database dir /var/folders/5q/xqlk9dzn5jz7_hn2j31brc0r0000gn/T/tmp.mRISPXGH
/var/folders/5q/xqlk9dzn5jz7_hn2j31brc0r0000gn/T/tmp.mRISPXGH ~
Initialized empty Git repository in /private/var/folders/5q/xqlk9dzn5jz7_hn2j31brc0r0000gn/T/tmp.mRISPXGH/.git/
[master (root-commit) 692e305] test
1 file changed, 1 insertion(+)
create mode 100644 bar
~
Starting com.docker.db %%VERSION%% ...
Test is still running
com.docker.db: [INFO] accepted a new connection on file:///var/tmp/foo
com.docker.db: [INFO] Using protocol TwoThousand msize 16384
Test is still running
Test is still running
Test is still running
Test is still running
Test has timed out
Failure
com.docker.db: [ERROR] Uncaught exception handling tag 72 Write(fid: 126, offset: 0, len(data): 6):
Exception: (Sys_error
"/var/folders/5q/xqlk9dzn5jz7_hn2j31brc0r0000gn/T/tmp.mRISPXGH/.git/lock/refs/heads: No such file or directory")
No backtrace available.
from datakit.
I did a simple bisection of the binary PR builds from CircleCI and have:
My guess is that it was introduced in #529
from datakit.
Related Issues (20)
- Invalid use of `${ROOT}` HOT 2
- check version
- Datakit using irmin store not writing to repository HOT 1
- Minor release with s/per_file/per_module/ ? HOT 3
- Unable to run CI example (permission denied) HOT 7
- Unable to run DataKit Local-Git bridge HOT 2
- Unix.ENAMETOOLONG when using datakit-bridge-local-git HOT 3
- Unix.EROFS while running self-ci locally HOT 8
- DataKitCI: configure docker run HOT 5
- CI: Rebuilding a job doesn't cancel its childen jobs HOT 5
- Can't create new live log on branch HOT 13
- Assert_failure ci/src/cI_monitored_pool.ml:44:2 HOT 5
- CI: local-bridge does not rebuild on new commit HOT 3
- CI: local state repo HOT 2
- CI: unable to use Git.run HOT 2
- Datakit-CI is rebuilding PR on title edit HOT 3
- Irmin.merge_head: conflict, retrying (223684) (100% CPU) HOT 10
- Irmin 2.0.0
- datakit-client is not compatible with result 1.5 on OCaml 4.08+
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from datakit.