Git Product home page Git Product logo

libchronicle's People

Contributors

dependabot[bot] avatar shuckc avatar wsargent avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

libchronicle's Issues

Index pages are ignored/not written

Appender code does not insert index pages nor maintain index2index.
Tailer codes does not use either to find correct replay point, does a linear scan.

Needs analysis of the upstream Java code to determine how to interpret the powers of two that control indexing.

rollover leaves previous queuefile without EOF entry

Our append function does not write EOF to the current queuefile when rolling to the next. So our own tailer will get stuck on the last entry of the old queuefile, even though the appender has continued ahead.

We consult the clock before polling the appender, and will switch queuefiles in preparation for the next write. This is not correct. instead:

  1. take write lock from the current queuefile
  2. sample the clock
  3. no roll:
    • write entry normally and release lock
  4. roll:
    • create new queuefile,
    • update global maxcycle, current cycle
    • write EOF to the current queuefile, releasing the writelock
    • re-attempt the write from step 1

Scala Native bindings

I would like to work on Scala Native bindings of your C library.
Could you please tell me which part of the API should be useful to be exposed for external use?
Thanks.

Running make does not compile

It looks like the k.h definition has changed in KxSystems/kdb and the code base no longer compiles:

❱ make 
wget https://raw.githubusercontent.com/KxSystems/kdb/master/c/c/k.h
--2020-09-12 14:57:10--  https://raw.githubusercontent.com/KxSystems/kdb/master/c/c/k.h
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.40.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.40.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 4126 (4.0K) [text/plain]
Saving to: ‘k.h’

k.h                                      100%[===============================================================================>]   4.03K  --.-KB/s    in 0s      

2020-09-12 14:57:10 (91.9 MB/s) - ‘k.h’ saved [4126/4126]

gcc -o obj/hpet.so hpet.c -DKXVER=3 -fPIC -I. -Wall -std=gnu99 -shared
gcc -o obj/shmipc.so shmipc.c -DKXVER=3 -fPIC -I. -Wall -std=gnu99 -shared
gcc -o obj/shmmain shmmain.c -DKXVER=3 -fPIC -I. -Wall -std=gnu99 -g -O0
In file included from shmmain.c:18:0:
mock_k.h:62:3: error: conflicting types for ‘dl’
 K dl(void* fnptr, int n) {
   ^~
In file included from ./shmipc.c:32:0,
                 from shmmain.c:15:
./k.h:74:118: note: previous declaration of ‘dl’ was here
 extern K ee(K),ktj(I,J),ka(I),kb(I),kg(I),kh(I),ki(I),kj(J),ke(F),kf(F),kc(I),ks(S),kd(I),kz(F),kt(I),sd1(I,K(*)(I)),dl(V*f,J),
                                                                                                                      ^~
shmmain.c: In function ‘main’:
shmmain.c:199:21: warning: format ‘%lld’ expects argument of type ‘long long int’, but argument 3 has type ‘uint64_t {aka long unsigned int}’ [-Wformat=]
    fprintf(tmp, "%lld %lld %lld\n", clock, bytes, r->j);
                  ~~~^
                  %ld
shmmain.c:214:24: warning: format ‘%lld’ expects argument of type ‘long long int *’, but argument 3 has type ‘uint64_t * {aka long unsigned int *}’ [-Wformat=]
    r = fscanf(tmp, "%lld %lld %lld\n", &clock, &bytes, &index);
                     ~~~^               ~~~~~~
                     %ld
shmmain.c:214:34: warning: format ‘%lld’ expects argument of type ‘long long int *’, but argument 5 has type ‘uint64_t * {aka long unsigned int *}’ [-Wformat=]
    r = fscanf(tmp, "%lld %lld %lld\n", &clock, &bytes, &index);
                               ~~~^                     ~~~~~~
                               %ld
shmmain.c:217:21: warning: format ‘%lld’ expects argument of type ‘long long int’, but argument 2 has type ‘uint64_t {aka long unsigned int}’ [-Wformat=]
     printf(" did %lld %lld %lld\n", clock, bytes, index);
                  ~~~^
                  %ld
shmmain.c:217:31: warning: format ‘%lld’ expects argument of type ‘long long int’, but argument 4 has type ‘uint64_t {aka long unsigned int}’ [-Wformat=]
     printf(" did %lld %lld %lld\n", clock, bytes, index);
                            ~~~^
                            %ld
Makefile:31: recipe for target 'obj/shmmain' failed
make: *** [obj/shmmain] Error 1

Chronicle Queue 5.x uses metadata.cq4t

There are references to directory-listing and not metadata:

https://github.com/TeaEngineering/libchronicle/blob/master/native/shmipc.c#L140

https://github.com/TeaEngineering/libchronicle/blob/master/bindings/kdb/shm.q#L23

This file is not used in Chronicle Queue 5:

They are almost the same, except files directory-listing.cq4t was in earlier versions of Chronicle Queue, while metadata.cq4t is applicable for Chronicle Queue 5.0 onwards.

https://github.com/OpenHFT/Chronicle-Queue/blob/master/docs/FAQ.adoc#what-are-the-differences-between-the-files-directory-listing-cq4t-and-metadata-cq4t

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.