c-cube / gen Goto Github PK

View Code? Open in Web Editor NEW

57.0 6.0 9.0 816 KB

Simple, efficient iterators for OCaml

Home Page: http://c-cube.github.io/gen/

License: BSD 2-Clause "Simplified" License

Makefile 0.75% OCaml 99.25%

ocaml gen generator iterator

gen's Introduction

Gen

Iterators for OCaml, both restartable and consumable. The implementation keeps a good balance between simplicity and performance.

The library is extensively tested using qtest. If you find a bug, please report!

The documentation can be found here the main module is Gen and should suffice for 95% of use cases.

Changelog

Native install

Installation in a native OCaml project, via opam:

$ opam install gen

or, manually, by building the library and running make install. Opam is recommended, for it keeps the library up-to-date.

BuckleScript install

Installation in JavaScript, via BuckleScript, in an npm project:

Install this package:
```
$ npm install bs-gen
```
Manually add bs-gen to your bsconfig.json's bs-dependencies:
```
"bs-dependencies": [
   ...
   "bs-gen"
],
```

Use

You can either build and install the library (see "Build"), or just copy files to your own project. The last solution has the benefits that you don't have additional dependencies nor build complications (and it may enable more inlining).

If you have comments, requests, or bugfixes, please share them! :-)

Build

There are no dependencies except for dune for building. This should work with OCaml>=4.02

$ make

To build and run tests (requires oUnit and qtest):

$ opam install oUnit qtest
$ make test

License

This code is free, under the BSD license.

gen's People

Contributors

Stargazers

Watchers

Forkers

struktured gbury rizo gmevel elliottcable genisysram fardalem edwardbetts lexifi

gen's Issues

some historic distfiles with wrong checksums in opam

Building /home/opam/opam-repository/archives/gen.0.2+opam.tar.gz
[gen.0.2] Downloading https://github.com/c-cube/gen/archive/0.2.tar.gz
[ERROR] Wrong checksum for /tmp/opam-41-d6d332/0.2.tar.gz:
  - 71df6f679f051a59e1c7d3fe71ed427d [expected result]
  - 4630f81c67531d217e9328eff9f3a11b [actual result]
This is surely due to outdated package descriptions and should be fixed by running `opam update`.
In case an update does not fix that problem, you can use the `--no-checksums` command-line option
to /bypass checking for invalid checksums.

[FR] allow marking generator as exhausted non-deterministically?

I have the following code to consume all stdin from user and abstract them over a genrator API:

let read_lineedit () = 
  print_string "> ";
  read_line ()

let gen_lines () = 
  let cache = ref [] in
  let eof_reached = ref false in
  let query_cache i = 
    let l = List.length !cache in
    if l <= i then
      None
    else 
      Some (List.nth !cache (l - i - 1))
  in
    
  let gen_at i = 
    match query_cache i with 
    | Some(g) -> g
    | None -> 
      if !eof_reached 
      then Gen.empty
      else begin
        while (List.length !cache) <= i && not !eof_reached do
          cache := 
          begin try 
              Gen.of_string (read_lineedit())
            with End_of_file -> 
              eof_reached := true; Gen.empty
            end :: !cache
        done;
        List.hd !cache
      end
  in
  Gen.init gen_at |> Gen.flatten

This is problematic, however, because the outer generator is infinite, and even we know inner generators can no longer produce any value, the outer generator still keep trying to generate inner generators.

skipping

i have not found a way to take only every nth element -- is this possible? have i missed it? more generally, one could think of a list of 'step lengths' or 'skip lengths' to apply cyclically as one moves over the generator, i.e. [0;2;1] means 'take next element, then skip two and take one, then skip one and take one, then take the next element, etc.,..'

Gen fails to build with opam 2.0.0 and brwap sandboxing

I've no problem building gen in the default switch, but it fails in a custom switch.

The problem is that my custom switch is mounted ro by the sandbox.sh
and the makefile tries to create the _build directory in this directory. Not sure if this is a
problem related to the sandbox script or gen. This is the first package I encounter to
show this problem.

This is the log.

$ opam install gen
The following actions will be performed:
  - install gen 0.5.1

<><> Gathering sources ><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
[gen.0.5.1] found in cache

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
[ERROR] The compilation of gen failed at "/home/abate/.opam/opam-init/hooks/sandbox.sh build
        jbuilder build @install".

#=== ERROR while compiling gen.0.5.1 ==========================================#
# context     2.0.0 | linux/x86_64 | ocaml-base-compiler.4.06.1 | https://opam.ocaml.org/2.0#ac6a2505
# path        ~/Projects/repos/tezos/_opam/.opam-switch/build/gen.0.5.1
# command     ~/.opam/opam-init/hooks/sandbox.sh build jbuilder build @install
# exit-code   1
# env-file    ~/.opam/log/gen-12256-05296d.env
# output-file ~/.opam/log/gen-12256-05296d.out
### output ###
# [...]
# + add_mounts rw /home/abate/Projects/repos/tezos/_opam/.opam-switch/build/gen.0.5.1
# + case "$1" in
# + B=--bind
# + for dir in "$@"
# + '[' -d rw ']'
# + for dir in "$@"
# + '[' -d /home/abate/Projects/repos/tezos/_opam/.opam-switch/build/gen.0.5.1 ']'
# + ARGS=("${ARGS[@]}" "$B" "$dir" "$dir")
# + add_ccache_mount
# + command -v ccache
# + exec bwrap --unshare-net --new-session --proc /proc --dev /dev --bind /tmp /tmp --tmpfs /run --tmpfs /var --ro-bind /usr /usr --ro-bind /bin /bin --ro-bind /lib /lib --ro-bind /lib64 /lib64 --ro-bind /etc /etc --ro-bind /opt /opt --ro-bind /home /home --ro-bind /home/abate/Projects/repos/tezos/_opam /home/abate/Projects/repos/tezos/_opam --bind /home/abate/Projects/repos/tezos/_opam/.opam-s[...]
# Error: mkdir: _build: Read-only file system



<><> Error report <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
+- The following actions failed
| - build gen 0.5.1
+- 
- No changes have been performed

Document how to get an infinite `int_range`?

Possibly worth mentioning: how do you produce an “infinite” range of integers?

This is what I did:

Gen.int_range 0 max_int

move to dune

Pervasives deprecated

Building with dune 2.5.1 and OCaml 4.09.1 on Arch Linux, I get the following errors:

File "src/gen.ml", line 671, characters 17-35:
671 | let lexico ?(cmp=Pervasives.compare) gen1 gen2 =
                       ^^^^^^^^^^^^^^^^^^
Error (alert deprecated): module Stdlib.Pervasives
Use Stdlib instead.

If you need to stay compatible with OCaml < 4.07, you can use the
stdlib-shims library: https://github.com/ocaml/stdlib-shims
File "src/gen.ml", line 851, characters 23-41:
851 | let intersection ?(cmp=Pervasives.compare) gen1 gen2 =
                             ^^^^^^^^^^^^^^^^^^
Error (alert deprecated): module Stdlib.Pervasives
Use Stdlib instead.

If you need to stay compatible with OCaml < 4.07, you can use the
stdlib-shims library: https://github.com/ocaml/stdlib-shims
[...]

Same thing here, should this be included into batteries?

add `peek_n`

peek_n : int -> 'a gen -> ('a * 'a list) gen, peek the n next elements.
implem: use an internal queue or something similar

META shows version 0.2.4

It looks like at least some of the oasis-generated files are not synced up. src/META lists the version number as 0.2.4, _oasis as 0.4. I ran into this while trying to constrain the version for a local project.

Labels version of the signature(s)

Following common conventions: ~f, ~init …

Upstream BuckleScript support?

Hey! (I meant to catch you one more time this week at ReasonConf, but I had to ditch early.)

I maintain a bs-gen fork of your work; it's literally just a few .json files added to the repository, and then published to npm.

Unfortunately, occasionally, I have to bump the version for npm- or BuckleScript-specific reasons — for instance, with the release last week of BuckleScript's support for 4.06, I've been going through my npm repackagings of opam packages and refreshing. Thing is, I don't want to decouple the version-number of bs-gen from your upstream release; it's such a simple repackaging …

Do you have any interest in letting me upstream those additions? I'm happy to keep carrying the maintenance burden of anything BuckleScript- or npm-specific. (And then bumping the opam version-number to 0.5.2, so it can stay in sync with the npm version.)

Installation failure with 4.06.0+flambda - gen v0.4 (opam)

ERROR while compiling gen.0.4
context      2.0.0~beta4 | linux/x86_64 | ocaml-variants.4.06.0+flambda | https://opam.ocaml.org/2.0#2017-11-03 19:07
path        ~ /.opam/4.06.0+flambda/.opam-switch/build/gen.0.4
command      make all
File "myocamlbuild.ml", line 518, characters 43-62:
Warning 3: deprecated: Ocamlbuild_plugin.String.uncapitalize
Use String.uncapitalize_ascii instead.
File "myocamlbuild.ml", line 531, characters 51-70:
Warning 3: deprecated: Ocamlbuild_plugin.String.uncapitalize
Use String.uncapitalize_ascii instead.
E: Failure("Command ''~/.opam/4.06.0+flambda/bin/ocamlbuild' src/gen.cma src/gen.cmxa src/gen.a src/gen.cmxs -tag debug' terminated with error code 10")
Makefile:16 : la recette pour la cible « all » a échouée
make: *** [all] Erreur 1

dune runtest fails

How is the dune runtest target supposed to work? This is what I get:

[   49s] Executing(%check): /bin/sh -e /var/tmp/rpm-tmp.ak8g3P
[   49s] + umask 022
[   49s] + cd /home/abuild/rpmbuild/BUILD
[   49s] + cd gen-0.5.2
[   49s] + dune_for_release=
[   49s] + test -f dune_release_pkgs-ocaml-gen-0.5.2-15.obc_408.2
[   49s] + read dune_release_pkgs
[   49s] + dune_for_release=--for-release-of-packages=gen
[   49s] + dune runtest --verbose --for-release-of-packages=gen
[   49s] # Workspace root: /home/abuild/rpmbuild/BUILD/gen-0.5.2
[   49s] Running[0]: /usr/bin/nproc > /tmp/dune3ce6d7.output 2> /dev/null
[   49s] # # Workspace root: /home/abuild/rpmbuild/BUILD/gen-0.5.2
[   49s] # Auto-detected concurrency: 8
[   49s] Running[1]: /usr/bin/ocamlc -config > /tmp/dune2dfa56.output
[   49s] # # # Workspace root: /home/abuild/rpmbuild/BUILD/gen-0.5.2
[   49s] # # Auto-detected concurrency: 8
[   49s] # Dune context:
[   49s] #  {name = "default";
[   49s] #    kind = "default";
[   49s] #    profile = "release";
[   49s] #    merlin = true;
[   49s] #    for_host = None;
[   49s] #    build_dir = "default";
[   49s] #    toplevel_path = None;
[   49s] #    ocaml_bin = External "/usr/bin";
[   49s] #    ocaml = External "/usr/bin/ocaml";
[   49s] #    ocamlc = External "/usr/bin/ocamlc";
[   49s] #    ocamlopt = None;
[   49s] #    ocamldep = External "/usr/bin/ocamldep";
[   49s] #    ocamlmklib = External "/usr/bin/ocamlmklib";
[   49s] #    env =
[   49s] #      map {"CAML_LD_LIBRARY_PATH" :
[   49s] #           "/home/abuild/rpmbuild/BUILD/gen-0.5.2/_build/install/default/lib/stublibs";
[   49s] #      "DUNE_CONFIGURATOR" : "/usr/bin/ocamlc"; "INSIDE_DUNE" : "1";
[   49s] #      "OCAMLFIND_IGNORE_DUPS_IN" :
[   49s] #      "/home/abuild/rpmbuild/BUILD/gen-0.5.2/_build/install/default/lib";
[   49s] #      "OCAMLPATH" :
[   49s] #      "/home/abuild/rpmbuild/BUILD/gen-0.5.2/_build/install/default/lib"};
[   49s] #    findlib_path = [External "/usr/lib64/ocaml"];
[   49s] #    arch_sixtyfour = true;
[   49s] #    natdynlink_supported = false;
[   49s] #    supports_shared_libraries = true;
[   49s] #    opam_vars = map {};
[   49s] #    ocaml_config =
[   49s] #      {version = "4.08.1";
[   49s] #        standard_library_default = "/usr/lib64/ocaml";
[   49s] #        standard_library = "/usr/lib64/ocaml";
[   49s] #        standard_runtime = "the_standard_runtime_variable_was_deleted";
[   49s] #        ccomp_type = "cc";
[   49s] #        c_compiler = "/usr/bin/gcc";
[   49s] #        ocamlc_cflags = ["-O2"; "-fno-strict-aliasing"; "-fwrapv"; "-fPIC"];
[   49s] #        ocamlopt_cflags = ["-O2"; "-fno-strict-aliasing"; "-fwrapv"];
[   49s] #        bytecomp_c_compiler =
[   49s] #          ["/usr/bin/gcc"; "-O2"; "-fno-strict-aliasing"; "-fwrapv"; "-fPIC";
[   49s] #          "-D_FILE_OFFSET_BITS=64"; "-D_REENTRANT"];
[   49s] #        bytecomp_c_libraries = ["-lm"; "-ldl"; "-lpthread"];
[   49s] #        native_c_compiler =
[   49s] #          ["/usr/bin/gcc"; "-O2"; "-fno-strict-aliasing"; "-fwrapv";
[   49s] #          "-D_FILE_OFFSET_BITS=64"; "-D_REENTRANT"];
[   49s] #        native_c_libraries = ["-lm"; "-ldl"];
[   49s] #        cc_profile = ["-pg"];
[   49s] #        architecture = "amd64";
[   49s] #        model = "default";
[   49s] #        int_size = 63;
[   49s] #        word_size = 64;
[   49s] #        system = "linux";
[   49s] #        asm = ["/usr/bin/as"];
[   49s] #        asm_cfi_supported = true;
[   49s] #        with_frame_pointers = false;
[   49s] #        ext_exe = "";
[   49s] #        ext_obj = ".o";
[   49s] #        ext_asm = ".s";
[   49s] #        ext_lib = ".a";
[   49s] #        ext_dll = ".so";
[   49s] #        os_type = "Unix";
[   49s] #        default_executable_name = "a.out";
[   49s] #        systhread_supported = true;
[   49s] #        host = "x86_64-suse-linux-gnu";
[   49s] #        target = "x86_64-suse-linux-gnu";
[   49s] #        profiling = true;
[   49s] #        flambda = false;
[   49s] #        spacetime = false;
[   49s] #        safe_string = false;
[   49s] #        exec_magic_number = "Caml1999X025";
[   49s] #        cmi_magic_number = "Caml1999I025";
[   49s] #        cmo_magic_number = "Caml1999O025";
[   49s] #        cma_magic_number = "Caml1999A025";
[   49s] #        cmx_magic_number = "Caml1999Y025";
[   49s] #        cmxa_magic_number = "Caml1999Z025";
[   49s] #        ast_impl_magic_number = "Caml1999M025";
[   49s] #        ast_intf_magic_number = "Caml1999N025";
[   49s] #        cmxs_magic_number = "Caml1999D025";
[   49s] #        cmt_magic_number = "Caml1999T025";
[   49s] #        natdynlink_supported = false;
[   49s] #        supports_shared_libraries = true;
[   49s] #        windows_unicode = false};
[   49s] #    which =
[   49s] #      map {"ocaml" : Some External "/usr/bin/ocaml";
[   49s] #      "ocamlc" : Some External "/usr/bin/ocamlc";
[   49s] #      "ocamlobjinfo" : Some External "/usr/bin/ocamlobjinfo"}}
[   49s] Running[2]: (cd _build/default/src && /usr/bin/qtest extract '${impl-files}' '${intf-files}') > _build/default/src/.gen.inline-tests/run.ml-gen
[   49s] Command [2] exited with code 2:
[   49s] $ (cd _build/default/src && /usr/bin/qtest extract '${impl-files}' '${intf-files}') > _build/default/src/.gen.inline-tests/run.ml-gen
[   49s] Extraction : `${impl-files}' Fatal error: exception Sys_error("${impl-files}: No such file or directory")
[   49s] + : make check failed