chores: port some of the official Neo4j driver tests to Bolt.Sips, using boltkit about bolt_sips HOT 7 CLOSED

Your code works, but we still need to feed a new script to boltstub for each test.
boltstub is just a dummy server that will match a client command and answer what the script told it to answer. And nothing more.
If you consider the following script:

!: AUTO INIT
!: AUTO RESET

C: RUN "RETURN 1 AS num" {}
   PULL_ALL
S: SUCCESS {"fields": ["num"]}
   RECORD [1]
SUCCESS {}

It will fail for any query that is not RETURN 1 AS num with out parameters.

-> It will be necessary to start new DbConnection
not needed either, if the scenario above works?!

boltstub is just a dummy server, we still need the client to perform connections and queries :)

And alas, it won't save us from using different "real" Neo4j versions, at least for writing the scenarii. We have those written by Neo4j teams but if we want more, we have to write them ourselves.... and know the subtle changes between the bolt version (and I'm not speaking of seemingly unused data in query/response)

But even with these considerations, it will be great for every new collaborator who work on the high level part of the driver to have those and not to have to run (as I do, as you may do) 3 (or more) Neo4j instances, event if it's easier now with Neo4j platform.

Considering a real server, boltkit has one-command server launch for single server, cluster and multicluster which is nice but require some AWS credentials....

from bolt_sips.

Very very nice idea.
I've already know about bolt_kit but get stuck at the point where I asked for AWS credentials to have a dev cluster running in one command but never get any answer back.
Anyway, I didn't look much into boltstub but it's really interesting, especially if we can use the scripts already written by Neo4j's team.

I've just spent some time today on the subject and it seems that there's a little more involved than expected:

• as you said, boltstub should run on background, this means something like a GenServer or something. I've made a quick test with Task.async (and a Process.sleep...) and it works almost fine. There's just some --- Logging error --- Python error popping
• It will be necessary to start new DbConnection, I don't know if this is problematic or not (especially if think about running tests concurrently -> port mess ahead maybe)
• ConfigAgent should be more flexible and not rely only on config

Some of these concerns are closely related to something I wanted to implement after you completed routing: the ability to connect to multiple database.
Then I don't know if now is the right to add the bolt_kit support or if it's better to wait for routing and all its induced changes.
First option leads to merge hell, the second leads to test pain for you.
Maybe the better should be to work directly on your dev branch (on branch checkout from your dev branch precisely) when you think it's stable enough.

from bolt_sips.

boltstub should run on background

actually I was thinking to wrap it in its own .sh scrip and/or starting it at the beginning of the tests, something like this:

options = [
  :binary,
  :stream,
  :use_stdio,
  :stderr_to_stdout,
  :exit_status,
  args: ["-v", 7687, "init_stubs.bolt"]
]

case System.find_executable("boltstub") do
  boltstub when not is_nil(boltstub) ->
    Port.open({:spawn_executable, boltstub}, options)
  _ ->
    {:error, :no_boltstub}
end

so no GenServer, in this case. Mind you, the above is not tested :)

It will be necessary to start new DbConnection

not needed either, if the scenario above works?!

ConfigAgent should be more flexible and not rely only on config

The ConfigAgent is gone :) The "routing" code acts as a "hub" for connections, and each connection maintains its own configuration; the latter inferred from the initial user's own. I hope I have this version stable enough very soon, as per our earlier chat.

I believe the approach I suggested above, using Port or maybe even System.cmd, could be experimented with without even needing to start the driver, or at least that's what I thought when I opened this request for help. But you're very right, given the new routing code, the changes are too many to reimport the master in, unless there are some hotfixes we absolutely need.

I totally support your concern, about timing. I'll need few more days and have a working prototype that can be used for adding more meat to it. Unfortunately, I'll probably have one next weekend, as I am not sure how much time I can find after my day-job schedule ends. So let's wait few more days :)

Thank you so much for feedback!!

from bolt_sips.

BTW since the boltkit is written in python, this fresh off the press story might be relevant to our story here: Mixing Python with Elixir?!

from bolt_sips.

hi just published this: https://github.com/florinpatrascu/fixex, maybe useful?! ;)

from bolt_sips.

closing this as I am able to use the official test stubs provided by the Python driver, with the FixEx. Still tinkering at it, but this how a simple test looks like now, with FixEx:

defmodule Bolt.Sips.PythonStubTest do
  use Fixex,
    adapter: Fixex.Python,
    config: [
      python_path: [
        to_charlist("test/stub"),
        to_charlist("test/stub/boltkit")
      ]
    ]

  describe "Boltkit" do
    @fix [:server, :start, ["test/stub/scripts/return_1_in_tx.script", 9001]]
    fix "test with stub", ctx do
      neo_conf = [
        url: "bolt://localhost:9001",
        role: :boltkit
      ]

      assert %{fix: true} = ctx
      assert {:ok, neo} = Bolt.Sips.start_link(neo_conf)
      assert conn = Bolt.Sips.conn(:boltkit)

      assert %Bolt.Sips.Response{results: [%{"1" => 1}]} 
               = Bolt.Sips.query!(conn, "return 1")
    end
  end
end

useful for covering a good amount of use cases.

from bolt_sips.

update:

• while the above works for some simpler use cases, and it's clean too, for the most complicated scenarios, the ones where the python support in boltkit is expecting generators, from us, the Erlport package fails - and since I am just an amateur in Python, I should probably delegate this to more Python-savvy troubleshooters. Will try using the stubs via Porcelain and see where I end up. Otherwise we might need to rewrite a lot of routing-related tests :/

from bolt_sips.