Stream Inspector about conduit HOT 7 CLOSED

Sidenote: I think it's a bit off-topic to talk about a CLI/REPL in this issue, it sounds like a whole separate feature by itself.

Yup for sure.

Here's a separate issue for that: #697.

from conduit.

Inspector will be nice, but I will raise you a REPL for this, it will be magnificent.

from conduit.

@lyuboxa what would a repl look like? Would you have to create a "conduit cli" to make this possible?

from conduit.

I think any tooling that can be used to inspect data real time and does not involve explicit parsing will be useful. When @hariso and I were discussing this something like this came to mind:

$ conduit-cli
> pipelines
<... print list of pipelines>
> c  = pipeline["abc"].connector["source"]
> c.peek(1) 
-> <message in structured format>
> c.processors["anonymize"].Peek(1)
-> <message in structured format> 

I think an interactive REPL will be useful, a console or whatever you want to call it.

from conduit.

@lyuboxa A REPL doesn't sound very composable to me 😕 Why not rather provide a simple CLI command to get the records? This would let you compose it with other tools like grep & co. Example:

$ conduit-cli inspect id:of:my:component --limit 2
<message>
<message>

Sidenote: I think it's a bit off-topic to talk about a CLI/REPL in this issue, it sounds like a whole separate feature by itself.

from conduit.

A REPL doesn't sound very composable to me .

Does it have to be? REPL are interactive tools for evaluation, not unix commands. Both options have their merits.
Former is more suitable for interactive work (which CLI is not tbh, but it does need a way to enter an interactive session).
However, if you only want to dump data and move on, then this is likely not the most optimal solution.

Sidenote: I think it's a bit off-topic to talk about a CLI/REPL in this issue, it sounds like a whole separate feature by itself.

Yup for sure.

from conduit.

All the tasks are done, closing the issue.

from conduit.