machinefi / w3bstream Goto Github PK

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Please notice that wsctl --version is not implemented. This error also shows up when you install wsctl using the installation script.

~ wsctl --version                                               

Error: unknown flag: --version
Usage:
  wsctl [command]

Available Commands:
  completion  Generate the autocompletion script for the specified shell
  config      config action
  help        Help about any command
  message     message action
  node        node service control
  version     Print the version number of wsctl

Flags:
  -h, --help   help for wsctl

Use "wsctl [command] --help" for more information about a command.

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

• persistence
• project
• task
• output

this four package need supplement unit test, for the coverage not reach 80% after enode refactor

When running tests, panic would happen.

That's a famous issue caused by gomonkey, does anyone here encounter?

For General Questions About Sprout

Please do not use GitHub issues for asking general questions about the Sprout project. Instead, we encourage you to seek support and ask questions in one of the following platforms:

Join Our Discord Community

For real-time discussions and community support, join our Discord workspace. This is a great place to get quick help, discuss features, and connect with other community members:
Join Sprout Discord

Ask on Stack Overflow

For more structured and detailed questions, consider using Stack Overflow. Many of Sprout's core and expert developers prefer this platform for its non-realtime format, which encourages well-structured and comprehensive questions.
Ask your question here: Stack Overflow - Sprout Tag

Remember, a good question on Stack Overflow typically requires some upfront effort. Check out Stack Overflow's guide on how to ask a good question.

Found a Bug or Have a Feature Request?

If you believe you've found a bug or if you have a proposal for a new feature, please use the appropriate issue template to report it.

Thank you for being a part of the Sprout community!

Is your feature request related to a problem? Please describe.
When the proof is sent to the router contract and the execution reverts, we have no info about the reason for the revert in the logs:

w3bstream-enode  | time=2024-02-26T17:00:36.741Z level=ERROR msg="output failed" error="execution simulation failed: status = 106\nestimate gas failed\ngithub.com/machinefi/sprout/output.(*ethereumContract).sendTX\n\t/go/src/output/ethereum.go:162\ngithub.com/machinefi/sprout/output.(*ethereumContract).Output\n\t/go/src/output/ethereum.go:121\ngithub.com/machinefi/sprout/task.(*Dispatcher).handleP2PData\n\t/go/src/task/dispatcher.go:91\ngithub.com/machinefi/sprout/p2p.(*pubSub).run\n\t/go/src/p2p/pubsub.go:132\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1650\ntransaction failed\ngithub.com/machinefi/sprout/output.(*ethereumContract).Output\n\t/go/src/output/ethereum.go:123\ngithub.com/machinefi/sprout/task.(*Dispatcher).handleP2PData\n\t/go/src/task/dispatcher.go:91\ngithub.com/machinefi/sprout/p2p.(*pubSub).run\n\t/go/src/p2p/pubsub.go:132\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1650"

Describe the solution you'd like
Include revert error message in the logs

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

When Sprout writes the proof on-chain, and the transaction reverts (because either the router or the recipient contract reverted), can we show the revert error message in the logs?

w3bstream-coordinator  | time=2024-04-17T13:11:33.323Z level=DEBUG msg="outputing to ethereum contract" "chain endpoint"=https://babel-api.testnet.iotex.io
w3bstream-coordinator  | time=2024-04-17T13:11:34.444Z level=ERROR msg="failed to output" error="execution simulation failed: status = 106\nestimate gas failed\ngithub.com/machinefi/sprout/output.(*ethereumContract).sendTX\n\t/go/src/output/ethereum.go:150\ngithub.com/machinefi/sprout/output.(*ethereumContract).Output\n\t/go/src/output/ethereum.go:109\ngithub.com/machinefi/sprout/task/internal/handler.(*TaskStateHandler).Handle\n\t/go/src/task/internal/handler/task_state_handler.go:62\ngithub.com/machinefi/sprout/task/internal/dispatcher.(*dispatcherTask).handleState\n\t/go/src/task/internal/dispatcher/dispatcher_task.go:26\ngithub.com/machinefi/sprout/task/internal/dispatcher.(*window).consume\n\t/go/src/task/internal/dispatcher/window.go:31\ngithub.com/machinefi/sprout/task/internal/dispatcher.(*ProjectDispatcher).Handle\n\t/go/src/task/internal/dispatcher/project_dispatcher.go:33\ngithub.com/machinefi/sprout/task.(*dispatcher).handleP2PData\n\t/go/src/task/dispatcher.go:45\ngithub.com/machinefi/sprout/p2p.(*subscriber).run\n\t/go/src/p2p/pubsub.go:74\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1650\nfailed to send transaction\ngithub.com/machinefi/sprout/output.(*ethereumContract).Output\n\t/go/src/output/ethereum.go:111\ngithub.com/machinefi/sprout/task/internal/handler.(*TaskStateHandler).Handle\n\t/go/src/task/internal/handler/task_state_handler.go:62\ngithub.com/machinefi/sprout/task/internal/dispatcher.(*dispatcherTask).handleState\n\t/go/src/task/internal/dispatcher/dispatcher_task.go:26\ngithub.com/machinefi/sprout/task/internal/dispatcher.(*window).consume\n\t/go/src/task/internal/dispatcher/window.go:31\ngithub.com/machinefi/sprout/task/internal/dispatcher.(*ProjectDispatcher).Handle\n\t/go/src/task/internal/dispatcher/project_dispatcher.go:33\ngithub.com/machinefi/sprout/task.(*dispatcher).handleP2PData\n\t/go/src/task/dispatcher.go:45\ngithub.com/machinefi/sprout/p2p.(*subscriber).run\n\t/go/src/p2p/pubsub.go:74\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1650" task_id=1
w3bstream-coordinator  | time=2024-04-17T13:11:34.461Z level=INFO msg="task finished" project_id=10002 task_id=1

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Can you provide instructions for developers on how to setup the dev environment to contribute? How to build and run the node without Docker?

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

We need a method for users to easily download the latest or a specific release of the node software from GitHub. This solution eliminates the need to use the (potentially unstable) docker-compose.yaml from the latest commit which most of the times is broken.

Ideally, this functionality should be integrated into wsctl itself:

# Only the first time, install the latest wsctl
curl https://raw.githubusercontent.com/machinefi/sprout/main/scripts/install_wsctl.sh | bash

# Download the desired release of sprout
wsctl znode get latest # or wsctl znode get 0.2.9
tar xzf sprout-0.2.9.tar.gz && cd sprout-0.2.9
docker compose up -d

This process should automatically retrieve all the necessary files for running that specific version on the user's operating system. These files currently include at least the following:

• docker-compose.yaml
• wsctl-0.2.9 executable
• Any test projects located in ./test/data/
• README.md

add unit test for module clients

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Is your feature request related to a problem? Please describe.
Project management with numeric ids is a bit hard to manage (it's easy to forget your project ID or mistype the number)

Describe the solution you'd like

• Allow to set a UTF8 unique ID when registering a project
• Allow project update in ioctl (e.g. ioctl-unstable ws project update --project-config-file project.json)
• Allow getting project info in ioctl (e.g. ioctl-unstable ws project info --project-id "my_project" which returns the ipfs URL, code size, output config, aggregation config, etc...)

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Is your feature request related to a problem? Please describe.
Currently the enode and znode fetch all projects that are registered in the project registry. The list can quickly become too long and some project configs can also be faulty and produce potential errors in Ws.

Describe the solution you'd like
Have a global setting somewhere to tell sprout "I'm only interested in project IDs [1,17,8]

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

From my understanding, code in ./client is provided as a client library. Since the implementation of Client in Typescript and Python might be needed, it would be better to be moved out.

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Can you have the latest Sprout L1 contact addresses listed in the README, or upload a deployments file? ProjectRegistry, router, fleet manager, etc...

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

add unit test for module tasks

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

https://github.com/machinefi/sprout/blob/0f56ad8ed16f57013062b87ec2f43713af91af79/types/task.go#L13

Can we also consider adding a generic payload []byte field in the Task protocol for the sequencer to fill out?

type Task struct {
	ID             uint64   `json:"id"`
	ProjectID      uint64   `json:"projectID"`
	ProjectVersion string   `json:"projectVersion"`
	Data           [][]byte `json:"data"`
	Payload        []byte   `json:"payload"` // <<<---
	Signature      string   `json:"signature"`
}

This would allow more flexibility for a DePIN to include whatever task data they require to be sent on-chain together with the proof.

For example, this can be used if we do not want to force Client ID in the W3bstream protocol: a DePIN project could include the client IDs or the Merkle root of all client ID contributions. Or they can add any other custom task metadata that they need on-chain.

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Is your feature request related to a problem? Please describe.
We need some more visibility on the current status of w3bstream tasks for a certain project.

Describe the solution you'd like
Return the current filled status of the task for a given project whose aggregation amount is >1 (i.e. how many messages have been currently queued in a Task before the aggregation amount is matched):

$ ioctl-unstable ws project query --project-id 17

{
  uri: ipfs://ipfs.mainnet.iotex.io/QmUQeVvwPtTPHia9WQqMfPhVzdVdwS6K6yoZkFqmguACHW, 
  hash: 0x686d2bf0e8b2985972f48fb3cd040e23b852a60f796d4754ae3fac1095e299f6, 
  paused:false,
  queued: 19
}

Describe alternatives you've considered
Any other solution that helps us understand the w3bstream state for a project (e.g., list task ids that belong to a project, and allow query the task status)

Additional context
Add any other context or screenshots about the feature request here.

add unit test for module vm

After the update of the risc0 image in docker-config.yaml from v0.8rc1 to v0.8.stark the proof generation is failing with error "failed to execute instance: failed to execute risc0 instance: rpc error: code = Canceled desc = stream terminated by RST_STREAM with error code: CANCEL"

w3bstream-node | [GIN-debug] Listening and serving HTTP on :9000
w3bstream-node | [GIN] 2023/11/14 - 10:58:43 | 200 |    2.438918ms |      172.28.0.1 | POST     "/message"
w3bstream-node | time=2023-11-14T10:58:43.484Z level=DEBUG msg="received your message, handling"
w3bstream-node | time=2023-11-14T10:58:43.485Z level=DEBUG msg="push message into sequencer"
w3bstream-node | time=2023-11-14T10:58:43.489Z level=DEBUG msg="message popped by proofer"
w3bstream-node | time=2023-11-14T10:58:43.722Z level=DEBUG msg="acquire risc0 instance success"
w3bstream-node | time=2023-11-14T10:58:43.801Z level=ERROR msg="failed to execute instance: failed to execute risc0 instance: rpc error: code = Canceled desc = stream terminated by RST_STREAM with error code: CANCEL"

PR #32 restores the working image.

add unit test for module persistence

add unit test for module p2p

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Environment: Ubuntu 22

./sprout$ curl https://raw.githubusercontent.com/machinefi/sprout/main/scripts/install_wsctl.sh | bash
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
curl: (60) SSL: no alternative certificate subject name matches target host name 'raw.githubusercontent.com'
More details here: https://curl.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

Another issue when executing the script directly

bash install_wsctl.sh
Download URL not found for wsctl-linux-amd64

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

add unit test for module output

add unit test for module project

https://nitayneeman.com/posts/understanding-semantic-commit-messages-using-git-and-angular/

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Describe the bug
When enode/znode doesn't connect to the database, other errors happen on start.
It seems that config values used to be in docker-compose.yaml have been removed and now they do not get applied.
This PR 5fcb8a0 is when the config has been removed from docker-compose.yaml. They have been moved to the Viper package, yet the default values are not applied to my system.

To Reproduce
Steps to reproduce the behavior:

Make sure you are in a clean system (i.e. Sprout config values such as CHAIN_ENDPOINT, DATABASE_DSN and similar config is not already set on your system)

git clone https://github.com/machinefi/sprout && cd sprout
git checkout v0.6.0
docker compose up -d

Check the Sprout logs (znode cannot connect to the DB, even though the DB seems to be running correctly). Check screenshots below.

Expected behavior
Be able to send messages to Sprout.

Screenshots

Versions (please complete the following information):

• Node Version [e.g. 0.6.0 or later]
• OS Name and Version: [e.g. Mac OS Sonoma 14.1.2]
• Architecture: Mac M1

Additional context
Add any other context about the problem here.

Is your feature request related to a problem? Please describe.
we are encountering issues with verifying the RISC0 proof on-chain. The challenge is due to W3bstream sending the proof to the smart contract as a single encoded JSON string, making it difficult (and costly) to process and extract the necessary values for the verifier (for RISC0, these include seal, imageId, postStateDigest, and journal).

Describe the solution you'd like
Is there an existing example of verifying the current RISC0 proof as it's sent by W3bstream?

Describe alternatives you've considered
Could it be more efficient for W3bstream to send these fields directly to the Router contract, instead of bundling them into a single encoded string? An alternative approach would be for the WS router contract to offer "submit" calls for different zk proofs (e.g., submitRiscZeroProof(bytes memory seal, bytes32 postStateDigest, bytes calldata journal, bytes calldata optionalPayload)). Depending on the prover used to process a task, W3bstream would then call the respective submit function in the router, simplifying the on-chain proof verification process.

Additional context
Add any other context or screenshots about the feature request here.

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Is your feature request related to a problem? Please describe.
It's not possible to create demos for Solana

Describe the solution you'd like
It would be very helpful to have an initial adapter for sending the proof to a Solana contract to initiate some PoC.

Describe alternatives you've considered
No alternatives

Additional context
None

add unit test for auth module

https://github.com/machinefi/sprout/blob/develop/test/clients/clients

After following the README instructions I found my node is failing for me with the send transaction failed: insufficient funds for gas * price + value error. However, I've used a private key that has funds on the IoTeX testnet (>1800 IOTX).

Can we have some additional log about the CHAIN_ENDPOINT configuration, the actual address the proof is being sent to and estimated gas in the error message?