support zeroconf for single subnet

Would you like any help on this project? If so, maybe we can use GitHub's issue-tracker, so that people don't work on the same thing at the same time.

depending on eventual RPC implementation, it may be advantageous to cache or pool the connections to avoid the initial setup/teardown expense.

Change doozer code to connect to any member of the cluster, and reconnect on failure to another member.

Since Ruby will get a gem, why not a library for Python?

Look at the signal implementation - see if anything can be added for convenience.

Update source code - all code - to use a permissive open source license like MIT, that only requires attribution and has no restrictions for usage.

We need a plan for testing.

1. Generators
2. Generated apps

Generator testing is pretty straightforward. It either generates what we want or not.

Generated app testing is harder. an idea

1. we create a skynet application much like the one in examples - with full tests - and use it as the extraction source for the generators

I'm open to any others. Skynet won't be useful without a suite of good tests, so this is a priority.

Benchmark suite to check operations and discourage performance regressions.

Let's abstract out the doozer calls, so that we do not depend on doozer for testing.

Finish work on Sky command

Admin logic, etc.

Code is generally ugly and needs refactoring, cleanup and full audit on error handling. Very few circumstances should allow a panic, most should handle any error condition gracefully from the end-user's perspective.

Create a detailed test script. Probably using a bash script. Start doozerd, start skynet instances. Test rpc, registration, deregistration, etc.

Publish skynet specification so that we can run nodes in any language that implements the spec and supports the RPC protocol.

Change the generator to be interactive. Instead of a bunch of ugly and hard to remember command line flags, let's use an interactive generator that builds up enough information to make a skynet mesh. This might be the highest bang for the buck right now.

Build a Wiki page on how to get started. Build a wiki page on how to contribute.

Create CI server and publish CI stats.

Add Twister, Web.Go, Resque and Stomp initiator examples

Each RPC service should export statistics on request timing, average, max, min, etc.

We should be able to see timing on each component of a composite service. If there are 10 RPC calls, then we should see timing on all ten as a piece of the composite.

Create more examples - more initiator types, more chained services. Some async, etc.

Create a library to enable timeouts on all RPC calls, configurable at the client.

Service should be marked as re-tryable. if service call fails, try again, up to limit or timeout.

I try to install the skynet and get these errors:

github.com/4ad/doozer

c:\go\src\pkg\github.com\4ad\doozer\conn.go:184: cannot use &t.req (type *reques
t) as type proto.Message in function argument:
*request does not implement proto.Message (missing ProtoMessage method)
c:\go\src\pkg\github.com\4ad\doozer\conn.go:198: cannot use &r (type *response)
as type proto.Message in function argument:
*response does not implement proto.Message (missing ProtoMessage method)

I use the google protobuf 2.4.1. Does the skynet need the lower version of protobuf ?

When a new process starts, it should pick a port to listen on if none is specified.

Add unit testing to Skylib

Create demon that runs on hosts and spawns instances of Skynet services. Should listen for commands on RPC interface on known port.

Create web interface to review logs. Should be searchable in various ways, and should live stream to the screen perhaps via websockets.

I have a problem compiled examples:

skynet/$ gb
(in examples/GetWidgets/initiators/mango) building cmd "mango"
(in examples/GetWidgets/router) building cmd "router"
router.go:15: can't find import: myStartup

By the way, I am using 'release' golang, not the latest, since I had a problem installing all libraries with the latest golang. But mango requires the latest golang, so I had to switch to an early version of mango. I'm curious how you manage to test everything. What versions are you using? If you try goinstall -clean -u on all dependencies, does it still work for you? (If that breaks for you, soon you'll be able to use my fork of mango. I learned that goinstall looks for a release tag if it exists.)

Brian, I think we should keep the README as simple as possible. We can skip the skygen/gb step by showing how to install/run GetWidgets or some other example. We can add tons of comments to that example for clarity.

Skygen can then be delineated on its own Wiki page, in much greater depth. Ok?

Right now, Skynet chooses a simple route randomly. May be very useful to choose routes other ways... perhaps by using a data center metric that gives affinity to closer nodes? More thought required on this.

Doozer is primary and makes most sense - but we should support some other options - via interface, selectable at generation time. Redis is a good choice because it supports push on changes. Others should be supported by implementing the interface.

DzNS doesn't remove downed instances. Harden our doozer discovery implementation

Once DzNS has been setup ensure that we handle BootUri's properly and have environment variables to support them.

Ensure that we handle doozer connection failures, and find a new master if doozer can only write to master.

When doozer instances die and are restarted they loose all information, we need to make sure they reregister themselves.

Export system metrics, load average, cpu, memory for collection by external process.

When specification is more stable, create a ruby gem that enables a ruby node to participate in Skynet.

Create named events based on lifecycle. Request start, complete, etc.

Create interface for event notification.

create implementation for event notification using a pub/sub (topic) message queue.

Copied from issue#19:

I am using 'release' golang, not the latest, since I had a problem installing all libraries with the latest golang. But mango requires the latest golang, so I had to switch to an early version of mango.

We've had a good discussion on goinstall in the golang forum. goinstall will pull from a tag or branch called release if it exists. A release branch is better than a tag because it's easier to move later. I just helped @paulbellamy with that in mango, and it works great.

It sounds a bit complicated, but that's only because golang changes so often. It also means that you can worry less about breaking things on master, since most people will be pulling from release.

Should be a pretty interface to edit routes, drag & drop -- should show a list of the available services.

Investigate creating dynamic request and response objects instead of currently strongly typed request and responses. Generic R & R would simplify a TON of code and allow all of the RPC stuff to be abstracted completely.

Create a screen cast of Skynet. Maybe do this after some of the UI issues are done - would be better impact with a visual dashboard showing nodes.

Start, Stop, Restart, Deregister services from sky command line interface.

EDITED: Change all nodes to support both GOB and JSON-RPC

Consolidate and display all of the fancy exported data that are under debug/expvar. Statistics are awesome - counts, averages, etc. Graphs are always good. Perhaps this is the start of a skynet dashboard.

Create an struct for service info that could be appended to an array of service infos. Then each service in the sky net chain can append metadata about the service request... processing time, node information, etc.

Show topology of the running services:

Datacenters
Hosts
Services by Service Version

At service version level, show metrics on response times.

Embed Graphite style dashboard for response time and request count monitoring, individually by service and aggregated by data center.

Skynet members die when dozer dies ... this is probably not so indestructible.

Implement the functionality of skylib in a Ruby gem.

Create interface for remote logging, and single implementation using mongodb capped collection. Interface should allow any implementation, first impl will be MongoDB capped collection.

Build Log viewer with searching into web UI.

To keep my java friends happy, add a Generator that generates generators of Factories and Factory Implementations.

FactoryImplFactoryGenerator

We have a reaper, but no spawner. Need a spawner (maybe one on each physical host??) that has minimum threshold counts for number of each type of skynet nodes. If threshold is met, new instance of that type of node is spawned. Find a way to do this that doesn't require a ton of configuration.

Much of this code is prototype quality. Refactor extensively. Fix pools, fix redundancy, fix inefficiencies, etc.