We will need to write a test for the AttestationRecord class based on these tests.

As we are converting this repository to typescript (and our other repos) and as we continue to build out the tooling necessary to implement our client, we will see an increase in the usage of packages in this repository. Currently, the repo is small and we have few (if any) outside packages in use and is not much of a concern. However, I think starting a discussion and thinking about what tools we can use to improve managing libraries early is important so that we have less surprises down the road.

I discovered this tool called Lerna that is being used for js-libp2p for managing JS projects with multiple packages.

Any thoughts?

CC'ing @GregTheGreek, @priom, @ansermino

Is your feature request related to a problem? Please describe.
By creating the startup functionality we will be able to begin testing the whole system as the pieces come together.

Describe the solution you'd like
Realistically, some functionalities will need to be stubbed, as we are still having some type related issues (primarily bytes #59).

How to
I am working on creating a start function (i.e the main entry point for the beacon chain) that will add an event listener to the deposit contract on ETH1.0. I'll have to get the old logs from the contract, and the continue to look for the new contracts. When the ChainStart log is emitted then my function will need to execute the get_initial_beacon_state function (not implemented, but I will have it implemented for my PR). At this point we will have the base state for the beacon chain and to avoid submitting an extensive pr i will stop at this point.

Is your feature request related to a problem? Please describe.
The current state from #71 and #73 would require us to pass around the state object on functions that mutate state which could become a bit difficult. If we switched to a class based approach we could access and manage state much easier.

Describe the solution you'd like
Ideally, if we have something like class BeaconChain {} then we can get acccess to state via this.state and guarantee we're mutating the correct state (you can imagine, if we have async and non async actions flying around then theres a chance we would mutate state in the incorrect order). That being said the actual startup functionality should probably be changed over to a command line argument based setup.

EDIT: expanded on the correct one.

In addition to #40 we should set up issue, feature request and PR templates for the repo. Would also be good to setup a contributors guidelines.

We need to complete tests before starting on the state transition functions. This will enable use to set up a flow for tests in this repo. Currently, there are (as of this writing) 2 helper functions that have tests.

The current library in the repo only supports 32 bytes and the spec requires a 64 bytes output

The following library seems to support 64 bytes and is in WASM :) https://github.com/nazar-pc/blake2.wasm

Hi @Mikerah, every time I am looking at this repo, I am irritated by the "alternative" parts in the headlines, as it suggests to me that you are doing an alternative implementation in the sense of "an alternative/different approach" - similar to the Casper FFG vs CBC research directions.

I assume that is not what you are intending and you just try to implement the (evolving) specifications in JavaScript (or do you want to indeed follow different paths than e.g. done in the Prysmatic Labs repo)?

Then I would suggest to drop the "alternative" in the header title and the initial section of the README and just state "provide an implementation of the beacon chain in JavaScript".

Cheers
Holger

Issue

getEmptyBitfield in bitfield.js is return an incorrect value. The following code returns strings like "x00x00x00x00x00".
https://github.com/ChainSafeSystems/lodestar_chain/blob/a270c7a577a3deda88f96cebb41bc3f173e26b1d/lodestar_chain/utils/bitfield.js#L19
The intention here is to initialize a byte array or some sort of bytes representation with the correct number of 0 bytes.

Proposed Implementation

Not sure the best way to represent bytes in javascript. Will need to dig around for best practices on this.

Due to the recent changes made to the states in the beacon chain, we will need to update the beacon block implementation. Here's a link: https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#beacon-chain-blocks

We need to find proper typescript types for (or write them ourselves):

• [ x ] uint's
• [ x ] bits
• [ x ] hash32

David's library should do the trick for uints. As for hash32 bits and bytes unsure.

In order to increase readability, minimize errors and onboarding outside contributions, we need to set up tools that can fix and check the formatting of the code that gets pushed to this repo. We can easily set up tslint for now but in the future, we should aim to have a tool like standard.js for typescript.

Write tests for the CrystallizedState class based on these tests.

Since we wish to both use the Node Buffer type and attempt to maintain in browser compatibility we must make sure we have a method for achieving both these goals. It appears that if we use browserify and this library: https://github.com/feross/buffer we should be able to achieve this with little difficulty.

Is your feature request related to a problem? Please describe.
Currently there is no standardized way to log anything.

Describe the solution you'd like
We should implement a custom logger.

While implementing ssz in JS, I have been representing bytes as arrays. Currently, in this repository, bytes are represented using strings.

I would like to start a discussion as to what is a good way to represent bytes so that while implementing both ssz and the beacon chain spec, we are consistent.

There seems to be some good discussion around this in the ethereumjs organization, as well as other JS blockchain projects. There is an additional barrier of entry, but it would seem that the benefits are worth it. We would have a huge advantage by making the switch now, although it would certainly could be reconsidered further down the road.

ethereumjs/organization#28
ethereumjs/ethereumjs-client#10
https://www.typescriptlang.org/docs/handbook/migrating-from-javascript.html

Is your feature request related to a problem? Please describe.
In order to keep track of the state we should add support for a DB.

Describe the solution you'd like
One option would be to implement level-js which is a key-value pairing that is an implementation of the google leveldb.

EDIT: Clarified this is a discussion.

Per @wemeetagain:

This may be a good opportunity to remove the uint/bytes duplicate types/vars from types/block.ts and types/state.ts and import/export these instead.
[...] it would be nice to import/export constants and enums from constants/index.ts like we're now doing with types/index.ts.

TODO:

• Consolidate and Remove duplicate alias types (probably move into constants) (PR: #91 )
• Import / Export constants/enums

@wemeetagain Assigned, you because im not sure if you already started or not, otherwise happy to take it on myself.

We have yet to convert the blake.js and bitfield.js files to typescript. This task is pretty straightforward.

The main beacon chain reference in Python makes heavy use of the py_ecc library that was written by Vitalik Buterian and Piper Merriam. Javascript doesn't have a robust library like this. The closest thing I found was rustbn.js but it's still not enough.

I am currently undertaking research on whether we should just implement this from scratch in Javascript or using WebAssembly/asm.js, take py_ecc and compile it to Javascript.

As was pointed out in #7, the repository currently contains code from previous specifications of the beacon chain.

We will need to compare and contrast what is in this repository, what is in the current specification and what is implemented in the reference python beacon chain.

@Mikerah and I had a chat with the team at ethereum-js/Shasper-js on moving forward and/or any collaboration. We came to the conclusion to migrate to typescript as per #24 and look into mimicking project structures from the ethereum-js client. Rational behind this is so contributors in the ecosystem should be able to easily navigate the javascript client codebases.

A simple test to validate that the runtime variables actually match up with the compile time interfaces.

The shuffle() helper function has been started but not finished:

https://github.com/ChainSafeSystems/lodestar_chain/blob/bbc223926df7d20e0fa6875cad051a4cfe790719/beaconChain/helpers/stateTransitionHelpers.ts#L25-L57

@GregTheGreek whats the blocker on this?

Related to #35
Is your feature request related to a problem? Please describe.
We are somewhat restricted in regards to what we can do with bytes.

Describe the solution you'd like
Ideally something native to vanilla JS, C/C++ bridge or WASM.

Describe alternatives you've considered
Webpack so we can use class Buffer() from node. Not ideal.

Write tests for the ShardAndCommittee class based on these tests.

We need to make sure that the blake2b hash function that is in use in the repository is consistent with the other implementations.

Issue

aggregateVote.numAggregateSig currently incorrectly uses the length of aggregate_sig to get the number of aggregate signatures.
https://github.com/ChainSafeSystems/lodestar_chain/blob/a270c7a577a3deda88f96cebb41bc3f173e26b1d/lodestar_chain/state/aggregateVote.js#L36-L38

aggregate_sig is in fact always just the length of 2 so this fails.

Proposed Implementation

You can use the number of 1s in the bitfield to get the length. If you need the length often, you might consider some sort of helper attribute that caches this results.

Is your feature request related to a problem? Please describe.
Licence is arguably invalid without proper instruction.

Describe the solution you'd like
Add GPL3 headers to every source code file.

Describe alternatives you've considered
N/A

Additional context
Suggested template:

Lodestar Chain
Copyright (C) 2019 ChainSafe Systems

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.

This isn't much of an issue at the moment, but any dependencies that we will be adding to this repo will need to be updated regularly. The Prysmatic Labs team uses Renovate to keep their packages up to date. The main issue I have with this is that it may break the code due to API changes,etc of a particular package.

Should we manually update packages in order to easily spot code breaks or should we use something like Renovate to automatically do it for us?

CC'ing @GregTheGreek, @priom, @ansermino

We need to write tests for the ActiveState of the beacon chain.
The tests will mostly follow the core devs's tests.

Write tests for the block class that will be a block of the beacon chain based on these tests.

The state transition function is effectively the core of the beacon chain spec. It is found here: https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#beacon-chain-processing. Its implementation is dependent on #29.

Consider using the LGPL.

This would protect derivative versions of the beacon chain under LGPL, but allow it to be imported/used as a library without forcing the linking project to be GPL.

While converting the code to typescript I will also be checking to ensure that we are still up-to-date with the current spec.

EDIT :: 25/11/18 - Since the spec has been changed quite a bit (noted by: @Mikerah & @ansermino) the primary focus will be to ensure that the spec is up-to-date as stated above, while also converting to typescript.

Note:
- I will be working alphabetically (descending) through core/state.
- Checked items are currently being worked on in the branch greg/typescript-conversion.

Currently working on:

- [x] Interfaces :: A collection of all the eth2.0 data structures

Completed:
none

Just going through the spec and re-implementing ground up!

Before implementing the state transition function, the helper functions that I badly implemented need to be updated and tested. It will need to be compatible with any choices made with regards to using buffers/bignumber.js and arrays and work in the browser.

This is apart of our bi-weekly spec update.

I've update the spec fork here and we can use this as our "source of truth" for verifying PR's etc... Unless major breaking changes occur then we should only reference our fork for consistency.

Note: Per last call -- they are only updating the spec on a weekly basis.

I've just cloned the repository, installed the dependencies and I'm getting this error when I run npm test:

[lodestar_chain](master)$ npm test

> [email protected] test /home/hugo/workspace/Sharding/lodestar_chain
> mocha --recursive

/home/hugo/workspace/Sharding/lodestar_chain/test/test_validator_registration.js:1
(function (exports, require, module, __filename, __dirname) { var ValidatorRegistration = artifacts.require('ValidatorRegistration');
                                                                                                    ^

ReferenceError: artifacts is not defined
    at Object.<anonymous> (/home/hugo/workspace/Sharding/lodestar_chain/test/test_validator_registration.js:1:101)
    at Module._compile (module.js:624:30)
    at Object.Module._extensions..js (module.js:635:10)
    at Module.load (module.js:545:32)
    at tryModuleLoad (module.js:508:12)
    at Function.Module._load (module.js:500:3)
    at Module.require (module.js:568:17)
    at require (internal/module.js:11:18)
    at /home/hugo/workspace/Sharding/lodestar_chain/node_modules/mocha/lib/mocha.js:250:27
    at Array.forEach (<anonymous>)
    at Mocha.loadFiles (/home/hugo/workspace/Sharding/lodestar_chain/node_modules/mocha/lib/mocha.js:247:14)
    at Mocha.run (/home/hugo/workspace/Sharding/lodestar_chain/node_modules/mocha/lib/mocha.js:576:10)
    at Object.<anonymous> (/home/hugo/workspace/Sharding/lodestar_chain/node_modules/mocha/bin/_mocha:637:18)
    at Module._compile (module.js:624:30)
    at Object.Module._extensions..js (module.js:635:10)
    at Module.load (module.js:545:32)
    at tryModuleLoad (module.js:508:12)
    at Function.Module._load (module.js:500:3)
    at Function.Module.runMain (module.js:665:10)
    at startup (bootstrap_node.js:187:16)
    at bootstrap_node.js:607:3
npm ERR! Test failed.  See above for more details. 

I couldn't find information about this error which is related to mocha, all the information I found is related to truffle.

Write tests for the CrossLinkRecord class based on these tests.

In order to simplify the protocol, a recent change was made to the specification to combine both the active state and crystallized state into a single state.

We will need to remove the active state and crystallized state classes and make a new beacon state class. Here's the reference in the spec: https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#beacon-chain-state

Development of ssz-js (https://github.com/ChainSafeSystems/ssz-js) is underway with the help of Darren Langley from Rocket Pool.

After the ssz is mostly done with tests and submitted to npm so that everyone can easily use it, we need to integrate it into this repository.

In Prague, there was a working group that was charged with standardizing testing clients in ETH2.0. Prysmatic Labs has just posted an uncompleted spec of this in their repo: https://github.com/prysmaticlabs/prysm/tree/master/beacon-chain/chaintest

We need to work on implementing these tests into Lodestar before the project gets to big.

Unfortunately, typescript interfaces do not appear to be available/exist at run time.
This means that we don't have access to the type information necessary to properly serialize the data.

There is a dependency here, between our "types" and our ssz library that we should be aware of.
Our ssz library assumes a type object with a fields key, which it uses to serialize each field. (See here)

Here is an example that can show the issue:

interface ProposalSignedData {
  // Slot number
  slot: uint64;
  // Shard number (`BEACON_CHAIN_SHARD_NUMBER` for beacon chain)
  shard: uint64;
  // Block hash
  blockHash: hash32;
}

const data: ProposalSignedData = {
  slot: 0,
  shard: 0,
  blockHash: 0,
};

const serializedData: Uint8Array = ssz.serialize(data, ???); // <--- problem here

Perhaps a way forward would be to create a "type" object for each interface.
Something that would play nicely with ssz-js:

const ProposalSignedData = {
  fields: [
    ['slot', 'uint64'],
    ['shard', 'uint64'],
    ['blockHash', 'hash32'],
  ],
};

// from above, problem fixed
const serializedData: Uint8Array = ssz.serialize(data, ProposalSignedData);

See also how the python implementation deals with this, eg: here

I browsed through the spec/our code, and found a few helper functions that are not yet implemented. I won't get to these for a bit as I'm currently working on writing further tests.

If you plan to work on one of the following please comment.

(Does not include validator helpers)

• hash
• hashTreeRoot
• slotToEpoch
• getPreviousEpoch
• getCurrentEpoch
• getEpochStartSlot
• isActiveValidator
• getActiveValidatorIndices
• getPermutedIndex
• split
• getEpochCommitteeCount
• getShuffling
• getPreviousEpochCommitteeCount
• getCurrentEpochCommitteeCount
• getNextEpochCommitteeCount
• getCrosslinkCommittessAtSlot
• getBlockRoot
• getRandaoMix
• getActiveIndexRoot
• generateRoot
• getBeaconProposerIndex
• merkleRoot
• getAttestationParticipants
• isPowerOfTwo
• intToBytes1, intToBytes2... (to be handled by Buffer)
• bytesToInt (to be handled by Buffer)
• getEffectiveBalance
• getTotalBalance
• getForkVersion
• getDomain
• getBitfieldBit
• verifyBitfield
• verifySlashableAttestation
• isDoubleVote
• isSurroundVote
• integerSquareRoot (intSqrt)
• getEntryExitEffectEpoch
• blsVerify
• blsVerifyMultiple
• blsAggregatePubkeys
• validateProofOfPossession
• processDeposit

Write tests for the RecentProposerRecord class based on these tests.

We should get a basic CI going, probably just to lint right now. Should enforce checks for PRs.

We have a few different milestones we can start assigning to this project.

I was thinking of having 2 for now:

• Beacon Chain POC
• Project Structure Cleanup (relation to #24 and #31)

It seems like the Validator Registration Contract is in this repo to remain consistent with the VMC contract defined in the current Beacon Chain specification. If this is the case, should it be updated to account for new contract fields?