unwriter / bitd Goto Github PK
View Code? Open in Web Editor NEWDEPRECATED. Replaced with Planaria https://planaria.network
DEPRECATED. Replaced with Planaria https://planaria.network
So it will be easy to read.
I don't think we can reduce data size with JSON. JSON is for human reading.
To reduce data size, we can forward it in protobuf later.
On startup, after going through "getBlockHashes" and then "getTransactionHeaders" up to the last block index, I get the following:
<--- Last few GCs --->
[25975:0x55c2e9434980] 235170 ms: Mark-sweep 1392.5 (1420.5) -> 1392.5 (1420.5) MB, 1170.8 / 0.0 ms (average mu = 0.123, current mu = 0.008) allocation failure scavenge might not succeed
[25975:0x55c2e9434980] 236404 ms: Mark-sweep 1393.4 (1420.5) -> 1393.4 (1421.0) MB, 1230.5 / 0.0 ms (average mu = 0.065, current mu = 0.002) allocation failure scavenge might not succeed
<--- JS stacktrace --->
==== JS stack trace =========================================
0: ExitFrame [pc: 0x12e56ffdc01d]
Security context: 0x29df7e11e549 <JSObject>
1: /* anonymous */ [0xaba2982291] [/.../bitd/bit.js:~191] [pc=0x12e5702b4fd7](this=0x38fd1e31aa09 <JSGlobal Object>,transactionHeader=0x32a71c38d5b9 <Object map = 0x2aedaddbef29>)
2: arguments adaptor frame: 3->1
3: StubFrame [pc: 0x12e56ffa6734]
4: map [0x29df7e106cc1](this=0x19d7aad022d1 <JSArray[7710467]>,0x00aba2982291 ...
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
1: 0x55c2e6ffd101 node::Abort() [node]
2: 0x55c2e6ffd14f [node]
3: 0x55c2e71bed72 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
4: 0x55c2e71befcb v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
5: 0x55c2e7553313 [node]
6: 0x55c2e7553454 [node]
7: 0x55c2e75630d3 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [node]
8: 0x55c2e7563a59 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
9: 0x55c2e7565b6d v8::internal::Heap::AllocateRawWithLigthRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [node]
10: 0x55c2e7565bc2 v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [node]
11: 0x55c2e7533d65 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [node]
12: 0x55c2e77a63ff v8::internal::Runtime_AllocateInNewSpace(int, v8::internal::Object**, v8::internal::Isolate*) [node]
13: 0x12e56ffdc01d
Aborted (core dumped)
I tried running node with --max-old-space-size=8000 option and I got this:
<--- Last few GCs --->
[26048:0x557923a42980] 234875 ms: Mark-sweep 1403.3 (1431.0) -> 1403.4 (1434.5) MB, 1505.2 / 0.0 ms (average mu = 0.210, current mu = 0.008) allocation failure scavenge might not succeed
[26048:0x557923a42980] 236244 ms: Mark-sweep 1407.0 (1434.5) -> 1407.2 (1442.5) MB, 1359.1 / 0.0 ms (average mu = 0.117, current mu = 0.007) allocation failure scavenge might not succeed
<--- JS stacktrace --->
Cannot get stack trace in GC.
FATAL ERROR: MarkCompactCollector: semi-space copy, fallback in old gen Allocation failed - JavaScript heap out of memory
1: 0x55792189a101 node::Abort() [node]
2: 0x55792189a14f [node]
3: 0x557921a5bd72 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
4: 0x557921a5bfcb v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
5: 0x557921df0313 [node]
6: 0x557921e1b478 v8::internal::EvacuateNewSpaceVisitor::Visit(v8::internal::HeapObject*, int) [node]
7: 0x557921e1fc56 void v8::internal::LiveObjectVisitor::VisitBlackObjectsNoFail<v8::internal::EvacuateNewSpaceVisitor, v8::internal::MajorNonAtomicMarkingState>(v8::internal::MemoryChunk*, v8::internal::MajorNonAtomicMarkingState*, v8::internal::EvacuateNewSpaceVisitor*, v8::internal::LiveObjectVisitor::IterationMode) [node]
8: 0x557921e27178 v8::internal::FullEvacuator::RawEvacuatePage(v8::internal::Page*, long*) [node]
9: 0x557921e16ada v8::internal::Evacuator::EvacuatePage(v8::internal::Page*) [node]
10: 0x557921e16e8b v8::internal::PageEvacuationTask::RunInParallel() [node]
11: 0x557921e0fe8f v8::internal::ItemParallelJob::Task::RunInternal() [node]
12: 0x557921e10a6e v8::internal::ItemParallelJob::Run(std::shared_ptr<v8::internal::Counters>) [node]
13: 0x557921e1c4da void v8::internal::MarkCompactCollectorBase::CreateAndExecuteEvacuationTasks<v8::internal::FullEvacuator, v8::internal::MarkCompactCollector>(v8::internal::MarkCompactCollector*, v8::internal::ItemParallelJob*, v8::internal::RecordMigratedSlotVisitor*, v8::internal::MigrationObserver*, long) [node]
14: 0x557921e25a37 v8::internal::MarkCompactCollector::EvacuatePagesInParallel() [node]
15: 0x557921e25be0 v8::internal::MarkCompactCollector::Evacuate() [node]
16: 0x557921e2654b v8::internal::MarkCompactCollector::CollectGarbage() [node]
17: 0x557921dffaa1 v8::internal::Heap::MarkCompact() [node]
18: 0x557921e00113 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [node]
19: 0x557921e00a59 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
20: 0x557921e02b6d v8::internal::Heap::AllocateRawWithLigthRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [node]
21: 0x557921e02bc2 v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [node]
22: 0x557921dd0d65 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [node]
23: 0x5579220433ff v8::internal::Runtime_AllocateInNewSpace(int, v8::internal::Object**, v8::internal::Isolate*) [node]
24: 0xc7c1fdc01d
Aborted (core dumped)
Builded while mongodb missing but could casue it this?
**~/bitdb/bitd# npm start**
> [email protected] start /root/bitdb/bitd
> node index
/root/bitdb/bitd/bit.js:41
let hash = message.toString('hex')
^^^
SyntaxError: Block-scoped declarations (let, const, function, class) not yet supported outside strict mode
at exports.runInThisContext (vm.js:53:16)
at Module._compile (module.js:373:25)
at Object.Module._extensions..js (module.js:416:10)
at Module.load (module.js:343:32)
at Function.Module._load (module.js:300:12)
at Module.require (module.js:353:17)
at require (internal/module.js:12:17)
at Object.<anonymous> (/root/bitdb/bitd/index.js:1:73)
at Module._compile (module.js:409:26)
at Object.Module._extensions..js (module.js:416:10)
npm ERR! Linux 4.15.0-23-generic
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "start"
npm ERR! node v4.5.0
npm ERR! npm v2.15.9
npm ERR! code ELIFECYCLE
npm ERR! [email protected] start: `node index`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] start script 'node index'.
Ubuntu 18.04 unable to install mongodb
In some cases a script is invalid but can exist on the blockchain. For example:
6a04534c500001010747454e455349534c004c004c004c0001004c000800000000000064
when deserialized is:
{ script:
{ b1: 'U0xQAA==',
s1: 'SLP\u0000',
b2: 'AQ==',
s2: '\u0001',
b3: 'R0VORVNJUw==',
s3: 'GENESIS',
b4: '',
s4: '',
b5: '',
s5: '',
b6: '',
s6: '',
b7: '',
s7: '',
b8: 'AA==',
s8: '\u0000',
b9: '',
s9: '',
b10: 'AAAAAAAAZA==',
s10: '\u0000\u0000\u0000\u0000\u0000\u0000d' },
scriptString: <Script: OP_RETURN 4 0x534c5000 1 0x01 7 0x47454e45534953 OP_PUSHDATA1 OP_PUSHDATA1 OP_PUSHDATA1 OP_PUSHDATA1 1 0x00 OP_PUSHDATA1 8 0x00000000000064> }
The last part says "push 8 bytes", but the actual bytes only contain 7 bytes.
Need to look further into this, but two possible solutions could be:
invalid
flag so that it preserves all the information (even if it's invalid)Example:
https://www.blockchain.com/btc/tx/ebc9fa1196a59e192352d76c0f6e73167046b9d37b8302b6bb6968dfd279b767
Reference (how others implement this)
The ZMQ broadcast from Bitcoin is flakey. Implement a solution that works even when ZMQ doesn't work.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.