blocknetdx / blocknet Goto Github PK
View Code? Open in Web Editor NEWOfficial Blocknet cryptocurrency wallet
Home Page: https://www.blocknet.org
License: MIT License
Official Blocknet cryptocurrency wallet
Home Page: https://www.blocknet.org
License: MIT License
Cloned from CORE-104
Scenario:
The XBridge protocol currently uses a mix of (float)
, double
, int64_t
and uint64_t
and as such we are seeing precision issues eg when partial orders are created (ref: #570)
Proposed solution:
Replace all uses of (float)
, double
, int64_t
and uint64_t
with a 'signed integer' type with 256 bits for all amounts (int256_t
), enabling XBridge support for tokens with 18 decimal places (being similar to what ETH uses (uint256
)). Further, using a signed type, as opposed to unsigned, would also prevent underflows in the event of a negative amount that results from a calculation.
To do:
int256_t
throughout the XBridge codeQA and Acceptance info to follow...
Add Segwit address support to core protocols.
To do:
I was tinkering with setting up the dex, when I noticed the program can rewrite the .conf files of the associated wallets (rpc-credentials). This also causes the wallets to become unresponsive, as they no longer accept commands from the -cli with pre-rewritten .conf info. But just the rewriting of the conf file seems like a big security flaw to me, but maybe I'm wrong.
Scenario: requesting xrGetBlockCount
via core console or cli returns unreliable results.
Issue:
There seems to be an upper limit on the queryCount parameter - possibly if the count is higher than the amount of nodes registered on the network, the query fails. Eg:
xrgetblockcount BLOCK 10
{
“reply”: 2244915,
“uuid”: “08ae7d53-49b7-498a-8d3d-9b3ece4d8a03”
}
xrgetblockcount BLOCK 100
{
“code”: 1032,
“error”: “Failed to find 100 service node(s) supporting BLOCK::xrGetBlockCount with config limits, found 0”,
“uuid”: “1933a5d2-f023-42d1-b154-8ed469d1a407”
}
To do:
To do:
Write XB unit tests, using https://github.com/blocknetdx/blocknet/tree/master/test, and implementing:
--- service node logic
--- connector logic
--- xb logic and trading tests
more info to follow...
The ServiceNodePing
(snp) message currently contains a lot of duplicate information. The entire service node object snode
is being serialized and broadcasted.
Current ServiceNodePing
structure:
CPubKey snodePubKey;
uint32_t bestBlock{0};
uint256 bestBlockHash;
uint32_t pingTime{0};
std::string config;
ServiceNode snode;
std::vector<unsigned char> signature;
Additionally, the entire service node config is broadcasted with every ping, even though it rarely changes.
Here three things a proposed:
Notify clients when order could not be created due to bad utxos.
Possibly requires new error code crBadUtxo
.
logs and more info to follow
Latest QA build: https://gitlab.com/blocknetdx/blocknet/-/pipelines/338714210
Incorrect balance shown in wallet.
Balance shown in GUI wallet, or daemon wallet via blocknet-cli should be correct.
Balance shown is incorrect and this does not appear to be the same as the "staking glitch" which can be (temporarily) fixed by restarting the wallet with zapwallettxes.
Balance shown by summing wallet addresses in explorer is correct. Balance shown by summing amounts in listunspent
output is correct, but the "missing" coins are shown with spendable: false, solvable: false
. The "missing" UTXOs were change outputs from regular txns.
Problem is not corrected by restarting with zapwallettxes, nor reindex/rescan. Attempts to spend the "missing" UTXOs by createrawtransaction, signrawtransaction result in an error message saying the wallet is unlocked for staking only. Yet the wallet is fully unlocked.
Blocknet 4.3.1, MX Linux 18.3, Intel core i5, 16GB RAM, SSD.
Scenario:
Right now fee's on BTC and ETH are dynamic, so having a hard-coded value in the config, like we currently have, doesn't work.
Instead, Service Nodes can query the blockchain and get the current fee dynamically, so the counter-parties can do the same and have proper fee values at time of trade execution.
Info: https://developer.bitcoin.org/reference/rpc/estimatesmartfee.html
To do:
dynamic
or static
How to test:
estimatesmartfee
eg. block/ltc, block/btc, ltc/btc and review tx details, noting fees, and check logs for fee methodQA pass:
to be confirmed…
estimatesmartfee
work as expected when fee is calculateddynamic
or static
QA build: https://gitlab.com/blocknetdx/blocknet/-/pipelines/368022475
dxGetOrders always returns null list when run on a servicenode
This behaviour is the same regardless of any dxnowallets or ShowAllOrders settings.
It would be helpful if it returned the full list of all open orders in the network.
Reproduces every time and very easily: on a servicenode run
blocknet-cli dxGetOrders
The result is always an empty list
[
]
Blocknet RPC client version v4.3.3.0-698e09fea on Ubuntu 18.04.
Describe the issue
When dxgetorderbook is called with detail level 2, the returned asks and bids array is filtered. Not all orders (as seen in detail level 3) with a different price are displayed. In my example, there was no pair in asks or bids that had the same price. Yet in detail level 2 some of these orders are missing, compared with detail level 3.
What behavior did you expect?
With detail level 2 I expect the same amount of bids and asks all with aggregation value 1 as detail level 3, if there is no pair that has the same price (on both bids and asks side).
What was the actual behavior (provide screenshots if the issue is GUI-related)?
Detail level 2
dxgetorderbook 2 BLOCK LTC

{
"detail": 2,
"maker": "BLOCK",
"taker": "LTC",
"asks": [
[
"0.045455",
"1.000000",
1
],
[
"0.039859",
"1.000000",
1
],
[
"0.038073",
"1.000000",
1
],
[
"0.037296",
"1.000000",
1
],
[
"0.034536",
"1.000000",
1
],
[
"0.032872",
"1.000000",
1
],
[
"0.029898",
"1.000000",
1
],
[
"0.024221",
"1.000000",
1
],
[
"0.021658",
"455.543029",
1
],
[
"0.021494",
"427.538326",
1
],
[
"0.021246",
"384.316901",
1
],
[
"0.020982",
"1.000000",
1
],
[
"0.020500",
"253.679699",
1
],
[
"0.020258",
"207.207289",
1
],
[
"0.017805",
"1.000000",
1
]
],
"bids": [
[
"0.020819",
"1.000000",
1
],
[
"0.016481",
"1.000000",
1
]
]
}
Detail level 3
dxgetorderbook 3 BLOCK LTC
{
"detail": 3,
"maker": "BLOCK",
"taker": "LTC",
"asks": [
[
"0.045455",
"1.000000",
"64b8b2abae4217b9f46f509998425d29c3dc94fa8a73b77a24da001739b2f756"
],
[
"0.040000",
"1.000000",
"adbf3968911f9932072f56898ae6690d5305248a1afb7395af6ce9f18b631c62"
],
[
"0.039859",
"1.000000",
"2a2a3f674db351fb9fe24e4c20ecc0bf5509e4cab8b22efa935449f0857846f4"
],
[
"0.038649",
"1.000000",
"42d6aad5bc61772a171c3590701b6f4a8c693219f10de9e0e1f4704d1597f332"
],
[
"0.038073",
"1.000000",
"e348bd187bc91a510218a40b28c805c408aec839e44ac76e0a71980af5fb1bf1"
],
[
"0.037511",
"1.000000",
"2a647d82058c18a5b2de4307c82b3cd4328b64fb288cd0ac7d61aeb533f7438f"
],
[
"0.037296",
"1.000000",
"8e8d4222077e4ed0f783473b89cbfa063e6f8037e412c4621047c35d44f3c071"
],
[
"0.035714",
"1.000000",
"b868bf58cab7906b4a8313e856a834f40a7a34f7984980527fda5761cc3be5ad"
],
[
"0.034536",
"1.000000",
"c36a603298ffdbf54a28a916b316f1686ab68ec51903ca674c86fcf9b9279c9b"
],
[
"0.033102",
"1.000000",
"8c5541704edf2712248b5c28482398d20d25d5262b79b36971d5a9d51a47b7d9"
],
[
"0.032872",
"1.000000",
"2c07255329412bc8062f99bb9749987423f1670c8ef564095d72e96cb20bd7ba"
],
[
"0.031858",
"1.000000",
"d6f6acf2a3018dc8ab6fdbbbf3f5c3efe6ac987ded21fe145588c4aaad86f1e1"
],
[
"0.029898",
"1.000000",
"7ce210ecf772d2824788b2cd983d45b5be9014d04fd3a08042b11c956476af0b"
],
[
"0.028010",
"1.000000",
"c6d8a11fe73e7146fd798835d0e348253084f8d490f3e6010fa6cdc3ddeda91f"
],
[
"0.024221",
"1.000000",
"a1ada389497e3748073b9fe733d462e05257dbf7503d3f8d3f27bbf7a52ba1ff"
],
[
"0.022569",
"1.000000",
"718bd855b9ca57ddb717a5937f5ed25bbde02bfb6a0d8f1bdfe4354f17374cf7"
],
[
"0.021658",
"455.543029",
"7fa232b6af5c02787a6873c516dbb7a3f13b74109c38ab9492a4bbec11358568"
],
[
"0.021494",
"427.538326",
"2cfd617cc55f07b4180d01d968d6ef7577eee35b9dd44722ebabfca04d96e37f"
],
[
"0.021438",
"1.000000",
"712d9746bcae6db1e0055990cb5d44a9efdd4be43af307af2cd503955f065d2e"
],
[
"0.021246",
"384.316901",
"fd879133ac9a4f2027f2436ceab58301f0c1521671db403283953eb76dc464f1"
],
[
"0.021143",
"366.153318",
"5d8066358b779fc8d5f67c6025b1a21357e1f71d6e81bcc49f41727cd0cb263a"
],
[
"0.020982",
"1.000000",
"a894707b1827b52fa8349d5394ebba990afc97e44fc05310fc1a8d293e1b7a82"
],
[
"0.020649",
"280.667394",
"f554b10b4eef86248fc3decac617a0e2f292bf4d25c072c7709f11f77d3ef166"
],
[
"0.020500",
"253.679699",
"553bf72a4ca23648cbb6b1ac72dfdc4651c8a732e243ac7f997abb1824dd6462"
],
[
"0.020362",
"230.867345",
"dab726525a2955478fc197beb41f01eb40d62c2df49b2b87bb375060e52861f9"
],
[
"0.020258",
"207.207289",
"d99101892a6ce61c9884e09a0cfe2c58cbd94ed4e73750e324ffa02f3bdca532"
],
[
"0.019646",
"105.458209",
"d7d7d93dfc57538df9ba97be453a456354b8357035b7d4311504aa6e9d82bc48"
],
[
"0.019241",
"35.185000",
"ffd76a8fb20ee13600b79ea5b97302521be55f13df63de0a7483b64d9a211555"
],
[
"0.017805",
"1.000000",
"8bc13e66f53e81c24825eecab132245b730d7d891c0c04b522f818bf2028aecd"
]
],
"bids": [
[
"0.020819",
"1.000000",
"bb5dd3256bf012f244d7cb8c91268664d17c06b7b5d7d4c0259165b353e860d2"
],
[
"0.016497",
"1.000000",
"23c422d0f255cf4be33a25c221f912d714a0b39c104d815f73060ada72306c96"
],
[
"0.016481",
"1.000000",
"2fcc97badbc3c70c1e70ed9acd9da9229e37b394add7e1cb953d3b5593608c2a"
],
[
"0.012500",
"15.200000",
"88281d7ba25ffacd53ff3aadaf1fe335bfa5185436cca04d98d1c72bc86165af"
]
]
}
Blocknet version
4.3.1
What type of machine are you observing the error on (OS/CPU and disk type)?
Windows 10 - 10.0.19041 Build 19041
STR:
dxmakepartialorder
(see https://api.blocknet.co/#dxmakepartialorder)created
in consoledxgetorder
with the order ID created in step 2 to verify order statusError result: XBridge error: order may be stuck, trying to submit order to previous snode
The error keeps looping and never seems to find a snode to submit the order to.
Expected behaviour: the order should be placed once xbridge is reconnected to the network, transitioning from new
to created
.
To do:
2021-Jul-14 14:16:35 [0x0x309a9b000] {"orderid":"38cc1fb72917327bf45ad7c6eebe994d8c353ed3b611b58aef8a0df5ebad3904","from_currency":"LTC","to_currency":"BLOCK","function":"checkAndRelayPendingOrders","msg":"order may be stuck, trying to submit order to previous snode"}
A bug occurs after voting for Superblock proposals where a UTXO (a certain amount of total balance) disappears from the wallet.
Fixing requires -zapwallettxes=2 to recover missing UTXO(s).
Steps to reproduce:
More logs to follow asap…
Scenario:
When dxMakePartialOrder
or dxMakeOrder
are called, there is currently no way to control where funds come from in the order and there is no way to prevent XBridge from autosplitting UTXOs on calls to dxMakePartialOrder
.
The use_all_funds
param is being added to allow for proper segregation and accounting of funds within a wallet, which in turn allows a trading bot to trade multiple pairs more conveniently.
To do:
use_all_funds
parameter to dxMakePartialOrder
true
(default) - dxMakePartialOrder uses funds from all available addressesfalse
- dxMakePartialOrder will try to use funds only from the address specified by the maker_address parameter.dxMakeOrder
true
(default) - dxMakeOrder
uses funds from all available addressesfalse
- dxMakeOrder
will try to use funds only from the address specified by the maker_address
parameter.use_all_funds = false
, XBridge needs to check if there are sufficient funds in the address as follows:use_all_funds
option = false, when an order is created by calling dxMakePartialOrder
or dxMakeOrderand
XBridge verifies there are not enough funds available in maker_address
to make the order, then return the error insufficient_funds
auto_split
parameter to dxMakePartialOrder
Important: no error log is needed when this parameter is set to false, and setting this parameter to false should not affect whether or not the order is successfully placed.
Goal: Prevent paying extra fees and incurring delays due to autosplitting UTXOs when using trade bots.
Currently a majority of log message print the wrong expected tx state.
Example:
if (xtx->state >= TransactionDescr::trInitialized)
{
UniValue log_obj(UniValue::VOBJ);
log_obj.pushKV("orderid", txid.GetHex());
log_obj.pushKV("state", xtx->state);
LogOrderMsg(log_obj, "wrong tx state, expecting initialized state", __FUNCTION__);
return true;
}
Check is xtx->state >= TransactionDescr::trCommited
, but the message should read "wrong tx state, expecting hold state".
Service nodes are the corner stones of the Blocknet network as they provide all sorts of services. However service nodes (and to some extend the service node list) aren't all that reliable. Therefore, I propose to hard harden the following requirements:
There is an intermittent bug with xbridge order posting that results in an 'invalid amount' error (see screenshot attached)
We need to find the route of this issue but the logs don't provide enough info to track it down. Currently we don't have the exact steps to reproduce but the following process can result in the error:
Steps to reproduce:
Result: error 1026 - invalid amount
Important note: this error is intermittent as you can see from the screenshots attached. Sometimes the same order works and others it fails.
To do:
dxmakepartialorder
is calledthe maximum number of utxos on the order was exceeded (UTXOs empty):
if (utxos.empty()) {...
xbridge::LogOrderMsg(log_obj, "failed to create order, the maximum number of utxos on the order was exceeded", __FUNCTION__);
return xbridge::Error::UTXOS_EMPTY;
}
insufficient funds on partial order:
xbridge::LogOrderMsg(log_obj, "failed to create order, insufficient funds on partial order", __FUNCTION__);
return xbridge::Error::INSUFFICIENT_FUNDS_PARTIAL_ORDER;
the maximum number of utxos on the order was exceeded:
if (ptr->usedCoins.size() > xBridgePartialOrderMaxUtxos) {...
xbridge::LogOrderMsg(log_obj, "failed to create order, the maximum number of utxos on the order was exceeded", __FUNCTION__);
return xbridge::Error::EXCEEDED_MAX_UTXOS;
case Error::UTXOS_EMPTY:
return "Failed to create order, the maximum number of utxos on the order was exceeded " + argument;
case Error::INSUFFICIENT_FUNDS_PARTIAL_ORDER:
return "failed to create order, insufficient funds on partial order " + argument;
case Error::EXCEEDED_MAX_UTXOS:
return "failed to create order, the maximum number of utxos on the order was exceeded " + argument;
Here's the issue content.
--header 'Content-Type: application/json' \
--header 'Authorization: Basic QmxvY2tEWEJsb2NrbmV0OjBmZjg4YTI4LWZjZTktNDc3MC1hMzFlLWE1ZTNlMWZhYzNiYg==' \
--data-raw '{
"method": "xrGetTransaction",
"params": [
"DASH",
"8efdea07690732d83b4b0e465a9616f6c62a64bb2ca0644d7dbbb93f7181cace",
5
],
"id": "dummyId"
}'
"result": {
"reply": "",
"uuid": "7C8D2D3E-4228-4565-B1C9-06E5F91C4988"
},
"error": null,
"id": "dummyId"
}
What is now latest update btc 0.18.0 at what bassed blocknet ? hv any plans to update to 0.19+?
QT wallet crashed with
Assertion failed: detected inconsistent lock order at sync.cpp:108, details in debug log
shortly after loading a different wallet from the debug console.
tail debug.log showed
2021-11-17T23:02:46Z Using wallet /home/mark/.blocknet/wallets/voting
2021-11-17T23:02:46Z BerkeleyEnvironment::Open: LogDir=/home/mark/.blocknet/wallets/voting/database ErrorFile=/home/mark/.blocknet/wallets/voting/db.log
2021-11-17T23:02:47Z init message: Loading wallet...
2021-11-17T23:02:54Z UpdateTip: new best=687795fd4b778b1896e8fe8e2a54a69c8dbad89845742518bd0a7e3f6eaf8d80 height=2243935 version=0x20000000 log2_work=66.805745 tx=4906931 date='2021-11-17T23:00:24Z' progress=1.000000 cache=0.0MiB(4txo)
2021-11-17T23:02:54Z BerkeleyEnvironment::Open: LogDir=/home/mark/.blocknet/wallets/voting/database ErrorFile=/home/mark/.blocknet/wallets/voting/db.log
2021-11-17T23:02:54Z [voting] nFileVersion = 4030300
2021-11-17T23:02:54Z [voting] Keys: 2 plaintext, 0 encrypted, 2 w/ metadata, 2 total. Unknown wallet records: 0
2021-11-17T23:02:54Z [voting] Wallet completed loading in 7115ms
2021-11-17T23:02:54Z UpdateTip: new best=c8c0f182de368c28091f1006854c4c4aa0bc7b3aa56c88f13402bdc75eef6a52 height=2243936 version=0x20000000 log2_work=66.805745 tx=4906934 date='2021-11-17T23:01:29Z' progress=1.000000 cache=0.0MiB(11txo)
2021-11-17T23:02:54Z init message: Rescanning...
2021-11-17T23:02:54Z [voting] Rescanning last 49072 blocks (from block 2194864)...
2021-11-17T23:02:54Z [voting] Rescan started from block 4690b938480a9dd945afd2aac16d826aeabbb1d987baa8f776f54c5d6c928643...
2021-11-17T23:02:54Z [voting] AddToWallet xxxxxx new
...
2021-11-17T23:02:57Z [voting] AddToWallet xxxxxx new
2021-11-17T23:02:57Z [voting] Rescan completed in 3665ms
2021-11-17T23:02:58Z [voting] setKeyPool.size() = 0
2021-11-17T23:02:58Z [voting] mapWallet.size() = 5697
2021-11-17T23:02:58Z [voting] mapAddressBook.size() = 6
2021-11-17T23:02:58Z UpdateTip: new best=5d3f5e34ab9aa0003bc11d2c9019f52371fed46cbba4c2d25402e70842ca9666 height=2243937 version=0x20000000 log2_work=66.805746 tx=4906936 date='2021-11-17T23:02:34Z' progress=1.000000 cache=0.0MiB(15txo)
2021-11-17T23:02:58Z New outbound peer connected: version: 70713, blocks=2243937, peer=2
2021-11-17T23:02:59Z POTENTIAL DEADLOCK DETECTED
2021-11-17T23:02:59Z Previous lock order was:
2021-11-17T23:02:59Z (1) cachedWalletMutex qt/transactiontablemodel.cpp:185
2021-11-17T23:02:59Z (2) cs_main interfaces/chain.cpp:148 (TRY)
2021-11-17T23:02:59Z Current lock order is:
2021-11-17T23:02:59Z (2) cs_main interfaces/chain.cpp:148
2021-11-17T23:02:59Z walletInstance->cs_wallet wallet/wallet.cpp:4647
2021-11-17T23:02:59Z (1) cachedWalletMutex qt/transactiontablemodel.cpp:81
wallets/voting/db.log was empty
core dump was produced, gdb showed
Core was generated by `blocknet-qt'.
Program terminated with signal SIGABRT, Aborted.
#0 0x00007fb0f10e07bb in raise () from /lib/x86_64-linux-gnu/libc.so.6
[Current thread is 1 (Thread 0x7fafba4c6700 (LWP 28151))]
(gdb) bt
#0 0x00007fb0f10e07bb in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007fb0f10cb535 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x00005555da4ffdcb in potential_deadlock_detected (mismatch={...},
mismatch={...}, s2=..., s1=...) at sync.cpp:109
#3 push_lock (locklocation=..., c=<optimized out>) at sync.cpp:133
#4 EnterCritical (pszName=pszName@entry=0x5555dccd578c "cs_main",
pszFile=pszFile@entry=0x5555dcd57ba9 "interfaces/chain.cpp",
nLine=nLine@entry=148, cs=<optimized out>, fTry=fTry@entry=true)
at sync.cpp:144
#5 0x00005555da9e9b87 in UniqueLock<AnnotatedMixin<std::recursive_mutex>, std::unique_lock<std::recursive_mutex> >::TryEnter (nLine=148,
pszFile=0x5555dcd57ba9 "interfaces/chain.cpp",
pszName=0x5555dccd578c "cs_main", this=0x7fb09803c0a8) at ./sync.h:135
#6 UniqueLock<AnnotatedMixin<std::recursive_mutex>, std::unique_lock<std::recursive_mutex> >::UniqueLock (fTry=true, nLine=148,
pszFile=0x5555dcd57ba9 "interfaces/chain.cpp",
pszName=0x5555dccd578c "cs_main", mutexIn=..., this=0x7fb09803c0a8)
at ./sync.h:148
#7 interfaces::(anonymous namespace)::LockingStateImpl::LockingStateImpl (
this=0x7fb09803c0a0) at interfaces/chain.cpp:140
#8 MakeUnique<interfaces::(anonymous namespace)::LockingStateImpl, AnnotatedMixin<std::recursive_mutex>&, char const (&) [8], char const (&) [21], int, bool&>
() at ./util/memory.h:16
#9 interfaces::(anonymous namespace)::ChainImpl::lock (this=<optimized out>,
try_lock=<optimized out>) at interfaces/chain.cpp:148
#10 0x00005555daa31faa in interfaces::(anonymous namespace)::WalletImpl::tryGetTxStatus (this=0x7fb0cd3f2000, txid=..., tx_status=...,
num_blocks=@0x7fafba4c5c44: 32688,
block_time=@0x7fafba4c5c48: 140392721571296) at ./wallet/wallet.h:830
#11 0x00005555da69dd0a in TransactionTablePriv::index (idx=<optimized out>,
wallet=..., this=0x7fb0cdd7ce10) at qt/transactiontablemodel.cpp:200
#12 TransactionTableModel::index (this=this@entry=0x7fb0cdd456b0, row=5407,
column=column@entry=0, parent=...) at qt/transactiontablemodel.cpp:673
#13 0x00005555da69bc4f in TransactionTableModel::updateConfirmations (
this=0x7fb0cdd456b0)
at /usr/include/x86_64-linux-gnu/c++/8/bits/gthr-default.h:778
#14 0x00005555da617d3c in WalletModel::pollBalanceChanged (this=0x7fb0cc002430)
at qt/walletmodel.cpp:108
#15 0x00007fb0f1341b2f in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#16 0x00007fb0f23eefa3 in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
#17 0x00007fb0f11a24cf in clone () from /lib/x86_64-linux-gnu/libc.so.6
Tried again, same error occurred.
Tried starting with
blocknet-qt -wallet=voting
which succeeded and I was able to vote.
Normally I use the daemon wallet and vote through blocknet-cli but with so many proposals this month thought I'd try the QT version instead. And every month starting with (I think) the first stake after voting, the wallet crashes. The crashes continue for about 3-7 days after the superblock and then stop. There are no clues in the debug.log and I've not collected a core dump for any of them. Will be interesting to see if I get the crashes this time around.
Self-compiled v4.3.3 with debug symbols, MX-Linux 19.3.
Add new Affinities
setting to the xbridge.conf
that allows to specify a list of service nodes ("snode") that should be selected first when creating orders. Affinities are only considered, if the snode supports are least one of the requested assets. Snodes are selected according to the order used in the list. A snode is identified by the public key.
Example:
Affinities=03f79d629c5c501aa87c71f885da8edfdbdaddf2cb502dc13be4519918101954c2,03f06b832f05c6ff04efe977bf7b562e94e92acc51d201e69c6733ac8db04e4539
This feature would be useful for both debugging purposes and node operators that want to use their own nodes for trading.
Scenario:
To support the continuation of Go-XRouter development (blocknetdx/go-xrouter#44), we need an analysis of the XRouter C++ code in order to tell whether there is an issue with the way snodelist works.
XRouter is behaving in a manner inconsistent with our understanding of how the snodelist
is acquired and used. However, this behaviour was not documented when it was originally implemented.
This issue is in order to analyse and document the actual behaviour of Blocknet Core around the snodelist
.
Before testing, our understanding was that each service node maintains an snodelist
, gossips it to peers upon request, and updates its list as peers on the list drop connections, or become available, or change score.
The following behaviours suggest either a misunderstanding of the Blocknet Core code, or that it has bugs.
snodelist
. This is surprising if the snodelist
in its entirety is gossiped.snodelist
up in successive runs after that.Both the above behaviours cause requests to fail either to return any responses, or enough responses to satisfy queryCount
. These are critical failures.
core wallet 4.3.3 (public) console, block cli
Note: in the C++ code, terminology isn’t consistent. snodelist
, sfnodelist
and NODE_SNODE_LIST
are all used.
Describe the issue
When dxgetorderbook is used with lowercase for maker and taker argument, the returned asks and bids array is empty (when it should not be empty). When uppercase is used, the call returns the correct asks and bids array.
What behavior did you expect?
I expect the same return value for both uppercase and lowercase as maker and taker input for this RPC call.
What was the actual behavior (provide screenshots if the issue is GUI-related)?
Upper case
dxgetorderbook 1 BLOCK LTC
{
"detail": 1,
"maker": "BLOCK",
"taker": "LTC",
"asks": [
[
"0.018617",
"91.642824",
1
]
],
"bids": [
]
}
Lower case
dxgetorderbook 1 block ltc
{
"detail": 1,
"maker": "block",
"taker": "ltc",
"asks": [
],
"bids": [
]
}
Blocknet version
4.3.1
What type of machine are you observing the error on (OS/CPU and disk type)?
Mac OS - Catalina 10.15.7
Windows 10 - 10.0.19041 Build 19041
Scenario:
Sending some XRouter commands via wallet console requires the parameters to be uppercase eg:
xrgetblockcount ltc
fails, but xrgetblockcount LTC
works as expected.
Given that all other console commands are case-insensitive, XRouter cmds should also be.
Env: Core wallet v4.3.3 public
To do:
xrgetblockcount ltc
or xrGetBlockCount LTC
should both work.QA:
Core 4.3.3 wallet release shows this info:
'This is a pre-release test build - use at your own risk - do not use for mining or merchant applications'}
To do:
QA:
Env: Core wallet 4.3.3, Mac, Standard GUI.
Env:
Blocknet core 4.3.3 public
Steps to reproduce:
dxsplitaddress BLOCK amount address
{
"error": "Bad Request address is invalid or not in the wallet for token BLOCK",
"code": 1004,
"name": "dxSplitAddress"
}
Expected:
{
"error": "Address split is not possible when the wallet is locked for token BLOCK. Please unlock the wallet and try again.",
"code": ???,
"name": "dxSplitAddress"
}
Scenario:
Currently the only way to receive funds is to navigate to ‘address book’ > create new address > copy it > and then manually send funds to that address. The UI doesn’t give the user any feedback on this process.
To do:
implement new quicker ‘receive funds’ pages as per designs attached (icon files: RequestFundsIcon)
replace the current ‘balances’ menu link icon with the new one attached: icon-balances.svg
keep to all current CSS states such as hover colour and font weight etc
json_spirit
was replaced with UniValue
in bitcoin core v0.12.0, because it is more lightweight and reduces compile time.
Currently, Blocknet/xbridge uses a mix of both. This causes compatibility issues and makes it difficult to change/update certain sections in the code.
Any remaining usage of json_spirit
should replaced with UniValue
to resolve those issues.
Most easy way explained, wallets has UTXO funds sitting on addresses, but Xbridge itself who is the only responsible for reserving/releasing UTXOs for orders, thinks some/all UTXOs are reserved but is not able to say what order UTXOs reserved for.
Bug impact/what is causing:
xbridge things UTXO is reserved/locked/used whatever you we name it, but is not able to provide
order id
but CLI of wallet ie DOGE/LTC sees UTXO as available/unlocked
utxos are disappearing from xbridge availability, blocknet daemon/wallet/bots need to be restarted
(like which coins ? Is is only DOGE ? Are you creating exact or partial orders? etc. - anything you think might help)
exact orders+ DOGE LTC pair twice, with both coins.
but saw other pairs also
summary:
Running
xbridge
scripts iedxmakerbot
which is usingdxmakeorder
anddxcancelorder
on pair ieDOGE vs LTC
causing that call DOGE/LTC wallets clilistunspent minconfirm=1
amount does not match xbridgedxgetUtxos coin=DOGE/LTC include_used=false
. Xbridge think UTXOs aremark as used even if they have no order ID
to be provided when callingdxgetutxos DOGE true
ordxgetutxos LTC true
.
ideally also provide the xbridge log from a day when it occurred
I was looking at logs, but as i mentioned before, i think we need debug/messages.
As i mentioned before idea was to update xpcxbridge.cpp code i was pointing on, especially that part of cpp code where locked and unlocked utxos are processing to get results which utxos are locked and which are unlocked.
What behavior did you expect?
Xbridge and wallet CLI locked/available amounts must match.
How reliably can you reproduce the issue, what are the steps to do so?
setup ie Blocknet + LTC + DOGE+ DxBot on LTC/DOGE, just let Bot create and cancel orders... will hit bug soon/later/6h.
bug seems like hit also when no orders are accepted... just creating and cancelling orders...
What version of Bitcoin Core are you using, where did you get it (website, self-compiled, etc)?
4.3.2 also 4.3.3 also self compiled also pre-build also gui also daemon.
What type of machine are you observing the error on (OS/CPU and disk type)?
Debian GNU/Llinux, lots of free space, lots of free ram, so not a missing resources bug.
Any extra information that might be useful in the debugging process.
Just for inspiration by
brainlesswojak
😁 >> Maybe adding some debug logs torpcxbridge.cpp
will be enough >>
dxMakeOrder() >> any error or cleanup we missed to handle ?
dxCancelOrder() >> any error or cleanup we missed to handle ?
dxGetUtxos() >> UTXO which is reserved for order should also have order id, but this RPC call in situation when bug is already hit, order is ""(empty string) >>
std::set<xbridge::wallet::UtxoEntry> excluded = xapp.getAllLockedUtxos(token);
std::vector<xbridge::wallet::UtxoEntry> unspent;
if (!conn->getUnspent(unspent, !includeUsed ? excluded : std::set<xbridge::wallet::UtxoEntry>{}))
return uret(xbridge::makeError(xbridge::BAD_REQUEST, __FUNCTION__, "failed to get unspent transaction outputs"));
UniValue r(UniValue::VARR);
for (const auto & utxo : unspent) {
UniValue o(UniValue::VOBJ);
o.pushKV("txid", utxo.txId);
o.pushKV("vout", static_cast<int>(utxo.vout));
o.pushKV("amount", xbridge::xBridgeStringValueFromPrice(utxo.amount, conn->COIN));
o.pushKV("address", utxo.address);
o.pushKV("scriptPubKey", utxo.scriptPubKey);
o.pushKV("confirmations", static_cast<int>(utxo.confirmations));
o.pushKV("orderid", "");
if (excluded.count(utxo) > 0) {
auto orderid = xapp.orderWithUtxo(utxo);
o.pushKV("orderid", orderid.IsNull() ? "" : orderid.GetHex());
}
r.push_back(o);
}
maybe also some debug in case when order is accepted... maybe we hit some corner case .... order been accepted but error happen and i missed that error in logs so some specific debug for this would be also good.
Adjust the xb protocol to allow for coin tickers to be longer than 7 characters.
Attempting to take an order can result in the following error:
not enough inputs to cover p2sh deposit fees
As a result the order is canceled by the taker and rolled back.
Edit:
It seems the code currently contains certain static fee assumptions. For instance, if the maker has significantly lower fee settings in their xbridge.conf
compared to the taker, it will result in the above error.
This is very likely to affect the dynamic fee PR.
The XBridgePacket
class is at the core of the xbridge communication layer. Currently, the implementation is quite complicated and error prone as the serialization and de-serialization happen all over the code base. Additionally, packages are often constructed "manually" with manual size calculations. This can easily introduce bugs when changing things and makes it difficult to review and test code properly.
I propose to simplify the class, consolidate relevant sections/methods into one place and find a better, more automated way to serialize the data (change, add, update certain data types),
Info:
There is a hard upper limit to queryCount
when appended to any XRouter call via console or cli (ie: when setting node_count
).
To do:
queryCount
or consensus
in the api docs to include the upper limit info (exact wording to TBD)An order was placed but not seen by other nodes or disappeared from the network-wide open orders list. The xbridge log shows the order was placed, confirmed by a servicenode and shows in the open orders list at the source node. Other nodes are not showing the order.
Upon investigation it turned out the servicenode was offline. It was positively last online about 30 hours previously.
Confirmed order should be visible network-wide.
Order was apparently confirmed by a servicenode but not visible elsewhere on the network. Perhaps the servicenode went offline immediately after confirming the order.
Never seen this before.
blockops@ops1:~/.blocknet/log$ parse-xbridge.py xbridgep2p_20211105.log
Order id: 47475484f68ff3a3d8d399deb0e7cc50e96e06fd94d2b613f8a37adef71b705b
[I] 2021-Nov-05 11:29:53 [0x7fe4677fe700]
{
"orderid": "47475484f68ff3a3d8d399deb0e7cc50e96e06fd94d2b613f8a37adef71b705b",
"snode_pubkey": "03b33d9186cc5ea9d30498316425cb615d5768358e2d12a93079d3bc122cb0bf39",
"function": "sendXBridgeTransaction",
"msg": "using servicenode for order"
}
[I] 2021-Nov-05 11:29:53 [0x7fe4677fe700] sendXBridgeTransaction order created
{
"orderid": "47475484f68ff3a3d8d399deb0e7cc50e96e06fd94d2b613f8a37adef71b705b",
"maker": "BLOCK",
"maker_size": "50.000000",
"maker_addr": "xxx",
"taker": "LTC",
"taker_size": "0.357770",
"taker_addr": "xxx",
"partial_allowed": false,
"partial_repost": false,
"partial_minimum": "0.000000",
"partial_orig_maker_size": "50.000000",
"partial_orig_taker_size": "0.357770",
"partial_parent_id": "0000000000000000000000000000000000000000000000000000000000000000",
"state": "new",
"block_hash": "b69ebdac62ebf342cca5495c214284fbf116de3c867d35de8fa48f99b3c3d48b",
"updated_at": "2021-11-05T10:29:53.498Z",
"created_at": "2021-11-05T10:29:53.498Z",
"err_msg": "",
"cancel_reason": "crNone",
"utxos": [
{...
}
[I] 2021-Nov-05 11:29:53 [0x7fe3f9d47700]
{
"orderid": "47475484f68ff3a3d8d399deb0e7cc50e96e06fd94d2b613f8a37adef71b705b",
"function": "processPendingTransaction",
"msg": "received confirmed order from snode, setting status to pending"
}
The node shows as offline
blockops@ops1:~/.blocknet/log$ blocknet-cli servicenodelist
[
...
{
"snodekey": "03b33d9186cc5ea9d30498316425cb615d5768358e2d12a93079d3bc122cb0bf39",
"tier": "SPV",
"address": "BapjD21wMqQR7BoEpmeNeChjMWwBDzgweB",
"timelastseen": 0,
"timelastseenstr": "1970-01-01T00:00:00.000Z",
"exr": false,
"status": "offline",
"score": 0,
"services": [
]
},
...
Currently xbridge logs go on for ever until disk is full. xbridge generates huge amounts of logs, maybe keep 7 days worth of logs
—
To do:
Currently, when an order is triggered both parties send their amount to a timelocked address, including two types of transaction fees
Example:
If one were to send 1 LTC (trade_amount
), the wallet would actually send
total = trade_amount + fee1 + fee2
.
The redeeming side would then be able to claim
total = trade_amount + fee2
,
while using fee2
to cover the tx costs, ending up with trade_amount
in their wallet.
All of these calculations happen without any user interaction. This works fairly well as long as fee calculations are static and both parties use very similar fee settings.
When moving towards a more dynamic (estimated) fees system, certain problems arise. This ticket makes attempt to highlight some of the shortcomings of the current implementation as well as present possible solutions.
Problem:
Transaction fees are currently somewhat of a black box system. Fees are automatically calculated and added to the trading amounts without any user interaction. Consequently, the user never sees nor confirms the actual fee amounts. This works well with static tx fees as costs are fairly predictable, however it poses a problem with dynamic fees, where this is no longer the case. Dynamic fee estimations might add quite high, even ridiculous fees, aren't very reliable and shouldn't blindly be trusted.
Problem:
Currently, fee1
and fee2
are calculated once during the order creation process and then never changed. Now, orders aren't necessary taken right away. In fact, a significant amount of time might pass between placing and taking an order. During that time estimated fees might drastically change leading to stuck/failed orders or to overpaid fees. In order to address this problem the client has to become more flexible when dealing with tx fees.
fee2
on the redeemer side (take from trade_amount
, add to fee2
, redeemer ends up with slightly less in their wallet)
fee2
, but this might lead to new problems (not enough funds, dust, etc)fee2
sufficient before triggering an order (would need a way of displaying this)Additionally:
The above list is not complete, but has the intention to start a discussion on the topic.
Related:
Scenario:
There is an intermittent bug with xbridge order posting that results in an 'invalid amount' error (see screenshot attached)
We need to find the route of this issue but the logs don't provide enough info to track it down. Currently we don't have the exact steps to reproduce but the following process can result in the error:
Steps to reproduce:
dxmakepartialorder
) for eg: dxMakePartialOrder LTC 0.010853 LNu2ja6eLrttWj4VwpABFAB4dBVeJBjSAB BLOCK 1 BVADycgu66et8PANEwLNY1uCHbBZFuC8Wp 0.001085 true
Error result: error 1026 - invalid amount
Important note: this error is intermittent as you can see from the screenshots attached. Sometimes the same order works and others it fails.
To do:
1. Investigate what causes xbridge to return the ‘invalid amount' error when dxmakepartialorder
is called
We need to add further logging to xbridge if this cannot be determined from the current code:
2. each UTXOs details should be available in the log
the maximum number of utxos on the order was exceeded (UTXOs empty)
=
if (utxos.empty()) {...
xbridge::LogOrderMsg(log_obj, "failed to create order, the maximum number of utxos on the order was exceeded", __FUNCTION__);
return xbridge::Error::UTXOS_EMPTY;
}
Insufficient funds on partial order
=
xbridge::LogOrderMsg(log_obj, "failed to create order, insufficient funds on partial order", __FUNCTION__);
return xbridge::Error::INSUFFICIENT_FUNDS_PARTIAL_ORDER;
the maximum number of utxos on the order was exceeded
=
if (ptr->usedCoins.size() > xBridgePartialOrderMaxUtxos) {...
xbridge::LogOrderMsg(log_obj, "failed to create order, the maximum number of utxos on the order was exceeded", __FUNCTION__);
return xbridge::Error::EXCEEDED_MAX_UTXOS;
3. add the above new errors in blocknet/src/xbridge/util/xbridgeerror.cpp
case Error::UTXOS_EMPTY:
return "Failed to create order, the maximum number of utxos on the order was exceeded " + argument;
case Error::INSUFFICIENT_FUNDS_PARTIAL_ORDER:
return "failed to create order, insufficient funds on partial order " + argument;
case Error::EXCEEDED_MAX_UTXOS:
return "failed to create order, the maximum number of utxos on the order was exceeded " + argument;
Ref: https://api.blocknet.co/#error-codes
Dev update after investigating issue: https://docs.google.com/document/d/1-cviFKPSC8wMJRNEP6cnoFkbVIEDXK3bxf99pBbCjbc/edit
Scenario: requesting xrGetBlockCount
via core console or cli with a queryCount > 50 returns an error.
xrgetblockcount BLOCK 10
{
“reply”: 2244915,
“uuid”: “08ae7d53-49b7-498a-8d3d-9b3ece4d8a03”
}
xrgetblockcount BLOCK 100
{
“code”: 1032,
“error”: “Failed to find 100 service node(s) supporting BLOCK::xrGetBlockCount with config limits, found 0”,
“uuid”: “1933a5d2-f023-42d1-b154-8ed469d1a407”
}
To do:
xrouter.conf
the following content:#! consensus is the minimum number of nodes you want your xrouter calls to query (1 or more)
#! Paid calls will send a payment to each selected service node.
consensus=3
should be replaced by:
#! consensus is the minimum number of nodes you want your xrouter calls to query (min=1, max=50)
#! Paid calls will send a payment to each selected service node.
consensus=3
{
“code”: 1032,
“error”: “Your request of X nodes is too high. Please set queryCount to >0 and <51”,
}
_where X = `queryCount`_
xbridge.conf
currently contains a variety of options/settings. However, many of these settings are very specific to a blockchain and should only have a very distinct value. The current setup isn't very user-friendly as miss-configuring these settings can lead to all sorts of problems and bugs. Given that many (end) users do not understand these settings, they would very likely depend on some pre-configured template. Therefore, it is proposed to split off the critical settings from the xbridge.conf
and move them to a new, developer reviewed and well maintained coin configuration file, which can then be shipped with blocknet releases.
Example of a coin section in xbridge.conf
:
[SYS]
Title=Syscoin
Ip=127.0.0.1
Port=8370
AddressPrefix=63
ScriptPrefix=5
SecretPrefix=128
COIN=100000000
MinimumAmount=0
TxVersion=1
DustAmount=0
CreateTxMethod=BTC
MinTxFee=5000
BlockTime=60
GetNewKeySupported=true
ImportWithNoScanSupported=true
FeePerByte=10
Confirmations=0
Username=BlockDXSyscoin
Password=XXX
Address=
TxWithTimeField=false
LockCoinsSupported=false
JSONVersion=
ContentType=
CashAddrPrefix=
At least the following settings should be moved:
AddressPrefix=63
ScriptPrefix=5
SecretPrefix=128
COIN=100000000
MinimumAmount=0
TxVersion=1
DustAmount=0
CreateTxMethod=BTC
BlockTime=60
Such a coin configuration file would have the additional benefit of being sharable between projects like Blocknet Core, XLite or Block-DX (those need/have similar coin configs).
Support for new coins could still be added with pull requests.
DX trades sometimes fail due to incorrect/unexpected locktimes.
For properly configured, properly behaving, synced nodes/wallets they should complete normally every time.
I tried to take 3 BLOCK/LTC orders on 1 October using CLI
blocknet-cli dxTakeOrder 674c069315400febdb722ff6304d686c6d1977119d3a65410fc9f2ab7cd9ff4e <send address> <receive address>
The trade did not complete. Checking the xbridgep2p log for this order showed that it failed with
...
[I] 2021-Oct-01 11:29:36 [0x7f0710672700]
{
"orderid": "674c069315400febdb722ff6304d686c6d1977119d3a65410fc9f2ab7cd9ff4e",
"expected_counterparty_locktime": 2132299,
"counterparty_locktime": 2132292,
"function": "processTransactionCreateB",
"msg": "bad locktime from counterparty, canceling"
}
[I] 2021-Oct-01 11:29:36 [0x7f0710672700]
{
"orderid": "674c069315400febdb722ff6304d686c6d1977119d3a65410fc9f2ab7cd9ff4e",
"cancel_reason": "crBadALockTime",
"function": "sendCancelTransaction",
"msg": "canceling order, initiated by me"
}
[I] 2021-Oct-01 11:29:36 [0x7f0710672700]
{
"orderid": "674c069315400febdb722ff6304d686c6d1977119d3a65410fc9f2ab7cd9ff4e",
"cancel_reason": "crBadALockTime",
"function": "processTransactionCancel",
"err_msg": "counterparty cancel request"
}
...
Taker BLOCK and LTC wallets were synced, unlocked and operating normally. I believe the same is true at the maker side. Tried 2 further times, order ids
Blocknet version v4.3.3.0-698e09fea on MX 19.3 and Ubuntu 18.04
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.