Is your feature request related to a problem? Please describe.
There are 2 problems I see with Koios-specific log files stored on the nodes:

1. They are not well structured (besides grest-poll.sh logs, the others are all appended to their respective single file). Inside those logfiles, the structure is sometimes hard to navigate as psql output intersects the cron script output.

2. They do not have an automatic rotation system.

Describe the solution you'd like
Some ideas:

1. Separate individual log files per time interval (daily, weekly or monthly) and improve logging inside the cron scripts.
2. A cron log clean-up job to keep only the last 30 days' worth of data?

Documentation:

Actual:

Describe the bug

Corresponding specs updates for cardano-community/guild-operators#1464

Documentation mentions:

Return information about blocks minted by a given pool in current epoch (or _epoch_no if provided)

However, instead of current epoch, data for epoch 294 is returned.

Example:

"https://api.koios.rest/api/v0/pool_blocks?_pool_bech32=pool10efuydxjnasr0309lqklmj9xahupdnzqd59r5dqjeh4twz06tr0"

does not return current pool blocks for current epoch, but for epoch 294.

Is your feature request related to a problem? Please describe.
cardano-db-sync [v10-v13] does not know of active stake from last epoch, and has to wait for a while before it's able to finish calculation of active stake from 'active' epoch (new epoch - 2) prior to epoch transition. This results in active stake being reported as null for few hours post entering new epoch for any stake addresses.
One option could be to have a special condition for stake distribution cache at epoch transition to fill the data in recent_epoch_stake table.

Describe the solution you'd like
Build an active stake cache dedicated to pre-filling the information after recently finished epoch, to have the data ready to read upon next epoch transition

Describe alternatives you've considered
Wait for fixes upstream, but that will likely be deferred to v14

Additional context
With the recent changes on node post 1.35, there will be increase in reliance on query layers like Koios' gRest (remotely or as local instance), it will also prevent being able to provide leadership schedule nonce right at epoch transition easily

Describe the bug

If a SPO updates his pool's metadata and then sends future updates that reverts metadata to a previous ID, the meta_id (or pmr_id reference) is not incremented, which means the ORDER BY statement for meta_id would cause trouble for such updates. This primarily impacts pool_info endpoint.

A simple solution would be removing meta_id from ordering, but that reaches back to previous issue (meta_id is populated with a new entry in pool_update table, but the entry in pool_offline_data for that ID is not instant - which could leave a null against a pool ID.

a. The simplest alternate that makes sense would be to use COALESCE to return latest pmr_id against a specified pool (as absence of an update in pool_offline_data table would mean previous data is visible. The question for this path really is whether showing a null for a new update to pool preferable over perhaps a previous entry (pro being null could really show specific state , con being the timing will depend on cache status and not be consistent between instances).
b. The alternative could be re-jig the pool_information_cache table to only provide a current view (not worry about history), but that would require update to pool_updates table (perhaps more) to use live data instead - impact needs to be evaluated thoroughly.

In either options, need to also keep in mind that it isnt a very common case, but good to be tackled - should not sacrifice UX.

(Thanks to @hodlonaut for reporting the issue)

Is your feature request related to a problem? Please describe.
No, it is not a problem. Just a few more fields required.

Describe the solution you'd like
In the 'pool_info' endpoint, could you include a field for 30-day ROS ie for the last 6 PAID epochs (ie 6 epochs preceding the current epoch-2). This would save having to hit all the pool_history endpoints to do the calculations for all the pools each epoch. It only needs to be done once per epoch after rewards are calculated/paid.

Less importantly, in the tokenomics 'totals' endpoint, it would be useful to have the current block reward as a field. As we know this is changing/reducing gradually. It would also only need to be calculated once per epoch. This is however an easy one to calculate with very few calls.

curl --location --request GET 'https://api.koios.rest/api/v0/genesis'

Response:

[
    {
        "networkmagic": "764824073",
        "networkid": "Mainnet",
        "activeslotcoeff": "0.05",
        "updatequorum": "5",
        "maxlovelacesupply": "45000000000000000",
        "epochlength": "432000",
        "systemstart": "2017-09-23T21:44:51Z", <---- should be UNIX Time
        "slotsperkesperiod": "129600",
        "slotlength": "1",
        "maxkesrevolutions": "62",
        "securityparam": "2160",
        "alonzogenesis": "{\"lovelacePerUTxOWord\":34482,\"executionPrices\":{\"prSteps\":{\"numerator\":721,\"denominator\":10000000},\"prMem\":{\"numerator\":577,\"denominator\":10000}},\"maxTxExUnits\":{\"exUnitsMem\":10000000,\"exUnitsSteps\":10000000000},\"maxBlockExUnits\":{\"exUnitsMem\":50000000,\"exUnitsSteps\":40000000000},\"maxValueSize\":5000,\"collateralPercentage\":150,\"maxCollateralInputs\":3,\"costModels\":{\"PlutusV1\":{\"sha2_256-memory-arguments\":4,\"equalsString-cpu-arguments-constant\":1000,\"cekDelayCost-exBudgetMemory\":100,\"lessThanEqualsByteString-cpu-arguments-intercept\":103599,\"divideInteger-memory-arguments-minimum\":1,\"appendByteString-cpu-arguments-slope\":621,\"blake2b-cpu-arguments-slope\":29175,\"iData-cpu-arguments\":150000,\"encodeUtf8-cpu-arguments-slope\":1000,\"unBData-cpu-arguments\":150000,\"multiplyInteger-cpu-arguments-intercept\":61516,\"cekConstCost-exBudgetMemory\":100,\"nullList-cpu-arguments\":150000,\"equalsString-cpu-arguments-intercept\":150000,\"trace-cpu-arguments\":150000,\"mkNilData-memory-arguments\":32,\"lengthOfByteString-cpu-arguments\":150000,\"cekBuiltinCost-exBudgetCPU\":29773,\"bData-cpu-arguments\":150000,\"subtractInteger-cpu-arguments-slope\":0,\"unIData-cpu-arguments\":150000,\"consByteString-memory-arguments-intercept\":0,\"divideInteger-memory-arguments-slope\":1,\"divideInteger-cpu-arguments-model-arguments-slope\":118,\"listData-cpu-arguments\":150000,\"headList-cpu-arguments\":150000,\"chooseData-memory-arguments\":32,\"equalsInteger-cpu-arguments-intercept\":136542,\"sha3_256-cpu-arguments-slope\":82363,\"sliceByteString-cpu-arguments-slope\":5000,\"unMapData-cpu-arguments\":150000,\"lessThanInteger-cpu-arguments-intercept\":179690,\"mkCons-cpu-arguments\":150000,\"appendString-memory-arguments-intercept\":0,\"modInteger-cpu-arguments-model-arguments-slope\":118,\"ifThenElse-cpu-arguments\":1,\"mkNilPairData-cpu-arguments\":150000,\"lessThanEqualsInteger-cpu-arguments-intercept\":145276,\"addInteger-memory-arguments-slope\":1,\"chooseList-memory-arguments\":32,\"constrData-memory-arguments\":32,\"decodeUtf8-cpu-arguments-intercept\":150000,\"equalsData-memory-arguments\":1,\"subtractInteger-memory-arguments-slope\":1,\"appendByteString-memory-arguments-intercept\":0,\"lengthOfByteString-memory-arguments\":4,\"headList-memory-arguments\":32,\"listData-memory-arguments\":32,\"consByteString-cpu-arguments-intercept\":150000,\"unIData-memory-arguments\":32,\"remainderInteger-memory-arguments-minimum\":1,\"bData-memory-arguments\":32,\"lessThanByteString-cpu-arguments-slope\":248,\"encodeUtf8-memory-arguments-intercept\":0,\"cekStartupCost-exBudgetCPU\":100,\"multiplyInteger-memory-arguments-intercept\":0,\"unListData-memory-arguments\":32,\"remainderInteger-cpu-arguments-model-arguments-slope\":118,\"cekVarCost-exBudgetCPU\":29773,\"remainderInteger-memory-arguments-slope\":1,\"cekForceCost-exBudgetCPU\":29773,\"sha2_256-cpu-arguments-slope\":29175,\"equalsInteger-memory-arguments\":1,\"indexByteString-memory-arguments\":1,\"addInteger-memory-arguments-intercept\":1,\"chooseUnit-cpu-arguments\":150000,\"sndPair-cpu-arguments\":150000,\"cekLamCost-exBudgetCPU\":29773,\"fstPair-cpu-arguments\":150000,\"quotientInteger-memory-arguments-minimum\":1,\"decodeUtf8-cpu-arguments-slope\":1000,\"lessThanInteger-memory-arguments\":1,\"lessThanEqualsInteger-cpu-arguments-slope\":1366,\"fstPair-memory-arguments\":32,\"modInteger-memory-arguments-intercept\":0,\"unConstrData-cpu-arguments\":150000,\"lessThanEqualsInteger-memory-arguments\":1,\"chooseUnit-memory-arguments\":32,\"sndPair-memory-arguments\":32,\"addInteger-cpu-arguments-intercept\":197209,\"decodeUtf8-memory-arguments-slope\":8,\"equalsData-cpu-arguments-intercept\":150000,\"mapData-cpu-arguments\":150000,\"mkPairData-cpu-arguments\":150000,\"quotientInteger-cpu-arguments-constant\":148000,\"consByteString-memory-arguments-slope\":1,\"cekVarCost-exBudgetMemory\":100,\"indexByteString-cpu-arguments\":150000,\"unListData-cpu-arguments\":150000,\"equalsInteger-cpu-arguments-slope\":1326,\"cekStartupCost-exBudgetMemory\":100,\"subtractInteger-cpu-arguments-intercept\":197209,\"divideInteger-cpu-arguments-model-arguments-intercept\":425507,\"divideInteger-memory-arguments-intercept\":0,\"cekForceCost-exBudgetMemory\":100,\"blake2b-cpu-arguments-intercept\":2477736,\"remainderInteger-cpu-arguments-constant\":148000,\"tailList-cpu-arguments\":150000,\"encodeUtf8-cpu-arguments-intercept\":150000,\"equalsString-cpu-arguments-slope\":1000,\"lessThanByteString-memory-arguments\":1,\"multiplyInteger-cpu-arguments-slope\":11218,\"appendByteString-cpu-arguments-intercept\":396231,\"lessThanEqualsByteString-cpu-arguments-slope\":248,\"modInteger-memory-arguments-slope\":1,\"addInteger-cpu-arguments-slope\":0,\"equalsData-cpu-arguments-slope\":10000,\"decodeUtf8-memory-arguments-intercept\":0,\"chooseList-cpu-arguments\":150000,\"constrData-cpu-arguments\":150000,\"equalsByteString-memory-arguments\":1,\"cekApplyCost-exBudgetCPU\":29773,\"quotientInteger-memory-arguments-slope\":1,\"verifySignature-cpu-arguments-intercept\":3345831,\"unMapData-memory-arguments\":32,\"mkCons-memory-arguments\":32,\"sliceByteString-memory-arguments-slope\":1,\"sha3_256-memory-arguments\":4,\"ifThenElse-memory-arguments\":1,\"mkNilPairData-memory-arguments\":32,\"equalsByteString-cpu-arguments-slope\":247,\"appendString-cpu-arguments-intercept\":150000,\"quotientInteger-cpu-arguments-model-arguments-slope\":118,\"cekApplyCost-exBudgetMemory\":100,\"equalsString-memory-arguments\":1,\"multiplyInteger-memory-arguments-slope\":1,\"cekBuiltinCost-exBudgetMemory\":100,\"remainderInteger-memory-arguments-intercept\":0,\"sha2_256-cpu-arguments-intercept\":2477736,\"remainderInteger-cpu-arguments-model-arguments-intercept\":425507,\"lessThanEqualsByteString-memory-arguments\":1,\"tailList-memory-arguments\":32,\"mkNilData-cpu-arguments\":150000,\"chooseData-cpu-arguments\":150000,\"unBData-memory-arguments\":32,\"blake2b-memory-arguments\":4,\"iData-memory-arguments\":32,\"nullList-memory-arguments\":32,\"cekDelayCost-exBudgetCPU\":29773,\"subtractInteger-memory-arguments-intercept\":1,\"lessThanByteString-cpu-arguments-intercept\":103599,\"consByteString-cpu-arguments-slope\":1000,\"appendByteString-memory-arguments-slope\":1,\"trace-memory-arguments\":32,\"divideInteger-cpu-arguments-constant\":148000,\"cekConstCost-exBudgetCPU\":29773,\"encodeUtf8-memory-arguments-slope\":8,\"quotientInteger-cpu-arguments-model-arguments-intercept\":425507,\"mapData-memory-arguments\":32,\"appendString-cpu-arguments-slope\":1000,\"modInteger-cpu-arguments-constant\":148000,\"verifySignature-cpu-arguments-slope\":1,\"unConstrData-memory-arguments\":32,\"quotientInteger-memory-arguments-intercept\":0,\"equalsByteString-cpu-arguments-constant\":150000,\"sliceByteString-memory-arguments-intercept\":0,\"mkPairData-memory-arguments\":32,\"equalsByteString-cpu-arguments-intercept\":112536,\"appendString-memory-arguments-slope\":1,\"lessThanInteger-cpu-arguments-slope\":497,\"modInteger-cpu-arguments-model-arguments-intercept\":425507,\"modInteger-memory-arguments-minimum\":1,\"sha3_256-cpu-arguments-intercept\":0,\"verifySignature-memory-arguments\":1,\"cekLamCost-exBudgetMemory\":100,\"sliceByteString-cpu-arguments-intercept\":150000}}}"
    }
]

curl --location --request GET 'https://testnet.koios.rest/api/v0/pool_stake_snapshot?_pool_bech32=pool1rcsezjrma577f06yp40lsz76uvwh7gne35afx3zrq2ktx50f8t8'

Bad request:

{
    "details": "unexpected \"p\" expecting \"not\" or operator (eq, gt, ...)",
    "message": "\"failed to parse filter (pool1rcsezjrma577f06yp40lsz76uvwh7gne35afx3zrq2ktx50f8t8)\" (line 1, column 1)"
}

For managing updates, we'd like to start using versioning as per below:

• Associate queries to add/update on project milestones on github.
• Save version information to control table.
• Add a check for version in grest-poll.
• Tag new versions on github when marking a release.
• Document changes (ones that require change to specs, seperate to non-breaking changes)

curl --location --request GET 'https://testnet.koios.rest/api/v0/asset_info?_asset_policy=5c2171471578441ab237b76531539b2d5bfa4193be4aab0466b817f4&_asset_name=54657374746f6b656e313233'

Bad Request:

{
    "hint": null,
    "details": null,
    "code": "21000",
    "message": "more than one row returned by a subquery used as an expression"
}

BF's Model

Koios Model

• PS: cip10 field is always null in BF so it might not be as important.

Describe the bug

Some endpoints return valid RFC3339/ISO8601 date and some not which makes it inconsistent to unmarshal json response.

To Reproduce

query /genesis "systemstart": "2017-09-23T21:44:51Z", has valid RFC3339/ISO8601 time layout while querying some other endpoints e.g. /epoch_info returns "start_time": "2022-02-09T21:44:51", (missing Z for Zero timezone offset) which is not valid RFC3339 layout.

Expected behavior

all date strings to have standard valid layout e.g. with "Z" or tz-numoffset

We have a use case where we'd want to drop some rewards for users registered against our pool.
We'd like to get the tx_hash for every delegation tx for https://api.koios.rest/#get-/pool_delegators - already discussed in Koios discussions group.

Could use bash (curl) and / or any other lang (eg: python class based program to use with testing/benchmarking suite in future)

ETA: 14th March 2022

Documentation:

Actual Response:

• go https://api.koios.rest/#get-/epoch_block_protocols
• click schema tab

**Is your feature request related to a problem?
I'm always frustrated when I can't find the delegators that are still staked in a retired pool.

Describe the solution you'd like

I would like there to be a way to find the delegators that are still staked in retired pools

Describe alternatives you've considered
Finding a way or adding another parameter that checks account status and dangling accounts

Additional context
Priyank helped me build this feature request.

• Reduce duplication of information
• Consider queries optimisations/filters where applicable
• Consider accepting batch parameters where applicable
• Ensure cache tables always report status to control table
• Remove use of Postgres Triggers - if any, instead use crontab

Describe the bug
A clear and concise description of what the bug is.
The stake snapshot values captured do not match the epoch_stake table.

To Reproduce
Steps to reproduce the behaviour:

cexplorer=# SELECT COUNT(*) FROM EPOCH_STAKE WHERE EPOCH_NO = 362;
  count
---------
 1215933
(1 row)

cexplorer=# SELECT COUNT(*) FROM grest.STAKE_SNAPSHOT_CACHE WHERE EPOCH_NO = 360;
  count
---------
 1215959
(1 row)

cexplorer=# SELECT SUM(amount) FROM EPOCH_STAKE WHERE EPOCH_NO = 362;
        sum
-------------------
 24724295922908154
(1 row)

cexplorer=# SELECT SUM(amount) FROM grest.STAKE_SNAPSHOT_CACHE WHERE EPOCH_NO = 360;
        sum
-------------------
 24724295975715173
(1 row)

We have some extra entries in the snapshot that should not be captured:

select sa.view, ph.view, ssc.* from grest.stake_snapshot_cache ssc inner join stake_address sa ON sa.id = ssc.addr_id INNER JOIN pool_hash ph on ph.id = ssc.pool_id where epoch_no = 360 and ssc.addr_id NOT IN (SELECT addr_id from epoch_stake where epoch_no = 362);
                            view                             |                           view                           | addr_id | pool_id |  amount  | epoch_no
-------------------------------------------------------------+----------------------------------------------------------+---------+---------+----------+----------
 stake1u8dxcszp4xnfafanzuduwusa9pjsy3x3hk3lurj6ck5z70ghlst7v | pool1ek2m5zuydqcgqv3s5mlh55tr8gp76yuxe4hj4s7xtxajyc94rx3 |    4470 |     221 |        0 |      360
 stake1u8n2cakupn552njqx6nth5q6a8ssm7js2xr4g0mfaupxngqrh6mtk | pool1ek2m5zuydqcgqv3s5mlh55tr8gp76yuxe4hj4s7xtxajyc94rx3 |   99360 |     221 |        0 |      360
 stake1u84v56pv66er9lxkmv33glk2s5uuhxa0dc6tfgkgnxa8z7ga5euwj | pool1ek2m5zuydqcgqv3s5mlh55tr8gp76yuxe4hj4s7xtxajyc94rx3 |   22023 |     221 |        0 |      360
 stake1u9tx44879lxa9vknvc2e60jv0lnwrj49pf3qmjkgldpv74ce2urfm | pool1ek2m5zuydqcgqv3s5mlh55tr8gp76yuxe4hj4s7xtxajyc94rx3 |   87938 |     221 |        0 |      360
 stake1u9yvmkcx2ftyq2g4lu060tjp929kxwgdv38mdgww36qjvzsf5ryhe | pool1ek2m5zuydqcgqv3s5mlh55tr8gp76yuxe4hj4s7xtxajyc94rx3 |   99445 |     221 |        0 |      360
 stake1u94ly60w6wx50kl5rzcrf6t4pf4ltgpwgc6haeglvcdxwhsqnvy7w | pool1ek2m5zuydqcgqv3s5mlh55tr8gp76yuxe4hj4s7xtxajyc94rx3 |   99458 |     221 |        0 |      360
 stake1uysd603jll45ev3lp7e63l3kzlj6mq6grgy2w9s667pccxcy8szhk | pool1ek2m5zuydqcgqv3s5mlh55tr8gp76yuxe4hj4s7xtxajyc94rx3 |   99489 |     221 |        0 |      360
 stake1u9amhdfvddtcfzzmvf2qurg27f9f945vvhgrqaa8vyr2z6cutpu6c | pool1ek2m5zuydqcgqv3s5mlh55tr8gp76yuxe4hj4s7xtxajyc94rx3 |   99512 |     221 |        0 |      360
 stake1u9pgavfu9p3dkpmqes7kr6gzmlwq0r0egvhcm4a04808nzg9x7tfq | pool1ek2m5zuydqcgqv3s5mlh55tr8gp76yuxe4hj4s7xtxajyc94rx3 |  100649 |     221 |        0 |      360
 stake1ux073fa7xs49jljgd8pceauymhgneqrc7vgsenhnqg93emgdlnjq4 | pool1ek2m5zuydqcgqv3s5mlh55tr8gp76yuxe4hj4s7xtxajyc94rx3 |  123512 |     221 |        0 |      360
 stake1u8v4uwrlr8aaeujlqp96k43hmzpcanawehga4p2fzukxjaqz884p7 | pool1ek2m5zuydqcgqv3s5mlh55tr8gp76yuxe4hj4s7xtxajyc94rx3 |  113216 |     221 | 20432252 |      360
 stake1uycf7qf4mlz3hhhrqgs65yeykn0nf3qq294spa432gc6trce2vvx7 | pool1ek2m5zuydqcgqv3s5mlh55tr8gp76yuxe4hj4s7xtxajyc94rx3 |  131745 |     221 |    13627 |      360
 stake1uylsedsn3ul90xyc23nxyee0k3g6lfyx50yds4m9xza9trqae8ae0 | pool1ek2m5zuydqcgqv3s5mlh55tr8gp76yuxe4hj4s7xtxajyc94rx3 |  129062 |     221 |   439724 |      360
 stake1uyq2qk7le8l9vwsd8l9s5khuhqrcdmta8mxvfjs35lcne5q4tcl59 | pool1ek2m5zuydqcgqv3s5mlh55tr8gp76yuxe4hj4s7xtxajyc94rx3 |  131889 |     221 |        0 |      360
 stake1u9v8gj329v47t9ew4w64g4m8vcps0nj7q6hpl6jl8wxu5jqgrrm4y | pool1ek2m5zuydqcgqv3s5mlh55tr8gp76yuxe4hj4s7xtxajyc94rx3 |  144510 |     221 |   230492 |      360
 stake1ux05gxdgq4adx5v325750f9rddcd7czzcsxzs864rl0623q7h3yez | pool1ek2m5zuydqcgqv3s5mlh55tr8gp76yuxe4hj4s7xtxajyc94rx3 |  181019 |     221 |   995534 |      360
 stake1u82neuuzrswsjm50kq5gxjkkz57lrhpthfvmx0pk737ms9s2f96w9 | pool1ek2m5zuydqcgqv3s5mlh55tr8gp76yuxe4hj4s7xtxajyc94rx3 |  177232 |     221 | 16312283 |      360
 stake1uxhsakr3nsnz4578cn88zkayk0gvjdrqq4qc9z5wvufvllgrmslvq | pool1ek2m5zuydqcgqv3s5mlh55tr8gp76yuxe4hj4s7xtxajyc94rx3 |  152656 |     221 |     6845 |      360
 stake1u8gl0c7n0c9uj4c4nlugxr5thn3r0mf5n85u53zezs269gszh5gtc | pool1ek2m5zuydqcgqv3s5mlh55tr8gp76yuxe4hj4s7xtxajyc94rx3 |  185599 |     221 |        0 |      360
 stake1uydj64zzepvap2k7ks8ut0y99dy08umap5aptfxjycs3raq582ng2 | pool1ek2m5zuydqcgqv3s5mlh55tr8gp76yuxe4hj4s7xtxajyc94rx3 |  208121 |     221 |        0 |      360
 stake1uyns7pa8p49rds0gan03kvg2qrgxdhv0fgwg8gwqge5vp0qxeus0m | pool1ek2m5zuydqcgqv3s5mlh55tr8gp76yuxe4hj4s7xtxajyc94rx3 |  208155 |     221 |        0 |      360
 stake1u928ahgs6hxymkw2neys9q0d64uhu652gll49d80xqycjqczgz5wh | pool15gdh4w26ax66aa2j5ehwcvm46as3jx0v7mhu678y7m5q5s80dwk | 1773110 |    2793 | 12596700 |      360
 stake1ux52497mc54pczpa5vewfltxvq4r08kckcea8dndjauxaesk34u48 | pool1ek2m5zuydqcgqv3s5mlh55tr8gp76yuxe4hj4s7xtxajyc94rx3 |  192769 |     221 |  1779562 |      360
 stake1uy0yhskzf9cmsyyuwu04fql0dnljc8el3k3vp4jvegx7xkq88syvw | pool1kprt6l4quz03dhqx8jw22spdjvp5dkshdnn567vlexqkj35hd8a | 1044924 |    2897 |        0 |      360
 stake1u9l7es8qafpkw5ezkjc46uunhv22ma57cs24z73nxpujsxqgzgdll | pool1ek2m5zuydqcgqv3s5mlh55tr8gp76yuxe4hj4s7xtxajyc94rx3 |  203876 |     221 |        0 |      360
 stake1uyrc2hvzj6g9wkpc29fgfnd66us433ekavvegw8lq4kj5xgehhm8j | pool1kprt6l4quz03dhqx8jw22spdjvp5dkshdnn567vlexqkj35hd8a | 1189739 |    2897 |        0 |      360
(26 rows)

The sum of these corresponds exactly to the difference we see in the SUM() value:

select sum(amount) from grest.stake_snapshot_cache ssc where epoch_no = 360 and ssc.addr_id NOT IN (SELECT addr_id from epoch_stake where epoch_no = 362);
   sum
----------
 52807019
(1 row)

Expected behaviour
A clear and concise description of what you expected to happen.

Values should match.

Describe the bug

Currently, Koios endpoints use a balanced approach for health checks to avoid scanning every endpoint, but still poll relevant ones. As such, we do not test pool_offline_data or asset registry cache as part of these polls, but since these do not depend on block/tx/etc, makes sense to add these individually.

Alternately, this would already be solv-able via approach from #26

Cant be merged until release is finalised and we've forked (PR: cardano-community/guild-operators#1437)

Changes captured by @DamjanOstrelic :

TxOut:

• columns added inlineDatumId and referenceScriptId (foreign keys for datumId and scriptId tables)
• consider what to do with those

CollateralTxOut/ReferenceTxIn:

• consider what to do with new tables

PoolUpdate:

• reward_addr was used raw in pool_info_cache.sql line 325, to be changed to use new reference reward_addr_id (address lookup required)

PoolOwner:

• pool_info_cache.sql line 104 uses removed column registered_tx_id
• pool_info_cache.sql line 104 uses removed columns registered_tx_id and pool_hash_id
• needs to get that data for cache using new pool_update_id reference (lookup required)

Redeemer:

• script_redeemers.sql line 35 uses removed column datum_id
• tx_info.sql line 501 uses removed column datum_id
• this is replaced by reference to new table (RedeemerData)

RedeemerData:

• consider additional uses for the new table, script endpoints are fairly limited

ExtraKeyWitness:

• consider use for new table and if inclusion is necessary to tx_info

ParamProposal:

• renamed column coins_per_utxo_word used in tx_info.sql

EpochParam:

• renamed column coins_per_utxo_word used in epoch_info_cache.sql
• renamed column entropy used in epoch_info_cache.sql (this column was NOT renamed in param_proposal table)

CostModel:

• consider use of new column hash

Few additions:

Blocks:

• Include proto_major and proto_minor in /blocks view and /block_info endpoint

Is your feature request related to a problem? Please describe.
I would like to limit the amount of requests needed to grab address information for all policy tokens. Right now you can grab all tokens to a policy but in order to see who owns them you have to query one by one.
Describe the solution you'd like
One request that would query a policy [asset_policy_info] and include [asset_address_list] for each token.
Describe alternatives you've considered
Hosting my own cardano-dbsync (again, its annoying though) or cardano-graphql to do own queries...
Additional context
Working on a d3js visualization for a few key areas of cardano ecosystem and for this specific feature, it would be for a policy(ies) overview. This shows all related accounts to specific policy(ies) to see the distribution of tokens visually.

Is your feature request related to a problem? Please describe.
When setting up grest on a network other than mainnet, the polling API spec definition will point to mainnet.

Describe the solution you'd like
When installing a network other than mainnet, grest-poll.sh should point to that network's API spec.

Describe alternatives you've considered
Keep manually editing the definition on new installs.

Description of Issue

Over a period of time, the size of table grest.account_active_stake_cache can be exceedingly large. Upon checking closely, the role this table plays is quite minimal. The contents served by this table is almost equivalent to public.epoch_stake itself, with the difference being instead of bech32 values, epoch_stake contains the primary keys for stake_address and pool_hash table (referenced as foreign key). Moreover, each of the column in epoch_stake table is already indexed.

Thus, it might make little sense to use this table simply for expansion of ID to bech32 values, especially when every observation (each active stake address for each epoch) will get repeated ~120 length of chars instead of 2 bigint(s) . Moreover, the primary key for this table is btree (stake_address, pool_id, epoch_no) which makes the index size to be almost the same as the table itself.

Lastly, there isnt an archival process here - so , especially visible on guild network where we have > 20K addresses, this almost ends up being a ~waste of storage for minimal performance gains.

Map for grest.account_active_stake_cache vs public.epoch_stake

Impact

It is primarily only used at two places:

1. Stake Distribution Cache to capture active stake of "current" epoch (only).
2. Account History - to get results that are fetch-able from epoch_stake table for filtered input of accounts.

Suggested change:

I believe if we do want to save expanded (bech32 view instead of ID) at all, we should preserve it only for the current state (thus, the primary key would get changed to stake_address, similarly - the insert statement would be updated for cache table). This should have neglible/no impact to SDC. While for account_history, we'd need to use epoch_stake table directly instead

Duplicate apispec.yml files and replace example values for guild/test

Essentially requests to sample queries generated dynamically against each network querying each endpoint, alongwith capturing the execution times.

For the initial phase, we are utilising schemathesis to perform checks against each endpoint for schema conformance, null-checks, not-server errors.

• Not a server error tests
• Response Schema conformance
• Response Header conformance
• Status code conformance
• Content Type conformance
• Performance testing
• Benchmark timings
• Data generation and accuracy

For using monitoring layer to be compatible as per here

• Depends on enough DNS participation to have atleast 1 local failover who is consistent for each location
• Need to use Geo-Location

num_confirmation is missing in Block Info Schema. The Data type is integer according to BF

Standalone utilities that can be served from API independent of DB:

• Hash verifier script as service (to verify tx, block hash formation)
• bech32 encode/decode
• Slot to time conversion
• Time to Slot conversion
• epoch start/end time for given epoch
• ADA price endpoint (refreshed on backend every 30s)
• Extract stake key from address if available

There are two new fields in Address Info Endpoint (UtxoSet field) in BF which are missing in Koios.
I understood that I can take them from TX Info Endpoint (I don't know how do they called exactly) but it would be best to have them also in Address Info Endpoint in order to decrease the number of requests involved using Cardano Java Client Library used by Java developers (Bloxbean).

Linked Issue in cardano-client-lib Repo: bloxbean/cardano-client-lib#148

Koios 1.0.7 - Testnet

curl --location --request GET 'https://testnet.koios.rest/api/v0/tx_metalabels'
<html><body><h1>404 Not Found</h1>
Requested endpoint with provided parameters was not found! Please verify syntax referring to examples from <a href="https://testnet.koios.rest">API Documentations</a> !!
</body></html>

Is your feature request related to a problem? Please describe.
Current pool_delegation endpoint lack certain data to know when the delegation took place. It also contains redundant information currently echoing back either the epoch used as input to query or the latest epoch, in each row.

Describe the solution you'd like
I propose to replace the current epoch_no in response with active_epoch_no. This is a much more useful data point and is requested by users of the service. This requires a join on delegation table but shouldn't add much processing and is fairly easy to implement.

Describe alternatives you've considered
Keep endpoint as-is.

Additional context

Epoch Protocol Parameters key_deposit and pool_deposit data type should be String as decided for lovelace fields

Many endpoints could leverage bulk query support (similar to tx endpoints) instead of 1 per request

Initial candidates:

• /block_transactions
• /address_info
• /address_assets
• /account_info
• /account_rewards
• /account_updates
• /account_addresses
• /account_assets
• /account_history

Less critical:

• /asset_policy_info
• /pool_delegators
• /pool_blocks
• /pool_history
• /pool_updates
• /script_redeemers

Note that we when we refer to hash, preferred hash is bech32 ID

• Info for specific contract (historical record of data script, redeemer, datum, collateral and time)

Add integration with pool_groups repository.

Need to put a bit of thinking into how clusters are represented (simple solution could be a Flag of Y/N against single pool - which wouldn't have to worry about descrepancy)

Is your feature request related to a problem? Please describe.
Trying to fetch the updated metadata of
https://pool.pm/902eaf4440531a68e8a00345f3c7455ad2c8c8f3bbd59355fa14b9aa.PomCyclr3757

However the metadata there is the most updated but koios only returns the mint metadata of a token

Describe the solution you'd like
To be able to fetch the latest metadata of a token instead of the mint one

Describe alternatives you've considered
No alternative solutions I can think of

Additional context
N/A

Description

Fetch historical information available with a dbsync (it may not be consistent across instances as it would depend upon offline metadata fetch), for a give pool to it's pool_updates.

Is your feature request related to a problem? Please describe.

cardano-community/guild-operators#1465

Hi!

I would like to suggest to improve asset_txs with the param _after_block similar to address_txs ( see https://api.koios.rest/#post-/address_txs). SQL Code: https://github.com/cardano-community/guild-operators/blob/alpha/files/grest/rpc/assets/asset_txs.sql

The goal is to only get the delta of transactions to speed up the query.

see discussion in tg: https://t.me/CardanoKoios/8500

thanks!
Sergio

Is your feature request related to a problem? Please describe.
So for my example I want to fetch royalty information of a policy / asset however I have to query every single asset of that policy using, asset_policy_info which is unnecessary data and leads to time outs and very long queries, the ability to fetch a single asset from a specific policy just to check the metadata would be nice so we could use asset_info but with a null asset_name as then we can fetch 1 asset from that policy instead of the 10,000 and so forth from the other one

Describe the solution you'd like
Null as an acceptable value on the asset_name field ( or for it to be optional ) so then it will fetch 1 asset, most likely the first one doesn't matter which one

Describe alternatives you've considered
None, this alternative was thought of by Priyank

Additional context
N/A

The asset array being hierarchical causes trouble for policies (as the volume of data cannot be natively be filtered by clients). We'd want to remove policy_id from response, and only return assets from the query. This is more of a bug-fix than a feature request (for those who use this option)

Similar to this from blockfrost:
https://cardano-mainnet.blockfrost.io/api/v0/scripts/datum/{datum_hash}
https://docs.blockfrost.io/#tag/Cardano-Scripts/paths/~1scripts~1datum~1{datum_hash}/get