steemit / conveyor Goto Github PK

Currently both adsorber and overseer is depending on it

Currently conveyer has the get_price API which is returning the conversion rates between STEEM and other currencies/tokens. Client apps/libraries are using this (along with an account and the global dynamic properties) to calculate the balances for an account.

Here is an example of one of these calculations: https://github.com/steemit/condenser/blob/e9ff0f7814bd62874327c5e8dd84ca0239a51df0/src/app/utils/StateFunctions.js#L52

Lots of client apps/libraries will want this account balance to be shown, but implementing and maintaining these calculations in each of the various languages is tedious and error-prone. It would be nice to have one API here that wrapped all this functionality up for clients to use.

Not strictly a conveyor problem (also observed on faucet which also uses RDS)

Some requests fail with getaddrinfo EAI_AGAIN when doing a db lookup, we might need a wrapper around db calls that retries a couple of times

Stored per user in S3 under bucket/steemitapi_${username}_flags.json

Methods:

get_feature_flags <username> - {"flag_name": boolean}
set_feature_flag <username> <flag>
get_feature_flag <username> <flag> - boolean

set_feature_flag_probability <flag> <probability>
get_feature_flag_probabilities- {"flag_name": float}

Flag Resolution:
user_json > flag_probabilities > service_config > false (not written to user json)

Should parse floats from string values gotten and throw on invalid, currently allows setting an invalid probability

The user accounts trie used in the conveyor.account_autocomplete relies on a static list of accounts. This should be changed to periodically update itself to reflect newer accounts.

Currently a simple regex is used, we should match what faucet is doing

if i’m reading the current implementation correctly, if we set the flag’s probability to 10%, then change it to 50%, some of those 10% will lose access. that shouldn’t happen.

also, if we set it to 50%, and 100 people use it, 50 being enabled, and we later change it to 10%, slowing the rollout, 40 of those 50 people will lose access, which shouldn’t happen.

whenever a feature flag is read for a user, it should be stored to a persistent datastore. i think an SQL table is appropriate for this because we will need to batch update sets of users who got false-valued feature flags set probabilistically when we roll things out to wider audiences.

Stored in s3 as file per user: conveyor_${username}_prefs.json

get_user_prefs <username> <key | undefined>
set_user_prefs <username> <object | key-value pair>

Both calls require signature of <username> or admin

We need to store user emails and phone numbers for accounts created when users signup via sc2/faucet

This means that faucet must be capable of taking a rpc endpoint that it should hit with that data once the user has signed up, also faucet should remove its own records of the data once the endpoint has been successfully called.

Endpoints

set_user_data <account> <{email, phone}>
requires auth, signed by account or an admin_role

get_user_data <account>
requires auth, signed by account or an admin_role

is_email_registered <email> responds with true if email is in db, otherwise false
requires auth, signed by an admin_role

Needs steemit/koa-jsonrpc#1

New name is conveyor

As a user,
I want to be able to see when posts and comments have been made by people at Steemit Inc,
so I can verify that the information can be trusted.

This new tag can be assigned to Steemit employees and contractors.

This will enable us to:

• Test new features in production (alongside our new feature flags)
• Add a badge / insignia to employee usernames and profile pages denoting their association with Steemit Inc.
• Possibly create functionality in the future which helps employees adhere to the new code of conduct rules

Add rpc methods to list/add/remove drafts, saved in s3

Data structure draft for drafts:

[
  {
    title: "draft post title",
    body: "I'll do it live damnit!",
    date: "2017-09-19T19:37:12.917Z",
    uuid: "3A60740F-06AC-4BE4-B017-96150DE1510F"
  }
  ..
]

Stored under bucket/steemitapit_drafts_.json

There should be a user setting that allows people to choose a layout style (e.g. list view or blog view) in the upcoming website restructure.

spec: (all calls require admin signature, this is an internal support system)

• call: define_tag(tagname, description)

• call: get_users_by_tags(tagname) # one tag

• call: get_users_by_tags([tagname]) # list of tags, ANDed

• call: add_tag_by_user(username, tag)

• call: remove_tag_by_user(username, tag)

• call: get_tags_for_user(username)

• table: taglist (id, name, desc, created)

• table: usertags (id, username, tagid)

The idea here is that redeemer and faucet and future blockchain followers and other things can add and remove tags to users based on their username, then other things (e.g. redeemer) can use the existence or nonexistence of a tag on a user to do stuff, e.g. not-re-delegate SP, or display a warning or error, or whatever.

Tags must be defined before they can be used, so that we can store a description of what that tag is specifically used for, because people cannot be trusted to use appropriately descriptive tag names.

Tags will probably be defined via admin CLI.

I'm sorry but I am not sure where this should go, if here, condenser or steem itself. Also, this could entirely be a FF issue or an issue with dsteem, but figured I post it FYI.

It seems that since yesterday if you try to use https://api.steemit.com for a site and view that site in FireFox 58 you'll get in the console:
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://api.steemit.com/. (Reason: CORS header ‘Access-Control-Allow-Origin’ missing).

Using another node, such as https://gtg.steem.house:8090/ works fine.
Using Chrome/Chromium works fine.
Using FF and going directly to https://api.steemit.com works fine.

Only difference in headers seems to be in the response from api.steemit.com:
content-security-policy : upgrade-insecure-requests
X-Firefox-Spdy : h2

Possibly one of these and some changes in FF 58 (that seems to have been released yesterday) is causing it.

My use case:
I fetch my steem blog using dsteem. This was working perfectly for the 3 weeks since I created it, but last evening I noticed it was not working. Have been troubleshooting today and this is what I managed to gather (I'm not usually doing web development so really have little to no clue).

I also tested using https://playground.steem.vc and same issue there if you change the node to api.steemit.com

Condenser is configured two ways right now:

1. defaults in codebase
2. overrides in environment

I would like to add a 3) runtime variables provided by conveyor

Condenser would have a set of "runtime config" keys that have defaults loaded in from the codebase. On startup and every 30 seconds thereafter, condenser queries conveyor for a set of condenser runtime setting overrides, which conveyor probably wants to store in redis or postgres.

Watt could then be upgraded to make signed calls to conveyor to modify these runtime settings which would take effect in condenser in under a minute.

Some items that could go in here:

bool maintenance_mode
steem_username[] currently_featured_communities
steem_tag[] currently_featured_tags

et c