1hive / apiary Goto Github PK

Make it possible for organisation editors to tag their organisations with a list of tags to improve discoverability.

The organisation list should be filterable by tags.

This requires #17 and #44 and #46

Since we're using Heroku to deploy Apiary for now, we might as well make use of their pipeline review apps feature.

This will make it easier for us to review new features.

It will also make it less painful when we add new stuff to the worker, since we can let it catch up before we promote it to production.

Is your feature request related to a problem? Please describe.

Currently the application detail view is surfaced in the client via the App Center, but this doesn't necessarily make sense in the context of Apiary which is User rather than Organization centric. When a user views apps on Apiary, they may want to see the application details but we don't want to make them navigate into any specific organization.

Describe the solution you'd like
It would be great to provide an app detail view directly in apiary, perhaps we can essentially use the same view that is provided in the App Center of the client but outside of the client itself.

We could then add a few additional summary stats related to installs, app score, and app mining payouts.

We could also include contextual discussions if we implement #77

Additional context
This feature would be a good way to improve application discovery

This is a large task.

At the moment, every 50 blocks a checkpoint is set by persisting the block number in Redis. This block number is used as a starting point if, for some reason, the worker crashes and has to restart.

Due to the asynchronous nature of how we process data, a checkpoint might be set at a block, while some other data processing saga fails before that specific checkpoint. This is because blocks are fetched in chunks, after which a checkpoint is set, regardless of how far we are with processing the data in those blocks.

One way to solve this is to only set checkpoints once the data in the blocks has actually been processed. Some inspiration on possible approaches:

• Saga composition
• Action pipelines

Is your feature request related to a problem? Please describe.

We want people to be able to go in and out of the Aragon Client interface without leaving Apiary. We also want to add functionality for managing subscriptions and notifications as an individual user. Some of this functionality exists within the Client (notifications, web3 auth).

Describe the solution you'd like

Ideally we would be able modularize the user focused elements of the client (web3 auth, notifications, local labels):

So that these can be provided by Apiary, and then simply embed the client without these features, to minimize compatibility/maintenance issues.

Describe alternatives you've considered
Not sure, but would totally open to other alternatives that meet the fundamental requirements.

Somewhere I accidentally set timestamp instead of created_at on organisations.

Add a feed of activity from organisations you have subscribed to and, of course, add the ability to subscribe to organisations.

Requires #17

The current approached used by the worker to track organization creation is by looking at logs for interactions with pre-deployed templates. In order for use to track a DAO we must have the template address configured in the application. As more custom templates are added or updated, this will become less accurate/scalable to maintain.

An alternative approach proposed by @onbjerg is to use an Archive node with tracing enabled so that we can analyze the bytecode of contracts as they are deployed. The process would look essentially like this:

1. new block comes in
2. traverse over all transactions and check if a contract has been deployed by checking the trace of the transaction
3. for each newly created contract, hash the bytecode and check if the hash of that bytecode corresponds to the hash of the KernelProxy bytecode (works for AppProxy too)
4. if it does then it's a dao

Then we only need to maintain a list of the hashes of important bytecodes as they are upgraded, and we will capture all DAOs created regardless of whether they use a specific template or not.

Make it possible to add additional information on organisations for a list of accounts, known as "editors".

Example information that could be added:

• Actual name (not ENS name)
• A short description (in plaintext)
• A long form description (in Markdown)
• A logo
• Social links (website, Twitter, ...)

Requires #17 and #44

In order to understand the size (in terms of number of participants), we should track the number of unique address interacting with an org an expose this as a column in the dataview component.

Whenever an app is updated (i.e. whenever setApp is called on the Kernel) we should track those updates and put the version number in the apps list of organisation objects.

Possible flow

Whenever a SetApp event is fired

• Update every app in the apps list on the organisation object corresponding to the address of the log origin, where the app ID of the entry in the list corresponds to the appId topic of the log.

It might be beneficial to have a template filter for now, until the app store is functional, since we're probably not going to see much divergence from these templates until then.

Tasks

• Expose template in API
• Add template filter in UI

It would be faster to run each component's tests and linting in parallel using CircleCI's fan-in/fan-out workflows.

It would be cool to support multiple networks such as Rinkeby where Aragon is deployed.

A simple implementation of this might be as follows:

• We run a worker for each network we want to support, each with an environment variable ETH_NETWORK set to their corresponding network name.
• Every top-level document (such as entries in the orgs and apps collections) have a property network attached to them
• The API should be able to filter on these top-level documents using a network name
• The website should have a simple dropdown with the supported networks

I estimate that if the feature is implemented like this we would be looking at around 4-5 hours of work with the caveat that infrastructure costs will scale linearly with the number of networks we support, since we have to run a worker for each network. Furthermore, if we want to eventually run archive nodes, then we'd have to run one for each network too.

Networks we should support:

• Mainnet
• Rinkeby
• Flora

Set up some sort of CI. For now it's OK to just run the lint pass.

Track activity (i.e. transactions and transaction paths) for apps tied to organisations.

Possible flow

• For each transaction that is directed towards a known AppProxy, look up the Radspec description for the method that was called and persist that in the database
• For each internal transaction as a result of above transaction, track do the exact same thing
• For each of the above steps, increment the appropriate activity counter for both the organisation and the apps in question. This is useful because we won't have to calculate the amount of activity for apps and organisations at request time.

In order to track internal transactions we need to run an archive node with tracing enabled.

Furthermore, we probably also want to persist the Radspec descriptions for each app when they are published (found in the artifact.json file) such that we don't have to fetch them from IPFS every time we track activity.

A lot of lint errors are caused by using tabs instead of spaces. An .editorconfig should mitigate this.

Budget 50K DAI

Mandate

To develop and launch an App Mining program where we reward publishers of Aragon Apps based on an agreed upon set of usage KPIs. The KPIs will be used to compute an Organization Score and from that an Application Score.

The following KPIs are what we will initially focus on implementing

To compute the organization score each KPI would be normalized using as a ratio across all organizations, then KPIs will be combined into a single score using a weighted average.

For example if an Org holds 100 ANT and the total amount of ANT held across all organizations is 1000 ANT, then the normalized ANT KPI would be 10%. The same process would be used to normalized AUM and Activity. To compute the organization score we would take ANT * .25 + AUM * .25 + Activity * .50. Let’s say the values for our example org are 10%, 20%, 30% respectively. The organization score would be 22.5%.

Then to compute the application score we would take the organization score and divide it by the number of applications installed in the organization, allocating the fractional amount to each installed application. We would compute a score for all applications regardless of whether the publisher has opted in to the App Mining program, but would exclude them publishers who have not opted in from payout calculation. If the example organization score is 22.5% and has 8 apps installed, the organization would contribute 2.81% to each app’s score.

What license should we go with for Apiary?

Luke checklist

• Determine license

Oliver checklist

• Add license to README
• Add license file
• Set license in worker
• Set license in API
• Set license in website

It would be useful to keep track of the total organization value in terms of the assets held in its vault/agent app. We can initially just look at DAI, though it may be interesting to look at other assets and convert to value to DAI terms.

When Apiary started as Daolist, it was a small solo hobby project, so tests were not a high priority.

Now that we're open source and the complexity of Apiary is increasing rapidly, we should start writing tests before the complexity becomes unmanageable.

Tests should be added for

• The worker (which is why this task is in the Discovery board)
• The API

We could test the front-end, too, but it doesn't matter too much.

Add addresses and signatures for the following new kits:

• Company
• Company Board
• Membership
• Reputation

This strips various stuff from React amongst other packages, reducing the bundle size.

Right now DAOs aren’t discoverable via search engines. I.e. when you search “Aragon Governance DAO”, there's no direct link returned to the DAO or a standard profile page.

We (Open Work Labs + Autark) are starting to work on organization settings and would love to help facilitate data standards and SEO. One thing specifically we’re wondering about is using something like schema.org to help index organization info and tags, and maybe later things like voting events and their associated proposals.

We’re curious to get a sense of the appetite for this. How important is it that DAOs are easily discoverable via web searches? What about the data inside them, like voting events?

It seems like the usage for "catch up" is quite high, but should be significantly lower under normal operation. It may make sense, once initially caught up to create backups to avoid the need to resync fully in the future.

Originally posted by @lkngtn in #1 (comment)

The same list view as the one we have for orgs should also be available for apps, but with the current metrics instead:

• App name
• # installs

Add something like prettier-standard or eslint to the tooling to ensure that the code looks consistent.

Add the currently selected filter and sorting to the query parameters in the URL, making your current search shareable.

It would be great to provide users with contextual discussions related to elements on Apiary and 3Box threads seems like it may be the simplest way to do that. https://docs.3box.io/build/web-apps/messaging/choose

Contextual discussions would be most relevant in a users activity feed where each activity represents a particular element (eg a vote in an organization), but it could also be used when viewing application details, which may be more immediately relevant to apiary.

Save the block at which a DAO with no name (i.e. a DAO that was not created with a kit).

Since address might be confusing as a property for apps, since people might think it refers to the implementation contract, we should rename it to the more appropriate repository, since the address actually just refers to the APM repository of the app.

The webworker seems to crash and/or hang after running for some time. Currently stuck at 565 DAOs with the most recent being "specialbank", compared to scouts reported 635 with most recent being "violeto"

Make it possible to sign in to Apiary using Web3. Signing in gives you the ability to persist your subscriptions across sessions.

I imagine the flow to be something like:

• Click "Sign in"
• A TOTP is sent from the server to the client
• The client signs the TOTP using Web3
• The signed TOTP is sent back to the server and verified
• A token is sent to the client that can be used for the remainder of the session

Another approach would be to possibly use the signed TOTP directly as the token.

If there are any other approaches I've missed then I'd like to hear them 👇

We should cache each of the node_modules directories in each of the components.

Some reading.

It would be nice to be able to see which accounts have voted for what in a DAO. It would also be nice to be able to see general voting activity in an organization (average % of participation, frequency of activity, stuff like that). This would be great to A) provide more transparency into the activities of an org, but also B) help people like me who can't remember if they voted in an open dot vote vote

It would be cool if there was an API to easily pull this data and/or a dashboard that showed it whenever you click on a DAO.

Considered asking for this as a feature request from the current teams building voting apps, but those teams seem stressed out af and not down to alter their roadmaps with feature requests. Also, having the data available at a "block explorer" level vs within the DAO means that this data can also be used to track the health of the Aragon ecosystem for App Mining or NestDAO purposes.

We can also move to a list view rather than cards to better allow for #5

The extra space on the right can be used to display aggregate/summary stats about the DAOs.

Mockup:

As a user it would be great to be able to sort and filter the list based on different properties.

Sorting

• By creation date

Filtering

• By installed application
• By creation date

The ENS name of apps is under the ens property, and it should be that too on organisations for two reasons:

• Consistency
• name should in actuality refer to the human name of the organisation, not the ENS name

In order to provide customisation features for organisations on Apiary, we need some way to prove that a specific account "owns" or "controls" an organisation.

This should be possible with the new Agent app.

Possible solution

Apiary would have a Registry contract somewhere that maps an organisation address to a list of "editor" addresses. In order to add an address to the list for your organisation you have to execute e.g. add(address editor) on this Registry contract.

Upon calling of this method, the contract will check if the caller is an Agent app and add the editor to the list of addresses for the kernel of the Agent app.

Apiary would periodically check for events (e.g. EditorAdded(address org, address editor)) on the Registry contract and give these addresses editing rights on Apiary to their respective organisations (see #17 for authentication).

Caveats: Right now it is only possible to use the Agent app using dao act in aragonCLI. Also, this approach requires organisations to have an Agent app.

There's a considerable amount of wasted screen space we should leverage. On smaller devices we should just remove the padding on Content entirely to have a full-width feel.

Currently Apiary is a centralized application, like a block explorer. We could potentially decentralize part of the backend by leveraging The Graph. Or we could stick with a centralized backend for simplicity and flexibility.

I lean towards the later, but wanted to make sure there was an issue to discuss this topic.

Whenever you visit Apiary, you might see a brief flash of the website with no styling and the text "Your browser does not support JavaScript [...]".

One of the most relevant metrics to sort on is the level of activity (# of transaction to or from an organization). We should track the number of these interactions and expose two new columns in the dataview: total # interactions, and rolling 30 day # interactions

We should also keep track of the most recent interaction (date), and expose that as a column and filter option.

As the data model sometimes changes it would be wise to have some migrations instead of migrating data manually.

It would be nice to make the worker runnable as a one-off task so we can do non-destructive migrations with no downtime.

The only thing that we would have to enable is that the worker can start from a custom block instead of the default one or the one in cache.

An example deployment of something like #9 (which is non-destructive since it only adds data) would be:

• Start the updated worker from the beginning block
• When this worker catches up, shut down the old one
• Replace the old worker with the new worker