Git Product home page Git Product logo

js-tableland's People

Contributors

brunocalza avatar carsonfarmer avatar chainguys avatar datadanne avatar dependabot[bot] avatar dtbuchholz avatar eightysteele avatar joewagner avatar sanderpick avatar textileio-machine avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

js-tableland's Issues

Notion integration has trouble parsing markdown

Expected Behavior

I can open an issue in github using the issue template

Actual Behavior

This is a test to see if I can open an issue using the template

Steps to Reproduce the Problem

Specifications

  • Version:
  • Platform:

#83 contains 2 unrelated problems. I'm splitting this into it's own issue.

Wallet can be connected to a different network than SDK

From discord:


strange, @njwags i'm getting this back from create on goerli:

{
  tableId: undefined,
  prefix: 'rigs_contract',
  chainId: 5,
  txnHash: '0xd78df58512c760846f8045ded312a2ff4932b9dd912850ae40d303ab00b05653',
  blockNumber: 27146217,
  name: 'rigs_contract_5_undefined'
}

txn hash doesn't actually exist if you look at goerli etherscan


The SDK is expecting to be connected to Goerli, but transaction was sent to Mumbai.
I believe the fix will be to enforce the signer's network to match the SDK in the siwe method

cc: @andrewxhill @sanderpick

Expose transfer method(s) on TablelandTables registry contract

Is your feature request related to a problem? Please describe.
The SDK should expose the transfer method(s) on the registry contract to support transferring ownership of tables.

Describe the solution you'd like
This can be able simple as creating a high-level API endpoint to expose the transfer method(s). An alternative would be to expose more of the contract methods in general, in a generalized way.

Describe alternatives you've considered
This is an important basic function, it should be done, and it should be relatively easy.

Additional context
This is the work required to enable tablelandnetwork/js-tableland-cli#20

Add enableTestnetRPCRelay flag

This flag will indicate if the smart contract interactions should go directly to the Smart Contract, or be relayed through the validator. The validator will initially refuse to relay on any mainnet due to cost.

`list` method is returning an Object

Expected Behavior

After connecting and calling siwe, calling list should return an Array of the callers tables. If the caller does not have any tables, then list should return an empty Array.

Actual Behavior

list is returning an empty Object.

Steps to Reproduce the Problem

  1. connect
  2. call siwe and login
  3. call list and inspect the response

This appears to be a bug in the v2 changes. The helper camelCaseKeys should only be called with an Object and in the case of the list method is being called with an Array.

Push `@tableland/sdk` to skypack to fix import error

Expected Behavior

I should be able to use @tableland/sdk in skypack and properly import connect.

However, Tableland doesn't appear on skypack (here's the skypack cdn link, for reference: here). Also, an error is thrown in the browser's console:

Uncaught SyntaxError: The requested module '/-/@tableland/[email protected]/dist=es2019,mode=imports/unoptimized/proxies.js' does not provide an export named 'proxies' (at connector-1be2aba6.js:3:9)

Steps to Reproduce the Problem

  1. Go to https://codepen.io/dtbuchholz/pen/RwQdwoe
  2. Open inspector/console
  3. Click Connect button to see error

Specifications

npm `prepublish` id depreciated

for reasons described here: npm/npm#10074 using the prepublish script is depreciated.
TL;DR; any time you run npm install while developing this lib the prepublish script is run. This is a bit confusing, and problematic if trying to coordinate changes between this lib and @tableland/eth

I think we want to move to prepublishOnly, but interested if others have any opinion?

Source map points to missing source files

The build of this code base that is published to npm contains .map files with reference to the src directory. The src directory is not published to npm which means that frontend tooling like Vite send out many Warnings anytime this package is used.

look in dist/esm/src/main.js.map, you will see the following JSON

{"version":3,"file":"main.js","sourceRoot":"","sources":["../../../src/main.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,OAAO,EAAE,MAAM,oBAAoB,CAAC;AAC7C,cAAc,iBAAiB,CAAC"}

Note that the "sources" property refers to ../../../src/main.ts. That file is not published to npm, hence the warnings from Vite

Creation of a table with a foreign key fails

Expected Behavior

Creation of tables with relations (with primary key and foreign key)

Actual Behavior

When I try to create a table with a foreign key property, it falis:
(node:2592) UnhandledPromiseRejectionWarning: Error: processing table registration: processing register table: exec CREATE statement: ERROR: relation "tablename" does not exist (SQLSTATE 42P01)

Specifications

  • @tableland/sdk: ^0.0.1

My Code

export const table1TableScript = () =>
	'CREATE TABLE Table1(id int,  primary key (id));';

export const table2TableScript = () =>
	'CREATE TABLE Table2(id int, table1_id int REFERENCES Table1(id),  primary key (id));'; //Also tried instead of 'Table1' -> 'table1_id'

First I create 'Table1' and afterwards run the 'table2TableScript' script which fails...

Appreciate your help.

The description option no longer works

Expected Behavior

When using the SDK to create a table, the caller used to be able to add a description to the table via an options Object

Actual Behavior

The description is ignored

Steps to Reproduce the Problem

  1. Call the create function with an options object containing a description property
  2. Call the list function with the same wallet as step one and notice that no description exists

Incorrect types for query responses

It looks like the Typescript types for query responses are currently incorrect. Whereas I would expect, based on the types that the following returns an object with columns and rows, I actually get an object with { data: { columns, rows } }. However, the Typescript compiler complains with Property 'data' does not exist on type 'ReadQueryResult'.. The fix is to either automatically destructure the returned object (internally) and return rows and columns, or to update the types to reflect the fact that the data is structured under the data key. The former seems like the nicer option.

const response = await connection.query(`select * from some_table;`);

Document delay

It would be useful to have documentation (especially around read/SELECT functions) discussing the delay between when an update transaction succeeds and when the change is likely to be reflected in the results of a table read / SELECT statement call. There are plenty of use cases where someone might take an action that appears to succeed, and then immediately reload a view expecting it to reflect the updated state. If that view gets data from a read/SELECT query, it won't reflect the changed data right away, leading the user to think the change was not effective, and possibly taking additional actions such as trying to repeat the state change, which might make things worse. There's currently no documentation about the source(s) or extent of this delay or if a developer can do anything to tell if it's over.

Tx takes forever to confirm & wait logic spams provider

Describe the bug
I'm creating a table using Polygon Mumbai. While waiting for the tx to confirm, Tableland makes ~1176 total requests to my provider (Alchemy) for a single table/tx. This is not ideal. I waited for 36 minutes before the process finally finished and the tx was included in a Polygon Mumbai block. I tested this out on both the SDK and CLI, and Tableland never exits the process.

Now, the obvious question is what the root cause is. It didn't look like Polygon Mumbai was down, but I'm not sure. I did this a few times to make sure I wasn't missing anything. The first in the list below was the one that took the longest (36 minutes). The others also took at least 5-10 mins. and resulted in a high number of provider requests as well:

Perhaps the getDefaultProvider and other logic related to using Polygon Mumbai is leading to some issue where the tx is never prioritized by miners and sits in the mempool? I'd have to look into how Mumbai works, but that could be part of the root cause -- gas price is too low?

To Reproduce
Steps to reproduce the behavior:

  1. Set up your env in Node.js and simply create a table on Polygon Mumbai; here's my setup where I run node <filename>: see gist. Alternatively, just use the CLI's create command (which is what was ultimately used in the primary example mentioned).
  2. Wait for the tx to complete.
  3. Check your Alchemy dashboard and be surprised.

Expected behavior
Ideally, waitConfirm logic should time out after X period of time or Y number of requests (by default). Or maybe in an ideal state, there's there's a way to listen to the validator for when the tx is seen & processed. Thus, no more spamming the node provider.

Perhaps this is an edge case, but I'd have to dig deeper to see how often Mumbai has this issue.

Screenshots
This screenshot includes 2 separate create attempts -- the first had 293 total (I think it was this tx), and the second led to 1176 requests (this tx). Peep the RPC calls.
Screen Shot 2022-08-20 at 5 21 20 PM

Desktop (please complete the following information):

  • OS: Mac
  • SDK: 3.1.0 (and for reference, CLI 3.1.1)
  • Node v18.7.0

"Dry Run" create table statements

This may have been discussed already, but I can't find reference to it on discord or notion.

When I use the SDK to create a table, the flow is as follows:

  1. Check if the caller's address is on the Tableland passlist
  2. Assuming the caller hasn't provided an ID, call registerTable which does a safeMint on the smart contract to get a tokenId
  3. Send and tableland_createTable method RPC request to the Validator with a SQL statement.

The issue I had while testing is that when the sql statement in step 3 is invalid the safeMint happens and the user pays for the tokenId, but then the table is not created. This isn't a big deal on rinkeby, but once we go to mainnet this will be a huge pain point for anyone making a mistake during a create call.

The issues with validating SQL purely in the SDK code we dealt with in #19 and #23 makes me think we should be asking the validator if the statement is going to work before doing safeMint on the SC.

For the sake of discussion I'll suggest that the Validator could provide a way to do a "dry run" of sql statements meant to be used with the tableland_createTable RPC method.

Need to add and expose a setController method

The SDK should enable easily setting the controller contract for tables. This can be done via a direct call to the smart contract (eth-calls.ts), the validator's RPC method (tableland-calls.ts), or potentially enable both and let the SDK user choose.

429 errors bubbling up too high

Expected Behavior

A tableland call (e.g. create()) which triggers a transaction that succeeds doesn't throw an error

Actual Behavior

Sometimes, MetaMask encounters an error inpage.js:1 MetaMask - RPC Error: Non-200 status code: '429' details: 'Rate limit exceeded: 40 per 1 second. Check respon…ticvigil.com/ to avoid hitting public ratelimits.', code: -32005, when checking the transaction status, and that particular status check fails. This error should be caught in the code which waits for a response, and continue waiting, as a subsequent status check can still succeed and reveal that the transaction was successfully mined into a block. The error should not be bubbled up to the calling context of create().

Steps to Reproduce the Problem

  1. In code running in a browser with MetaMask, connect to tableland and make a create() call from the SDK. It is an intermittent issue affecting <50% of calls, but still problematic.

The workaround is to catch the error in the calling context and re-trigger creation of the table, but the gas/resources wasted on successfully creating the first are lost due to poor error handling in the SDK.

Specifications

Large number handling

Using the SDK, the results of a SELECT statement includes a rows object with an array of row arrays. If one of the selected values was an int type during table creation, it is present in the appropriate position of each row array as a JavaScript number.
This works fine in most cases.
However, my understanding is that the equivalent Solidity type is uint256.
One common use case would be to have a numeric integer primary key column of type uint256 and include a reference to this id in a second table as a foreign key reference, even if not explicitly declared.

Does Tableland's SDK actually store and return large numbers correctly? I was trying to build a test case to see what would be returned in the row value when a value was outside of JavaScript's integer range (specifically, I tried the number 18446744073709551616 (2^64) added into the query string without having to be stored numerically in JavaScript) and it looked like the change wasn't actually applied. Reading the docs, it looks like the type int64 might be more appropriate than uint256 as the EVN version of Tableland numbers, but even this has values well outside Javascript's safe range for accurate computation.

Other methods of interacting with blockchains tend to use some BigNumber implementation for numeric values input to or output from blockchain functions; this is one that does not. Should it?

calling the `write` method with valid sql that violates a table constraint should throw an error when the caller is waiting for confirmation

Describe the bug
When you create a table with a primary key, then try to insert duplicate values for that primary key using the write method, the method does not throw an error even though the writes do not succeed.

To Reproduce

  1. Create a table with a primary key.
  2. Insert the same value for the primary key twice.

Expected behavior
Assuming the caller is using the default value for skipConfirm an error should be thrown after the second insert fails.

See chain of messages in Discord for more detail: https://discord.com/channels/592843512312102924/902337528902860820/1012107267816034334

cc: @carsonfarmer

Multiple INSERT ops in one tx?

Is it possible to have multiple INSERT operations (e.g. writing data to two tables) in the same EVN transaction?
I seem to remember reading that it's not possible to have two CREATE TABLE operations in one transaction, though the docs search feature seems broken and I can't find exactly where, and the limits section doesn't say anything about even that restriction.

I am trying to run a smart contract with two functions (that matter for this discussion): one that does an INSERT query, and another that does a different INSERT query on a conceptually related table and then calls the first one for a second INSERT. Calling just the first function by itself works fine, with results visible on a subsequent SELECT. On calling the second, I can see the first INSERT take effect with the INSERTed data returned in a subsequent SELECT. However, the second INSERT seems to silently fail when it's called from the second function. The first function still proceeds to emit an event indicating successful update, in which the parameters appear correct, but the data isn't reflected in any subsequent read operation.

The only thing I can think of at the moment as a possible reason is an undocumented restriction against writing to more than one table in one query. Such a restriction (especially undocumented!) would be pretty severe limit on the use of the relational data table model.

It's also a bit concerning that the Tableland INSERT call succeeds without revert() even when the call is failing and the data is not getting added - is there any good way to detect that in the EVM environment?

Under "walkthroughs" on the left there's one for the Javascript SDK but not for EVN / Solidity. At the bottom (e.g. of this page there's a link to "Hop into Discord" which is a completely empty server, visible only after completing Discord's full sign-up process if needed:
image
It's the same result when using the Discord link on the Community page.

The website repo, visited in the hopes that GitHub search might work better than the on-site search, reports that it's archived with reference to a different Discord link, which renders like this:
image

Therefore I'm posting here, but figured I'd include those notices to provide a better idea of what Tableland's new user experience is currently like.

unsupported getDefaultProvider network for Polygon Mumbai

Describe the bug
We have now switched to Polygon Mumbai as our default network... but there is no default provider available in ethersjs for Polygon networks. This means that folks that don't have a provider API key for Infura, Alchemy, or Etherscan run into issues when trying to use Polygon.

To Reproduce
To reproduce, simply try to issue a mutating call (such as create or write) using the SDK when connected to Polygon or while simply using the defaults in NodeJS.

Expected behavior
There should at least be a sane default, or a handy error indicating how to fix this.

Additional context
This is due to our recent move to Polygon by default. We should try to have a default Polygon provider that we use internally to avoid this bug/issue.

Typescript Svelte + Rollup incompatibility

Expected Behavior

When using Svelte with the default bundler, Rollup, This library should not cause warnings.

Actual Behavior

Rollup complains with "The following packages did not export their package.json", and "this has been rewritten to undefined"

Screen Shot 2022-05-27 at 9 53 52 AM

Steps to Reproduce the Problem

  1. Follow the Svelte docs example using degit and create a Svelte app from template that includes Typescript support
  2. Import this SDK in the app
  3. run the app with npm run dev

Allow specifying Bearer token as option to connect

A user on discord is asking to be able to pass the value of token in to the call to connect.
Apps are able to see the token, so unless we are planning to make that private, I don't see the harm of letting them pass in the token.
e.g.

const tableland = await connect();
// The app would save this value somewhere i guess?  My thought is that this is not best practices, but I would need to do some research
console.log(tableland.token);

Later on after page refresh...

const token = getUsersTokenSomehow();
const tableland = await connect({token: token});

The person asking for this on Discord wants to use localStorage, which is going to be available to the entire page. This is probably not best practices, but I'm not sure if it's the place of this library to determine that?

@carsonfarmer @awmuncy any thoughts? The PR would be a one liner, I'm happy to create it if we all think doing this makes sense?

Alchemy Polygon provider returns underpriced gasPrice

Some setup to show the issue inside registerTable:

async function registerTable(
  this: Connection,
  query: string
): Promise<ContractReceipt> {
  this.signer = this.signer ?? (await getSigner());
  const address = await this.signer.getAddress();

  const contractAddress = this.options.contract;

  const contract = TablelandTables__factory.connect(
    contractAddress,
    this.signer
  );

  const oldGasPrice = await this.signer.getGasPrice();
  console.log("oldGasPrice", oldGasPrice.toNumber());

  const feeData = await this.signer.getFeeData();
  console.log("gasPrice", feeData.gasPrice?.toNumber());
  console.log("maxFeePerGas", feeData.maxFeePerGas?.toNumber());
  console.log("maxPriorityFeePerGas", feeData.maxPriorityFeePerGas?.toNumber());

  const tx = await contract.createTable(address, query);

  return await tx.wait();
}

This issue (ethers-io/ethers.js#2439 (reply in thread)) is unrelated, but has a relevant comment that provider.getGasPrice should no longer be used. However, on both Optimism and Polygon, both provider.getGasPrice and feeData.gasPrice are the same. Following that commenter's logic, we get a giant (too big maxFeePerGas).

In any case, at the bottom of the thread is a comment from somebody else saying that they must always specify gasPrice. I suspect they are hitting the same issue we are hitting.

The gasPrice returned by Alchemy (using either method mentioned above) is too low compared to what metamask uses (see https://gas-api.metaswap.codefi.network/networks/137/suggestedGasFees) or what is reported on Polyscan: https://polygonscan.com/gastracker.

If I bump the Alchemy-suggested gasPrice by 10%, I can create tables:

async function registerTable(
  this: Connection,
  query: string
): Promise<ContractReceipt> {
  this.signer = this.signer ?? (await getSigner());
  const address = await this.signer.getAddress();

  const contractAddress = this.options.contract;

  const contract = TablelandTables__factory.connect(
    contractAddress,
    this.signer
  );

  const oldGasPrice = await this.signer.getGasPrice();
  console.log("oldGasPrice", oldGasPrice.toNumber());

  const feeData = await this.signer.getFeeData();
  console.log("gasPrice", feeData.gasPrice?.toNumber());
  console.log("maxFeePerGas", feeData.maxFeePerGas?.toNumber());
  console.log("maxPriorityFeePerGas", feeData.maxPriorityFeePerGas?.toNumber());

  let gasPrice;
  if (feeData.gasPrice) {
    gasPrice = Math.floor(feeData.gasPrice.toNumber() * 1.1);
  }
  console.log("bumped gasPrice", gasPrice);

  const tx = await contract.createTable(address, query, {
    gasPrice,
  });

  return await tx.wait();
}

Perhaps we should query the metamask or polyscan API for suggested gasPrice instead of relying on the provider (Alchemy in my case).

Need clarity on how the SDK will connect to Tableland when there are multiple Validators

Creating this issue to capture questions from the discussion in #112 involving how the SDK should connect when we have multiple Validators.

outstanding questions and thoughts:

  • Where does the uri value that is used in a siwe message come from? A possibility is to use the value of the host connection option, but if we go this route how would a host uri map to a Validator.
  • What does it mean for a Validator to validate the value or uri if the Validator doesn't have a specific DNS address?
  • What is the relationship between Validator, Gateway, and DNS address?
  • Will SDK users be required to specify which Validator should be connected to, i.e. a value for the host option? If not, the default value will probably end up receiving the large majority of requests.
  • Does some kind of Validator load balancing behind a single URL (Gateway) make sense?

Note: to match the siwe spec uri would need to be a string that conforms to RFC 3986

A lot of these questions seem related to Validator incentives/payment mechanisms

Nits

  • A number of deps need updating
  • We should add the eslint / prettier flow from evm-tableland and add npm run lint and npm run prettier to CI checks (as in evm-tableland)

`fetch is not defined` error due to missing `fetch` import in `tableland-calls.ts`

Describe the bug

I'm trying to simply use Tableland in an ES module, but I always get the error fetch is not defined, stemming from tableland-calls.js (using @tableland/[email protected]). I use node v18.7.0 and npm 8.15.0. My package.json has a type: module defined, and I successfully import fetch from "node-fetch" and use fetch elsewhere in my code. I try replacing my fetch import with the following 1-liner in my code (as recommended per the JS SDK docs for CJS/require), but it has no impact:

const fetch = (...args) => import("node-fetch").then(({ default: fetch }) => fetch(...args))

The only way to stop this error in my application is (usually) by reverting to CJS over ESM and then using the 1-liner/polyfill, which is not ideal. Thus, I'm assuming the issue must originate from Tableland, not my code.

After adding this 1-liner to tableland-calls.js in dist, the error is resolved. Upon digging into the SDK code a bit, it looks like the error stems from the fact tableland-calls.ts does not import node-fetch. If you import node-fetch into tableland-calls.ts and build, everything is fixed; no more fetch is not defined error.

Disclaimer: I know we use a modern version of Node, but I'm using Node v18 and still having this issue...so I'm not sure exactly why this issue is happening, to be honest. I would expect that I wouldn't have to import node-fetch with Node 18. Albeit, I'm not an expert on these intricacies. The fix described below resolves the error, and I've heard this problem pop up a few times from others, before. Not exactly sure if they were using CJS or ESM or what version of Node, though.

To Reproduce

  1. Create some Node.js ESM (using JS!) and install deps node-fetch as well as @tableland/sdk; call the file index.js.
  2. Import these into your index.js:
import fetch from "node-fetch"
import { connect } from "@tableland/sdk"
  1. Try to use the SDK's read query after connecting. You will get an error:
file:///Users/db/Tableland/projects/discord-table-bot/node_modules/@tableland/sdk/dist/esm/src/lib/tableland-calls.js:22
        const res = yield fetch(`${this.options.host}/rpc`, {
                    ^
ReferenceError: fetch is not defined
  1. Try to fix it with the 1-liner within index.js, and nothing is resolved.
  2. Manually add it in your node_modules/@tableland/sdk/dist/esm/src/lib/tableland-calls.js, and things work without errors:
const fetch = (...args) => import("node-fetch").then(({ default: fetch }) => fetch(...args))

Alternatives:

Here's the fix:

  1. In tableland-calls.ts, add @types/node-fetch:
npm i --save-dev @types/node-fetch --force
  1. Import node-fetch into tableland-calls.ts:
import fetch from 'node-fetch'
  1. Build. Now, the index.js file mentioned above works as expected without any messy node-fetch issues.

Note: in step (2) of the fix, if you don't --force, then ethers and siwe won't let you install, throwing this error:

npm ERR! code ERESOLVE
npm ERR! ERESOLVE could not resolve
npm ERR! 
npm ERR! While resolving: [email protected]
npm ERR! Found: [email protected]
npm ERR! node_modules/ethers
npm ERR!   ethers@"^5.6.9" from the root project
npm ERR! 
npm ERR! Could not resolve dependency:
npm ERR! peer ethers@"5.5.1" from [email protected]

Expected behavior:

Be able to use Tableland in a JS ESM without polyfills or workarounds for node-fetch.

Environment:

  • OS: Mac
  • @tableland/sdk 3.1.0
  • JS, not TS
  • node v18.7.0
  • npm 8.15.0

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.