tablelandnetwork / js-tableland Goto Github PK
View Code? Open in Web Editor NEWDevelopment has moved to https://github.com/tablelandnetwork/tableland-js/
License: MIT License
Development has moved to https://github.com/tablelandnetwork/tableland-js/
License: MIT License
I can open an issue in github using the issue template
This is a test to see if I can open an issue using the template
#83 contains 2 unrelated problems. I'm splitting this into it's own issue.
From discord:
strange, @njwags i'm getting this back from create on goerli:
{
tableId: undefined,
prefix: 'rigs_contract',
chainId: 5,
txnHash: '0xd78df58512c760846f8045ded312a2ff4932b9dd912850ae40d303ab00b05653',
blockNumber: 27146217,
name: 'rigs_contract_5_undefined'
}
txn hash doesn't actually exist if you look at goerli etherscan
The SDK is expecting to be connected to Goerli, but transaction was sent to Mumbai.
I believe the fix will be to enforce the signer's network to match the SDK in the siwe
method
Is your feature request related to a problem? Please describe.
The SDK should expose the transfer method(s) on the registry contract to support transferring ownership of tables.
Describe the solution you'd like
This can be able simple as creating a high-level API endpoint to expose the transfer method(s). An alternative would be to expose more of the contract methods in general, in a generalized way.
Describe alternatives you've considered
This is an important basic function, it should be done, and it should be relatively easy.
Additional context
This is the work required to enable tablelandnetwork/js-tableland-cli#20
This flag will indicate if the smart contract interactions should go directly to the Smart Contract, or be relayed through the validator. The validator will initially refuse to relay on any mainnet due to cost.
After connecting and calling siwe
, calling list
should return an Array of the callers tables. If the caller does not have any tables, then list
should return an empty Array.
list
is returning an empty Object.
siwe
and loginlist
and inspect the responseThis appears to be a bug in the v2 changes. The helper camelCaseKeys
should only be called with an Object and in the case of the list
method is being called with an Array.
I should be able to use @tableland/sdk
in skypack and properly import connect
.
However, Tableland doesn't appear on skypack (here's the skypack cdn link, for reference: here). Also, an error is thrown in the browser's console:
Uncaught SyntaxError: The requested module '/-/@tableland/[email protected]/dist=es2019,mode=imports/unoptimized/proxies.js' does not provide an export named 'proxies' (at connector-1be2aba6.js:3:9)
Connect
button to see errorRight now we have duplicated content. This could lead to errors and missed changes/updates. Let's stick to a single source of truth on the docs site.
There should be doc strings
There are no doc strings
for reasons described here: npm/npm#10074 using the prepublish
script is depreciated.
TL;DR; any time you run npm install
while developing this lib the prepublish script is run. This is a bit confusing, and problematic if trying to coordinate changes between this lib and @tableland/eth
I think we want to move to prepublishOnly
, but interested if others have any opinion?
The build of this code base that is published to npm contains .map
files with reference to the src directory. The src directory is not published to npm which means that frontend tooling like Vite send out many Warnings anytime this package is used.
look in dist/esm/src/main.js.map
, you will see the following JSON
{"version":3,"file":"main.js","sourceRoot":"","sources":["../../../src/main.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,OAAO,EAAE,MAAM,oBAAoB,CAAC;AAC7C,cAAc,iBAAiB,CAAC"}
Note that the "sources" property refers to ../../../src/main.ts
. That file is not published to npm, hence the warnings from Vite
Tableland calls ( list and create) should successfully execute in nodejs
Tableland calls (list and create) fail with the error ReferenceError: fetch is not defined
Currently we always set the token to expire in 10 hours. We might want to consider allowing the SDK caller to specify this value
Related to #56. Add support for other missing methods:
setController
getController
lockController
Creation of tables with relations (with primary key and foreign key)
When I try to create a table with a foreign key property, it falis:
(node:2592) UnhandledPromiseRejectionWarning: Error: processing table registration: processing register table: exec CREATE statement: ERROR: relation "tablename" does not exist (SQLSTATE 42P01)
export const table1TableScript = () =>
'CREATE TABLE Table1(id int, primary key (id));';
export const table2TableScript = () =>
'CREATE TABLE Table2(id int, table1_id int REFERENCES Table1(id), primary key (id));'; //Also tried instead of 'Table1' -> 'table1_id'
First I create 'Table1' and afterwards run the 'table2TableScript' script which fails...
Appreciate your help.
As a side effect of updating to siwe v2 the SDK now relies on an implementation of Buffer being available on the global scope.
More discussion here: #112 (comment)
When using the SDK to create a table, the caller used to be able to add a description to the table via an options Object
The description is ignored
create
function with an options object containing a description propertylist
function with the same wallet as step one and notice that no description existsIt looks like the Typescript types for query responses are currently incorrect. Whereas I would expect, based on the types that the following returns an object with columns
and rows
, I actually get an object with { data: { columns, rows } }
. However, the Typescript compiler complains with Property 'data' does not exist on type 'ReadQueryResult'.
. The fix is to either automatically destructure the returned object (internally) and return rows and columns, or to update the types to reflect the fact that the data is structured under the data key. The former seems like the nicer option.
const response = await connection.query(`select * from some_table;`);
We don't use registerTable
anywhere else in Tableland, may as well change it here
It would be useful to have documentation (especially around read/SELECT functions) discussing the delay between when an update transaction succeeds and when the change is likely to be reflected in the results of a table read / SELECT statement call. There are plenty of use cases where someone might take an action that appears to succeed, and then immediately reload a view expecting it to reflect the updated state. If that view gets data from a read/SELECT query, it won't reflect the changed data right away, leading the user to think the change was not effective, and possibly taking additional actions such as trying to repeat the state change, which might make things worse. There's currently no documentation about the source(s) or extent of this delay or if a developer can do anything to tell if it's over.
Describe the bug
I'm creating a table using Polygon Mumbai. While waiting for the tx to confirm, Tableland makes ~1176 total requests to my provider (Alchemy) for a single table/tx. This is not ideal. I waited for 36 minutes before the process finally finished and the tx was included in a Polygon Mumbai block. I tested this out on both the SDK and CLI, and Tableland never exits the process.
Now, the obvious question is what the root cause is. It didn't look like Polygon Mumbai was down, but I'm not sure. I did this a few times to make sure I wasn't missing anything. The first in the list below was the one that took the longest (36 minutes). The others also took at least 5-10 mins. and resulted in a high number of provider requests as well:
Perhaps the getDefaultProvider
and other logic related to using Polygon Mumbai is leading to some issue where the tx is never prioritized by miners and sits in the mempool? I'd have to look into how Mumbai works, but that could be part of the root cause -- gas price is too low?
To Reproduce
Steps to reproduce the behavior:
node <filename>
: see gist. Alternatively, just use the CLI's create
command (which is what was ultimately used in the primary example mentioned).Expected behavior
Ideally, waitConfirm
logic should time out after X period of time or Y number of requests (by default). Or maybe in an ideal state, there's there's a way to listen to the validator for when the tx is seen & processed. Thus, no more spamming the node provider.
Perhaps this is an edge case, but I'd have to dig deeper to see how often Mumbai has this issue.
Screenshots
This screenshot includes 2 separate create attempts -- the first had 293 total (I think it was this tx), and the second led to 1176 requests (this tx). Peep the RPC calls.
Desktop (please complete the following information):
Testing the Notion GH action.
This may have been discussed already, but I can't find reference to it on discord or notion.
When I use the SDK to create a table, the flow is as follows:
tableland_createTable
method RPC request to the Validator with a SQL statement.The issue I had while testing is that when the sql statement in step 3 is invalid the safeMint happens and the user pays for the tokenId, but then the table is not created. This isn't a big deal on rinkeby, but once we go to mainnet this will be a huge pain point for anyone making a mistake during a create call.
The issues with validating SQL purely in the SDK code we dealt with in #19 and #23 makes me think we should be asking the validator if the statement is going to work before doing safeMint
on the SC.
For the sake of discussion I'll suggest that the Validator could provide a way to do a "dry run" of sql statements meant to be used with the tableland_createTable
RPC method.
Capturing this after a call today. The util.ts file tracks the addresses that the TablelandTables registry contracts are deployed to for each chain. Soon evm-tableland will publish these addresses, and once that happens we can import them instead of manually editiing the util.ts file
The SDK should enable easily setting the controller contract for tables. This can be done via a direct call to the smart contract (eth-calls.ts), the validator's RPC method (tableland-calls.ts), or potentially enable both and let the SDK user choose.
expose changes from tablelandnetwork/go-tableland#66 to consumer
A tableland call (e.g. create()
) which triggers a transaction that succeeds doesn't throw an error
Sometimes, MetaMask encounters an error inpage.js:1 MetaMask - RPC Error: Non-200 status code: '429' details: 'Rate limit exceeded: 40 per 1 second. Check respon…ticvigil.com/ to avoid hitting public ratelimits.', code: -32005,
when checking the transaction status, and that particular status check fails. This error should be caught in the code which waits for a response, and continue waiting, as a subsequent status check can still succeed and reveal that the transaction was successfully mined into a block. The error should not be bubbled up to the calling context of create()
.
create()
call from the SDK. It is an intermittent issue affecting <50% of calls, but still problematic.The workaround is to catch the error in the calling context and re-trigger creation of the table, but the gas/resources wasted on successfully creating the first are lost due to poor error handling in the SDK.
Using the SDK, the results of a SELECT statement includes a rows
object with an array of row
arrays. If one of the selected values was an int
type during table creation, it is present in the appropriate position of each row
array as a JavaScript number.
This works fine in most cases.
However, my understanding is that the equivalent Solidity type is uint256
.
One common use case would be to have a numeric integer primary key column of type uint256
and include a reference to this id in a second table as a foreign key reference, even if not explicitly declared.
Does Tableland's SDK actually store and return large numbers correctly? I was trying to build a test case to see what would be returned in the row
value when a value was outside of JavaScript's integer
range (specifically, I tried the number 18446744073709551616 (2^64) added into the query string without having to be stored numerically in JavaScript) and it looked like the change wasn't actually applied. Reading the docs, it looks like the type int64
might be more appropriate than uint256
as the EVN version of Tableland numbers, but even this has values well outside Javascript's safe range for accurate computation.
Other methods of interacting with blockchains tend to use some BigNumber implementation for numeric values input to or output from blockchain functions; this is one that does not. Should it?
As a fix for #33 we published the typescript source. This introduced an new issue because the source in now in violation of the eslint rule node/no-unpublished-import
ref: https://github.com/mysticatea/eslint-plugin-node/blob/master/docs/rules/no-unpublished-import.md
A solution that should meet all the requirements is to add eslint disables for that rule.
Describe the bug
When you create a table with a primary key, then try to insert duplicate values for that primary key using the write
method, the method does not throw an error even though the writes do not succeed.
To Reproduce
Expected behavior
Assuming the caller is using the default value for skipConfirm
an error should be thrown after the second insert fails.
See chain of messages in Discord for more detail: https://discord.com/channels/592843512312102924/902337528902860820/1012107267816034334
cc: @carsonfarmer
This makes it difficult to use with none browser-based providers, or signers that don't require a remote provider.
The line above cuts the first 2 characters off of the passed in tableId, which is a v4 uuid.
I think this was an attempt to fix an encoding issue, but when an app calls await runQuery('SELECT *', tableId);
where tableId is a valid uuid they won't want the first 2 characters cut off.
Is it possible to have multiple INSERT operations (e.g. writing data to two tables) in the same EVN transaction?
I seem to remember reading that it's not possible to have two CREATE TABLE operations in one transaction, though the docs search feature seems broken and I can't find exactly where, and the limits section doesn't say anything about even that restriction.
I am trying to run a smart contract with two functions (that matter for this discussion): one that does an INSERT query, and another that does a different INSERT query on a conceptually related table and then calls the first one for a second INSERT. Calling just the first function by itself works fine, with results visible on a subsequent SELECT. On calling the second, I can see the first INSERT take effect with the INSERTed data returned in a subsequent SELECT. However, the second INSERT seems to silently fail when it's called from the second function. The first function still proceeds to emit an event indicating successful update, in which the parameters appear correct, but the data isn't reflected in any subsequent read operation.
The only thing I can think of at the moment as a possible reason is an undocumented restriction against writing to more than one table in one query. Such a restriction (especially undocumented!) would be pretty severe limit on the use of the relational data table model.
It's also a bit concerning that the Tableland INSERT call succeeds without revert() even when the call is failing and the data is not getting added - is there any good way to detect that in the EVM environment?
Under "walkthroughs" on the left there's one for the Javascript SDK but not for EVN / Solidity. At the bottom (e.g. of this page there's a link to "Hop into Discord" which is a completely empty server, visible only after completing Discord's full sign-up process if needed:
It's the same result when using the Discord link on the Community page.
The website repo, visited in the hopes that GitHub search might work better than the on-site search, reports that it's archived with reference to a different Discord link, which renders like this:
Therefore I'm posting here, but figured I'd include those notices to provide a better idea of what Tableland's new user experience is currently like.
Describe the bug
We have now switched to Polygon Mumbai as our default network... but there is no default provider available in ethersjs for Polygon networks. This means that folks that don't have a provider API key for Infura, Alchemy, or Etherscan run into issues when trying to use Polygon.
To Reproduce
To reproduce, simply try to issue a mutating call (such as create or write) using the SDK when connected to Polygon or while simply using the defaults in NodeJS.
Expected behavior
There should at least be a sane default, or a handy error indicating how to fix this.
Additional context
This is due to our recent move to Polygon by default. We should try to have a default Polygon provider that we use internally to avoid this bug/issue.
When using Svelte with the default bundler, Rollup, This library should not cause warnings.
Rollup complains with "The following packages did not export their package.json
", and "this
has been rewritten to undefined
"
npm run dev
Since you don't actually write your own create statements if you are using the SDK (or the CLI), we should just match that API in the validation function as well. Additionally, hash is slightly misleading, because the name doesn't reflect what the API is really designed for... validation.
TLDR; the connect test "throws error if provider network is not supported", should lookup supported networks in the util.ts file
See discussion here: #44 (comment)
A user on discord is asking to be able to pass the value of token
in to the call to connect
.
Apps are able to see the token, so unless we are planning to make that private, I don't see the harm of letting them pass in the token.
e.g.
const tableland = await connect();
// The app would save this value somewhere i guess? My thought is that this is not best practices, but I would need to do some research
console.log(tableland.token);
Later on after page refresh...
const token = getUsersTokenSomehow();
const tableland = await connect({token: token});
The person asking for this on Discord wants to use localStorage
, which is going to be available to the entire page. This is probably not best practices, but I'm not sure if it's the place of this library to determine that?
@carsonfarmer @awmuncy any thoughts? The PR would be a one liner, I'm happy to create it if we all think doing this makes sense?
We should have at least a few tests that make actual http requests and smart contract calls to a publicly available Tableland Network. I'll suggest that this can be the "staging" Tableland Network, but testnet could work too, or we could add the tooling to spin up a network via https://github.com/textileio/local-tableland
Some setup to show the issue inside registerTable
:
async function registerTable(
this: Connection,
query: string
): Promise<ContractReceipt> {
this.signer = this.signer ?? (await getSigner());
const address = await this.signer.getAddress();
const contractAddress = this.options.contract;
const contract = TablelandTables__factory.connect(
contractAddress,
this.signer
);
const oldGasPrice = await this.signer.getGasPrice();
console.log("oldGasPrice", oldGasPrice.toNumber());
const feeData = await this.signer.getFeeData();
console.log("gasPrice", feeData.gasPrice?.toNumber());
console.log("maxFeePerGas", feeData.maxFeePerGas?.toNumber());
console.log("maxPriorityFeePerGas", feeData.maxPriorityFeePerGas?.toNumber());
const tx = await contract.createTable(address, query);
return await tx.wait();
}
This issue (ethers-io/ethers.js#2439 (reply in thread)) is unrelated, but has a relevant comment that provider.getGasPrice
should no longer be used. However, on both Optimism and Polygon, both provider.getGasPrice
and feeData.gasPrice
are the same. Following that commenter's logic, we get a giant (too big maxFeePerGas
).
In any case, at the bottom of the thread is a comment from somebody else saying that they must always specify gasPrice
. I suspect they are hitting the same issue we are hitting.
The gasPrice
returned by Alchemy (using either method mentioned above) is too low compared to what metamask uses (see https://gas-api.metaswap.codefi.network/networks/137/suggestedGasFees) or what is reported on Polyscan: https://polygonscan.com/gastracker.
If I bump the Alchemy-suggested gasPrice
by 10%, I can create tables:
async function registerTable(
this: Connection,
query: string
): Promise<ContractReceipt> {
this.signer = this.signer ?? (await getSigner());
const address = await this.signer.getAddress();
const contractAddress = this.options.contract;
const contract = TablelandTables__factory.connect(
contractAddress,
this.signer
);
const oldGasPrice = await this.signer.getGasPrice();
console.log("oldGasPrice", oldGasPrice.toNumber());
const feeData = await this.signer.getFeeData();
console.log("gasPrice", feeData.gasPrice?.toNumber());
console.log("maxFeePerGas", feeData.maxFeePerGas?.toNumber());
console.log("maxPriorityFeePerGas", feeData.maxPriorityFeePerGas?.toNumber());
let gasPrice;
if (feeData.gasPrice) {
gasPrice = Math.floor(feeData.gasPrice.toNumber() * 1.1);
}
console.log("bumped gasPrice", gasPrice);
const tx = await contract.createTable(address, query, {
gasPrice,
});
return await tx.wait();
}
Perhaps we should query the metamask or polyscan API for suggested gasPrice
instead of relying on the provider (Alchemy in my case).
Creating this issue to capture questions from the discussion in #112 involving how the SDK should connect when we have multiple Validators.
uri
value that is used in a siwe message come from? A possibility is to use the value of the host
connection option, but if we go this route how would a host uri map to a Validator.uri
if the Validator doesn't have a specific DNS address?host
option? If not, the default value will probably end up receiving the large majority of requests.Note: to match the siwe spec uri
would need to be a string that conforms to RFC 3986
A lot of these questions seem related to Validator incentives/payment mechanisms
Right now the default is Goerli, which is our slowest chain. Mumbai has been really popular at hackathons and it's fast.
This repo hasn't been updated to the pattern used elsewhere: don't mix linting and formating
Once @tableland/eth is published with new name @tableland/evm we need to make that change here too.
Ideally, the host argument to the connect function is optional, and would fallback on network or even just signer if possible.
Right now, the host argument is required in the interface. But this need not be the case.
We should just make it optional.
evm-tableland
and add npm run lint
and npm run prettier
to CI checks (as in evm-tableland
)I'm trying to simply use Tableland in an ES module, but I always get the error fetch is not defined
, stemming from tableland-calls.js
(using @tableland/[email protected]
). I use node v18.7.0
and npm 8.15.0
. My package.json
has a type: module
defined, and I successfully import fetch from "node-fetch"
and use fetch
elsewhere in my code. I try replacing my fetch
import with the following 1-liner in my code (as recommended per the JS SDK docs for CJS/require
), but it has no impact:
const fetch = (...args) => import("node-fetch").then(({ default: fetch }) => fetch(...args))
The only way to stop this error in my application is (usually) by reverting to CJS over ESM and then using the 1-liner/polyfill, which is not ideal. Thus, I'm assuming the issue must originate from Tableland, not my code.
After adding this 1-liner to tableland-calls.js
in dist
, the error is resolved. Upon digging into the SDK code a bit, it looks like the error stems from the fact tableland-calls.ts
does not import node-fetch
. If you import node-fetch
into tableland-calls.ts
and build, everything is fixed; no more fetch is not defined
error.
Disclaimer: I know we use a modern version of Node, but I'm using Node v18 and still having this issue...so I'm not sure exactly why this issue is happening, to be honest. I would expect that I wouldn't have to import node-fetch
with Node 18. Albeit, I'm not an expert on these intricacies. The fix described below resolves the error, and I've heard this problem pop up a few times from others, before. Not exactly sure if they were using CJS or ESM or what version of Node, though.
node-fetch
as well as @tableland/sdk
; call the file index.js
.index.js
:import fetch from "node-fetch"
import { connect } from "@tableland/sdk"
read
query after connecting. You will get an error:file:///Users/db/Tableland/projects/discord-table-bot/node_modules/@tableland/sdk/dist/esm/src/lib/tableland-calls.js:22
const res = yield fetch(`${this.options.host}/rpc`, {
^
ReferenceError: fetch is not defined
index.js
, and nothing is resolved.node_modules/@tableland/sdk/dist/esm/src/lib/tableland-calls.js
, and things work without errors:const fetch = (...args) => import("node-fetch").then(({ default: fetch }) => fetch(...args))
Here's the fix:
tableland-calls.ts
, add @types/node-fetch
:npm i --save-dev @types/node-fetch --force
node-fetch
into tableland-calls.ts
:import fetch from 'node-fetch'
index.js
file mentioned above works as expected without any messy node-fetch
issues.Note: in step (2) of the fix, if you don't --force
, then ethers
and siwe
won't let you install, throwing this error:
npm ERR! code ERESOLVE
npm ERR! ERESOLVE could not resolve
npm ERR!
npm ERR! While resolving: [email protected]
npm ERR! Found: [email protected]
npm ERR! node_modules/ethers
npm ERR! ethers@"^5.6.9" from the root project
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer ethers@"5.5.1" from [email protected]
Be able to use Tableland in a JS ESM without polyfills or workarounds for node-fetch
.
@tableland/sdk 3.1.0
node v18.7.0
npm 8.15.0
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.