Comments (6)
IMHO, dedicated
executemany
method would simplify API. It is easier to writeawait conn.executemany('...', data)
than the whole loop. If this was implemented at Cython level, wouldn't it be faster as well?
No, high-level APIs in asyncpg are not Cythoned, and even if they were, the perf difference would be minimal.
I hope you do not mind, I will repeat the question from my first comment - are the rows iterated in the loop sent one by one or are the statements batched?
There is no such thing as batching, psycopg2
does the same row-by-row statement execution.
That said, event loop overhead needs to be benchmarked for this case. If it turns out to be significant, we will look into implementing the executemany
method.
Meanwhile you can create a helper method using the loop technique mentioned above.
from asyncpg.
According to https://magic.io/blog/asyncpg-1m-rows-from-postgres-to-python/ asyncpg
supports arrays and composite types, so I can imagine that determining semantics of value
in con.excute(query, value)
can be tricky.
However, it would be good to have at least executemany
method, so
for row in data:
await con.execute(query, *row)
can be executed as
await con.executemany(query, data)
where data
could be any iterable.
BTW. How con.execute(query, *row)
executed multiple times works exactly? Does it send each statement one by one over network or are multiple rows bundled into a batch?
from asyncpg.
Also, looking at http://initd.org/psycopg/docs/usage.html
# Pass data to fill a query placeholders and let Psycopg perform
# the correct conversion (no more SQL injections!)
>>> cur.execute("INSERT INTO test (num, data) VALUES (%s, %s)",
... (100, "abc'def"))
so asyncpg
connection execute
method seems to be incompatible with DB-API?
from asyncpg.
What about changing execute
method signature to
async def execute(self, query: str, *args, data=None, timeout: float=None) -> str:
If args
is non-empty then raise deprecation warning. If both args
and data
are specified, then raise TypeErrror
. After few asyncpg
versions, the args
positional parameters could be removed.
from asyncpg.
asyncpg explicitly supports prepared statements, so there is no need for a dedicated executemany
method. Simply iterate over a loop, calling the prepared statement:
ps = await conn.prepare('INSERT INTO test(num, data) VALUES($1, $2)')
for row in data:
await ps.fetchval(*row)
Or, since the statements are prepared by default, you can use even simpler form:
for row in data:
await conn.execute('INSERT INTO test(num, data) VALUES($1, $2)', *row)
from asyncpg.
IMHO, dedicated executemany
method would simplify API. It is easier to write
await conn.executemany('...', data)
than the whole loop. If this was implemented at Cython level, wouldn't it be faster as well?
I hope you do not mind, I will repeat the question from my first comment - are the rows iterated in the loop sent one by one or are the statements batched?
from asyncpg.
Related Issues (20)
- Asyncpg (asyncio/asyncpg.py, line 62, in get_query) get_query throws index error.
- no binary format encoder for type jsonb
- Whenever i create a table with a `real` column and read the value i get something different then what i saved HOT 1
- insufficient data in buffer
- GCP CloudSQL Connector with asyncpg Pool HOT 1
- cursor prefetch with bounded lag
- asyncpg.connect: ValueError: invalid literal for int() with base 10 HOT 5
- Connection secrets displayed in clear text in logs
- RDS Proxy Pinning HOT 1
- Any ETA on releasing the latest version with GSS? HOT 1
- Timeout configuration for multi-host config
- Terminate function does not work when a stream is cancelled prematurely when in an anyio task group HOT 2
- Extremely long delay grabbing type info for string array (and likely other types) on CockroachDB HOT 11
- SELECT query to database returns wrong decimal values. HOT 3
- max_inactive_connection_lifetime param not respected in a conn pool HOT 1
- CockroachDB first query huge 7 seconds delay!! HOT 2
- Connection pool docs inconsistency? HOT 2
- Not accepting str as value for interval/date columns
- Does asyncpg support using `pg_service.conf` for connection parameters?
- No rows returned by fetch() when for DELETE rewritten to UPDATE using rule
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from asyncpg.