Comments (4)
You could write:
async def test():
f = aiofile.async_open('filename.txt', mode='w')
try:
await f.file
await f.write('123')
finally:
pass
# await f.close()
asyncio.run(test())
But I don't really understand the need for this from your example. The only thing I can think of would be that maybe you'd want to return f
to whatever called test
so it can do more writes before closing the file but then, why not something like this instead:
@asynccontextmanager
async def foo():
async with aiofile.async_open('filename.txt', mode='w') as afp:
await afp.write('123')
yield afp
async def bar():
async with foo() as afp:
await afp.write('456')
asyncio.run(bar())
This seems safer to ensure afp
is eventually closed.
from aiofile.
You could write:
Thank you -- is helpful.
This seems safer to ensure
afp
is eventually closed.
Sure - but many applications are not so trivial and the thing writing to the file may not have async_open in its stack.
For instance:
def logMessage(self, msg)
async with self.writeLock:
...
await self.aiofile.write(msg)
...
def onRecvX(self, msg): # Runs concurrently as a task
...
await self.logMessage(msg)
def onRecvY(self, msg): # Runs concurrently as a task
...
await self.logMessage(msg)
def onRecvZ(self, msg): # Runs concurrently as a task
...
await self.logMessage(msg)
Simply, the write call might not be in the stack of onRecv...
. The file may be opened lazily in logMessage. The file might also be closed - it may be log-rotated, for instance. I don't want to unwind the stack and cancel all running asyncio workers that have started since the file was opened just so that I can open/reopen the file.
from aiofile.
Thank you, I understand your requirements a little better. Here's how I'd do it with the log rotate example:
import asyncio
from datetime import datetime
from pathlib import Path
from aiofile import AIOFile, Writer
class AsyncRotatedLog:
def __init__(self, directory='./logs/'):
self._aio_file = None
self._current_log_path = None
self._writer = None
self._directory = Path(directory).resolve()
self._lock = asyncio.Lock()
async def __aenter__(self):
return self
async def __aexit__(self, exc_type, exc_val, exc_tb):
if self._aio_file:
await self._aio_file.close()
async def _get_writer(self):
async with self._lock:
filename = self._directory / f'{datetime.now().strftime("%Y.%m.%d_%H:%M:%S")}.log'
if filename != self._current_log_path:
if self._aio_file:
await self._aio_file.close()
self._aio_file = await AIOFile(filename, 'w')
self._current_log_path = filename
self._writer = Writer(self._aio_file)
return self._writer
async def write(self, msg: str):
msg = msg.strip('\n') + '\n'
writer = await self._get_writer()
await writer(msg)
class Foo:
def __init__(self, async_logger):
self.async_logger = async_logger
async def log_message(self, msg):
await self.async_logger.write(msg)
async def on_x(self, msg): # Runs concurrently as a task
...
await self.log_message(msg)
async def on_y(self, msg): # Runs concurrently as a task
...
await self.log_message(msg)
async def on_z(self, msg): # Runs concurrently as a task
...
await self.log_message(msg)
async def listen_for_xyz(self):
await self.on_x('X happened')
await asyncio.sleep(0.2)
await self.on_y('Y happened')
await asyncio.sleep(1.5)
await self.on_z('Z happened')
async def main():
async with AsyncRotatedLog() as logger:
foo = Foo(logger)
await foo.listen_for_xyz()
asyncio.run(main())
Basically, encapsulate your log rotation logic in its own class and make it an async context manager (this part is optional, you could just as easily replace the __aexit__(self)
with a close(self)
method and wrap your calling code in a try... finally
. The important thing here is that you ensure your file descriptor are always closed on exit.
Disclaimer: this is simple example code. It's working but it may have bugs (I didn't spend much time on it). Use at your own risks.
from aiofile.
@stuz5000 this will be fixed in version 3.8.0.
Code example:
import asyncio
import atexit
import os
from tempfile import mktemp
from aiofile import async_open
TMP_NAME = mktemp()
atexit.register(os.unlink, TMP_NAME)
async def main():
afp = await async_open(TMP_NAME, "w")
await afp.write("Hello")
await afp.close()
asyncio.run(main())
assert open(TMP_NAME, "r").read() == "Hello"
from aiofile.
Related Issues (20)
- Can't install on Amazon Linux HOT 2
- Function not implemented HOT 1
- async_open does not create file if file does not exist in mode 'a+' HOT 1
- Add support for StringIO and BytesIO HOT 1
- Race condition in `AIOFile.open()` HOT 4
- Some sort of changelog / release notes?
- async_open doesn't fully mimic the behavior of Python file objects HOT 5
- Manual context management example on readme has a bug HOT 1
- How to work with NFS? HOT 1
- Tag the source HOT 1
- AIOFile context manager looses data HOT 3
- aiofile LineReader does a read for every line in spite of having multiple lines in CHUNK_SIZE HOT 2
- Which of the methods are coroutine-safe? HOT 2
- Why is fsync calling fdsync HOT 2
- No flush API and no way to set unbuffered writes in binary mode HOT 3
- Add performance benchmarks to the readme HOT 3
- Unable to call close() twice
- Memory leak during import
- TextFileWrapper.read reads more than requested
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from aiofile.