tchaloupka / during Goto Github PK
View Code? Open in Web Editor NEWdlang io_uring wrapper
Home Page: https://tchaloupka.github.io/during/during.html
License: Boost Software License 1.0
dlang io_uring wrapper
Home Page: https://tchaloupka.github.io/during/during.html
License: Boost Software License 1.0
Hello @tchaloupka
Can you please help? Below you'll find simple code which demonstrate my problem. This code has two threads - main thread just submits NOP entries to SQ in a tight loop and Consumer thread consume these NOPs from CQ.
Expected result - program finish with next output:
Running ./test
completions=10000000
submissions=10000000
overflow= 0
but instead I get some number of overflows and consumer loop never ends.
If I increase RING_SIZE to 4096 then I get expected result. So the question is - how can I avoid overflows and event loss.
Thanks!
$ uname -a
Linux igor-Zen 5.3.0-53-generic #47~18.04.1-Ubuntu SMP Thu May 7 13:10:50 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
import std.stdio;
import std.datetime;
import core.thread;
import during;
static long submissions, completions;
enum iterations = 10_000_000;
enum RING_SIZE = 32;
void main()
{
Uring io;
io.setup(RING_SIZE);
void Consumer()
{
// consume NOP events
while(completions<iterations)
{
int r = io.wait(1);
if(r==0)
{
io.popFront;
completions++;
}
}
writefln("completions=%d", completions);
}
Thread consumer = new Thread(&Consumer).start;
// produce NOP events
foreach(i;0..iterations)
{
SubmissionEntry e;
e.prepNop();
e.user_data = i;
while(io.full)
{
// short sleep while sq is full
Thread.sleep(1.hnsecs);
}
io.put(e);
io.submit();
submissions++;
}
writefln("submissions=%d", submissions);
writefln("overflow= %d", io.overflow);
consumer.join();
}
Would you accept a PR?
liburing is conforming to memory ordering requirements of the kernel using it's hefty o_uring_smp._.. inline functions.
https://github.com/axboe/liburing/blob/master/src/queue.c#L223
https://github.com/axboe/liburing/blob/master/src/queue.c#L387
https://github.com/axboe/liburing/blob/master/src/queue.c#L53
https://github.com/axboe/liburing/blob/a209abe65befe603311b6fcd861e14d28e3e2f2d/src/include/liburing/barrier.h#L8
Didn't notice anything like it in during.
Hi, @tchaloupka!
do you plane update this library for newer kernel version (5.7 for example)?
The only field that can be set is the flags one but if I want to control the sq_thread_cpu
I can't.
I think it would make sense to have a second setup function that will take the whole struct as an argument and copy it over for the setup.
First off, this is lovely. I have plans to integrate uring into the concurrency library and build servers based on that.
I'll probably have to support windows iocp too, and the static assert would be giving trouble.
Would you consider to refactor to just version(linux):
?
In any case, thanks for a good library.
Thank you for writing such a nice interface to uring. I like the API a lot. However I cannot get it to work. Take this very simple example, adjusted from the readme. I should write "test" to a file opened with std.stdio : File
:
#!/usr/bin/env dub
/+ dub.sdl:
name "during-test"
dependency "during" version="~>0.2.0"
+/
module examples.test_during;
import during;
import std.range : drop, iota;
import std.algorithm : copy, equal, map;
import std.stdio : File, writefln, writeln;
import std.string : representation;
import std.exception : enforce;
import std.array;
void main()
{
Uring io;
auto res = io.setup();
enforce(res >= 0, "Error initializing IO");
File f = File("/tmp/test.txt", "wb");
ubyte[] buffer = new ubyte[4];
buffer[] = "test".representation[0 .. 4];
writeln(buffer);
SubmissionEntry entry;
prepWrite(entry, f.fileno, buffer, 0);
entry.user_data = 1;
// chain operations
res = io
.put(entry) // whole entry as defined by io_uring
.submit(1); // submit operations and wait for at least 1 completed
enforce(res == 1); // 3 operations were submitted to the submission queue
io.wait(1);
enforce(!io.empty); // at least one operation has been completed
foreach (ref CompletionEntry cqe; io) {
writefln("user data=%s, result=%s", cqe.user_data, cqe.res);
}
}
I get EFAULT (-14) as a result code for the operation. Do you have any pointers on what I am doing wrong?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.