mas-bandwidth / yojimbo Goto Github PK
View Code? Open in Web Editor NEWA network library for client/server games written in C++
License: BSD 3-Clause "New" or "Revised" License
A network library for client/server games written in C++
License: BSD 3-Clause "New" or "Revised" License
In addition, I suggest changing platform_sleep for a varied time like this.
`const int logicFrame = 60;
double accumilatedTime = 0;
double deltaTime = platform_time();
double logicFrameLength = 1.0/logicFrame;
while( !quit )
{
double now = platform_time();
accumilatedTime += now-deltaTime;
deltaTime = now;
while( accumilatedTime > logicFrameLength )
{
server.SendPackets();
serverTransport.WritePackets();
serverTransport.ReadPackets();
server.ReceivePackets();
server.CheckForTimeOut();
time += logicFrameLength;
server.AdvanceTime( time );
serverTransport.AdvanceTime( time );
accumilatedTime -= logicFrameLength;
}
platform_sleep( logicFrameLength-accumilatedTime );
}`
I understand that this is built on netcode.io and reliable.io but I'm not sure what exactly this provides over those two libraries.
Also would either this or netcode.io be suitable for a turn based game like worms? From my understanding netcode.io would be but im not sure what yojimbo offers in addition... is it more action game focused?
While browsing through your code I've noticed a few odd programming choices.
For example, most of your classes have user defined destructors. Due to a very bad (and deprecated) C++ rule the compiler will implicitly generate copy ctors/assignment that do bitwise copy. If a user accidentally copies an object, they will have a double free.
Is there a special reason why you haven't disabled the copy operations?
Note that issue would have been resolved if you've used something similar to std::unique_ptr because it won't require any custom destructors.
Some of the classes are also perfect examples of C++11 move-only types. I know that game prograers have these bias against any newer C++ features but move semantics are useful.
I've also noted that you aren't using the C++ standard library but instead reimplement stuff like the min/max/randint or swap in a (partly) worse way. And you already require the C standard library and a C++ compiler, so there is the C++ library available as well.
Disclaimer: I come from /r/cpp but want to understand more of your reasons not just blindly bash you.
Currently there is no way to completely disable simulator when using NetworkTransport.
Maybe worth adding that flag to NetworkTransport constructor, but I'm not sure about argument order - if allocateNetworkSimulator flag is the last, then I have to specify all other buffer size related args.
More options:
NetworkSimulator eats up significant amount of memory and drains some CPU even when network conditions at defaults.
Is there any kind of flow-control/congestion avoidance in libyojimbo?
This post talks about a simple binary congestion avoidance algorithm, but it doesn't seem to be implemeneted in libyojimbo; the packet flow rate seems to be controlled by the frequency in which clients/server call SendPackets()/WritePackets()
.
Until "snapshots/transport-level packet fragmentation" is implemented, it seems that the maxPacketSize
for a Connection should be configured to be less than the MTU, yes? I'm a bit thrown off because the DefaultMaxPacketSize = 4 * 1024;
is larger than standard MTU of 1500.
In the code documentation for ReliableOrderedChannel
, it mentions:
Only one message block may be in flight over the network at any time, so blocks stall out message delivery slightly.
Does single message block in flight restriction hold also for unreliable-unordered channels as well?
Server crashed when client sent message that cannot be serialized. I think it should be compatible with this dirty data. Reading stream was broke exceptionally caused by the wrong numMessages. I've been reading your code for a week, and I'm trying to fix it. It works. Did I mess it up? This is my backtrace and modification.
(gdb) bt
#0 0x00007f6664949625 in raise () from /lib64/libc.so.6
#1 0x00007f666494ae05 in abort () from /lib64/libc.so.6
#2 0x00007f666494274e in __assert_fail_base () from /lib64/libc.so.6
#3 0x00007f6664942810 in __assert_fail () from /lib64/libc.so.6
#4 0x00000000004214b5 in yojimbo::ChannelPacketData::Free (this=0x226c220, messageFactory=...) at yojimbo_channel.cpp:39
#5 0x0000000000417e5b in yojimbo::ConnectionPacket::~ConnectionPacket (this=0x226c2b0, __in_chrg=) at yojimbo_connection.cpp:45
#6 0x000000000041c494 in yojimbo::PacketFactory::DestroyPacket (this=0x1d24200, packet=0x226c2b0) at yojimbo_packet.cpp:309
#7 0x000000000041b7d5 in yojimbo::Packet::Destroy (this=0x226c2b0) at yojimbo_packet.cpp:40
#8 0x000000000041c0a6 in yojimbo::ReadPacket (info=..., buffer=0x7fff34e07d00 "", bufferSize=93, errorCode=0x7fff34e07c98) at yojimbo_packet.cpp:239
#9 0x000000000042b284 in yojimbo::PacketProcessor::ReadPacket (this=0x1d22080, packetData=0x7fff34e07d00 "", sequence=@0x7fff34e08d50, packetBytes=93,
encrypted=@0x7fff34e08d5f, key=0x0, encryptedPacketTypes=0x1d24120 "\001\001\001\001\001\001\001\001",
unencryptedPacketTypes=0x1d24120 "\001\001\001\001\001\001\001\001", streamAllocator=..., packetFactory=...) at yojimbo_packet_processor.cpp:237
#10 0x000000000041e9d0 in yojimbo::BaseTransport::ReadPackets (this=0x7fff34e08df0) at yojimbo_transport.cpp:367
#11 0x0000000000401c9b in ServerMain () at tests/server.cpp:145
#12 0x0000000000401e22 in main () at tests/server.cpp:181
template bool SerializeOrderedMessages( Stream & stream, MessageFactory & messageFactory, int & numMessages, Message ** & messages, int maxMessagesPerPacket )
{
const int maxMessageType = messageFactory.GetNumTypes() - 1;
bool hasMessages = Stream::IsWriting && numMessages != 0;
serialize_bool( stream, hasMessages );
if ( hasMessages )
{
serialize_int( stream, numMessages, 1, maxMessagesPerPacket );
int * messageTypes = (int*) alloca( sizeof( int ) * numMessages );
uint16_t * messageIds = (uint16_t*) alloca( sizeof( uint16_t ) * numMessages );
memset( messageTypes, 0, sizeof( int ) * numMessages );
memset( messageIds, 0, sizeof( uint16_t ) * numMessages );
if ( Stream::IsWriting )
{
assert( messages );
for ( int i = 0; i < numMessages; ++i )
{
assert( messages[i] );
messageTypes[i] = messages[i]->GetType();
messageIds[i] = messages[i]->GetId();
}
}
else
{
Allocator & allocator = messageFactory.GetAllocator();
messages = (Message**) allocator.Allocate( sizeof( Message* ) * numMessages );
}
serialize_bits( stream, messageIds[0], 16 );
for ( int i = 1; i < numMessages; ++i )
serialize_sequence_relative( stream, messageIds[i-1], messageIds[i] );
for ( int i = 0; i < numMessages; ++i )
{
if ( maxMessageType > 0 )
{
serialize_int( stream, messageTypes[i], 0, maxMessageType );
}
else
{
messageTypes[i] = 0;
}
if ( Stream::IsReading )
{
messages[i] = messageFactory.Create( messageTypes[i] );
if ( !messages[i] )
return false;
messages[i]->AssignId( messageIds[i] );
}
assert( messages[i] );
if ( !messages[i]->SerializeInternal( stream ) )
{
numMessages = i+1; // fix processed messages amount
return false;
}
}
}
return true;
}
Thank you for your great job. It's helpful. I will continue to pay attention to.
I made a echo server to test sending messages in the real network with 37ms delay. The latancy was as high as 80ms until I raised the loop frequency to 150 to reduce the latancy to 50ms.
Packet loss? What should I do?
I've grabbed the similar detection code from RapidJson. It checks for __BYTE_ORDER__
and <endian.h>
first, looks more robust.
This can be tricky to debug.
The repository already has a few pieces of sample code, but seems to lack any other kind of documentation.
I'm genuinely curious (as some games will demand more than 64 players per area):
MaxPlayers
?Hi again,
A new problem arises, using InsecureConnect, with every server that is not hosted on my machine, the server will spam
"ignored connection request packet. connect token expired"
when I try to connect to it, other people are managing to connect to the server.
Any idea what could be causing this ?
Client log:
client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client connect failed. connect token expired client disconnected client changed state from 'sending connection request' to 'connect token expired'
Thanks,
Max
Edit: Added client log
I'm somewhat confused about how messages are carried on packets with regard to the frame concept and time.
Let's say I setup an unreliable, unordered channel between client and server. I want my client to send one message every 20 ms to the serve -- the message size is smaller than the max packet size. After the connection and channel is established between client and server, I would think only one packet would be need to be sent to carry that message.
Based on the code the function PumpClientServerUpdate()
in tests/test.cpp
, I made this test:
Example client code:
// After connection established
...
client->SendMsg(message);
client->SendPackets();
transport->WritePackets();
...
Example server code:
// After connection established
...
while (true) {
...
transport.ReadPackets();
server.ReceivePackets();
...
}
In my tests, the message is not received by the server when only one packet is sent from the client. In fact, it requires ~300 packets to be sent from the client before the server is able to receive the message.
What's the correct usage of SendMessage()
vis-a-vis SendPackets()
for a game frame update?
It seems that SendMessage()
queues up the message and SendPackets()
adds to the transport's packet queue a single connection packet with a single message from the message queue. But in my tests, it seems fore every SendMessage()
called, I have to call SendPackets()
~300 times for the message to be received.
Also if I want to send multiple messages every frame (calling SendMessage()
multiple times), it seems that a single call to SendPackets()
every frame will only send to the transport a single packet with a single message during that frame.
Practically speaking, If I want to send multiple messages every game frame update, do I have to call SendPackets()
at a higher frequency than the frame update frequency?
Hi, could we collect a list of games using yojimbo? It's always nice to just see something already working, which can be tweaked immediately.
BaseTransport::Reset() crashes when allocateNetworkSimulator is false.
I'm trying to understand how to use the library, but have found the myself confused by the example client and server in tests/
.
In the loop in tests/client.cpp
(similar one in tests/server.cpp
), the client establishes a connection to the server and continuously they then continuously exchange packets of type TEST_PACKET_CONNECTION
. No user packets are exchanged however (type=TEST_USER_PACKET
).
while ( !quit )
{
client.SendPackets();
clientTransport.WritePackets();
clientTransport.ReadPackets();
client.ReceivePackets();
client.CheckForTimeOut();
if ( client.IsDisconnected() )
break;
time += deltaTime;
client.AdvanceTime( time );
clientTransport.AdvanceTime( time );
if ( client.ConnectionFailed() )
break;
platform_sleep( deltaTime );
}
Conceptually, what functionality does the client perform compared to the transport? What does client.SendPackets()
do exactly as compared toclientTransport.WritePackets()
?
How can user packets (type=TEST_USER_PACKET
) be exchanged once the connection is setup?
Also, my understanding is that various user packet formats are defined in shared.h
and a factory for these types is generated with YOJIMBO_PACKET_FACTORY_START/STOP
macro block?
It'd be great to have some kind of HOWTO on using the library!
I'm seeing some strange behavior where mbedtls is receiving a truncated https response from the matcher.go. I haven't seen this before and have no idea why it has started happening today (no changes made to yojimbo).
I updated to a latest Docker version and noticed this today, so I think it might be related to that.
I'm still investigating and don't know what it is yet so I'm putting this up so people know what's going on if they see this themselves.
Symptoms are:
"premake5 connect" fails to connect and says "failed to connect to matcher, is the matcher running?" even though the matcher is running.
Windows only. Seems fine on MacOS
Hello, I'm interested in using yojimbo with an unreliable-sequenced stream of object updates with acknowledgements for which messages are actually received by the client. (Something like the UNRELIABLE_SEQUENCED_WITH_ACK_RECEIPT
message type from RakNet.)
I've looked through the code, and this seems pretty hard to implement since all channels are multiplexed into one packet with one sequence number. I can easily do sequencing on the client, but I don't think I can use the current system (Channel::onAck
) to detect ignored out-of-order packets on the server - there's the possibility of out-of-order acks sent by the client becoming in-order on the way to the server. If I skip sending acks from the client for out-of-order packets, I'd cause unnecessary strain on any reliable channels.
Just curious about your thoughts. If there's a reasonably easy way to do it, I might give implementing it into the library proper a shot. If not, I can just do the sequencing manually inside my messages.
Thanks for the library, good luck with the startup!
Hi, is there any plan to consider other language bindings in the future? Since there are also some other languages used in game dev, I think it will be useful to have those or plan so that it's easier to create bindings(C interface?)
Personally I will be using Rust and willing to work on the binding.
I ran into a snag trying to make the project work with Unity.
I got as far as getting a connection token from the "web backend" only for my connection request to time out.
We used the matcher to generate the token but when we attempt to connect using it the server outputs the following message:
"ignored connection request packet. connect token failed to decrypt"
I wanted to use the build guildlines outlined in BUILDING.md, but I am unable to create the .sln file from premake. (I maybe doing it wrong)
I installed premake and pulled the latest release (0.6).
Opened up CMD and cd to yojimbo.
Then I typed 'premake5 solution'.
When it runs, I get an error that states: "Windows cannot find 'Yojimbo.sln'. Make sure you typed the name correctly, and then try again."
Am I suppose to make a .sln first? I am using VS 2015 and/or 2017
Due to apparent issues with mobile networks and ipv4, I am attempting to get a client to connected to a server using ipv6. Using my local ipv6 address I can't seem to get the insecure server to see any traffic from the insecure client. I can, however, get the server to detect garbage UDP packets while running from the ipv6 address. I pass the same address to both the client and the server and the client times out while attempting to connect. The same results occur while trying to connect to a remote server.
I can connect successfully using the while running using ipv4, both locally and remotely.
Can you see any obvious flaws that I'm missing?
I'm curious as to why you have a return
here, where all the other branches have a goto cleanup;
https://github.com/networkprotocol/libyojimbo/blob/master/yojimbo_matcher.cpp#L128
if ( ( ret = mbedtls_ssl_config_defaults( &m_internal->conf,
MBEDTLS_SSL_IS_CLIENT,
MBEDTLS_SSL_TRANSPORT_STREAM,
MBEDTLS_SSL_PRESET_DEFAULT ) ) != 0 )
{
m_status = MATCHER_FAILED;
return; // <-- All the other error branches are `goto cleanup;`
}
// ....
cleanup:
mbedtls_ssl_close_notify( &m_internal->ssl );
I don't actually know enough about how the different mbedtls functions work to say this is a bug, but it looks suspicious to have only one error path return directly.
bool WouldReadPastEnd( int bits ) const
{
return m_bitsRead + bits > m_numBits;
}
In WouldReadPastEnd
it is possible to overflow the lhs of the comparison if a crafted packet gives an extremely large value for bits
. This is pretty much impossible in the codebase as-is. The only place where the user can give a large bits value is serialise_string and only if the buffer_size
parameter is unreasonably large.
However it seems that it's probably best to remove the potential overflow from WouldReadPastEnd
.
Similarly bits is signed, so a negative number would also cause the same issues, however I think it's quite a lot easier to screw up an upper bound than it is to accidentally allow negative lengths though.
Something like the following would address both issues, provided I've not messed up along the way?
bool WouldReadPastEnd( unsigned bits ) const
{
return bits > (unsigned)(m_numBits - m_bitsRead);
}
Heyo,
Apologies for using github issues to ask questions. For the most part I've been able to solve everything from looking at the source, but this one is a little trickier.
In short, does it make sense to send variable length reliable RPC messages using BlockMessage? I know you've previously mentioned that these messages are generally better for downloading a large chunk of data once (like downloading a level) because they only allow one to be in-flight at a time. So I assume the answer is "don't use BlockMessage" for this, but I figured I'd double check.
The longer version of the story is that I have an app that sends a large amount of reliable RPC messages (90-180/sec). Each RPC contains a small buffer of data that varies in size. Most of the time, these buffers are very small (20-40 bytes), however, occasionally we'll see a single message here and there that's larger than the WriteStream buffer (8 * 1024 according to the default connection config).
Architecturally, I'd love to just have a single message type that is a subclass of BlockMessage. However, it looks like I should probably make two subclasses. One that's a subclass of Message and one BlockMessage that I use whenever the buffer is larger than the WriteStream buffer size.
Anyway, I wasn't sure if Yojimbo makes an exception if the entire block message fits inside of a single packet.
Max
Heyo,
I just wanted to check that the only difference between the secure_client/secure_server and the regular client/server examples is the token from the matcher.
I'm using yojimbo for a game where the sessions can be quite long in duration (possibly days). I have an accounts service that generates a session token (instead of a matcher instance). The user then connects to a yojimbo instance via client.InsecureConnect() and gives the server its session token (which is then used to verify if that user can join this match). I do this because the session token needs to be continually evaluated to make sure it's valid throughout the duration of the session. (Someone can get banned mid-game)
Is this still going to be secure? As you can imagine calling client.InsecureConnect scares me a little. I assume data is still being encrypted with libsodium.
Max
When Windows.h is included, any function named SendMessage gets macro expanded to SendMessageW or SendMessageA.
I'm not sure if this is something you want to handle in the library or let the user deal with, just seems like something other people would hit and be confused about like me.
I personally worked around this by using a MSVC pragma and renaming the function in my inherited Client.
#pragma push_macro("SendMessage")
#undef SendMessage
void GameClient::SendGameMessage( GameMessage* Message )
{
Assert( IsConnected() );
Assert( m_messageFactory );
Assert( m_connection );
m_connection->SendMessage( Message );
}
#pragma pop_macro("SendMessage")
Hello,
I am trying to run a simple server in a docker instance, mapping the same port in the host machine as the container's binding port works like a charm.
docker run -it -p 10578:10578/udp is perfect.
As soon as I try to run
docker run -it -p 10579:10578/udp it won't work anymore.
It is something I require to run multiple containers on the same machine, is there something in yojimbo that might be causing this behavior ?
Thanks,
Max
Not sure how hard it is to support json web tokens (JWT) as a token authentication method to https servers.
For example being able to use Google accounts / Facebook accounts / Github accounts / Run your own openidc server with keycloak accounts.
We are having an occasional issue sending ordered messages. Most of the time everything works fine for an extended run, but sometimes we are hitting the assert in ReliableOrderedChannel::AddMessagePacketEntry.
We are sending reliable ordered messages fairly infrequently and the sequence number that gets passed to the Insert function seems to be getting incremented every time Server::SendPackets is called. We can reliably reproduce the assert by waiting ~32768 frames (i.e SendPacket calls) between sending a reliable ordered message.
It seems like the sequence passed to the sequence buffer should only increment when a message is sent rather than every frame, or else reliable messages need to be sent more often. Is this a bug or are we misusing something?
I was wondering what the feasibility is of being able to use a custom socket implementation, i.e. I am working on a restricted hardware platform where instead of normal sockets, there is a vendor provided socket implementation (different functions/params but very similar), which has the same properties of UDP (however I'd like to have reliable, ordered packets). Essentially, rather than passing in an address for yojimbo to open, I'd like to open the socket myself, and then pass yojimbo my socket (as a void* or whatever), and then register functions with yojimbo for sending, receiving and closing said socket.
What are your thoughts?
Able to build on iOS but run into runtime issues.
Run into memory access issues (EXC_BAD_ACCESS
) involving the allocator/deallocator macros, e.g.
#define YOJIMBO_NEW( a, T, ... ) ( new ( (a).Allocate( sizeof(T), __FILE__, __LINE__ ) ) T(__VA_ARGS__) )
Stack trace:
#0 0x0000000103c6b7fc in yojimbo::ClientServerPacketFactory::CreatePacket(int) at /Users/Dom/Desktop/libyojimbo/./yojimbo_client_server_packets.h:263
#1 0x0000000103c7fb04 in yojimbo::PacketFactory::Create(int) at /Users/Dom/Desktop/libyojimbo/yojimbo_packet.cpp:266
#2 0x0000000103c64b54 in yojimbo::Client::CreatePacket(int) at /Users/Dom/Desktop/libyojimbo/yojimbo_client.cpp:305
#3 0x0000000103c655f0 in yojimbo::Client::SendPackets() at /Users/Dom/Desktop/libyojimbo/yojimbo_client.cpp:321
#4 0x00000001000596d8 in ::copr_pump_frame(double *, double) at /Users/Dom/Desktop/interspace-ios/Classes/copr_client.cpp:268
#5 0x0000000100059510 in ::copr_connect(double *, double, uint32_t, uint64_t, const char *, uint16_t) at /Users/Dom/Desktop/interspace-ios/Classes/copr_client.cpp:125
#6 0x00000001001e9b20 in ::CoprClient_copr_connect_m3420134110(Il2CppObject *, double *, double, uint32_t, uint64_t, String_t *, uint16_t, const MethodInfo *) at /Users/Dom/Desktop/interspace-ios/Classes/Native/Bulk_Assembly-CSharp_0.cpp:23636
#7 0x00000001001e668c in ::CoprClient_CoprThreadRun_m141907639(Il2CppObject *, const MethodInfo *) at /Users/Dom/Desktop/interspace-ios/Classes/Native/Bulk_Assembly-CSharp_0.cpp:23067
#8 0x00000001031a4b54 in ::ThreadStart_Invoke_m225334093(ThreadStart_t124146534 *, const MethodInfo *) at /Users/Dom/Desktop/interspace-ios/Classes/Native/Bulk_mscorlib_7.cpp:45558
#9 0x0000000103927838 in RuntimeInvoker_Void_t2863195528(MethodInfo const*, void*, void**) at /Users/Dom/Desktop/interspace-ios/Classes/Native/Il2CppInvokerTable.cpp:1065
#10 0x00000001045ff064 in il2cpp::vm::Runtime::DelegateInvoke(Il2CppDelegate*, void**, Il2CppException**) ()
#11 0x00000001045d9248 in ::RunThreadWrapper() at /Users/builduser/buildslave/unity/build/Runtime/Threads/Thread.cpp:57
#12 0x00000001045e16e0 in ::RunThreadWrapper() at /Users/builduser/buildslave/unity/build/Runtime/Threads/Thread.cpp:52
#13 0x00000001045e74ec in il2cpp::os::ThreadImpl::ThreadStartWrapper(void*) ()
#14 0x000000018480575c in _pthread_body ()
#15 0x000000018480566c in _pthread_start ()
#16 0x0000000184802d84 in thread_start ()
In a hack, replaced the allocator macros with just plain new
and delete
which seems to get around the EXC_BAD_ACCESS
issue, but clients are unable to establish connections with the server.
The code works on Mac (x86_64), Linux (x86_64) and Windows (x86_64).
in yojimbo_bitpack.h, there are few assert
with m_numWords
. m_numWords
is defined when DEBUG is defined in the same file, but seems like assert is specified to check NDEBUG, not DEBUG. http://en.cppreference.com/w/cpp/error/assert
so should we change to
#ifndef NDEBUG
int m_numWords;
#endif // #ifdef NDEBUG
I don't like double negative, but seems like this is more correct way? I encountered this issue when I was compiling yojimbo with Rust's cpp interop crate.
Just a quick question. I was reviewing this line here: https://github.com/networkprotocol/libyojimbo/blob/master/yojimbo_sequence_buffer.h#L71
T * Insert( uint16_t sequence )
{
if ( sequence_greater_than( sequence + 1, m_sequence ) )
{
RemoveEntries( m_sequence, sequence );
m_sequence = sequence + 1;
}
// THIS LINE HERE
else if ( sequence_less_than( sequence, m_sequence - m_size ) )
{
return NULL;
}
const int index = sequence % m_size;
m_entry_sequence[index] = sequence;
return &m_entries[index];
}
I've been trying to understand what this is checking exactly. My best guess it to catch old packets that would possibly have wrapped around the buffer size, but have yet to wrap around a full uint16_t. Could you possibly shed a little light here? Thank you!
Hey guys,
It's been a long time so I've worked in C/C++ so please bear with me. I've started working on a proof-of-concept for some hobbyist game code using yojimbo. It compiles and runs well on my Windows development box using your premake5 compile options for VC++. Unfortunately, when I try running it on any other machine it crashes on startup saying that it's missing the appropriate VC++ runtimes. Since installing the runtimes doesn't seem to fix it, I instead tried compiling & linking staticly (using /MT and /MTd rather than /MD and /MDd) to bake any missing runtimes into the .EXE and thus avoid the problem entirely, and now the wheels are coming off the bus. I'm obviously doing something really silly cause I'm getting errors like these...
...numerous warnings...
2>yojimbo.lib(dhm.obj) : error LNK2019: unresolved external symbol __imp_fread referenced in function dhm_check_range
2>yojimbo.lib(hmac_drbg.obj) : error LNK2001: unresolved external symbol __imp_fread
2>yojimbo.lib(entropy.obj) : error LNK2001: unresolved external symbol __imp_fread
2>yojimbo.lib(ctr_drbg.obj) : error LNK2001: unresolved external symbol __imp_fread
2>yojimbo.lib(md.obj) : error LNK2001: unresolved external symbol __imp_fread
2>yojimbo.lib(pkparse.obj) : error LNK2001: unresolved external symbol __imp_fread
2>yojimbo.lib(md.obj) : error LNK2019: unresolved external symbol __imp_ferror referenced in function mbedtls_md_file
2>yojimbo.lib(bignum.obj) : error LNK2019: unresolved external symbol __imp_fgets referenced in function mbedtls_mpi_read_file
2>D:\Projects\FrankGame01\FrankGame01\bin\x64\Debug\FrankGame01.exe : fatal error LNK1120: 3 unresolved externals
2>Done building project "FrankGame01.vcxproj" -- FAILED.
Any suggestions? I'm sure it's something basic. Thanks in advance for any help.
Hi,
I tried to compile yojimbo on pi 3 for some udp communication. I managed to build premake5, sodium and mbedtls. After the execution of premake on yojimbo i tried to use make to build it but it said that there are 2 unknown flags. -m64 and -msse2. Sure, thats because of the arm architecture. I removed these flags and yojimbo compiled. But linking against the resulting libyojimbo.a file results in a linker error on calling the initialize function of yojimbo.
Would be incredible if there are plans to support raspberry pi!
Thank you very much!
I think that is very important to create a wrapper for 3D Engine: Unity 3D will be the perfect 3D engine to start.
It is possible to start a Patreon campaign where I would like to contribute.
Hey Glenn,
how to build yojimbo for Android on Windows?
Alex
I can't get the submodules for the project, as the version specified for netcode.io isn't valid (https://github.com/networkprotocol/netcode.io/tree/7b405ab738ce07f224bdfab2bc48b024f9d17f13).
Also thanks for this library. I look forward to playing around with it...
Would you like to replace any double quotes by angle brackets around file names for include statements?
Would you accept a cmake version of the build system? If I made one? Not sure how to deal with two versions of the build system though because one be outdated.
Hello Glenn! Thanks for the killer libraries. Just updated to your latest commit. Cleaned up so nicely.
I've been implementing a Unity plugin with yojimbo. Works really well overall. My biggest hangup is understanding how to tune the ordered reliable channel. For example, with unordered I can get a smooth 90 msgs per sec (256 bytes). With ordered, I am seeing a limit of 8 msgs per second for a 120ms server and about 60 msgs per second on LAN. To get the fastest rate I had to tweak the main loop Hz, min resend time and send rate. I've read your articles before starting. Maybe I will re-read them. Otherwise, any hints would be greatly appreciated!
Geo
Currently, if you try to send a message with a size that happens to exceed the packet budget on it's channel, then the message itself will be silently dropped after failing to fit into a packet.
This is impossible to see from the applications point of view, as no counters are affected, no error state is set, and the packet itself sends correctly - albeit without the oversized message.
Some kind of notification when this happens would be appreciated, instead of the oversized message silently disappearing into the ether.
Macros from yojimbo_serialize.h look like functions in namespace, but defines ignore namespaces. I get name clash because of write_int, write_float, etc (a really common names).
Maybe rename all of them to UPPERCASE with yojimbo prefix to match the style?
Encountered the following error while running premake5 gmake && make all
:
Building configurations...
Running action 'gmake'...
Generated Makefile...
Generated test.make...
Generated connect.make...
Generated info.make...
Generated yojimbo.make...
Generated client.make...
Generated server.make...
Generated secure_server.make...
Generated client_server.make...
Generated soak.make...
Generated profile.make...
Generated simple_messages.make...
Done (43ms).
==== Building yojimbo (debug_x64) ====
Creating bin
Creating obj/x64/Debug/yojimbo
yojimbo.cpp
yojimbo_address.cpp
yojimbo_channel.cpp
yojimbo_client_server.cpp
yojimbo_common.cpp
yojimbo_connection.cpp
yojimbo_encryption.cpp
yojimbo_matcher.cpp
yojimbo_network.cpp
yojimbo_packet.cpp
yojimbo_packet_processor.cpp
yojimbo_platform.cpp
yojimbo_simulator.cpp
yojimbo_sockets.cpp
yojimbo_transport.cpp
Linking yojimbo
==== Building test (debug_x64) ====
Creating obj/x64/Debug/test
test.cpp
Linking test
ld: warning: directory not found for option '-L/usr/lib64'
ld: unknown option: --start-group
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [bin/test] Error 1
make: *** [test] Error 2
From what I can tell, there isn't a lib64
and clang targeting Darwin doesn't support --start-group
/--end-group
on OSX.
premake5 --version
outputs: premake5 (Premake Build Script Generator) 5.0.0-alpha9
make -v
outputs: GNU Make 3.81
clang -v
outputs:
Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn)
Target: x86_64-apple-darwin13.4.0
Thread model: posix
Curious if libyojimbo would be appropriate for carrying voice/audio streams as an alternative to something like RTP, especially if libyojimbo is already being use to carry game state updates?
Hi Glenn,
I tried to integrate libyojimbo with unreal engine, after successfuly sending/receiving message with bare windows client to linux server, it's failed doing excactling the same thing in Unreal Engine tutorial scene. I did some network sniffing with Wireshark, it's weird the client side sends 10 UDP packets of 18 bytes payload to server, and server logged with "ignored encrypted packet, no read packet key for this address", any ideas? Here's my code (in Unreal).
// Copyright 1998-2017 Epic Games, Inc. All Rights Reserved.
#include "physxClient.h"
#include "physxClientGameMode.h"
#include "physxClientHUD.h"
#include "physxClientCharacter.h"
#include "../ThirdParty/libyojimbo/include/yojimbo.h"
#include "../ThirdParty/libyojimbo/include/shared.h"
#include <signal.h>
using namespace yojimbo;
AphysxClientGameMode::AphysxClientGameMode()
: Super()
{
// set default pawn class to our Blueprinted character
static ConstructorHelpers::FClassFinder<APawn> PlayerPawnClassFinder(TEXT("/Game/FirstPersonCPP/Blueprints/FirstPersonCharacter"));
DefaultPawnClass = PlayerPawnClassFinder.Class;
// use our custom HUD class
HUDClass = AphysxClientHUD::StaticClass();
}
double ti = 100.0;
Client* client ;
void AphysxClientGameMode::StartPlay() {
Super::StartPlay();
InitializeYojimbo();
OgameMessageFactory messageFactory(GetDefaultAllocator());
uint64_t clientId = 1;
Address clientAddress("0.0.0.0", 6667);
Address serverAddress("192.168.0.8", 6666);
ClientServerConfig config;
config.numChannels = 1;
config.channel[0].type = CHANNEL_TYPE_UNRELIABLE_UNORDERED;
client = new Client(GetDefaultAllocator(), clientAddress, config, adapter, ti);
uint8_t privateKey[KeyBytes];
memset(privateKey, 0, KeyBytes);
client->InsecureConnect(privateKey, clientId, serverAddress);
client->AdvanceTime(ti);
}
void AphysxClientGameMode::Tick(float f) {
Super::Tick(f);
const double deltaTime = 0.1;
JoinMessage* message = (JoinMessage*)(client->CreateMessage(JOIN_MESSAGE));
message->teamId = 1;
client->SendMessage(0, message);
client->SendPackets();
client->ReceivePackets();
ti += deltaTime;
client->AdvanceTime(ti);
//messageFactory.ReleaseMessage(message);
yojimbo_sleep(deltaTime);
return;
}
make all
==== Building yojimbo (debug_x64) ====
==== Building test (debug_x64) ====
Linking test
ld: library not found for -lsodium
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [bin/test] Error 1
make: *** [test] Error 2
macOS 10.12.3
Apple LLVM version 8.0.0 (clang-800.0.42.1)
Target: x86_64-apple-darwin16.4.0
Thread model: posix
homebrew config
HOMEBREW_VERSION: 1.1.11
ORIGIN: https://github.com/Homebrew/brew.git
HEAD: 394f9fa0aaa5854aa52bc589708a079665dcf462
Last commit: 3 weeks ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 146a4bf1580b0cc2ea22b6df6320fab43f3f3e76
Core tap last commit: 23 hours ago
HOMEBREW_PREFIX: /usr/local
HOMEBREW_REPOSITORY: /usr/local/Homebrew
HOMEBREW_CELLAR: /usr/local/Cellar
HOMEBREW_BOTTLE_DOMAIN: https://homebrew.bintray.com
CPU: quad-core 64-bit sandybridge
Homebrew Ruby: 2.0.0-p648
Clang: 8.0 build 800
Git: 2.10.1 => /Applications/Xcode.app/Contents/Developer/usr/bin/git
Perl: /usr/bin/perl
Python: /usr/bin/python
Ruby: /Users/apple/.rvm/rubies/ruby-1.9.3-p194/bin/ruby
Java: N/A
macOS: 10.12.3-x86_64
Xcode: 8.2.1
CLT: N/A
X11: 2.7.7 => /opt/X11
brew ls libsodium
/usr/local/Cellar/libsodium/1.0.12/include/sodium/ (57 files)
/usr/local/Cellar/libsodium/1.0.12/include/sodium.h
/usr/local/Cellar/libsodium/1.0.12/lib/libsodium.18.dylib
/usr/local/Cellar/libsodium/1.0.12/lib/pkgconfig/libsodium.pc
/usr/local/Cellar/libsodium/1.0.12/lib/ (2 other files)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.