dcs-grpc / rust-server Goto Github PK
View Code? Open in Web Editor NEWDCS gRPC server written in Rust. Get data out of DCS and send commands into DCS.
License: GNU Affero General Public License v3.0
DCS gRPC server written in Rust. Get data out of DCS and send commands into DCS.
License: GNU Affero General Public License v3.0
The speed and heading of units is not being updated in the units stream.
Proving it will require a huge dump of stats so to illustate the point I am going to put just a few entries demonstrating it.
(Altitude doesn't change in these three samples but I did observer changes in other units so altitude is fine, this is an AI unit in a racetrack which is why it is so steady)
{
"unit": {
"id": 7013,
"name": "RUSSIA air 13 unit1",
"callsign": "106",
"coalition": "COALITION_RED",
"type": "C-101CC",
"position": {
"lat": 43.89192219804639,
"lon": 40.93609502901023,
"alt": 5486.39990234375
},
"groupName": "RUSSIA air 13",
"numberInGroup": 1,
"speed": 119.97181015273709,
"heading": 126.50408556324253
}
},
{
"unit": {
"id": 7013,
"name": "RUSSIA air 13 unit1",
"callsign": "106",
"coalition": "COALITION_RED",
"type": "C-101CC",
"position": {
"lat": 43.88827413932173,
"lon": 40.94169167458405,
"alt": 5486.39990234375
},
"groupName": "RUSSIA air 13",
"numberInGroup": 1,
"speed": 119.97181015273709,
"heading": 126.50408556324253
}
},
{
"unit": {
"id": 7013,
"name": "RUSSIA air 13 unit1",
"callsign": "106",
"coalition": "COALITION_RED",
"type": "C-101CC",
"position": {
"lat": 43.8702459604217,
"lon": 40.969337211586804,
"alt": 5486.39990234375
},
"groupName": "RUSSIA air 13",
"numberInGroup": 1,
"speed": 119.97181015273709,
"heading": 126.50408556324253
}
}
Some users have asked us to include a way to stream weapons and ballistics.
Opening this issue for discussion. Two options I see.
Advantage of having separate streams is no unecessary extra work that might not be used by clients (e.g. OverlordBot does not care about weapons).
Since we have an SRS integration now, we could establish an SRS Connection that listens for SRS events and broadcasts them on the gRPC event stream.
Possible events might be:
I am not sure if the mapping from an SRS client to a unit works well enough. I know that we are getting the unit id, but finding a unit just by its id is a mess (there is no getById
).
Adapt mmiller84@8fcbc84 but instead of having a separate API, add them as events into the existing event stream we use for all the DCS events.
Therefore we should make it easy to retrieve the map and mission name via DCS-gRPC.
Follow-up from the main TTS Work.
Add an API that, when called, returns a boolean that indicates if DCS-gRPC is currently transmitting on a certain frequency. This would be callable over network and from MSE in lua.
Optionally, add an API that, when called, blocks if DCS-gRPC is currently transmitting on a certain frequency and doesn't return until the transmission has finished. This would be network only.
These two APIs can be ersatz mutexes to help stop clients stepping over each other on the radios.
Add or update the README.md to fulfil this purpose
In order to improve debuggability output the ip and port that the DCS-gRPC is binding to in the dcs.log / grpc.log
Using dcs-grpc-stubs
as a dependency in other Rust-based projects is a bit inconvenient as it requires this workaround to get it to build using its bundled protoc.
An alternative might be to pre-built the porto files instead of having them as part of the normal build process. Here is an example of another project doing something like this: open-telemetry/opentelemetry-rust#881
Follow-up for #194
Future improvement: keep SRS client connections open for a bit to re-use them for subsequent requests
Background, some servers are suffering issues where ballistics objects appear to not be cleaned up by DCS. Being able to track this count of objects would be very handy.
Add the ability to send ballistics object information to clients.
This would have to be done using LoGetWorldObjects("ballistic")
in the export environment and, given the number of objects that can exist at a time and the potential performance impact I am not sure if we would want to export them as part of the getUnitStream
or some synchronous GetBallistics
API that is regularly called by the client?
This ticket is mainly for record-keeping and pondering.
When deserialization fails in a next
call this error gets returned from rust:
Lines 151 to 158 in 4a04655
Error retrieving next command: attempt to concatenate a userdata value
. This is very misleading.mlua:
function that passes the error to lua: callback_error
function that converts the mlua error into a string: init_error_registry
Split the API services into separate grpc pacakges so that we can rationalise the APIs and remove conflicts due to duplication, it also encourages proper separation of concerns.
Out of scope:
rpc.rs
file into package specific rust files. (Done)https://github.com/DCS-gRPC/rust-server/blob/main/lua/DCS-gRPC/exporters/object.lua#L28-L30
The heading
returned for a unit is only correct if the unit is moving exactly forward. This is because getVelocity()
is being used to calculate the "heading". The result is actually a 2d "velocity vector", which is the correct velocity vector. If the unit is moving forward it should match or be very close to the units heading.
However, it is not the units heading. A unit moving backwards such as a helicopter will result in "heading" from the unit exporter equal to the direction the unit is moving.
The correct way to get the units heading appears to be with the unit:getPosition()
function (hoggit wiki) using the same algo in use.
This is supported by this forum thread: https://forum.dcs.world/topic/148896-converting-a-units-orientation-to-a-quotheadingquot-script-help/
I'm able to confirm this with CustomService.Eval
(I'm working in typescript, you can do the same in any language. Below code is a snippet)
const lua = `return Unit.getByName("${name}"):getPosition()`
custom.eval({ lua }, (error, result) => {
if (error) {
console.log('failed to get unit position', error)
return
}
const pos = JSON.parse(result!.json!)
let heading = deg(Math.atan2(pos.x.z, pos.x.x))
if (heading < 0) {
heading = heading + 360
}
// here, heading value is always correct, even when moving backwards
})
function deg(radians: number): number {
return radians * (180 / Math.PI)
}
warning: opinions below:
As a consumer of dcs-grpc, I want the units actual heading, In addition to the units velocity. (speed, velocity vector, etc)
I propose that instead of (or in addition to) extracting the heading and speed, expose the units raw velocity vec3. In addition to the units position vector
There seems to be some attempt to expose the data available in lua outside of lua, but for units it doesn't align with the lua stuff very well. I had to figure out what heading
was here. And currently have no access to raw vecs to do my own math
Add comments to the .proto files that can then be used to auto-generate documentation
When the gRPC server is shutdown, it closes all opened streams (events, units, ...) as follows:
It is possible that (1) is not run again until the client read the previous item from the stream from (3). If this theory holds true, it would be possible to block the server shutdown by establishing a connection but never reading items from it.
Implement a basic spawn api that will let a client spawn a group with at least one unit.
Required
Optional
Notes
The spawn API can range from simple to very complex so I am happy to break this down into multiple stages across multiple merge requests, potentially across multiple milestones.
Left-over from #40 (comment)
All settings set on the GRPC.
table in a mission only apply to the mission and to hook scripting env, but they should also apply to the latter.
Comparing recent DCS version, I've noticed the following new events that we could add:
S_EVENT_BDA = 37
S_EVENT_AI_ABORT_MISSION = 38
S_EVENT_DAYNIGHT = 39
S_EVENT_FLIGHT_TIME = 40
S_EVENT_PLAYER_SELF_KILL_PILOT = 41
S_EVENT_PLAYER_CAPTURE_AIRFIELD = 42
S_EVENT_EMERGENCY_LANDING = 43
when AI aircraft lands on belly or ditch on water. Support for player will be added later
S_EVENT_UNIT_CREATE_TASK = 44
S_EVENT_UNIT_DELETE_TASK = 45
Hi ya'll, using the latest version it looks like the server never starts. The only mention in the log is:
2022-10-03 09:56:22.708 INFO [GRPC-Hook]: Initializing ...
2022-10-03 09:56:22.708 INFO [GRPC-Hook]: Checking optional config at Config/dcs-grpc.lua
...
2022-10-03 09:56:22.708 INFO [GRPC-Hook]: Config/dcs-grpc.lua
successfully read
2022-10-03 09:56:22.710 INFO [GRPC-Hook]: Initialized ...
There is no gRPC.log being generated. Our dcs_grpc.lua is as follows:
autostart = true
-- Whether the Eval
method is enabled or not.
evalEnabled = false
-- The host the gRPC listens on (use "0.0.0.0" to listen on all IP addresses of the host).
host = '0.0.0.0'
-- The port to listen on.
port = 50052
-- Whether debug logging is enabled or not.
debug = true
-- Limit of calls per second that are executed inside of the mission scripting environment.
throughputLimit = 600
This issue is all about versioning and how we want to handle it.
gRPC is designed with the philosophy that you maintain backwards compatibility as much as possible and that, when you cannot, you provide a new API while maintaining the old API (at least until you are sure everyone has migrated).
Therefore we need to think about how we will handle this in DCS-gRPC
To this end the recommended (and kind of required by Java and Go clients apparently) namespace format is. package.version
so, for example. Our atmosphere
package should actually be atmosphere.v1
and if we ever add a breaking change to any of the APIs then we add an atmosphere.v2
package and clone the .proto and make our changes in the new proto. (This is also reflected in the folder structure).
However if we completely embraces this concept then we need to do two things.
GRPC["methods"]["atmosphere"]["v1"]["getWind"] = function ...
and mirroring the proto folder structure in that respect.The issue with this is that we would end up loading all of the different versions of lua into the scripting environment which kind of goes against our concept of "more performance" but I am not sure if loaded but unexecuted lua is actually that much of a problem. There might be issues with streaming if there are streams coming from two different versions and we are trying to optimised but that scenario might not happen (Or we can make streams the exception in that we will only support 1 version at a time).
We only create a new version if we change the public interface. Changes to the lua implementation will not prompt a version upgrade requirement. This allows us to always have just one implementation of lua instead of multiple versions.
We do not do any gRPC / code versioning and rely on semantic versioning to allow clients to decide if they want to upgrade their servers and clients. I did this with the demo apps and, with the help of an IDE admittedly, it wasn't that painful.
I am not actually against any of the above options. I think the full versioning option is the proper option if we think we will have many different clients (which is actually a hope of mine so that adds weight to this argument).
Otherwise I think we should just go with no versioning. I think the "interface change only" option brings extra complexity without any appropriate benefit.
No matter what options we choose I think we need to do the v1
package naming to make things easier for Go and Java clients anyway and I think we should do it now since we are making big refactoring changes anyway.
Converting an event to a Rust struct is a costly operation (not as costly as Lua -> JSON -> Rust would be though). We should thus update the implementation to only make the conversion if necessary.
Implementation:
Move the DCT functions out of the custom
namespace and into a dct
namespace. All future Custom
, framework specific, work will be done in an isolated namespace.
As part of this we will rename the lua methods with a prefix as suggested by @justin-lovell to help with isolation and reduce chances of conflict. (i.e. dct.joinMission
). We may eventually rename all existing core methods as well but that is out of scope for this.
The DCS dedicated server will crash with the current DCS-gRPC master branch code on player connection:
Steps to replicate.
Note that this does not happen:
I am not sure when this started happening, there is at least one version of the server running on the Hoggit Syria At War server with no issues so we know of so this commit is working correctly.
If you cannot think of anything obvious then I wil do a git bisect
although that will take a bit of time.
Stack trace below:
2021-08-21 02:27:34.651 INFO EDCORE: signal caught: 'Abnormal termination'
2021-08-21 02:27:34.651 INFO EDCORE: try to write dump information
2021-08-21 02:27:34.653 INFO EDCORE: # -------------- 20210821-022734 --------------
2021-08-21 02:27:34.654 INFO EDCORE: DCS/2.7.5.10869 (x86_64; Windows NT 10.0.19042)
2021-08-21 02:27:34.656 INFO EDCORE: C:\Windows\System32\KERNELBASE.dll
2021-08-21 02:27:34.657 INFO EDCORE: # 00000000 STATUS_SUCCESS at 73F44ED9 00:00000000
2021-08-21 02:27:34.663 INFO EDCORE: SymInit: Symbol-SearchPath: '.;C:\Program Files\Eagle Dynamics\DCS World OpenBeta Server;C:\Program Files\Eagle Dynamics\DCS World OpenBeta Server\bin;C:\Windows;C:\Windows\system32;SRV*C:\websymbols*https://msdl.microsoft.com/download/symbols;', symOptions: 528, UserName: 'CowServer'
2021-08-21 02:27:34.665 INFO EDCORE: OS-Version: 10.0.19042 () 0x100-0x1
2021-08-21 02:27:35.025 INFO EDCORE: 0x0000000000034ED9 (KERNELBASE): RaiseException + 0x69
2021-08-21 02:27:35.025 INFO EDCORE: 0x00000000000BB677 (edCore): ed::core::InitVFS::~InitVFS + 0x6B7
2021-08-21 02:27:35.025 INFO EDCORE: 0x0000000000071881 (ucrtbase): raise + 0x1E1
2021-08-21 02:27:35.026 INFO EDCORE: 0x0000000000072851 (ucrtbase): abort + 0x31
2021-08-21 02:27:35.026 INFO EDCORE: 0x000000000000360D (VCRUNTIME140): _is_exception_typeof + 0x136D
2021-08-21 02:27:35.026 INFO EDCORE: 0x000000000000BB64 (VCRUNTIME140): __C_specific_handler + 0x334
2021-08-21 02:27:35.026 INFO EDCORE: 0x0000000000003029 (VCRUNTIME140): _is_exception_typeof + 0xD89
2021-08-21 02:27:35.026 INFO EDCORE: 0x000000000000C021 (VCRUNTIME140): __CxxFrameHandler3 + 0x71
2021-08-21 02:27:35.027 INFO EDCORE: 0x00000000000A21FF (ntdll): __chkstk + 0x19F
2021-08-21 02:27:35.027 INFO EDCORE: 0x0000000000030939 (ntdll): RtlUnwindEx + 0x339
2021-08-21 02:27:35.027 INFO EDCORE: 0x000000000000C80A (VCRUNTIME140): __report_gsfailure + 0x26A
2021-08-21 02:27:35.027 INFO EDCORE: 0x0000000000006ED1 (lua): lua_getinfo + 0x1291
2021-08-21 02:27:35.028 INFO EDCORE: 0x0000000000006CE2 (lua): lua_getinfo + 0x10A2
2021-08-21 02:27:35.028 INFO EDCORE: 0x0000000000006950 (lua): lua_getinfo + 0xD10
2021-08-21 02:27:35.028 INFO EDCORE: 0x0000000000017BB5 (lua): luaS_newlstr + 0x4075
2021-08-21 02:27:35.028 INFO EDCORE: 0x0000000000002AA0 (lua): lua_concat + 0x40
2021-08-21 02:27:35.028 INFO EDCORE: 0x000000000035A82A (dcs_grpc_server): luaopen_dcs_grpc_server + 0x1FCFBA
2021-08-21 02:27:35.028 INFO EDCORE: 0x0000000000357DD3 (dcs_grpc_server): luaopen_dcs_grpc_server + 0x1FA563
2021-08-21 02:27:35.029 INFO EDCORE: 0x0000000000007AC5 (lua): luaD_growstack + 0x845
2021-08-21 02:27:35.029 INFO EDCORE: 0x0000000000018D94 (lua): luaS_newlstr + 0x5254
2021-08-21 02:27:35.029 INFO EDCORE: 0x0000000000007E04 (lua): luaD_growstack + 0xB84
2021-08-21 02:27:35.029 INFO EDCORE: 0x0000000000006F7F (lua): lua_getinfo + 0x133F
2021-08-21 02:27:35.029 INFO EDCORE: 0x000000000000812E (lua): lua_yield + 0x9E
2021-08-21 02:27:35.030 INFO EDCORE: 0x000000000001EEAC (lua): luaL_newstate + 0x21FC
2021-08-21 02:27:35.030 INFO EDCORE: 0x0000000000007AC5 (lua): luaD_growstack + 0x845
2021-08-21 02:27:35.030 INFO EDCORE: 0x0000000000018D94 (lua): luaS_newlstr + 0x5254
2021-08-21 02:27:35.030 INFO EDCORE: 0x0000000000007E04 (lua): luaD_growstack + 0xB84
2021-08-21 02:27:35.030 INFO EDCORE: 0x0000000000006F7F (lua): lua_getinfo + 0x133F
2021-08-21 02:27:35.030 INFO EDCORE: 0x000000000000812E (lua): lua_yield + 0x9E
2021-08-21 02:27:35.031 INFO EDCORE: 0x0000000000002576 (lua): lua_pcall + 0x66
2021-08-21 02:27:35.031 INFO EDCORE: 0x00000000000F2DF9 (edCore): ED_lua_pcall + 0x59
2021-08-21 02:27:35.031 INFO EDCORE: 0x00000000000EE8F4 (edCore): Lua::Config::call_func + 0xF4
2021-08-21 02:27:35.031 INFO EDCORE: 0x00000000007D3C37 (DCS): SW + 0x4E3107
2021-08-21 02:27:35.031 INFO EDCORE: 0x00000000008886C3 (DCS): SW + 0x597B93
2021-08-21 02:27:35.032 INFO EDCORE: 0x000000000088125D (DCS): SW + 0x59072D
2021-08-21 02:27:35.032 INFO EDCORE: 0x000000000087E612 (DCS): SW + 0x58DAE2
2021-08-21 02:27:35.032 INFO EDCORE: 0x00000000000C6A57 (Flight): woATC::updateCoalition + 0x687
2021-08-21 02:27:35.032 INFO EDCORE: 0x000000000009D2DA (Flight): woATC::Control + 0x51A
2021-08-21 02:27:35.032 INFO EDCORE: 0x00000000000576CC (Flight): wAirdrome::Control + 0xCC
2021-08-21 02:27:35.032 INFO EDCORE: 0x000000000005B3E3 (Flight): wAirdrome::Init + 0xF33
2021-08-21 02:27:35.033 INFO EDCORE: 0x0000000000003AD6 (World): wSimTrace::CommandsTraceDiscreteIsOn + 0x466
2021-08-21 02:27:35.033 INFO EDCORE: 0x0000000000003F3D (World): wSimCalendar::DoActionsUntil + 0x1FD
2021-08-21 02:27:35.033 INFO EDCORE: 0x00000000007ECA22 (DCS): SW + 0x4FBEF2
2021-08-21 02:27:35.033 INFO EDCORE: 0x00000000007EC78E (DCS): SW + 0x4FBC5E
2021-08-21 02:27:35.033 INFO EDCORE: 0x00000000008022EB (DCS): SW + 0x5117BB
2021-08-21 02:27:35.033 INFO EDCORE: 0x00000000007D3194 (DCS): SW + 0x4E2664
2021-08-21 02:27:35.034 INFO EDCORE: 0x00000000007D356D (DCS): SW + 0x4E2A3D
2021-08-21 02:27:35.034 INFO EDCORE: 0x0000000001C7332F (DCS): AmdPowerXpressRequestHighPerformance + 0xE7D32B
2021-08-21 02:27:35.034 INFO EDCORE: 0x0000000000A0B23E (DCS): SW + 0x71A70E
2021-08-21 02:27:35.034 INFO EDCORE: 0x0000000000017034 (KERNEL32): BaseThreadInitThunk + 0x14
2021-08-21 02:27:35.034 INFO EDCORE: 0x0000000000052651 (ntdll): RtlUserThreadStart + 0x21
2021-08-21 02:27:35.243 INFO EDCORE: Minidump created.
2021-08-21 02:27:35.243 INFO Lua::Config: stack traceback:
Return parent objects intead of references.
For example, when returning a unit we don't return a group_name
or group_id
field. We return the parent group object. We also return the player
object (If one exists).
This will flesh out the information and allow for more advanced processing by event consumers and reduce the chance of race-conditions.
Note that we will only ever return the parent object. We wil not return children of the parent object since that will result in, at worst, an infinite loop (e.g. we will not return a unit with a group with a list of units, each with a group etc.).
DArt, the creator of LotATC is interested in potentially using DCS-gRPC as an alternative back-end to get data out of, and commands into, DCS.
He did a bit of searching in his codebase and determined he uses the following APIs. If we want to enable DCS-gRPC as a back-end for LotATC then we would need to provide these APIs or an equivalent of. Some of them we already provide or have alternatives to; and some of them I have never heard of.
Timer.GetTime
DCS-gRPC APIget_player_list
get_player_list
PlayerSendChatEvent
message in the event streamThe following are temporarily assessed as probably not needed if using DCS-gRPC:
It looks like errors returned from Rust to Lua might lead to DCS crashing once a mission is stopped. So either figure out what is wrong with those errors, or don't return errors at all and simply log them with log::error!
instead.
Noting here for reference.
Running the following via the gRPC hook eval API terrain.getRadio()[1]
returns a gRPC error response with the following in the dcs.log
.
2022-04-20 11:45:00.730 ERROR [GRPC-Hook]: Error retrieving next command: callback error
stack traceback:
[C]: in function 'next'
[string "C:\Users\jeff\source\repos\DCS-gRPC\rust-server\lua\DCS-gRPC\grpc.lua"]:244: in function <[string "C:\Users\jeff\source\repos\DCS-gRPC\rust-server\lua\DCS-gRPC\grpc.lua"]:242>
[C]: in function 'pcall'
[string "C:\Users\jeff\source\repos\DCS-gRPC\rust-server\lua\DCS-gRPC\grpc.lua"]:259: in function 'onSimulationFrame'
[string "C:\Users\jeff\source\repos\DCS-gRPC\rust-server\lua\DCS-gRPC\grpc-hook.lua"]:34: in function <[string "C:\Users\jeff\source\repos\DCS-gRPC\rust-server\lua\DCS-gRPC\grpc-hook.lua"]:32>
[C]: in function 'pcall'
[string "MissionEditor/loadUserScripts.lua"]:131: in function <[string "MissionEditor/loadUserScripts.lua"]:126>
(tail call): in function <(tail call):-1>
caused by: Failed to deserialize result for method hookEval: deserialize error: invalid type: integer `2`, expected a string key
{
"role" = {
1 = "ground",
2 = "tower",
3 = "approach",
},
"callsign" = {
1 = {
"common" = {
1 = "OMAA",
2 = "OMAA",
},
},
},
"sceneObjects" = {
1 = "t:60392560",
},
"radioId" = "airfield22_0",
"frequency" = {
2 = {
1 = 0,
2 = 119200000,
},
},
}
rust-server/lua/DCS-gRPC/methods/unit.lua
Lines 112 to 114 in 15f8064
Table is populated with index values, but not with any values or keys.
Allow users to (optionally) specify the IP and port to bind to. This will allow users to run multiple instances on one server long with the DCS instance.
The changes should enabled something like the following (Using the README.md init example)
package.cpath = package.cpath..[[M:\Development\DCS-gRPC\rust-server\target\debug\?.dll;]]
GRPC = { basePath = [[M:\Development\DCS-gRPC\rust-server\lua\]] }
+GRPC.host = '192.168.1.10' -- '*' for everything, '127.0.0.1' default
+GRPC.port = 54763 -- 50051 default.
dofile(GRPC.basePath .. [[grpc.lua]])
There are two areas where we can possibly clean up APIs we are exporting.
There is inconsistency in the the use of weaponName
string fields and weapon
tables in the lua events. We could get rid of the weaponName
field and move everything into the weapon
table for events that have both (hit/kill events being good examples).
For the APIs that take mission filenames (LoadMission
, GetMissionFileName
) and which take file paths (For some reason) we could do things such as splitting the paths and filenames into two separate fields.
See #11 (comment)
This issue is the master issue for getting group spawning working and will probably have more things added to it over time as we realise what needs to be done:
The API for spawning addGroup
in lua is basically "Pass in a huge lua table and hope you get it right" which is not the most user-friendly experience. Therefore we should try and add validation to our gRPC API (still in lua) where possible to remove some of the uncertainty around calling it.
It appears that in some situations (especially on very fast hardware) DCS races the DCS-gRPC initialization and causes an unrecoverable scripting error trying to call world.addEventHandler.
An example of the log output for this situation:
2022-01-09 06:01:17.040 ERROR SCRIPTING: [GRPC] Checking optional config at `Config/dcs-grpc.lua` ...
2022-01-09 06:01:17.040 INFO SCRIPTING: [GRPC] Optional config at `Config/dcs-grpc.lua` not found (can't open "C:\Users\andrei\Saved Games\DCS.openbeta_server\Config\dcs-grpc.lua" for 'r')
2022-01-09 06:01:17.040 INFO SCRIPTING: [GRPC] Writing C:\Users\andrei\Saved Games\DCS.openbeta_server\Data\dcs-grpc.lua
2022-01-09 06:01:17.041 INFO SCRIPTING: [GRPC] auto starting
2022-01-09 06:01:17.047 INFO SCRIPTING: [GRPC] mission loading ...
2022-01-09 06:01:17.064 INFO SCRIPTING: [GRPC] Limit request execution at max. 18 calls every 0.03s (โ throughput of 600)
2022-01-09 06:01:17.064 ERROR Lua::Config: load error Scripts/MissionScripting.lua:[string "C:\Users\andrei\Documents\GitHub\rust-server\lua\DCS-gRPC\grpc.lua"]:239: attempt to call field 'addEventHandler' (a nil value).
A simple workaround is to delay calling this function for a little bit (1 second worked for me, but we could experiment further).
timer.scheduleFunction(function ()
GRPC.logInfo("Adding world event handler")
world.addEventHandler(eventHandler)
end, {}, timer.getTime() + 1)
I've only encountered this on my desktop PC (i9-11900k), other instances of DCS running in VMs / on slower hardware do not encounter this problem.
The following is in our log:
2023-01-02 19:34:05.364 INFO Scripting (Main): event:type=kill,target_unit_type=BRDM-2,target_coalition=1,weapon=AGM-65L,t=5703.05,initiator_ws_type1=1,targetMissionID=1002613,initiator_unit_type=AV8BNA,target_ws_type1=2,initiator_object_id=16812805,initiator_coalition=2,target=BRDM-2,initiatorPilotName=TDCS | Pilot Pyles,initiatorMissionID=2166,target_object_id=17092097,
2023-01-02 19:34:05.364 ERROR SCRIPTING (Main): [GRPC] Error in event handler: [string "C:\Users\TacticalDCS\Saved Games\instance2\Scripts\DCS-gRPC\exporters\object.lua"]:45: attempt to index a nil value
stack traceback:
[string "C:\Users\TacticalDCS\Saved Games\instance2\Scripts\DCS-gRPC\exporters\object.lua"]:45: in function <[string "C:\Users\TacticalDCS\Saved Games\instance2\Scripts\DCS-gRPC\exporters\object.lua"]:27>
(tail call): ?
[string "C:\Users\TacticalDCS\Saved Games\instance2\Scripts\DCS-gRPC\methods\mission.lua"]:43: in function 'typed_exporter'
[string "C:\Users\TacticalDCS\Saved Games\instance2\Scripts\DCS-gRPC\methods\mission.lua"]:341: in function <[string "C:\Users\TacticalDCS\Saved Games\instance2\Scripts\DCS-gRPC\methods\mission.lua"]:63>
(tail call): ?
[C]: in function 'xpcall'
[string "C:\Users\TacticalDCS\Saved Games\instance2\Scripts\DCS-gRPC\grpc.lua"]:226: in function 'onEvent'
[string "Scripts/World/EventHandlers.lua"]:13: in function <[string "Scripts/World/EventHandlers.lua"]:11>
This links to this line in object.lua:
groupName = Unit.getGroup(unit):getName(),
Allow for clients to tune the number of times DCS-gRPC can call into the DCS Mission Scripting Environment from the initialization lua. Optionally also set the TPS for which the gRPC server is polled for incoming requests (This must be equal or lower to the MSE TPS limit).
Something along the lines of
package.cpath = package.cpath..[[M:\Development\DCS-gRPC\rust-server\target\debug\?.dll;]]
GRPC = { basePath = [[M:\Development\DCS-gRPC\rust-server\lua\]] }
+GRPC.setMaxMseTps(10)
+GRPC.setMaxGrpcTps(10)
dofile(GRPC.basePath .. [[grpc.lua]])
Hello!
I am having the following problem after the server was running for a couple of hours:
Copied the log out of the Air defense logging, but my own application gets the same error.
Could it be that I did not properly close the connection i.e when forcefully shutting down my debugging session in Visual Studio?
I have a session pool that only allows one active session at a time so it cannot be that too many sessions are created.
I will have a look if there is a difference between me shutting it down using the ShutdownAsync function and forcefully closing and will list the results here.
Many thanks in advance!
IP replaced with ***
Es konnte keine Verbindung hergestellt werden, da der Zielcomputer die Verbindung verweigerte. = Connection refused by the remote host.
[WRN] gRPC Exception
Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Es konnte keine Verbindung hergestellt werden, da der Zielcomputer die Verbindung verweigerte. () SocketException: Es konnte keine Verbindung hergestellt werden, da der Zielcomputer die Verbindung verweigerte.", DebugException="System.Net.Http.HttpRequestException: Es konnte keine Verbindung hergestellt werden, da der Zielcomputer die Verbindung verweigerte. ()
---> System.Net.Sockets.SocketException (10061): Es konnte keine Verbindung hergestellt werden, da der Zielcomputer die Verbindung verweigerte.
Since DCS-gRPC is hooked into the hook environment and gets called on each simulation frame; we can calculate the FPS of the server and emit that value as an event every second. This will allow clients to be able to see the server FPS which is useful for diagnostic purposes.
One solid use-case is to use Telemachus
to record FPS metrics for display and alarming.
Note: Need to verify that a paused server still calls OnSimulationFrame
or not. If it still calls the hook when the server is paused then we could use this as a way to see if the server has frozen or not.
Since DCS-gRPC clients have a lifecycle outside of the DCS server we need a way for them to know if a mission has restarted or a new one loaded. At the moment the only thing a client will know is that it got disconnected and then reconnected.
After bouncing some ideas around with @rkusa I think what we want is:
Mission.GetSessionId
(Namespace up for debate) DCS-gRPC
method.dcs_grpc_session_id
os.time(os.date("!*t"))
and return the value.This will generate a new session unique ID each time a mission is loaded and return the same value until another one is loaded. Clients can then use this ID to determine if a session has changed after getting disconnected and perform appropriate processing.
Documenting this here in case anyone wants to give it a go, otherwise I will get around to it at some point. This would be a good, simple, API to implement for a newcomer.
In browsing the source code, I may have generated a conflict without realizing it at the time. Here are two functions which both define getPlayers
:
rust-server/lua/DCS-gRPC/methods/coalitions.lua
Lines 153 to 160 in b1d3906
rust-server/lua/DCS-gRPC/methods/net.lua
Lines 24 to 43 in c437fb4
I would like to propose to avoid naming conflicting, let's prefix the method name with some annotation of the proto file as a namespace.
E.g.
- GRPC.methods.getPlayers = function(params)
+ GRPC.methods.net_getPlayers = function(params)
and
- GRPC.methods.getPlayers = function(params)
+ GRPC.methods.coalition_getPlayers = function(params)
An alternative is setup a localized Lua table with the following pseudo:
local coalition = {}
GRPC.methods.coalition = coalition
coalition.getPlayers = function() {
-- ...
}
and the Rust code would perform a function call similar to:
- self.request("getPlayers", request)
+self.request("coalition", "getPlayers", request)
Add the timestamp to the StreamUnitsResponse so that clients know exactly when the update occurred. This will bring it inline with the EventStreamResponse which already has the time.
The use-case provided for this was to be able to replay event streams offline for testing and other purposes.
EDIT
I think the easiest thing to do so far is to just use the MIST code where the lua will literally just be reading the mist.dbs.aliveUnits
table, maybe converting them using the existing exporters and returning the table with the exported versions.
The question is how to have the rust code call this method on a timer and stream the objects to all clients. Since this is rust area I have very little idea of what needs to be done
Putting this in a Github Issue for discussion and brain storming.
We would like to export unit position information as a gRPC stream to clients. We want to do this in a way that does not block the lua thread in large missions. We want to do it at a use specified refresh rate which would be coarse to the level of something like every 1 second at highest refresh rate but could be as slow as every 10 seconds.
Looking for ideas since some of this will require more rust coding than just acting as an RPC interface I think (The perform in chunks option especially).
The basic process is:
This will work but it will get all units for each coalition in one go which may end up being many units and that could block the lua thread.
Possible ways around this....
coalition.getGroups
for coalitions 0, 1, 2 and add to a united groups
table which is returned to rustGroup.getUnits
and return a lua table that contains the units to the rust code that iterates and serializes and streams each unit.This is the same as the basic process but we use lua coroutines and add some yields
in the code when it iterates groups (or even units). I have no idea if coroutines will work as expected when the method is being called from rust or even if they will work properly.
We could look at how MIST does it ( https://github.com/dcs-liberation/dcs_liberation/blob/b8e6c2fe78ca3a3d49d302239403777e46bd12ad/resources/plugins/base/mist_4_4_90.lua ) or even just use MIST for the implementation by default.
Here is a hook that can be used to inject the gRPC server into all missions of the DCS server:
Saved Games\DCS.openbeta\Scripts\Hooks\gRPC.lua
:
local function inject()
log.write("[gRPC]", log.INFO, "Injecting gRPC into mission")
local result = net.dostring_in("server", [[
if GRPC == nil then
package.cpath = package.cpath.."M:\\Development\\DCS-gRPC\\rust-server\\target\\debug\\?.dll;"
GRPC = { basePath = "M:\\Development\\DCS-gRPC\\rust-server\\lua\\" }
dofile(GRPC.basePath .. "grpc.lua")
end
]])
end
local function init()
local handler = {}
function handler.onSimulationResume()
local ok, err = pcall(inject)
if not ok then
log.write("[gRPC]", log.ERROR, "Failed to inject: "..tostring(err))
end
end
DCS.setUserCallbacks(handler)
end
local ok, err = pcall(init)
if not ok then
log.write("[gRPC]", log.ERROR, "Failed to init: "..tostring(err))
end
Adding it as an issue here to keep track of it. We can think of somehow adding it to the repo in the future.
Add github CI stage / actions to create a downloadable zip file that contains the required DCS-gRPC files as well as installation guide and documentation
I know there are certain DCS servers like Enigmas that don't use LotATC or Tacview server side for performance reasons, and I know that DCS-gRPC can help server performance by offloading expensive calculations. My initial thought is DCS-gRPC might not help improve server performance when used as the basis for unit position streaming/recording applications.
Presumably though, there are some expectations of improved performance with #131, so are there similar expectations if DCS-gRPC were used to create an alternate Tacview exporter?
Before 2.8, DCS did not have an API to retrieve the magnetic variation at a coordinate. Since 2.8 (might have also been available before, just unknown), there is finally one.
Our current implementation calculates the magvar outside of DCS based on the International Geomagnetic Reference Field (IGRF). The result is very close to what DCS calculates, but can now be replaced by the new API provided with DCS 2.8:
local magvar = require('magvar')
magvar.get_mag_decl(lat, lon)
For some more details around the API, see:
https://discord.com/channels/696721363276267590/911042786122399824/1041889659174793256
Necessary changes:
igrf
dependencysteps to reproduce:
WorldService.GetMarkPanels
error from the logs:
stack traceback:
[string "<redacted>\Saved Games\DCS\Scripts\DCS-gRPC\exporters\object.lua"]:28: in function 'unit'
[string "<redacted>\Saved Games\DCS\Scripts\DCS-gRPC\exporters\object.lua"]:122: in function 'markPanel'
[string "<redacted>\Saved Games\DCS\Scripts\DCS-gRPC\methods\world.lua"]:36: in function <[string "<redacted>\Saved Games\DCS\Scripts\DCS-gRPC\methods\world.lua"]:31>
(tail call): ?
[C]: in function 'xpcall'
[string "<redacted>\Saved Games\DCS\Scripts\DCS-gRPC\grpc.lua"]:167: in function <[string "<redacted>\Saved Games\DCS\Scripts\DCS-gRPC\grpc.lua"]:163>
[C]: in function 'next'
[string "<redacted>\Saved Games\DCS\Scripts\DCS-gRPC\grpc.lua"]:200: in function <[string "<redacted>\Saved Games\DCS\Scripts\DCS-gRPC\grpc.lua"]:198>
[C]: in function 'pcall'
[string "<redacted>\Saved Games\DCS\Scripts\DCS-gRPC\grpc.lua"]:211: in function <[string "<redacted>\Saved Games\DCS\Scripts\DCS-gRPC\grpc.lua"]:209>
2022-06-04 22:42:46.069 ERROR dcs_grpc: Error retrieving next command: attempt to concatenate a userdata value```
Log performance information to allow for server owners to have the information needed to see how DCS-gRPC is impacting their server.
Every minute write to the dcs.log file with the following information calculated from metrics of the last minute.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.