gsteinltu / roboscapesimulator Goto Github PK
View Code? Open in Web Editor NEWSimple online simulation of robots for demonstration/education purposes
Simple online simulation of robots for demonstration/education purposes
So far, environments have only used the reset button for everything, but it would be nice to have options defined by each environment's needs. For example, a "start" button for a race environment that toggles to a "stop" button, with some features changed during the "race" period (e.g. locking out reset/encrypt) so an "official" run can have separate rules from non-official runs. This will also need work on the client-side.
There seems to be some sort of issue with JIT where the first rooms launched by the server misbehave and need a reset. Having the server startup include opening a room and running it for a bit should fix this.
His request:
This sounds simple but implementing it properly will need a few changes to how rooms work to make them more flexible. It would probably be a good refactor to give more flexibility around physics engines in the future if Bepu needs replacing or gets a major breaking update.
Ben requested this.
Latency (and possibly other network conditions) for robot communications should be able to be set by the room's owner. This allows for simulating distance between robot and user, which would be useful both for teaching and for testing performance of other code under poor network conditions.
It would be nice to be able to use something like geckos.io for UDP-based networking, or even just newer versions of socket.io, but the C# library for socket.io being used only supports an older version. My current plan for this is to have a Node program handle the networking, with messages passed back and forth with some form of IPC, but this will require a lot of testing to make sure it is feasible/reliable. This may solve #68 if that is due to the library used.
Current tests do not provide much coverage. Need to look into the best mocking options for socket.io in order to create a larger test suite. Client-side tests may also be feasible to add.
Some scenarios represent activities that currently rely a bit on the honor system for students to not cheat by resetting or encrypting the robots mid-course. For cybersecurity lessons especially this is not desirable as they encrypt button would be too powerful and is meant to represent having the robot physically.
Would require a client-side addition to display the buttons as disabled, although it could potentially be implemented as the server "claiming" the robots itself. #116 is likely a prereq for this to create a state where it makes more sense to disable reset, but activities targeting cybersecurity and using the race timer could potentially just disable encrypt while the timer is running.
After running for some amount of time (a week?), the server becomes semi-unresponsive regardless of usage level.
Clients appear to connect (the server logs them connecting/disconnecting), but will not receive any responses from the server.
Previously had:
Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt. malloc(): corrupted top size
Today had:
Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
Repeat 15401 times:
--------------------------------
at BepuPhysics.Trees.Tree.SplitSubtreesIntoChildrenBinned(BepuPhysics.Trees.BinnedResources ByRef, Int32, Int32, Int32, Int32 ByRef, Single ByRef)
at BepuPhysics.Trees.Tree.CreateStagingNodeBinned(BepuPhysics.Trees.BinnedResources ByRef, Int32, Int32, Int32 ByRef, Single ByRef)
--------------------------------
at BepuPhysics.Trees.Tree.BinnedRefine(Int32, BepuUtilities.Collections.QuickList`1<Int32> ByRef, Int32, BepuUtilities.Collections.QuickList`1<Int32> ByRef, BepuPhysics.Trees.BinnedResources ByRef, BepuUtilities.Memory.BufferPool)
at BepuPhysics.Trees.Tree.RefitAndRefine(BepuUtilities.Memory.BufferPool, Int32, Single)
at BepuPhysics.CollisionDetection.BroadPhase.Update(BepuUtilities.IThreadDispatcher)
at BepuPhysics.Simulation.CollisionDetection(Single, BepuUtilities.IThreadDispatcher)
at BepuPhysics.DefaultTimestepper.Timestep(BepuPhysics.Simulation, Single, BepuUtilities.IThreadDispatcher)
at BepuPhysics.Simulation.Timestep(Single, BepuUtilities.IThreadDispatcher)
at RoboScapeSimulator.SimulationInstance.Update(Single)
at RoboScapeSimulator.Room.Update(Single)
at Program+<>c__DisplayClass0_1.<<Main>$>b__1(System.Object)
at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(System.Threading.Thread, System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
at System.Threading.TimerQueueTimer.Fire(Boolean)
at System.Threading.ThreadPoolWorkQueue.Dispatch()
at System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart()
May need a better check for this sort of failure to auto-restart, should also talk to bepu people about this.
UDP makes more sense for this application, switching to Geckos.io would provide this.
A long term goal will see the simulator running on multiple server instances to both distribute load and allow for scaling. The main instance will need to coordinate with other servers to set up a web socket connection on the client to where the room is hosted, and, if necessary/possible, start additional servers to meet demand.
It may be necessary to set up a "budget" for each server to ensure quality of service and reduce costs. Then room creation could allow for more limited rooms (lower robot count, lower tick-rate, etc) if necessary.
It would be nice if some portion of the environment could be created from the NetsBlox side, even if it's just specifying where some blocks/robots would go. Things like triggers could even be included, although they would likely need to talk to NetsBlox code instead of C# code. The resource limits of this would need to be figured out (e.g. how many robots are allowed?), but if it can be combined with a "create room" block, custom environments could be entirely contained in a NetsBlox project. Some of this functionality could also allow for things like #111 depending on how this functionality is exposed.
New boxes appear as pushed off, with timer and count.
Timer that instead of starting on room start, starts when robot starts moving or gets first command.
Eventually it would be nice to have a countdown that locks out robots, but this is sufficient for now.
Current default environment doesn't show many features (although it's pretty lightweight, which is desirable), and the random boxes/robots sometimes have issues. It would be better to have a default environment that's more reliable and a little more interesting.
Clipboard API won't work in all contexts, it should be hidden to prevent confusion from a non-functional button.
Robots occasionally get stuck on corners they should be fine going around, the hitboxes may be slightly too wide.
Need the ability to remove entities.
Quadcopter style drone simulation
Goals:
Big question is how to control it:
Environments would be more useful if they could keep score for the games played inside them. For example, a robot soccer environment where trigger areas on each goal add a point to their respective teams and move the ball back to the middle.
Possible actions on trigger brainstorming:
There will need to be some more general filters available for robots/pushables as a class.
This may require a client-side UI to display some environment-specific information and may eventually evolve into a full scripting interface for environments, which comes with its own set of challenges.
Need to figure out how best to test things, the physics engine might be light enough to run tests inside of it accelerated.
Also add getPosition RPC that returns x y z tuple.
It's minor, but tests cause robots to show up on netsblox.org by default and it would be nicer if they didn't.
This is a longer term goal, especially since #5 is a prerequisite for it.
One of the big challenges with using the robots IRL is assessment. Competitions are "fun" and mean students do more than just wait their turn to demonstrate, but tournaments are hard to manage while running a class, and not all tournament systems are good for all groups. Some may want a quick elimination bracket, some may have time for a Swiss tournament which has more "fairness", some may want a full round-robin.
With the robots reset when the round begins, it also eliminates the usual real-life difficulty of making sure the robots have been reset and were turned on at the same time.
Ideally, the system would allow for students to be given a "practice" section of the environment to work with (see #7), and the students in the current competition round would be alerted and have their robots moved to the main area so they can start.
Current sensors are all polling-based, but to enable some educational possibilities (mostly to make them vulnerable to sensor spoofing), it would be good to have an option to subscribe to updates from a sensor, and to have sensors be able to have one of the modes disabled based on scenario.
For IDs this could be a bit trickier to avoid collisions, especially with physical robots (querying the NetsBlox server may be the solution there) and may be unnecessary with the "robots in room" block, but users should be able to at least choose the robot "model" among compatible meshes, possibly with a fully customizable color that would be consistent between rooms. Color "collisions" could also be a problem, but it's not a big issue.
Need to have help content (possibly available through the wiki/pages here) and a way to access relevant info in use. Tooltips would be helpful.
May need some brainstorming of exactly what would be helpful, but being able to get a log of commands or speeds/positions could be useful.
Current ideas:
LIDAR and PositionSensor
Varying numbers
With/without walls
There needs to be a system to control who is allowed to create rooms, eventually even who can see/join rooms. At some point teachers would have access codes to create rooms, but how this whole system will work will probably require input from Akos.
Four-way intersection, traffic light at center with each direction controlled by an IoTScape service. Autonomous cars spawn and attempt to cross. A traffic sensor accessible through IoTScape detects vehicles waiting on each direction. When the light in a direction is green/yellow, cars will cross in that direction. When it is red, they will stop.
Could be a use for null physics engine if adding triggers to it is not too difficult (depending on implementation it could be very difficult or not at all) and handling the car-car collisions isn't too hard.
Also requires the ability to update visualinfo (#112) after start.
Now that it is available, we should take advantage of its improvements. Native AOT could make for much better improvement if added to the build process. System.Text.Json has some improvements which could be useful.
Reusable code to create noise/error/drift/latency/non-linearity/bucketing/etc for sensors to make them more realistic could be useful for more advanced scenarios.
This will have #5 as a prereq.
We will need to test if this is more efficient than having multiple separate rooms. The idea is that the room's environment would be cloned and translated, so students can have separate "sub rooms" for their own robots, which can be created/merged/destroyed by the instructor.
A cool visualization (for the instructor/classroom view) would show all of the sub-environments in a grid so what all the students are working on is visible at the same time. A camera able to zoom/pan between them smoothly would be nice on top of that.
Room creators should be able to change settings for their rooms, reset the environment, kick users/robots, etc.
The solution used for #4 will likely change details of how this is implemented, as an account system would be the more straightforward way to decide who the owner of a room is.
This new release provides many fixes and performance improvements, but may require some changes to switch to.
A student had PositionSensor fail to work, but the Waypoint service was still working.
Allow for directly connecting to a room with a link, with a share room button in the sidebar.
Current events are not compatible with how the Unity plugin sends messages. This will need to be changed.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.