doctorray117 / minecraft-ondemand Goto Github PK
View Code? Open in Web Editor NEWTemplates to deploy a serverless Minecraft Server on demand in AWS
License: Apache License 2.0
Templates to deploy a serverless Minecraft Server on demand in AWS
License: Apache License 2.0
I'm trying to get this set up for various worlds/modpacks, and it's not clear to me how to use an existing world, or upload a modpack zip for minecraft-docker to use, or add datapacks to an existing world
I'm pretty inexperienced with AWS, and I went through the guide a few times, but the Lambda errors out when trying to start the service. I'm getting this error.
"errorMessage": "An error occurred (AccessDeniedException) when calling the DescribeServices operation: User: arn:aws:sts::XXXX:assumed-role/minecraft-launcher-role-mijbep5x/minecraft-launcher is not authorized to perform: ecs:DescribeServices on resource: arn:aws:ecs:us-west-1:XXXX:service/minecraft/minecraft-server because no identity-based policy allows the ecs:DescribeServices action",
I have the polices attached to the role, but I don't know what the assumed-role part of the arn means, or how to attach policies to it.
I've tried to deploy it on the M1 Mac.
When I deployed it without customization code, I've got tasks failed to start
issue on AWS ECS.
The reason was the failure of the watchdog container.
The error message was just exit code 1
on the container in the failed task.
https://github.com/doctorray117/minecraft-ondemand/blob/main/cdk/lib/minecraft-stack.ts#L226-L232
- image: isDockerInstalled()
- ? ecs.ContainerImage.fromAsset(
- path.resolve(__dirname, '../../minecraft-ecsfargate-watchdog/')
- )
- : ecs.ContainerImage.fromRegistry(
- 'doctorray/minecraft-ecsfargate-watchdog'
- ),
+ image: ecs.ContainerImage.fromRegistry(
+ 'doctorray/minecraft-ecsfargate-watchdog'
+ ),
If the script builds a watchdog image from the asset, you get an arm64 build image.
The image doesn't work because the current ECS Task definition use x86 CPU architecture. Otherwise need to config: AWS ECS Doc: Specifying the ARM architecture in your task definition
So, I changed the code to get the image from the registry always.
Related issue: #27
Just share my solution to resolve the same issue of others and to improve the code to support arm64 machine efficiently.
Ideally this will provide us with two methods for deployment:
This work is currently in progress, placing here to avoid duplicated efforts.
first off, well done, this is a great piece of work and clearly a lot of effort gone in.
I've made a few modifications as I wanted to run a pufferpanel server rather than just a single standalone minecraft server. Also puffer panel then gives easy access to the files through sftp to install plugins and tools to manage the server.
I noticed that the watchdog doesn't seem to work (well I can't seem to see it working) for bedrock connections. Not sure about the netstat commands etc in the script but from what I can see (including local testing) it never detects any udp connections.
I tried everything I could to get it to work, but couldn't. So in the end I gave up and just decided to write a small node script that use the minecraft protocol to get the status of the server directly. Seems to be a bit more robust as it gets the status and number of players online directly. I'd be happy to share it if you are open to taking PRs?
Something is causing the watchdog process to terminate early, setting the task count to zero (maybe a SIGTERM, or something else. This seems to consistently happen 15-20 seconds after the watchdog container listens for connections.
I've set up the guide on two AWS accounts from scratch and encounter the same issue. Not sure if it's me or the guide, but something's missed.
Timestamp | Message |
---|---|
2021-09-09T12:31:08.332+01:00 | Waiting for Minecraft RCON to begin listening for connections |
2021-09-09T12:31:24.423+01:00 | Setting desired task count to zero. |
Hi, your project inspired me to work on a Terraform implementation instead of using the AWS CDK. I have worked with CloudFormation for a couple of years, I can hardly bring up the patience for its quirks and features anymore. Terraform isn't perfect either, but it gives a whole lot of options that CFN just doesn't bring to the table. I happily imported my EFS file system from the CDK stack into my terraform state.
Everything has been implemented except for the Twilio variables. I'm looking to add discord integration as well.
Appreciate the work you've put into this, it's a great setup.
I've currently used your setup for a minecraft server which has been great, but I'm trying to wander off into the weeds by setting up a valheim server.
It uses 2456-2457/udp.
I was looking at using something like a python library to query the server direct, but then I realised your bedrock code is detecting connections on UDP.
Spent the evening trying to understand it, currently lost on the bedrockping string you build.
Basically just seeing if you'd be willing to shed some light on it before I spend some more time on it :)
Thanks again.
Hi
This is a fun project :)
I have 3 questions
Could the Watchdog not run in a Lambda? triggered every 10 minutes or so by EventBridge schedule that is created by the error handling lambda (and deleted again when the container is terminated)? This should further optimise cost and make it more serverless.
This is not a scaling solution, right? it is 1 container that needs to be big enough for the expected number of users.
if the container is launched after a hit on the domain name, at some point you will get a lot of false positives from bots and such, right? is there a (serverless) way to make sure it's a Minecraft client asking for the link before launching anything?
thanks!
Hello,
Can you advise on how the hosted zone could be reused for multiple instances (i.e. define in a outer stack and share to multiple sub-stacks) or how an existing HZ should be used in it's place (via lookup).
I would like to run multiple servers and would like to avoid paying HZ fees for every instance, when it may be possible to have them all under different sub-sub domains under a common HZ.
Thank you for any guidance you can give
Cloudformation Error: Resource handler returned message: "The runtime parameter of nodejs12.x is no longer supported for creating or updating AWS Lambda functions. We recommend you use the new runtime (nodejs18.x)
Hi,
I've deployed the CDK stack to my account this weekend but it was much more complicated than expected.
This sadly was caused by the recent AWS transition to the version 2 of CDK bootstrapping module.
To make it work, remember to export the CDK_NEW_BOOTSTRAP
environment variable:
export CDK_NEW_BOOTSTRAP=1
If you agree I can create a PR, adding this in the documentation.
Thanks for the help and thank you again for this awesome piece of art.
Hosting a serverless Minecraft server is so cool :)
Just did a fresh setup via the CDK, and when I was walking through the manual steps under Option 2: DataSync and S3 section of the readme, there's a couple things missing that might've changed since the last time these steps were updated:
minecraft-server-stack/Vpc/PublicSubnet1
and minecraft-server-stack/Vpc/IsolatedSubnet3
and they both work (as long as the next setting is correct)minecraft-server-stack-ServiceSecurityGroup
, using the other SG created (minecraft-server-stack-FileSystemEfsSecurityGroup
) or a default SG will not work OOTB/
in front of the file names, which is weird but the form fails validation if any of the fields are missing it (personally, I just use the "Entire source location" option to avoid the hassle)I'm running Spigotmc / Geysermc and the watch dog scripts cannot detect bedrock client connection. Thus my server is shutting down after 10 minutes even I'm playing and connecting. The easy way to mitigate the issue is assuming that Java server can host both of bedrock clients (UDP ports) and Java clients. Could we have such change?
PS. thanks for awesome documentation and the script. I'm AWS newbie and it only took 2 hours to setup the ECS cluster and lambda stuffs. Everything is working great except the bedrock client detection.
I was able to get the EFS stuff working with editing files but I'd like to restart the server to see my changes. How do I do that without waiting 20 minutes?
Hi,
I'm pretty new to aws, and I followed your wonderful instructions, but unfortunately I had some issues I couldn't figure out why (and they are not in the troubleshooting section). I hope this is the right place to seek for help.
Basically the lambda works, the watchdog can trigger the lambda, and both tasks will try to run, but they will die immediately with the following error (they couldn't even print out any log, except just the error):
ResourceInitializationError: failed to validate logger args: create stream has been retried 1 times: failed to create Cloudwatch log stream: AccessDeniedException: User: arn:aws:sts::zzzzzzzzzzzz:assumed-role/ecs.task.minecraft-server/123456a999ee4b919...
Those two tasks are both running docker images, thus I have no idea why they have to create Cloudwatch logs. I can verify ecs.task.minecraft-server
has all four policies added,
Any suggestions will be appreciated. Thank you.
EDIT: I forgot to mention, the desired task count is always 1, even after both tasks failed. Had to manually update it to 0.
Simply amazing...
You, Sir, are a gentleman and a scholar.
Bravo!
Hello,
I was wondering if there was a way to have my minecraft server hosted on my root domain, instead of subdomain.domainname. For example, instead of connecting to the server with acme.example.com, I just want to be able to type in example.com to connect to the server. I tried removing some code related to the subdomain in the cdk code, but it ended up breaking and not being able to deploy anymore. Does anyone know the correct way to do this? Thanks!
Your tutorial guides are very well written but do not mention how to use this with Pterodactyl Panel, which makes managing and operating a minecraft server much easier. I would love to see a guide for Pterodactyl On Demand!
Hello, and thank you for putting together this tutorial! I don't have much AWS experience, but I was able to get about 99% of the way to an ondemand modded Minecraft server with your help. The server spins itself up and shuts itself down according to activity as advertised, and I've been able to log into them.
Unfortunately, I've encountered an issue that I'm not sure how to resolve. It seems like the cluster containers aren't using the EFS as a data volume somehow. Nothing carries over between sessions, and when I manually mount into the EFS between active sessions, there's nothing there besides the ops.json I manually created, inserted into the S3 and then transferred into the EFS. Judging by that, I know that the DataSync tasks are working and the FS and FSAP IDs are correct.
Cluster Task Definition Storage Configuration
efs.rw.minecraft-data IAM policy
Roles with efs.rw.minecraft-data attached
Logs from Cluster tasks showing files and directories being referenced
EC2 Console After Mounting EFS and using ls to check contents
I've been tweaking a number of parameters including minecraft version, java flags, data packs, and server specs. All seem to perform worse than what I'm used to for similar server specs in terms of chunk loading (both new and old chunks). I'm wondering if anyone else is experiencing this issue, and if it could perhaps be EFS at fault.
@ryansonshine While testing CDK set up and tear down, the npm run destroy failed unless I manually deleted the A record in the subzone. Is that behavior that you would expect? I wonder if it's because it wasn't set at the default 192.168.1.1 anymore?
Just wondering if anyone has had this happen. After a while of having the system setup, the service started coming up and down like a yo yo when we weren't using it. I checked the logs and the domain name is getting hit all the time.
I assumed it's just hackers port scanning and trawling to find servers etc, but it's kinda made the auto start feature unusable. But just wondering if I'm missing something?
Any other ideas on easy ways to setup some sort of autostart function?
Cheers
@all-contributors add @doctorray117 code
Insufficient permissions, disregard
Can I create multiple instances using multiple tasks?
Is load balancing possible?
And how do you check the health of each container?
This would make the 400ln install process as simple as calling terraform apply
First off, this is very cool and I'm currently using it without issue! My only concern is scaling with number of players or general preference, since afaik Minecraft performance is almost entirely based on the single core performance. I believe this can be achieved using an EC2 Launch Type instead of Fargate, is that correct? Could most of the configuration remain the same?
I'm happy to do most of the work, if nothing else I'm mainly looking for guidance/advice.
I have encountered issues getting the CDK to run successfully for the minecraft-server-stack
CloudFormation stack.
After a bit of troubleshooting I was able to identify that my memory and virtual CPU configurations were not supported. I think it'd be great to add some documentation around this and provide some example configurations.
I'd be open to making this change!
As a minecraft player, I want to use this procedure with bedrock edition. Right now, bedrock communicates on a UDP port which makes the current netstat+grep+wc method of determining connected player count inadequate for bedrock users.
https://github.com/itzg/mc-monitor is a project that offers server querying for both bedrock and java edition servers, which will return server status as well as an online player count. While available as a standalone container, fargate is not set up in a way that we can fire off individual containers for the purpose of querying server status. Nor can we do Docker inside of Docker within this setup.
Potential options are:
go get
function within this container build, call it against the local server host, and parse the quantity of online players. Advantages is that it works out of the box for java and bedrock, disadvantage is that it will likely increase the container size to over 1gb.Switching to this will allow the easy addition of bedrock servers.
I didn't try, but if I want to change some settings, like Twilio keys or the vCPUs and RAM. Do I need to change the .env
files and run npm run build && npm run deploy
? I haven't tried that yet, but I am worrying about that it's going to create a new stack and a completely new server? Or maybe I was wrong?
Hi there, even after allocating 4 CPU units and 8GB of ram (which may not even be needed/helpful), I still seem to be having server issues. While my ping is the range of 100, it spikes up to 200 when breaking blocks, with TPS being between 12 and 13 on average. Any suggestions on how to improve performance, or have I misconfigured something?
Hi,
Fantastic project, the kids love it and its fun to play with it.
Do you have a hint re: CDK deploy, how to deploy multiple instances into the same account (aka bedrock + java or multiple worlds for each) - tried just to clone the repo into different directories, with separate configs/domains etc. but I think its still mixing/breaking each other. Would it be an option to include the domain name into the resource names?
Thanks,
Max
When my Fargate Service runs, both containers deploy fine (after some wrangling), but now I'm stuck on the following error when the minecraft-server contain starts to Run:
Failed to bulk-set properties in file: unable to access properties file server.properties: open server.properties: permission denied
Then the container immediately goes to STOPPED while the watchdog container keeps running for a while longer as it "waits"...
I've checked/doublechecked Security Groups/Subnets, etc for the EFS mounts, and am about to give up. I even had this working at one point before this error reared it's ugly head. Anything I can look for?
Thanks for the awesome project!
Hi there,
I have it working pretty well so far, with a mounted EFS, route56, and plugins manually installed.
I am now setting up the plugins' configurations from an EC2 with the EFS mounted to it. I need to test and restart continously the server to load the new configuration.
I've been doing so by stopping and start the service's task over and over again. I tried to use the ECS Exec to send the /restart command to the server's terminal but it wasn't recognized. (I may have configured it wrong).
However I realized I could use /restart from the in game chat. I did, and the server shutdown but never restarted and in the end I had to restart the task.
I've seen a significant increase of cost when I have to restrat multiple times the server, so I was wondering if there's any way to restart it without stopping the task.
Thanks.
Pepe
I want to highlight it as an easy way to get up and running quickly while leaving the full docs for anyone who desires to learn more about how all this stuff works in AWS. Will be working on it between work items this week.
According to the step-by-step:
Includes for Java edition: add three:
*.json (this will include banned-ips, banned-players, ops, usercache, whitelist)
but the problem is that, despite AWS describing this in their own documentation, it's not possible to define file patterns like this because:
Unless I'm doing something completely wrong here?
Just wondering what's the best way to avoid the IPv4 charge that will come into place in February for VPC gateways?
Will everything still work with an IPv6 gateway? Not sure how the watchdog container handles the address...
Maybe to make a end to end solution where you use amazon SNS instead of twilio. I would be willing to code this solution into this project. Maybe you could point me where the notification code is and we can make the option of using amazon sns instead if twilio?
Hi, considering Microsoft Azure even provides free container services for everyone, I'm thinking if it is also possible to put the server on Azure. (Also, Azure provides free credits for students, trial users, etc.)
Hello Sir!
Good job on this!
Simple question, is it an option to implement the GeyserMC into the supported stack?
Regards
you will need to add the following to the Environment variables in the .env
MINECRAFT_IMAGE_ENV_VARS_JSON = { "EULA": "TRUE", "MEMORY": "512M" }
as instructed here: https://github.com/itzg/docker-minecraft-server/blob/master/README.md#memory-limit
Now that AWS supports arm64 on ECS Fargate, it would be nice to push builds of the watchdog container to support that.
As of #6 the image needs no additional changes, and I have a test image up at stealthii/minecraft-ecsfargate-watchdog
.
Here's the line I used to build (you'll likely want to multi-arch the build):
docker buildx build --platform=linux/arm64 --tag stealthii/minecraft-ecsfargate-watchdog --push minecraft-ecsfargate-watchdog
I've been using arm64 successfully on the Lambda tasks, the watchdog container, and the 1.18.x Fabric server running under this project's configuration!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.