doctorray117 / minecraft-ondemand Goto Github PK

I'm trying to get this set up for various worlds/modpacks, and it's not clear to me how to use an existing world, or upload a modpack zip for minecraft-docker to use, or add datapacks to an existing world

I'm pretty inexperienced with AWS, and I went through the guide a few times, but the Lambda errors out when trying to start the service. I'm getting this error.

"errorMessage": "An error occurred (AccessDeniedException) when calling the DescribeServices operation: User: arn:aws:sts::XXXX:assumed-role/minecraft-launcher-role-mijbep5x/minecraft-launcher is not authorized to perform: ecs:DescribeServices on resource: arn:aws:ecs:us-west-1:XXXX:service/minecraft/minecraft-server because no identity-based policy allows the ecs:DescribeServices action",

I have the polices attached to the role, but I don't know what the assumed-role part of the arn means, or how to attach policies to it.

Description

I've tried to deploy it on the M1 Mac.
When I deployed it without customization code, I've got tasks failed to start issue on AWS ECS.
The reason was the failure of the watchdog container.
The error message was just exit code 1 on the container in the failed task.

How to fix

https://github.com/doctorray117/minecraft-ondemand/blob/main/cdk/lib/minecraft-stack.ts#L226-L232

-        image: isDockerInstalled()
-          ? ecs.ContainerImage.fromAsset(
-              path.resolve(__dirname, '../../minecraft-ecsfargate-watchdog/')
-            )
-          : ecs.ContainerImage.fromRegistry(
-              'doctorray/minecraft-ecsfargate-watchdog'
-            ),
+        image: ecs.ContainerImage.fromRegistry(
+              'doctorray/minecraft-ecsfargate-watchdog'
+            ),

If the script builds a watchdog image from the asset, you get an arm64 build image.
The image doesn't work because the current ECS Task definition use x86 CPU architecture. Otherwise need to config: AWS ECS Doc: Specifying the ARM architecture in your task definition
So, I changed the code to get the image from the registry always.

Related issue: #27

Just share my solution to resolve the same issue of others and to improve the code to support arm64 machine efficiently.

Ideally this will provide us with two methods for deployment:

1. AWS experienced users can deploy with the CDK CLI
2. Non-experienced users or those who just want something quick can do a one-click deploy with the slightly modified CloudFormation template we will generate.

This work is currently in progress, placing here to avoid duplicated efforts.

first off, well done, this is a great piece of work and clearly a lot of effort gone in.

I've made a few modifications as I wanted to run a pufferpanel server rather than just a single standalone minecraft server. Also puffer panel then gives easy access to the files through sftp to install plugins and tools to manage the server.

I noticed that the watchdog doesn't seem to work (well I can't seem to see it working) for bedrock connections. Not sure about the netstat commands etc in the script but from what I can see (including local testing) it never detects any udp connections.

I tried everything I could to get it to work, but couldn't. So in the end I gave up and just decided to write a small node script that use the minecraft protocol to get the status of the server directly. Seems to be a bit more robust as it gets the status and number of players online directly. I'd be happy to share it if you are open to taking PRs?

Something is causing the watchdog process to terminate early, setting the task count to zero (maybe a SIGTERM, or something else. This seems to consistently happen 15-20 seconds after the watchdog container listens for connections.

I've set up the guide on two AWS accounts from scratch and encounter the same issue. Not sure if it's me or the guide, but something's missed.

Hi, your project inspired me to work on a Terraform implementation instead of using the AWS CDK. I have worked with CloudFormation for a couple of years, I can hardly bring up the patience for its quirks and features anymore. Terraform isn't perfect either, but it gives a whole lot of options that CFN just doesn't bring to the table. I happily imported my EFS file system from the CDK stack into my terraform state.

Everything has been implemented except for the Twilio variables. I'm looking to add discord integration as well.

Appreciate the work you've put into this, it's a great setup.

Hello,

I am trying to set up a server but my task keeps stopping in ECS.

I am new to AWS and I have no idea why this happened. I ran the cdk code and I thought it was supposed to set everything up, but I am just stuck at this one thing. Does anyone know how I can fix this?

I've currently used your setup for a minecraft server which has been great, but I'm trying to wander off into the weeds by setting up a valheim server.

It uses 2456-2457/udp.

I was looking at using something like a python library to query the server direct, but then I realised your bedrock code is detecting connections on UDP.

Spent the evening trying to understand it, currently lost on the bedrockping string you build.

Basically just seeing if you'd be willing to shed some light on it before I spend some more time on it :)

Thanks again.

Hi
This is a fun project :)

I have 3 questions

1. Could the Watchdog not run in a Lambda? triggered every 10 minutes or so by EventBridge schedule that is created by the error handling lambda (and deleted again when the container is terminated)? This should further optimise cost and make it more serverless.

2. This is not a scaling solution, right? it is 1 container that needs to be big enough for the expected number of users.

3. if the container is launched after a hit on the domain name, at some point you will get a lot of false positives from bots and such, right? is there a (serverless) way to make sure it's a Minecraft client asking for the link before launching anything?

thanks!

Hello,
Can you advise on how the hosted zone could be reused for multiple instances (i.e. define in a outer stack and share to multiple sub-stacks) or how an existing HZ should be used in it's place (via lookup).

I would like to run multiple servers and would like to avoid paying HZ fees for every instance, when it may be possible to have them all under different sub-sub domains under a common HZ.

Thank you for any guidance you can give

Cloudformation Error: Resource handler returned message: "The runtime parameter of nodejs12.x is no longer supported for creating or updating AWS Lambda functions. We recommend you use the new runtime (nodejs18.x)

Hi,
I've deployed the CDK stack to my account this weekend but it was much more complicated than expected.
This sadly was caused by the recent AWS transition to the version 2 of CDK bootstrapping module.
To make it work, remember to export the CDK_NEW_BOOTSTRAP environment variable:

export CDK_NEW_BOOTSTRAP=1

If you agree I can create a PR, adding this in the documentation.
Thanks for the help and thank you again for this awesome piece of art.
Hosting a serverless Minecraft server is so cool :)

Just did a fresh setup via the CDK, and when I was walking through the manual steps under Option 2: DataSync and S3 section of the readme, there's a couple things missing that might've changed since the last time these steps were updated:

1. Under "Step 2: Create an EFS -> S3 DataSync Task", there's 2 additional fields that the form requires for the EFS location:
   1. Subnet - I think any of the subnets created during setup will work, I've tested with my minecraft-server-stack/Vpc/PublicSubnet1 and minecraft-server-stack/Vpc/IsolatedSubnet3 and they both work (as long as the next setting is correct)
   2. Security Group - this must be the one that starts with minecraft-server-stack-ServiceSecurityGroup, using the other SG created (minecraft-server-stack-FileSystemEfsSecurityGroup) or a default SG will not work OOTB
2. For the files included, you have to specify a / in front of the file names, which is weird but the form fails validation if any of the fields are missing it (personally, I just use the "Entire source location" option to avoid the hassle)

I'm running Spigotmc / Geysermc and the watch dog scripts cannot detect bedrock client connection. Thus my server is shutting down after 10 minutes even I'm playing and connecting. The easy way to mitigate the issue is assuming that Java server can host both of bedrock clients (UDP ports) and Java clients. Could we have such change?

PS. thanks for awesome documentation and the script. I'm AWS newbie and it only took 2 hours to setup the ECS cluster and lambda stuffs. Everything is working great except the bedrock client detection.

I was able to get the EFS stuff working with editing files but I'd like to restart the server to see my changes. How do I do that without waiting 20 minutes?

Hi,

I'm pretty new to aws, and I followed your wonderful instructions, but unfortunately I had some issues I couldn't figure out why (and they are not in the troubleshooting section). I hope this is the right place to seek for help.

Basically the lambda works, the watchdog can trigger the lambda, and both tasks will try to run, but they will die immediately with the following error (they couldn't even print out any log, except just the error):

ResourceInitializationError: failed to validate logger args: create stream has been retried 1 times: failed to create Cloudwatch log stream: AccessDeniedException: User: arn:aws:sts::zzzzzzzzzzzz:assumed-role/ecs.task.minecraft-server/123456a999ee4b919...

Those two tasks are both running docker images, thus I have no idea why they have to create Cloudwatch logs. I can verify ecs.task.minecraft-server has all four policies added,

Any suggestions will be appreciated. Thank you.

EDIT: I forgot to mention, the desired task count is always 1, even after both tasks failed. Had to manually update it to 0.

Simply amazing...
You, Sir, are a gentleman and a scholar.
Bravo!

Hello,

I was wondering if there was a way to have my minecraft server hosted on my root domain, instead of subdomain.domainname. For example, instead of connecting to the server with acme.example.com, I just want to be able to type in example.com to connect to the server. I tried removing some code related to the subdomain in the cdk code, but it ended up breaking and not being able to deploy anymore. Does anyone know the correct way to do this? Thanks!

Your tutorial guides are very well written but do not mention how to use this with Pterodactyl Panel, which makes managing and operating a minecraft server much easier. I would love to see a guide for Pterodactyl On Demand!

Hello, and thank you for putting together this tutorial! I don't have much AWS experience, but I was able to get about 99% of the way to an ondemand modded Minecraft server with your help. The server spins itself up and shuts itself down according to activity as advertised, and I've been able to log into them.

Unfortunately, I've encountered an issue that I'm not sure how to resolve. It seems like the cluster containers aren't using the EFS as a data volume somehow. Nothing carries over between sessions, and when I manually mount into the EFS between active sessions, there's nothing there besides the ops.json I manually created, inserted into the S3 and then transferred into the EFS. Judging by that, I know that the DataSync tasks are working and the FS and FSAP IDs are correct.

FA and FSAP IDs

FSAP Details

Cluster Task Definition Storage Configuration

efs.rw.minecraft-data IAM policy

Roles with efs.rw.minecraft-data attached

Logs from Cluster tasks showing files and directories being referenced

EC2 Console After Mounting EFS and using ls to check contents

I've been tweaking a number of parameters including minecraft version, java flags, data packs, and server specs. All seem to perform worse than what I'm used to for similar server specs in terms of chunk loading (both new and old chunks). I'm wondering if anyone else is experiencing this issue, and if it could perhaps be EFS at fault.

@ryansonshine While testing CDK set up and tear down, the npm run destroy failed unless I manually deleted the A record in the subzone. Is that behavior that you would expect? I wonder if it's because it wasn't set at the default 192.168.1.1 anymore?

Just wondering if anyone has had this happen. After a while of having the system setup, the service started coming up and down like a yo yo when we weren't using it. I checked the logs and the domain name is getting hit all the time.

I assumed it's just hackers port scanning and trawling to find servers etc, but it's kinda made the auto start feature unusable. But just wondering if I'm missing something?

Any other ideas on easy ways to setup some sort of autostart function?

Cheers

@all-contributors add @doctorray117 code

Insufficient permissions, disregard

Can I create multiple instances using multiple tasks?
Is load balancing possible?

And how do you check the health of each container?

This would make the 400ln install process as simple as calling terraform apply

First off, this is very cool and I'm currently using it without issue! My only concern is scaling with number of players or general preference, since afaik Minecraft performance is almost entirely based on the single core performance. I believe this can be achieved using an EC2 Launch Type instead of Fargate, is that correct? Could most of the configuration remain the same?

I'm happy to do most of the work, if nothing else I'm mainly looking for guidance/advice.

I have encountered issues getting the CDK to run successfully for the minecraft-server-stack CloudFormation stack.

After a bit of troubleshooting I was able to identify that my memory and virtual CPU configurations were not supported. I think it'd be great to add some documentation around this and provide some example configurations.

I'd be open to making this change!

As a minecraft player, I want to use this procedure with bedrock edition. Right now, bedrock communicates on a UDP port which makes the current netstat+grep+wc method of determining connected player count inadequate for bedrock users.

https://github.com/itzg/mc-monitor is a project that offers server querying for both bedrock and java edition servers, which will return server status as well as an online player count. While available as a standalone container, fargate is not set up in a way that we can fire off individual containers for the purpose of querying server status. Nor can we do Docker inside of Docker within this setup.

Potential options are:

1. include the mc-monitor code as a go get function within this container build, call it against the local server host, and parse the quantity of online players. Advantages is that it works out of the box for java and bedrock, disadvantage is that it will likely increase the container size to over 1gb.
2. Write a new curl and/or socat request to query the server and just pull the online player count.
3. Grep the logfile for connect/disconnect messages to determine player count.

Switching to this will allow the easy addition of bedrock servers.

I didn't try, but if I want to change some settings, like Twilio keys or the vCPUs and RAM. Do I need to change the .env files and run npm run build && npm run deploy? I haven't tried that yet, but I am worrying about that it's going to create a new stack and a completely new server? Or maybe I was wrong?

Hi there, even after allocating 4 CPU units and 8GB of ram (which may not even be needed/helpful), I still seem to be having server issues. While my ping is the range of 100, it spikes up to 200 when breaking blocks, with TPS being between 12 and 13 on average. Any suggestions on how to improve performance, or have I misconfigured something?

Hi,

Fantastic project, the kids love it and its fun to play with it.

Do you have a hint re: CDK deploy, how to deploy multiple instances into the same account (aka bedrock + java or multiple worlds for each) - tried just to clone the repo into different directories, with separate configs/domains etc. but I think its still mixing/breaking each other. Would it be an option to include the domain name into the resource names?

Thanks,

Max

When my Fargate Service runs, both containers deploy fine (after some wrangling), but now I'm stuck on the following error when the minecraft-server contain starts to Run:
Failed to bulk-set properties in file: unable to access properties file server.properties: open server.properties: permission denied

Then the container immediately goes to STOPPED while the watchdog container keeps running for a while longer as it "waits"...

I've checked/doublechecked Security Groups/Subnets, etc for the EFS mounts, and am about to give up. I even had this working at one point before this error reared it's ugly head. Anything I can look for?

Thanks for the awesome project!

Hi there,

I have it working pretty well so far, with a mounted EFS, route56, and plugins manually installed.
I am now setting up the plugins' configurations from an EC2 with the EFS mounted to it. I need to test and restart continously the server to load the new configuration.

I've been doing so by stopping and start the service's task over and over again. I tried to use the ECS Exec to send the /restart command to the server's terminal but it wasn't recognized. (I may have configured it wrong).
However I realized I could use /restart from the in game chat. I did, and the server shutdown but never restarted and in the end I had to restart the task.

I've seen a significant increase of cost when I have to restrat multiple times the server, so I was wondering if there's any way to restart it without stopping the task.

Thanks.

Pepe

I want to highlight it as an easy way to get up and running quickly while leaving the full docs for anyone who desires to learn more about how all this stuff works in AWS. Will be working on it between work items this week.

According to the step-by-step:

Includes for Java edition: add three:
*.json (this will include banned-ips, banned-players, ops, usercache, whitelist)

but the problem is that, despite AWS describing this in their own documentation, it's not possible to define file patterns like this because:

Unless I'm doing something completely wrong here?

Just wondering what's the best way to avoid the IPv4 charge that will come into place in February for VPC gateways?

Will everything still work with an IPv6 gateway? Not sure how the watchdog container handles the address...

Maybe to make a end to end solution where you use amazon SNS instead of twilio. I would be willing to code this solution into this project. Maybe you could point me where the notification code is and we can make the option of using amazon sns instead if twilio?

Hi, considering Microsoft Azure even provides free container services for everyone, I'm thinking if it is also possible to put the server on Azure. (Also, Azure provides free credits for students, trial users, etc.)

Hello Sir!

Good job on this!

Simple question, is it an option to implement the GeyserMC into the supported stack?

Regards

you will need to add the following to the Environment variables in the .env

MINECRAFT_IMAGE_ENV_VARS_JSON = { "EULA": "TRUE", "MEMORY": "512M" }

as instructed here: https://github.com/itzg/docker-minecraft-server/blob/master/README.md#memory-limit

Now that AWS supports arm64 on ECS Fargate, it would be nice to push builds of the watchdog container to support that.

As of #6 the image needs no additional changes, and I have a test image up at stealthii/minecraft-ecsfargate-watchdog.

Here's the line I used to build (you'll likely want to multi-arch the build):

docker buildx build --platform=linux/arm64 --tag stealthii/minecraft-ecsfargate-watchdog --push minecraft-ecsfargate-watchdog 

I've been using arm64 successfully on the Lambda tasks, the watchdog container, and the 1.18.x Fabric server running under this project's configuration!