debian12 cpu very hight 100% and Always restart about gitea HOT 27 CLOSED

on debian12 alway Starting gitea.service and Stopped gitea.service。 and reboot server cpu very hight 100%

There could be 2 problems:

1. do not use "notify" in the systemd service, just use simple
2. cpu 100% seems to be caused by some initialization tasks, apply step 1 and make Gitea run enough time

from gitea.

on debian12 alway Starting gitea.service and Stopped gitea.service。 and reboot server cpu very hight 100%

There could be 2 problems:

1. do not use "notify" in the systemd service, just use simple
2. cpu 100% seems to be caused by some initialization tasks, apply step 1 and make Gitea run enough time

I refer to this to configure, I all the installation of the official way to configure. https://github.com/go-gitea/gitea/blob/release/v1.21/contrib/systemd/gitea.service

[Unit]
Description=Gitea (Git with a cup of tea)
After=network.target
#如果是mysql这打开
#Wants=mysql.service
#After=mysql.service
[Service]
RestartSec=2s
Type=simple
User=git
Group=git
WorkingDirectory=/var/lib/gitea/
ExecStart=/usr/local/bin/gitea web --config /etc/gitea/app.ini
Restart=always
Environment=USER=git HOME=/home/git GITEA_WORK_DIR=/var/lib/gitea
# Git LFS support
Environment=PATH=/path/to/git/bin:/bin:/sbin:/usr/bin:/usr/sbin
[Install]
WantedBy=multi-user.target

from gitea.

Could you try this?

sudo -u git USER=git HOME=/home/git GITEA_WORK_DIR=/var/lib/gitea /usr/local/bin/gitea web --config /etc/gitea/app.ini

It should do the same work as systemctl start, so maybe there could be more logs?

from gitea.

Could you try this?

sudo -u git USER=git HOME=/home/git GITEA_WORK_DIR=/var/lib/gitea /usr/local/bin/gitea web --config /etc/gitea/app.ini

It should do the same work as systemctl start, so maybe there could be more logs?

This is OK, it does not restart all the time,

but it will not work if you use systemctl start, including versions 1.21.10 and 1.20.6

from gitea.

I still think there is something wrong in your systemd service file. According to your message: Apr 28 21:13:54 debian systemd[1]: gitea.service: start operation timed out. Terminating., it shouldn't happen whey Type=simple.

Have you reloaded your systemd service?

from gitea.

I still think there is something wrong in your systemd service file. According to your message: Apr 28 21:13:54 debian systemd[1]: gitea.service: start operation timed out. Terminating., it shouldn't happen whey Type=simple.

Have you reloaded your systemd service?

"Have you reloaded your systemd service?" yes ,also reboot server

Although there are errors, it is OK to access http://127.0.0.1:3000, However, sometimes after reboot server, the cpu takes up 100%

I configure the systemd service by referring to the official documentation.

I had no problem with rocky9.3, gitea 1.20.06. However, using gitea 1.21.10 also shows the above error

from gitea.

I had no problem with rocky9.3, gitea 1.20.06. However, using gitea 1.21.10 also shows the above error

It shouldn't be related to linux distro.

Although there are errors, it is OK to access http://127.0.0.1:3000, However, sometimes after reboot server, the cpu takes up 100%

I have no more idea at the moment. Maybe you could try to figure which process takes 100% CPU and what it is doing (there are many linux utils for such diagnosis)

And maybe you could just use Docker, it is much easier to use, and no need to worry about conflicting with host OS.

from gitea.

I configure the systemd service by referring to the official documentation.

Some old documents used Type=notify (the document was just fixed by #28722), which would exactly cause the problem in your screenshot.

from gitea.

I configure the systemd service by referring to the official documentation.

Some old documents used Type=notify (the document was just fixed by #28722), which would exactly cause the problem in your screenshot.

This is my gitea.service configuration on debian 12

gitea1.21.10
sudo vim /etc/systemd/system/gitea.service

[Unit]
Description=Gitea (Git with a cup of tea)
After=network.target
#如果是mysql这打开
#Wants=mysql.service
#After=mysql.service
[Service]
RestartSec=2s
Type=simple
User=git
Group=git
WorkingDirectory=/var/lib/gitea/
ExecStart=/usr/local/bin/gitea web --config /etc/gitea/app.ini
Restart=always
Environment=USER=git HOME=/home/git GITEA_WORK_DIR=/var/lib/gitea
# Git LFS support
Environment=PATH=/path/to/git/bin:/bin:/sbin:/usr/bin:/usr/sbin
[Install]
WantedBy=multi-user.target

gitea 1.20.06

sudo vim /etc/systemd/system/gitea.service

[Unit]
Description=Gitea (Git with a cup of tea)
After=syslog.target
After=network.target

#如果是mysql就打开
#Wants=mysql.service
#After=mysql.service

[Service]
# Uncomment the next line if you have repos with lots of files and get a HTTP 500 error because of that
# LimitNOFILE=524288:524288
RestartSec=2s
Type=notify
User=git
Group=git
WorkingDirectory=/var/lib/gitea/

ExecStart=/usr/local/bin/gitea web --config /etc/gitea/app.ini
Restart=always
Environment=USER=git HOME=/home/git GITEA_WORK_DIR=/var/lib/gitea
WatchdogSec=30s

# Git LFS support
Environment=PATH=/path/to/git/bin:/bin:/sbin:/usr/bin:/usr/sbin

[Install]
WantedBy=multi-user.target

apply

sudo systemctl daemon-reload
sudo systemctl enable gitea
sudo systemctl start gitea

from gitea.

I can see "Type=notify" in your "gitea 1.20.06" systemd service. It is not right for an uninstalled instance (restarting forever).

I guess you mixed something. Please make sure every step is clear and you could clearly know which file/config is used.

from gitea.

I can see "Type=notify" in your "gitea 1.20.06" systemd service. It is not right for an uninstalled instance (restarting forever).

I guess you mixed something. Please make sure every step is clear and you could clearly know which file/config is used.

我是按照官方提供的配置文件设置的，不同的版本使用不同的配置，还是说1.20.06版本也可以使用最新版本的配置，下面是官方提供的link
https://github.com/go-gitea/gitea/blob/release/v1.21/contrib/systemd/gitea.service

https://github.com/go-gitea/gitea/blob/release/v1.20/contrib/systemd/gitea.service

此外我是用esxi 安装debian 12 ，做了快照，每次切换gitea版本时候，都是通过快照还原成最新安装的系统，以保证系统是干净的，所以不存在混用的情况，我再安装1.21.10版本复现一下这个cpu占用100%的情况

from gitea.

1. I have told you "Type=notify" is not right. 1.20 is outdated so its documents are never updated.
2. I have done these steps:

➜  cat /etc/debian_version
12.5

# Use your systemd service:
➜ vim /etc/systemd/system/gitea.service
[Unit]
Description=Gitea (Git with a cup of tea)
After=network.target
#如果是mysql这打开
#Wants=mysql.service
#After=mysql.service
[Service]
RestartSec=2s
Type=simple
User=git
Group=git
WorkingDirectory=/var/lib/gitea/
ExecStart=/usr/local/bin/gitea web --config /etc/gitea/app.ini
Restart=always
Environment=USER=git HOME=/home/git GITEA_WORK_DIR=/var/lib/gitea
# Git LFS support
Environment=PATH=/path/to/git/bin:/bin:/sbin:/usr/bin:/usr/sbin
[Install]
WantedBy=multi-user.target

# Install Gitea
➜  ~ cd /usr/local/bin
➜  bin wget https://dl.gitea.com/gitea/1.21.10/gitea-1.21.10-linux-amd64
➜  bin mv gitea-1.21.10-linux-amd64 gitea
➜  bin chmod a+x gitea

# run adduser & mkdir & chdown & chmode provided by the document
...

# start gitea
➜  systemctl restart gitea
➜  date
Mon Apr 29 04:36:03 AM UTC 2024
➜  systemctl status gitea
● gitea.service - Gitea (Git with a cup of tea)
     Loaded: loaded (/etc/systemd/system/gitea.service; disabled; preset: enabled)
     Active: active (running) since Mon 2024-04-29 04:32:18 UTC; 3min 48s ago
   Main PID: 182187 (gitea)
      Tasks: 6 (limit: 482)
     Memory: 110.2M
        CPU: 747ms
     CGroup: /system.slice/gitea.service
             └─182187 /usr/local/bin/gitea web --config /etc/gitea/app.ini
...
Apr 29 04:32:18 vultr.guest gitea[182187]: 2024/04/29 04:32:18 cmd/web.go:117:showWebStartupMessage() [I] Prepare to run install page
Apr 29 04:32:19 vultr.guest gitea[182187]: 2024/04/29 04:32:19 cmd/web.go:304:listen() [I] Listen: http://0.0.0.0:3000
Apr 29 04:32:19 vultr.guest gitea[182187]: 2024/04/29 04:32:19 cmd/web.go:308:listen() [I] AppURL(ROOT_URL): http://localhost:3000/
Apr 29 04:32:19 vultr.guest gitea[182187]: 2024/04/29 04:32:19 ...s/graceful/server.go:70:NewServer() [I] Starting new Web server: tcp:0.0.0.0:3000 on PID: 182187

Now, Gitea runs stablely for minutes, and I don't see 100% CPU either.

If you couldn't provide reproducible steps, I couldn't have more ideas about this problem. And I do not think there is any problem TBH.

from gitea.

1. I have told you "Type=notify" is not right. 1.20 is outdated so its documents are never updated.
2. I have done these steps:

➜  cat /etc/debian_version
12.5

# Use your systemd service:
➜ vim /etc/systemd/system/gitea.service
[Unit]
Description=Gitea (Git with a cup of tea)
After=network.target
#如果是mysql这打开
#Wants=mysql.service
#After=mysql.service
[Service]
RestartSec=2s
Type=simple
User=git
Group=git
WorkingDirectory=/var/lib/gitea/
ExecStart=/usr/local/bin/gitea web --config /etc/gitea/app.ini
Restart=always
Environment=USER=git HOME=/home/git GITEA_WORK_DIR=/var/lib/gitea
# Git LFS support
Environment=PATH=/path/to/git/bin:/bin:/sbin:/usr/bin:/usr/sbin
[Install]
WantedBy=multi-user.target

# Install Gitea
➜  ~ cd /usr/local/bin
➜  bin wget https://dl.gitea.com/gitea/1.21.10/gitea-1.21.10-linux-amd64
➜  bin mv gitea-1.21.10-linux-amd64 gitea
➜  bin chmod a+x gitea

# run adduser & mkdir & chdown & chmode provided by the document
...

# start gitea
➜  systemctl restart gitea
➜  date
Mon Apr 29 04:36:03 AM UTC 2024
➜  systemctl status gitea
● gitea.service - Gitea (Git with a cup of tea)
     Loaded: loaded (/etc/systemd/system/gitea.service; disabled; preset: enabled)
     Active: active (running) since Mon 2024-04-29 04:32:18 UTC; 3min 48s ago
   Main PID: 182187 (gitea)
      Tasks: 6 (limit: 482)
     Memory: 110.2M
        CPU: 747ms
     CGroup: /system.slice/gitea.service
             └─182187 /usr/local/bin/gitea web --config /etc/gitea/app.ini
...
Apr 29 04:32:18 vultr.guest gitea[182187]: 2024/04/29 04:32:18 cmd/web.go:117:showWebStartupMessage() [I] Prepare to run install page
Apr 29 04:32:19 vultr.guest gitea[182187]: 2024/04/29 04:32:19 cmd/web.go:304:listen() [I] Listen: http://0.0.0.0:3000
Apr 29 04:32:19 vultr.guest gitea[182187]: 2024/04/29 04:32:19 cmd/web.go:308:listen() [I] AppURL(ROOT_URL): http://localhost:3000/
Apr 29 04:32:19 vultr.guest gitea[182187]: 2024/04/29 04:32:19 ...s/graceful/server.go:70:NewServer() [I] Starting new Web server: tcp:0.0.0.0:3000 on PID: 182187

Now, Gitea runs stablely for minutes, and I don't see 100% CPU either.

If you couldn't provide reproducible steps, I couldn't have more ideas about this problem. And I do not think there is any problem TBH.

my gitea install step

login debian user

sudo -i

wget -O gitea https://dl.gitea.com/gitea/1.21.10/gitea-1.21.10-linux-amd64
chmod +x gitea


apt install git -y
apt install git-lfs -y

adduser \
   --system \
   --shell /bin/bash \
   --gecos 'Git Version Control' \
   --group \
   --disabled-password \
   --home /home/git \
   git


mkdir -p /var/lib/gitea/{custom,data,log}
chown -R git:git /var/lib/gitea/
chmod -R 750 /var/lib/gitea/
mkdir /etc/gitea
chown root:git /etc/gitea
chmod 770 /etc/gitea

cp gitea /usr/local/bin/gitea

sudo vim /etc/systemd/system/gitea.service

[Unit]
Description=Gitea (Git with a cup of tea)
After=network.target
#如果是mysql这打开
#Wants=mysql.service
#After=mysql.service
[Service]
RestartSec=2s
Type=simple
User=git
Group=git
WorkingDirectory=/var/lib/gitea/
ExecStart=/usr/local/bin/gitea web --config /etc/gitea/app.ini
Restart=always
Environment=USER=git HOME=/home/git GITEA_WORK_DIR=/var/lib/gitea
# Git LFS support
Environment=PATH=/path/to/git/bin:/bin:/sbin:/usr/bin:/usr/sbin
[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable gitea
sudo systemctl start gitea

login http://127.0.0.1:3000 setup

login gitea

sudo reboot

After the restart, wait (you can try multiple restarts), and then execute the top command
top

systemctl status gitea.service

top

top -Hp 631

update
perf top -g -p 631

Press Enter Key

Press k

At this point, if I perform a restart

sudo systemctl stop gitea
sudo systemctl start gitea

The cpu usage will be down, as long as reboot server, the cpu is very high

from gitea.

By your example, it is still unclear to me:

1. Does it still "always restart"?
2. If it still "always restart", is the high CPU usage related to the restarting?
3. If no, could you provide provide the "diagnosis report" from admin panel -> monitor/stacktrace ?

from gitea.

By your example, it is still unclear to me:

1. Does it still "always restart"?
2. If it still "always restart", is the high CPU usage related to the restarting?
3. If no, could you provide provide the "diagnosis report" from admin panel -> monitor/stacktrace ?

1、yes，it still "always restart"
2、I'm not sure it has anything to do with gitea restart. Although it "always restart", the cpu usage is not high，The cpu usage is high when the server restarts
3、
gitea-diagnosis-20240429-021158.zip

from gitea.

OK, I think we should focus on "always restart" first.

1. If you disable the systemd service, reboot system, start the service manually after some time (eg: 5sec, 30sec, 5min, etc), does the "always restart" problem still occur?
   • If yes, what's the difference between it and your "manually run it in the terminal"? It's said that if "manually run it in the terminal" then everything is fine.
   • If no, what happens between the duration?

from gitea.

OK, I think we should focus on "always restart" first.

1. If you disable the systemd service, reboot system, start the service manually after some time (eg: 5sec, 30sec, 5min, etc), does the "always restart" problem still occur?

   • If yes, what's the difference between it and your "manually run it in the terminal"? It's said that if "manually run it in the terminal" then everything is fine.
   • If no, what happens between the duration?

start the service manually after some time, is not "always restart". cpu is also run ok!

sudo su git
GITEA_WORK_DIR=/var/lib/gitea/ /usr/local/bin/gitea web -c /etc/gitea/app.ini

from gitea.

start the service manually after some time, is not "always restart". cpu is also run ok!

I mean, if you disable the systemd service, and then "systemctl start gitea" after reboot, does the "always restart" problem still occur?

from gitea.

start the service manually after some time, is not "always restart". cpu is also run ok!

I mean, if you disable the systemd service, and then "systemctl start gitea" after reboot, does the "always restart" problem still occur?

yes，it also "always restart"

from gitea.

start the service manually after some time, is not "always restart". cpu is also run ok!

I mean, if you disable the systemd service, and then "systemctl start gitea" after reboot, does the "always restart" problem still occur?

yes，it also "always restart"

Then what's the difference between it and starting from terminal? If UID is right, environment is right, Type=simple is right, then these 2 methods should be almost exactly the same.

from gitea.

Maybe you could also try to use systemctl cat gitea to see the real systemd service file you are using.

from gitea.

Maybe you could also try to use systemctl cat gitea to see the real systemd service file you are using.

from gitea.

Maybe you could also try to use systemctl cat gitea to see the real systemd service file you are using.

TBH, no idea from my side, I could only guess at the moment:

1. Is it related to file number limit? Try to set to a large value. https://github.com/go-gitea/gitea/blob/main/contrib/systemd/gitea.service#L52C3-L52C14
2. Could you enable the logs and see whether there is anything wrong? https://docs.gitea.com/help/support#more-config-options-for-logs
3. Could you wrap the gitea command into a bash script, just like you start it manually, and redirect the output to a file, the see whether there is anything wrong in the output file /tmp/gitea-start.log:

#!/usr/bin/env bash
GITEA_WORK_DIR=/var/lib/gitea/ /usr/local/bin/gitea web -c /etc/gitea/app.ini > /tmp/gitea-start.log 2>&1

Then chmod a+x and set ExecStart to this script, then systemctl reload&restart

from gitea.

也许您也可以尝试使用来查看您正在使用的真实 systemd 服务文件。systemctl cat gitea

TBH，我这边不知道，我此刻只能猜测：

1. 它与文件数量限制有关吗？尝试设置为较大的值。https://github.com/go-gitea/gitea/blob/main/contrib/systemd/gitea.service#L52C3-L52C14
2. 你能启用日志看看是否有任何问题吗？https://docs.gitea.com/help/support#more-config-options-for-logs
3. 您能否将 gitea 命令包装成 bash 脚本，就像手动启动它一样，并将输出重定向到文件，查看输出文件中是否有任何错误：/tmp/gitea-start.log

#!/usr/bin/env bash
GITEA_WORK_DIR=/var/lib/gitea/ /usr/local/bin/gitea web -c /etc/gitea/app.ini > /tmp/gitea-start.log 2>&1

然后设置为此脚本，然后 systemctl 重新加载并重新启动chmod a+x``ExecStart

Maybe you could also try to use systemctl cat gitea to see the real systemd service file you are using.

TBH, no idea from my side, I could only guess at the moment:

1. Is it related to file number limit? Try to set to a large value. https://github.com/go-gitea/gitea/blob/main/contrib/systemd/gitea.service#L52C3-L52C14
2. Could you enable the logs and see whether there is anything wrong? https://docs.gitea.com/help/support#more-config-options-for-logs
3. Could you wrap the gitea command into a bash script, just like you start it manually, and redirect the output to a file, the see whether there is anything wrong in the output file /tmp/gitea-start.log:

#!/usr/bin/env bash
GITEA_WORK_DIR=/var/lib/gitea/ /usr/local/bin/gitea web -c /etc/gitea/app.ini > /tmp/gitea-start.log 2>&1

Then chmod a+x and set ExecStart to this script, then systemctl reload&restart

I suggest you review the problem on debian12 instead of having me try it over and over again here.

download debian-12.5.0-amd64-netinst.iso

from gitea.

I suggest you review the problem on debian12 instead of having me try it over and over again here.

download debian-12.5.0-amd64-netinst.iso

I have showed my Debian 12 test in #30754 (comment) , I can not reproduce. And 1.20/1.21 both have been released for long time, there are many debian users, you are the first to report such problem, so I have no idea.

from gitea.

I suggest you review the problem on debian12 instead of having me try it over and over again here.
download debian-12.5.0-amd64-netinst.iso

I have showed my Debian 12 test in #30754 (comment) , I can not reproduce. And 1.20/1.21 both have been released for long time, there are many debian users, you are the first to report such problem, so I have no idea.

我已经切换成gitea 1.20.06版本，在rocky9上安装了。我不想再折腾了，先这样吧？后续有时间我在重新安装debian系统试试。

from gitea.

Thank you for your report.

from gitea.