stjudewashere / seonaut Goto Github PK

Describe the bug

This bug report is more of a "Heads up" than anything else.

Getting seonaut to run on podman took some finagling. All that was needed was adding docker.io/ to the image designators, and adding the ports for mysql to the compose file.

The error is the result of the Mysql container requiring AVX support, which the system did not have.

After reviewing the documentation for go-sql-driver/mysql I exchanged mysql with mariadb, which does not require AVX. The problem is that although mariadb is up, seonaut still does not recognize that it is up. Which is what I am working on at this moment.

Designating IP addresses, hostnames, and adding extra_hosts entries to the docker-compose.yml file allowed docker-compose-wait to successfully recognize when the database client has become available. This has allowed seonaut to run and be accessible on localhost, but for some reason, podman is not allowing external access right now. Which is something to work out at a later time.

Use:
-windows 10
-Docker version 26.1.4, build 5650f9b
When run
docker-compose up -d --build get error:
=> ERROR [app builder 5/5] RUN CGO_ENABLED=0 GOOS=linux go build -o seonaut cmd/server/main.go 0.6s

[app builder 5/5] RUN CGO_ENABLED=0 GOOS=linux go build -o seonaut cmd/server/main.go:
0.324 package cmd/server/main.go is not in std (/usr/local/go/src/cmd/server/main.go)

failed to solve: process "/bin/sh -c CGO_ENABLED=0 GOOS=linux go build -o seonaut cmd/server/main.go" did not complete successfully: exit code: 1

I have these alerts: 2 URLs Duplicate description and 2 URLs duplicate title, but they are about the same url.

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Is your feature request related to a problem? Please describe.
Once the crawling job is complete, it should send notification to email or any other platform with summary information and required reports.

Hi,

Can we remove a project (or free a slot for a new website) on seonaut.org ? If not, it seems to be an priority feature for usability :) To avoid abuse, you could add a limit of maximum project change per month (that could be a metric).

Thank you :)

Describe the bug
When basic auth is active, seonaut fails to find the sitemap and robots.txt.

To Reproduce
Steps to reproduce the behavior:

1. generate a simple file structure to serve: index.html, robots.txt, sitemap.xml in a directory
2. create a vhost to serve the test directory. Do not use basic auth
3. run seonaut against the test vhost
4. check that robots.txt and sitemap.xml were found
5. edit the vhost and put everything under / behind basic auth
6. run seonaut again
7. check that robots.txt and sitemap.xml were not found

Expected behavior
Both files are found regardless of basic auth

Sample vhost directory content

index.html

hi

sitemap.xml

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">

  <url>
    <loc>http://example.com/</loc>
    <lastmod>2023-10-04</lastmod>
    <changefreq>monthly</changefreq>
  </url>

  <url>
    <loc>http://example.com/nope</loc>
    <lastmod>2023-10-04</lastmod>
    <changefreq>monthly</changefreq>
  </url>

</urlset>

robots.txt

User-agent: *
Allow: /

Sitemap: http://example.com/sitemap.xml

Is your feature request related to a problem? Please describe.
The Docker config publishes the database port to the host in the standard mysqld port. This can (as in my case) conflict with an already running mysqld and prevent the container from starting. I commented out the published ports in the docker config, which seems to have no effect on the running seonaut instance later.

Describe the solution you'd like
If there's no need to expose the DB to the host: remove the port publishing for the DB.
If the DB needs to be exposed to the host: use a port other than 3306 on the host.

Describe alternatives you've considered

Additional context

Describe the bug
Cloning the repo (checked out at 1ed77d7) and trying to run a docker build fails with the following npm error

 => ERROR [front 4/4] RUN npm install esbuild && ./node_modules/esbuild/bin/esbuild ./app/web/css/style.css  --bundle  --minify  --outdir=./app/web/static  --public  2.7s
------
 > [front 4/4] RUN npm install esbuild && ./node_modules/esbuild/bin/esbuild ./app/web/css/style.css 	--bundle 	--minify 	--outdir=./app/web/static 	--public-path=/resources 	--loader:.woff=file 	--loader:.woff2=file:
#0 2.604 npm notice
#0 2.604 npm notice New minor version of npm available! 9.6.7 -> 9.7.1
#0 2.604 npm notice Changelog: <https://github.com/npm/cli/releases/tag/v9.7.1>
#0 2.604 npm notice Run `npm install -g [email protected]` to update!
#0 2.604 npm notice
#0 2.605 npm ERR! code 1
#0 2.605 npm ERR! path /home/node/node_modules/esbuild
#0 2.605 npm ERR! command failed
#0 2.605 npm ERR! command sh -c node install.js
#0 2.605 npm ERR! node:internal/errors:496
#0 2.605 npm ERR!     ErrorCaptureStackTrace(err);
#0 2.605 npm ERR!     ^
#0 2.605 npm ERR!
#0 2.606 npm ERR! <ref *1> Error: spawnSync /home/node/node_modules/esbuild/bin/esbuild ETXTBSY
#0 2.606 npm ERR!     at Object.spawnSync (node:internal/child_process:1117:20)
#0 2.606 npm ERR!     at spawnSync (node:child_process:871:24)
#0 2.606 npm ERR!     at Object.execFileSync (node:child_process:914:15)
#0 2.606 npm ERR!     at validateBinaryVersion (/home/node/node_modules/esbuild/install.js:98:28)
#0 2.606 npm ERR!     at /home/node/node_modules/esbuild/install.js:285:5 {
#0 2.606 npm ERR!   errno: -26,
#0 2.606 npm ERR!   code: 'ETXTBSY',
#0 2.606 npm ERR!   syscall: 'spawnSync /home/node/node_modules/esbuild/bin/esbuild',
#0 2.606 npm ERR!   path: '/home/node/node_modules/esbuild/bin/esbuild',
#0 2.606 npm ERR!   spawnargs: [ '--version' ],
#0 2.606 npm ERR!   error: [Circular *1],
#0 2.606 npm ERR!   status: null,
#0 2.606 npm ERR!   signal: null,
#0 2.606 npm ERR!   output: null,
#0 2.606 npm ERR!   pid: 0,
#0 2.606 npm ERR!   stdout: null,
#0 2.606 npm ERR!   stderr: null
#0 2.606 npm ERR! }
#0 2.606 npm ERR!
#0 2.606 npm ERR! Node.js v20.3.0
#0 2.607
#0 2.609 npm ERR! A complete log of this run can be found in: /root/.npm/_logs/2023-06-20T17_22_19_474Z-debug-0.log
------
failed to solve: executor failed running [/bin/sh -c npm install esbuild && ./node_modules/esbuild/bin/esbuild ./app/web/css/style.css 	--bundle 	--minify 	--outdir=./app/web/static 	--public-path=/resources 	--loader:.woff=file 	--loader:.woff2=file]: exit code: 1

At 4878c78 the issue is not present.

To Reproduce
Steps to reproduce the behavior:

1. checkout at 1ed77d7
2. run docker-compose up -d --build

Expected behavior
Build finishes successfully.

PS: thanks a lot for this projects. I just started testing it and it looks terrific!

Describe the bug
I've started a crawl, but it's stuck on "Connecting to the server, please wait..."
I mistakenly selected HTTP auth, but there isn't any on the site.

To Reproduce
Steps to reproduce the behavior:

1. Create a new project (that doesn't need basic auth)
2. Select HTTP Basic Auth
3. Put in some details
4. Run, and see the error

Expected behavior
Either to timeout after a while, or the option to manually cancel a crawl.

I installed it on my web server that is available publicly, but I'm worrying about security, first it is http then everybody can sign up and use it !
Can I protect the access to it without having to install a proxy ? Is there php files I can change somewhere to prevent signups ?

Describe the bug
Got a "Mismatching language" alert for pages that have the following meta tags :
URL : https://example.com/fr/contact/
Canonical : https://example.com/fr/contact/
Language : fr
Hreflang :
fr https://example.com/fr/contact/
en https://example.com/en/contact/
x-default https://example.com/fr/contact/

To Reproduce
Steps to reproduce the behavior:

1. Set up meta tags for a multilingual site as in the exemple
2. Crawl site on SEOnaut
3. Go to site issues
4. See error "Mismatching language"

Expected behavior
I shouldn't get the "Mismatching language" alert since the meta tags are properly set

Is your feature request related to a problem? Please describe.
I want to be sure that the build I'm going to distribute passes an SEO audit, but now it has to be manual. I need to deploy the code and, after deploying, run the audit.

Describe the solution you'd like
Add integration to GitHub Action to act as a checker. Code cannot be deployed if the PR generates any critical issues.

Is your feature request related to a problem? Please describe.
Currently as per my understanding, it allows crawling on demand, is there any possibility to add feature to schedule the crawling process based on given time intervals.

Describe the solution you'd like
It should allow kind of cronjob scheduler service to enable scheduling the crawling job.

Describe the bug
A crawl was started but it doesn't ever finish. The UI only shows "Crawling...". This status has lasted for over a week now.

Fully stopping docker and starting it again didn't help. I'm not sure where to find debug information for this issue, therefore excuse the vagueness. Please let me know which information I can add.

To Reproduce
No steps to reproduce, happens randomly. So far only twice, both projects during the same day. Since then it hasn't occurred again.

Expected behavior
Crawl either finishes successfully or breaks on no activity after a certain amount of time or after a configured max timeout. An option to cancel the crawl would be a nice feature to have (as in #4). The option to delete the project (and thus also cancel the crawl) could also be available at all times, even when a crawl is running.

Anchors for in-text internal links does not display correctly like it is the case for external links. I guess this is a known issues, just reporting it in case. I spend some time during my lunch digging a bit further on your tool and it is excellent.

Congratulations again.