When running make, right after the TestFlush test, it stops with the following error.
I'm not experienced in go, what should I do to build it?

2024/03/13 09:40:24 Updating database schema from 0 to 1... (this might take a while)
2024/03/13 09:40:24 Updating database schema from 1 to 2... (this might take a while)
2024/03/13 09:40:24 Updating database schema from 2 to 3... (this might take a while)
=== RUN   TestTorrentMetadata_MarshalJSON
--- PASS: TestTorrentMetadata_MarshalJSON (0.00s)
=== RUN   TestNewStatistics
--- PASS: TestNewStatistics (0.00s)
=== RUN   TestParseISO8601
=== PAUSE TestParseISO8601
=== RUN   TestDaysOfMonth
--- PASS: TestDaysOfMonth (0.00s)
=== RUN   TestIsLeap
--- PASS: TestIsLeap (0.00s)
=== RUN   TestPostgresDatabase_ExecuteTemplate
--- PASS: TestPostgresDatabase_ExecuteTemplate (0.00s)
=== RUN   TestPostgresDatabase_OrderOn
--- PASS: TestPostgresDatabase_OrderOn (0.00s)
=== RUN   Test_sqlite3Database_DoesTorrentExist
=== PAUSE Test_sqlite3Database_DoesTorrentExist
=== RUN   Test_sqlite3Database_GetNumberOfTorrents
=== PAUSE Test_sqlite3Database_GetNumberOfTorrents
=== RUN   Test_sqlite3Database_AddNewTorrent
=== PAUSE Test_sqlite3Database_AddNewTorrent
=== RUN   Test_sqlite3Database_QueryTorrents
=== PAUSE Test_sqlite3Database_QueryTorrents
=== RUN   Test_sqlite3Database_GetTorrent
=== PAUSE Test_sqlite3Database_GetTorrent
=== RUN   Test_sqlite3Database_GetFiles
=== PAUSE Test_sqlite3Database_GetFiles
=== RUN   Test_sqlite3Database_GetStatistics
=== PAUSE Test_sqlite3Database_GetStatistics
=== CONT  Test_sqlite3Database_DoesTorrentExist
=== CONT  Test_sqlite3Database_QueryTorrents
=== RUN   Test_sqlite3Database_QueryTorrents/Test_Relevance
=== RUN   Test_sqlite3Database_DoesTorrentExist/Test_Empty
--- FAIL: Test_sqlite3Database_QueryTorrents (0.00s)
    --- FAIL: Test_sqlite3Database_QueryTorrents/Test_Relevance (0.00s)
=== CONT  Test_sqlite3Database_AddNewTorrent
=== RUN   Test_sqlite3Database_AddNewTorrent/Test_Nil
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x58 pc=0xc51f6b]

goroutine 3 [running]:
testing.tRunner.func1.2({0xddf080, 0x1382280})
	/usr/lib/go/src/testing/testing.go:1631 +0x3f7
testing.tRunner.func1()
	/usr/lib/go/src/testing/testing.go:1634 +0x6b6
panic({0xddf080?, 0x1382280?})
	/usr/lib/go/src/runtime/panic.go:770 +0x132
github.com/tgragnato/magnetico/persistence.Test_sqlite3Database_QueryTorrents.func1(0xc0000b2000)
	/opt/magnetico/persistence/sqlite3_test.go:174 +0xab
testing.tRunner(0xc0000b2000, 0xc0000ac000)
	/usr/lib/go/src/testing/testing.go:1689 +0x21f
created by testing.(*T).Run in goroutine 34
	/usr/lib/go/src/testing/testing.go:1742 +0x826
FAIL	github.com/tgragnato/magnetico/persistence	0.046s
FAIL
make: *** [Makefile:15: test] Error 1

Would you mind expanding readme with instructions on running this? Especially in docker for windows.

Using the instructions in your readme to pull the "latest" tag (linux/amd64) seems to pull a non-functional image that results in a SIGSEGV when run. Conversely, pulling the "next" tag (from instructions here: https://github.com/tgragnato/magnetico/pkgs/container/magnetico) released 2 hours ago (at time of writing) pulled a different image that works

I find this project - bitmagnet.io.
This seems to be a more complete solution for users. Maybe we can join them? ;)

Its clear that this is not an "Easy installation" as the readme would suggest.

After 4 hours of my time, 20+ years of computing experience, added googling and prompting, here is what I've been able to do so far from a fresh install of Ubuntu (24):

sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

# DB
sudo docker pull postgres
sudo docker run --net=host --name magnetico-postgres -e POSTGRES_USER=magnetico -e POSTGRES_PASSWORD=magnetico -e POSTGRES_HOST_AUTH_METHOD=trust -d postgres
sudo docker exec -it magnetico-postgres bash
psql -U magnetico
CREATE EXTENSION pg_trgm;
quit

# magnetico
git clone https://github.com/tgragnato/magnetico
cd ./magnetico
docker build -t mag .
sudo docker run --net=host --rm -it --name contmag --entrypoint "/bin/sh" prjmag -c /usr/bin/magneticod

Then to get the web server, magneticow, running, start another terminal and then:

sudo docker exec -it contmag /usr/bin/magneticow --no-auth

I am super happy to report that this seems to be working after a lot of messing around.

However, as from the origonal repo, and related press coverage, I would have expected it to be easier to run?!

Can this be simplifed at all? In the origonal repo they simple state the you can run make docker that it sets everything up and runs both serives, however it never worked for me.

Hello!

Looking at what is communicated using wireshark it seems like magnetico always uses node-id 0000...

Code seems to agree with this behaviour:

Is there a reason for using the 0000 id or is this a bug?

During use, it was found that the addNode function occupied a large amount of memory over time and was not released, eventually leading to system memory exhaustion.

git : ac5125f

pprof:

I think the problem is caused by slicing.

I am not sure if it is appropriate to use the copy() function to solve this problem through online query.

Why is the checked length equal to 1 here, if the second element is also used?

if len(peerAddrs) = 1 then ms.incomingInfoHashes[infoHash] = peerAddrs[1:] will cause an error or nothing (because empty slice)?

I think the check needs to be corrected to len(peerAddrs) < 2

if len(peerAddrs) < 1 {
    return
}

similar behavior occurs further:

Fix G601 (CWE-118): Implicit memory aliasing in for loop

https://github.com/securego/gosec

https://pkg.go.dev/golang.org/x/vuln/cmd/govulncheck

Hi, and many thanks for taking care of magnetico!

I'm a long time user of it (since the python version in fact), and still using an SQLite database, because I found that performance and storage space of PostgreSQL doesn't fit my need.

I saw that you removed SQLite in 5ed7d99

Do you plan to support it, at least for scraping the DHT? I guess that you removed it because it didn't fit your needs for the web interface, but… it's just a guess :)

After running for a while the leechers are all used up and will not allow addition of new leeches.
You can test this by adding some debug output and running with --leech-max-n=1.

The issue lies in the Sink which leaves zero-length peer lists in the ms.incomingInfoHashes when it handles the last peer of an infohash. This should be cleared by onError.

I.e a delete is needed here:

However, adding that breaks the current tests due to race-conditions.

What do you mean by checking nodes?

Could not add new torrent to the database. tx.Exec (INSERT INTO files)
ERROR: invalid byte sequence for encoding "UTF8": 0x00 (SQLSTATE 22021)

Experiment with actions using go-test-coverage
https://github.com/marketplace/actions/go-test-coverage

Have a nice day at work!

I noticed that you are actively developing the "magnetico" project.

Importing info-hashes from the specified tracker (OpenTracker) is very important to me.

If possible, add an alternative behavior for "magneticoD".

For tests, I can provide my home OpenTracker, which I use. There aren't many torrents there, but this should be enough to check.

Please, additionally read my correspondence with the author of "magnetico":

My letter to Bora:

Good day!

I am an application administrator and I'm not strong in development.
"Magneticow" works very good, however, "magneticod" works only for global DHT. I need to configure the "magneticod" to work "only for local network".

I have "OpenTracker" ( source from https://erdgeist.org/arts/software/opentracker/ ) and several "Transmission" clients running on my local network.
"Curl" can request a list of announced "Info-hashes" from "OpenTracker".
For example:

• announce URL of my "OpenTracker" is "http://ds:6969/announce".
• top10 URL"http://ds:6969/stats?mode=top10" (text and info-hash)
• top100 URL "http://ds:6969/stats?mode=top100" (text and info-hash)
• tpbs "http://ds:6969/stats?mode=tpbs&format=txt" (info-hash, seed and leech)
• statedump "http://ds:6969/stats?mode=statedump" (info-hash and text)
• scrape "http://ds:6969/scrape" (binary)

Help me set up the "magneticod" for my purposes.
How to run "magneticod" for index my "OpenTracker"?

Thanks for any help provided. Have a good day!

Reply from Bora:

Hi,

No worries at all. In fact, you reminded me to do something that I've been postponing for a long time and I just archived the project today.
Regarding your request, I would have gladly helped if it was simple but I am afraid it isn't. I really cannot see how you can trawl a DHT of localhosts only, except that you can change the bootstrapping nodes (you need to hosts those yourself of course) and add some iptables rules to ensure that magnetico doesn't try to connect hosts outside of your local/private network. I would think it's doable, but would require lots of tuning I reckon. Hope it helps!

I think this can be removed (it seems to be legacy of code that was used for debugging)

According to this convention, imports are typically grouped into three sections, and each section is separated by a blank line:

1. Standard library imports
2. Third-party package imports
3. Your own package imports

Ref: 3d346ad

Edit: I called it deadlock first, but first line indicates concurrent read/write of map:
fatal error: concurrent map iteration and map write

Not sure how to help debug, but the output is attached.
trace.txt