Git Product home page Git Product logo

Comments (6)

beniz avatar beniz commented on May 26, 2024 1

First investigation reveals this is faiss crashing:

Thread 37 "dede" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffe50ffd700 (LWP 10616)]
0x0000555555c49c41 in faiss::OnDiskInvertedLists::OngoingPrefetch::Thread::one_list() ()
(gdb) bt
#0  0x0000555555c49c41 in faiss::OnDiskInvertedLists::OngoingPrefetch::Thread::one_list() ()
#1  0x0000555555c49d7b in faiss::OnDiskInvertedLists::OngoingPrefetch::prefetch_list(void*) ()
#2  0x00007fffda3366db in start_thread (arg=0x7ffe50ffd700) at pthread_create.c:463
#3  0x00007fffd20aa71f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

from deepdetect.

dgtlmoon avatar dgtlmoon commented on May 26, 2024

I tried indexing 600 images, with a much lower train_samples and nprobe but the same problem..

    "index_type": "IVF20,SQ8",
    "train_samples": 10,
    "nprobe": 8

in model/

-rw-r--r-- 1 dgtlmoon dgtlmoon    181948 Nov 15 08:49 index.faiss
-rw-r--r-- 1 dgtlmoon dgtlmoon   4194304 Nov 15 08:49 index_mmap.faiss
[2021-11-15 07:51:19.101] [torchlib] [info] Network initialization done.
[2021-11-15 07:51:19.102] [test] [info] Using pre-trained weights from /var/www/xxx/web/files/trainer/simsearch/model/model_iter_13500.caffemodel
[2021-11-15 07:51:19.408] [torchlib] [info] Attempting to upgrade batch norm layers using deprecated params: /var/www/xxx/web/files/trainer/simsearch/model/model_iter_13500.caffemodel
[2021-11-15 07:51:19.408] [torchlib] [info] Successfully upgraded batch norm layers using deprecated params.
[2021-11-15 07:51:19.512] [test] [info] Net total flops=3858534272 / total params=26063936
[2021-11-15 07:51:19.512] [test] [info] detected network type is classification
[2021-11-15 07:51:19.513] [api] [info] HTTP/1.1 "PUT /services/test" <n/a> 201 508ms
open existing index db
[2021-11-15 07:51:23.846] [torchlib] [info] Opened lmdb /var/www/xxx/web/files/trainer/simsearch/model//names.bin
bash: line 1:     7 Segmentation fault      (core dumped) ./dede -host 0.0.0.0

from deepdetect.

dgtlmoon avatar dgtlmoon commented on May 26, 2024

Maybe the version of FAISS changed, and it worked with an earlier lib version?

from deepdetect.

dgtlmoon avatar dgtlmoon commented on May 26, 2024

The cmake doesnt seem to be tied to any particular version

      message(STATUS "Fetching FAISS")
      add_definitions(-DUSE_FAISS)
      add_definitions(-DUSE_SIMSEARCH)
      ExternalProject_Add(
        faisslib
        PREFIX faiss
        SOURCE_DIR ${CMAKE_BINARY_DIR}/faiss/src/faiss
        GIT_REPOSITORY https://github.com/facebookresearch/faiss
        GIT_TAG main
        UPDATE_DISCONNECTED 1
        CONFIGURE_COMMAND cmake ${CONFIGURE_OPTS} -B build .
        BUILD_COMMAND make -C build -j${N}
        INSTALL_COMMAND ""
        BUILD_IN_SOURCE 1
      )

from deepdetect.

beniz avatar beniz commented on May 26, 2024

This is fixed with #1369, disabling MMAP, no alternative solution found at the moment :/ Thanks for the report.

from deepdetect.

dgtlmoon avatar dgtlmoon commented on May 26, 2024

@beniz no probs, this will make its way into a docker build at some time?

from deepdetect.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.