naim94a / lumen Goto Github PK

Describe the bug
Hey Naim! At first I wanted to run lumen without a doker, but I didn't understand where schema.sql is located. but I managed to build the lumen and connect the ida to it. when trying to send the idb received an error

lumen: database error; please try again later..
Failed to execute the "LuminaViewAllMds" command

and on server
failed to extract, db: the "funcs" relation does not exist.
So I had to resort to docker. But even here I was in for a setback. I used the default docker-compose file. At startup, compilation started, but ended with an error too.

root@lumen-srv:/lumen# docker-compose up Building lumina Sending build context to Docker daemon 452.6kB Step 1/22 : FROM rust:1.68.2-slim-buster ---> 352f3bc068fa Step 2/22 : ARG DEBIAN_FRONTEND=noninteractive ---> Using cache ---> 95e5c1cdf4c0 Step 3/22 : RUN apt-get update && apt-get install -y --no-install-recommends --no-install-suggests ca-certificates pkg-config libssl-dev libpq-dev ---> Using cache ---> ad80587b6d79 Step 4/22 : ENV CARGO_REGISTRIES_CRATES_IO_PROTOCOL=sparse ---> Using cache ---> a8ae4f5cf938 Step 5/22 : RUN --mount=type=cache,target=$CARGO_HOME/registry cargo install diesel_cli --version 2.0.1 --no-default-features --features postgres the --mount option requires BuildKit. Refer to https://docs.docker.com/go/buildkit/ to learn how to build images with BuildKit enabled ERROR: Service 'lumina' failed to build : Build failed

To Reproduce
docker-compose up -d

Server (please complete the following information):

• OS: Debian 12 bookworm
• Architecture: x86_64
• Docker: 20.10.24+dfsg1, build 297e128
• docker-copose: 1.29.2, build unknown

Client (please complete the following information):

• OS: Windows 11
• Architecture: x86_64
• IDA Version: 7.7.220118

first time making a bug report so bear with me.

the database im trying to pull on is SoTGame_dump.exe (1,865,212KB).

I get this error whenever I try to pull metadata. I'm on IDA 7.7.220118 (Windows x64).

lumina: lumen: db error; please try again later..

Command "LuminaPullAllMds" failed

Environment: Using IDA version 8.3 on a Windows system.

Setup: Installed and configured socat and openssl for Windows. Successfully created the 'hexrays.crt' file in the IDA 8.3 installation directory using openssl.

Issue: Encountering an SSL handshake failure error when attempting to connect to Lumina via socat server in powershell.

Steps to Reproduce:

1. Executed the command to start a socat server: socat -s -dd tcp4-listen:1234,fork,reuseaddr openssl:lumen.abda.nl:1235,cafile=hexrays.crt.
2. Ran a .bat file with the command set LUMINA_TLS=false to disable TLS for Lumina and then started ida64.exe through cmd using the bat file.
3. In IDA, navigated to the Lumina tab under options, entered "localhost" as the server, "1234" for the port, and used "guest" for both username and password.

Error Observed:
On applying these settings, an error was generated in the socat server running in PowerShell, indicating a problem with SSL handshake and certificate:
2023/12/27 03:41:57 socat[2296] E SSL_connect(): error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
2023/12/27 03:41:57 socat[2296] E no peer certificate
2023/12/27 03:41:57 socat[33544] W waitpid(): child 2296 exited with status 255

I cannot makje docker-compose to work. If trying to run it it will die here:

lumina-postgres | 2022-04-01 14:54:11.359 UTC [1] LOG:  listening on IPv6 address "::", port 5432
lumina-postgres | 2022-04-01 14:54:11.361 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
lumina-postgres | 2022-04-01 14:54:11.364 UTC [53] LOG:  database system was shut down at 2022-04-01 14:54:11 UTC
lumina-postgres | 2022-04-01 14:54:11.367 UTC [1] LOG:  database system is ready to accept connections
lumina_1  | nc: connect to db port 5432 (tcp) failed: Connection timed out
lumina_1  | nc: connect to db port 5432 (tcp) failed: Connection timed out
lumina_1  | nc: connect to db port 5432 (tcp) failed: Connection timed out

The nc: connect message will loop forever. Any idea how to fix this? I tried it on 2 different ubuntu host systems. I am also open to suggestions for the host (debian?) if you have any.

The following metadata types aren't parsed at the moment:

• 0x01: typeinfo (assuming it's the function's signature)
• 0x08: set stack points
• 0x09: function frame
• 0x0a: set operand types

It seems that some users are having trouble connecting to unofficial lumina servers with TLS enabled. It seems that the current workarounds aren't simple enough.

We should have a plugin that securely connects to any lumen instance by approving preset certificates in addition to the builtin ones.

Describe the bug
Using the latest GTA V 3095 dump, a lot of people like to take the source code that was leaked and match it up to functions and push all that data into the Lumen server. On top of the guy who seems to love pushing all the decompiled natives into it (which is another 10,000 functions) there can be as many as 100,000 functions that it needs to pull. All of this data causes the following error:

`lumina: lumen: query took too long to execute.

Command "LuminaPullAllMds" failed`

To Reproduce
Dump GTA V's latest EXE from memory during the legal screen when the natives are decrypted.
Load it into IDA.
Try to do a Lumina -> Pull All.

Expected behavior
Should pull all the data, so long as nothing was stuck or faulted.

Server (please complete the following information):

• OS: Whatever lumen.abda.nl runs.
• Architecture: Whatever lumen.abda.nl runs.

Client (please complete the following information):

• OS: Windows 11 Pro 23H2 22631.3155
• Architecture: x64
• IDA Version: 8.3.230608

Additional context
Maybe @tomrus88 can replace the timeout variable with his plugin?

Any push/pull results in:

thread 'tokio-runtime-worker' panicked at 'index out of bounds: the len is 0 but the index is 0', /home/ace/lumen/common/src/rpc/mod.rs:95:5
stack backtrace:
   0:     0x55e0a78469c0 - std::backtrace_rs::backtrace::libunwind::trace::h746c3e9529d524bc
                               at /rustc/f74583445702e2e27ec4415376f2c540a83d7ded/library/std/src/../../backtrace/src/backtrace/libunwind.rs:90:5
   1:     0x55e0a78469c0 - std::backtrace_rs::backtrace::trace_unsynchronized::h84373278bfb39e0c
                               at /rustc/f74583445702e2e27ec4415376f2c540a83d7ded/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0x55e0a78469c0 - std::sys_common::backtrace::_print_fmt::h517324efde750597
                               at /rustc/f74583445702e2e27ec4415376f2c540a83d7ded/library/std/src/sys_common/backtrace.rs:67:5
   3:     0x55e0a78469c0 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hf594ab77fac89284
                               at /rustc/f74583445702e2e27ec4415376f2c540a83d7ded/library/std/src/sys_common/backtrace.rs:46:22
   4:     0x55e0a7766c9c - core::fmt::write::h3868db8542c90941
                               at /rustc/f74583445702e2e27ec4415376f2c540a83d7ded/library/core/src/fmt/mod.rs:1078:17
   5:     0x55e0a7845fc1 - std::io::Write::write_fmt::h3f6656f045fa877f
                               at /rustc/f74583445702e2e27ec4415376f2c540a83d7ded/library/std/src/io/mod.rs:1519:15
   6:     0x55e0a7845a00 - std::sys_common::backtrace::_print::hda7655c057c24dcc
                               at /rustc/f74583445702e2e27ec4415376f2c540a83d7ded/library/std/src/sys_common/backtrace.rs:49:5
   7:     0x55e0a7845a00 - std::sys_common::backtrace::print::h546a6c8431d46287
                               at /rustc/f74583445702e2e27ec4415376f2c540a83d7ded/library/std/src/sys_common/backtrace.rs:36:9
   8:     0x55e0a7845a00 - std::panicking::default_hook::{{closure}}::h006dd083853faf51
                               at /rustc/f74583445702e2e27ec4415376f2c540a83d7ded/library/std/src/panicking.rs:208:50
   9:     0x55e0a78452b3 - std::panicking::default_hook::hf0f9afb1017317fc
                               at /rustc/f74583445702e2e27ec4415376f2c540a83d7ded/library/std/src/panicking.rs:225:9
  10:     0x55e0a78452b3 - std::panicking::rust_panic_with_hook::hb7a19826c029b1d6
                               at /rustc/f74583445702e2e27ec4415376f2c540a83d7ded/library/std/src/panicking.rs:591:17
  11:     0x55e0a7863e5d - std::panicking::begin_panic_handler::{{closure}}::hde71edcd925d0c5e
  12:     0x55e0a7863dcc - std::sys_common::backtrace::__rust_end_short_backtrace::h8a3c7d6cea578919
                               at /rustc/f74583445702e2e27ec4415376f2c540a83d7ded/library/std/src/sys_common/backtrace.rs:141:18
  13:     0x55e0a7863d7d - rust_begin_unwind
                               at /rustc/f74583445702e2e27ec4415376f2c540a83d7ded/library/std/src/panicking.rs:493:5
  14:     0x55e0a7764490 - core::panicking::panic_fmt::h20225113c4a2f8fd
                               at /rustc/f74583445702e2e27ec4415376f2c540a83d7ded/library/core/src/panicking.rs:92:14
  15:     0x55e0a7764321 - core::panicking::panic_bounds_check::h5752306d250b5620
                               at /rustc/f74583445702e2e27ec4415376f2c540a83d7ded/library/core/src/panicking.rs:69:5
  16:     0x55e0a7702f07 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::hb4f310b474cb9da1
  17:     0x55e0a77055aa - lumen::serve::{{closure}}::{{closure}}::h07416f8232903818
  18:     0x55e0a77ab71b - tokio::runtime::task::raw::poll::h1705a3383fc32a6c
  19:     0x55e0a7886cb7 - tokio::runtime::thread_pool::worker::Context::run_task::h59b2fa03a65ecc27
  20:     0x55e0a787e36e - tokio::runtime::task::raw::poll::ha196a4abbf2823db
  21:     0x55e0a787cfc3 - std::sys_common::backtrace::__rust_begin_short_backtrace::h4d117004d6fda901
  22:     0x55e0a7872297 - core::ops::function::FnOnce::call_once{{vtable.shim}}::ha186c7af46b2d32a
  23:     0x55e0a786bf35 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hea1090dbdcecbf5a
                               at /rustc/f74583445702e2e27ec4415376f2c540a83d7ded/library/alloc/src/boxed.rs:1328:9
  24:     0x55e0a786bf35 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h8d5723d3912bd325
                               at /rustc/f74583445702e2e27ec4415376f2c540a83d7ded/library/alloc/src/boxed.rs:1328:9
  25:     0x55e0a786bf35 - std::sys::unix::thread::Thread::new::thread_start::hc17a425ca2995724
                               at /rustc/f74583445702e2e27ec4415376f2c540a83d7ded/library/std/src/sys/unix/thread.rs:71:17
  26:     0x7f5d29eb9fa3 - start_thread
  27:     0x7f5d29c604cf - clone
  28:                0x0 - <unknown>

Not using TLS. IDA 7.5. The hosted server you run works fine, just trying to run my own doesn't.

Describe the bug
Earlier today, I noticed that my local Lumen server appeared to have stopped working, with IDA reporting db error; please try again later... Went to check the docker compose log, it appears that something perhaps related to the migration changes made back in Feb has broken something in the connection between the client and the Postgres db. When attempting to pull information, the following is printed to the log (note that this occurs even after clearing the postgres volume/creating a brand new db),

lumina-postgres  | 2023-04-10 09:39:48.490 UTC [1] LOG:  starting PostgreSQL 15.1 on x86_64-pc-linux-musl, compiled by gcc (Alpine 12.2.1_git20220924-r4) 12.2.1 20220924, 64-bit
lumina-postgres  | 2023-04-10 09:39:48.490 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
lumina-postgres  | 2023-04-10 09:39:48.490 UTC [1] LOG:  listening on IPv6 address "::", port 5432
lumina-postgres  | 2023-04-10 09:39:48.494 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
lumina-postgres  | 2023-04-10 09:39:48.500 UTC [52] LOG:  database system was shut down at 2023-04-10 09:39:48 UTC
lumina-postgres  | 2023-04-10 09:39:48.504 UTC [1] LOG:  database system is ready to accept connections
lumina-postgres  | 2023-04-10 09:39:57.884 UTC [56] ERROR:  relation "funcs" does not exist at character 68
lumina-postgres  | 2023-04-10 09:39:57.884 UTC [56] STATEMENT:  WITH best AS (
lumina-postgres  |                  select chksum,MAX(rank) as maxrank from funcs f1
lumina-postgres  |                  WHERE chksum = ANY($1)
lumina-postgres  |                  GROUP BY chksum
lumina-postgres  |              )
lumina-postgres  |              SELECT f2.name,f2.len,f2.metadata,f2.chksum FROM best
lumina-postgres  |              LEFT JOIN funcs f2 ON (best.chksum=f2.chksum AND best.maxrank=f2.rank)

To Reproduce

1. Start a new Lumen instance
2. Pull/push to/from IDA.
3. Failure.

Expected behavior
Successful metadata exchange

Server (please complete the following information):

• OS: Ubuntu Ubuntu 22.04.2 LTS
• Architecture: x86_64

Client (please complete the following information):

• OS: Windows 11
• Architecture: x64
• IDA Version: 8.2

Additional context
Add any other context about the problem here.
Please add a panic backtrace if available.

Describe the bug
On IDA Pro 8.1, deleting a function metadata fails with the following error:

lumina: lumen: error: invalid data.
Command "LuminaFunctionsDeleteMd" failed

This works when trying to the the same with a private Lumina server.

To Reproduce

• Rename a function
• Click "Push current function metadata"
• Click "Pull current function metadata"
• Click "Delete current function metadata"

Expected behavior
The function's metadata is successfully deleted

Server (please complete the following information):

• OS: Debian 11
• Architecture: x86_64

Client (please complete the following information):

• OS: Windows 10
• Architecture: x86_64
• IDA Version: 8.1

Describe the bug
lumina: Certificate verification failed: certificate has expired

To Reproduce

• client

lumina: Certificate verification failed: certificate has expired
Command "LuminaPullAllMds" failed
lumina: Certificate verification failed: certificate has expired
Command "LuminaPullAllMds" failed

• server

 2021-02-04T15:45:03.857Z INFO  lumen > starting private lumen server...
 2021-02-04T15:45:03.884Z INFO  lumen > starting http api server on 0.0.0.0:8082
 2021-02-04T15:45:03.884Z INFO  lumen > listening on 0.0.0.0:1234 secure=true
 2021-02-04T15:45:21.174Z WARN  lumen > err: IOError(Custom { kind: UnexpectedEof, error: "early eof" })
 2021-02-04T15:45:28.010Z WARN  lumen > err: IOError(Custom { kind: UnexpectedEof, error: "early eof" })
 2021-02-04T15:51:36.887Z WARN  lumen > err: IOError(Custom { kind: UnexpectedEof, error: "early eof" })
 2021-02-04T15:51:40.747Z WARN  lumen > err: IOError(Custom { kind: UnexpectedEof, error: "early eof" })
 2021-02-04T15:55:57.878Z WARN  lumen > err: IOError(Custom { kind: UnexpectedEof, error: "early eof" })
 2021-02-04T15:56:16.972Z WARN  lumen > err: IOError(Custom { kind: UnexpectedEof, error: "early eof" })

• server config

[lumina]
# address that lumen will listen on for IDA to connect to
bind_addr = "0.0.0.0:1234"
# indicates if TLS should be used for connections, if true the `lumina.tls` section is required.
# <<<<<<<<<<<<<<<<<
use_tls = true
# server display name; appears in IDA output window
server_name = "lumen"

# only required when `use_tls` is set to true.
[lumina.tls]
# Specify the server's certificate. 
# Clients connecting to the server must match this certificate.
# If the certificate is password protected, the password can be specified in the `PKCSPASSWD` environment variable.
# <<<<<<<<<<<<<<<<<
server_cert = "/root/lumen5/lumen/lumen.p12"


[database]
# Specifies a postgresql connection string. All variables can be found here: https://docs.rs/tokio-postgres/0.6.0/tokio_postgres/config/struct.Config.html
connection_info = "host=127.0.0.1 user=test111 password=test111"
# Sets if the database connection should be made using TLS.
use_tls = false
# If the database requires a secure connection, paths to server-ca and client-id certificates can be set here:
server_ca = "db_ca.pem"
client_id = "db_id.p12"

# comment out this section to disable api server
# api server allows to query the database for comments by file or function hash.
[api_server]
bind_addr = "0.0.0.0:8082"

• I checked the certificate time is normal

root@user123:~/lumen5/lumen# cat lumen_crt.pem | openssl x509 -noout -enddate
notAfter=Feb  4 15:27:45 2022 GMT
root@user123:~/lumen5/lumen# openssl pkcs12 -in lumen.p12 -clcerts -nodes | openssl x509 -noout -enddate
Enter Import Password:
notAfter=Feb  4 15:27:45 2022 GMT

i'm wondering cuz i found it is so hard to upload funcs in lib one by one

Describe the bug
HelloResult response is not fully implemented
Related #144

To Reproduce
Some features are not enabled in IDA when using lumen server, such as Delete metadata

Expected behavior
Server should respond correctly with all needed information

Server (please complete the following information):

• OS: whatever lumen.abda.nl uses
• Architecture: whatever lumen.abda.nl uses

Client (please complete the following information):

• OS: Windows 11
• Architecture: x86_64
• IDA Version: 8.3

Additional context
Correct structure of HelloResult is as following:

// this struct contains string data from ida.key file
struct license_info_t
{
    qstring id; // license id
    qstring name; // license name
    qstring email; // license email
};

// lumina_user_t structure also used in some other unimplemented packets, such as:
// GET_LUMINA_STATS_RESULT
// SHOW_USERS_RESULT
// USER_OPERATION
// SHOW_PUSHES_RESULT
struct lumina_user_t
{
    license_info_t license_info;
    qstring name; // login user name
    uint32_t karma; // whatever that means lol
    uint64_t last_active; // unix time?
    uint32_t features; // flags: 0x1 is_admin, 0x2 appears to be can_delete_metadata, official private lumina server sends 0x3 for admin accounts and 0x2 for regular accounts here, and 0 when using official public lumina server
};

struct helo_result_t
{
    lumina_user_t user;
};

It's my own mistake(😓

Describe the bug
Lumen service doesn't run because of a missing file.

lumina_1         | Connection to db 5432 port [tcp/*] succeeded!
lumina_1         | sh: 1: /lumen/docker-init.sh: not found
lumina_1 exited with code 127

To Reproduce
docker-compose up

Expected behavior
The lumen container runs.

Describe the bug
Function names look garbled, or are using some strange format. What is the likely cause of this?

99% of my functions imported look like this no matter the library they are from.

To Reproduce
Lumina -> View All

Expected behavior
Clear function names like TiXmlDeclaration instead of ??0TiXmlDeclaration@@QAE@PBD00@Z

Server (please complete the following information):
lumen.abda.nl

Client (please complete the following information):
windows 11, 64bit, 8.3

When i try pulling meta data i got:

lumina: Invalid license
Command "LuminaPullAllMds" failed

A bit more research is required to determine how IDA calculates function signatures. Calculating signatures independently would allow other tools (such as radare2, binary-ninja, Ghidra) to use the lumen server.

The signature is the md5 of the function's bytes and a bitmap of the same length.

Ideally, something like this:

fn calc_chksum(fn_bytes: &[u8]) -> [u8; 16] {
    let mut md5 = Md5::new();
    md5.update(fn_bytes);
    let bitmap = // work needed
    md5.update(&bitmap);
    md5.digest()
}

IDA 8.2
Lumina -> push all metadata

ERROR lumen > push failed, db: error encoding message to server: value too large to transmit

Describe the bug
I accidentally pushed a bunch sub_XXXXXX functions.

To Reproduce
1, Select multiple sub_XXXXX in the functions window.
2, Right click
3, Click Lumina
4, Click Push functions metadata

Expected behavior
Don't push sub_XXXXXX metadata.

Server (please complete the following information):

• OS: Debian 10
• Architecture: x86_64

Client (please complete the following information):

• OS: Windows 7
• Architecture: x86_64
• IDA Version: 7.3sp3

Hello,

I'm part of a malware research and analysis team that has taken steps to perform our research and analysis in an airgapped environment, with our own simulated "internet" and the likes of it. We came across lumen as an offline alternative to lumina, and we are fairly interested to see if it's possible for the database to be made open source and downloadable?

As means of re-contribution, we are also open to any suggestions of how we can obtain the changes we make in our own environment and push back to the community database?

Depending on how much metadata you have pushed into the database you can get the wrong results when pulling. For example, when pulling metadata for IDA's QT5Gui.dll I get metadata for CryptoPP and 7-zip that I have uploaded into my database the past week.

If each library and program has its own database, I could compile QT5 and upload it into the QT5 database. Since I'm decompiling IDA's QT5Gui.dll I could select the QT5 database and pull metadata from it, without the possibility of getting metadata from unrelated programs and libraries.

Maybe even create separate databases per OS/Architecture, maybe even compiler versions:

windows/x86/qt5.sql
windows/x64/qt5.sql
linux/arm64/qt5.sql
linux/x86/gcc-6.4.0/qt5.sql
linux/x86/gcc-6.5.0/qt5.sql

As far as switching between databases, I have no idea how it would work.

Describe the bug
Following error in IDA output window appears

Failed to get func histories from secondary lumina server: lumen: error: invalid data.

Failed to get func histories from primary lumina server: lumen: error: invalid data.

Command "LuminaMetadataHistory" failed

To Reproduce
Open context menu inside some function and select option Lumina -> Metadata History

Expected behavior
New window should open which shows historical information about function, such as:

• Revision (1,2,3,...)
• Timestamp (date & time)
• Name (function name)
• Author
• IDB path

There's also 2 empty bottom panes which probably should show something as well...

Server (please complete the following information):

• Public server at lumen.abda.nl

Client (please complete the following information):

• OS: Windows 11
• Architecture: x86_64
• IDA Version: 8.3

Additional context

• N/A

when I try to pull or push my IDA freezes indefinitely, cert isn't expired either idk whats the problem.

Describe the bug
When trying to use Pull All Metadata i have this error:
lumina: Invalid remote certificate
Command "LuminaPullAllMds" failed

Client

• OS: Windows 11
• Architecture: x86_64
• IDA Version: 7.5 SP3 x64

I did edit the ida.cfg with those edits:
LUMINA_HOST = "lumen.abda.nl"; // This semicolon is important!
LUMINA_PORT = 1235

and install the hexrays.crt into the IDA directory

Describe the bug

root@user123:~/lumen/lumen/bin/release# ./lumen -c config.toml
 2021-02-04T07:14:42.313Z INFO  lumen > starting private lumen server...
 2021-02-04T07:14:42.339Z INFO  lumen > starting http api server on 0.0.0.0:8082
thread 'main' panicked at 'there is no reactor running, must be called from the context of a Tokio 1.x runtime', lumen/src/main.rs:297:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Aborted

To Reproduce
CONFIG.tomL

[lumina]
# address that lumen will listen on for IDA to connect to
bind_addr = "0.0.0.0:1234"
# indicates if TLS should be used for connections, if true the `lumina.tls` section is required.
use_tls = false
# server display name; appears in IDA output window
server_name = "lumen"

# only required when `use_tls` is set to true.
[lumina.tls]
# Specify the server's certificate. 
# Clients connecting to the server must match this certificate.
# If the certificate is password protected, the password can be specified in the `PKCSPASSWD` environment variable.
server_cert = "path/to/server_crt"

[database]
# Specifies a postgresql connection string. All variables can be found here: https://docs.rs/tokio-postgres/0.6.0/tokio_postgres/config/struct.Config.html
connection_info = "host=127.0.0.1 user=my_new_lunmne password=my_new_lunmne "
# Sets if the database connection should be made using TLS.
use_tls = false
# If the database requires a secure connection, paths to server-ca and client-id certificates can be set here:
server_ca = "db_ca.pem"
client_id = "db_id.p12"

# comment out this section to disable api server
# api server allows to query the database for comments by file or function hash.
[api_server]
bind_addr = "0.0.0.0:8082"

Server (please complete the following information):

• OS: Debian 10
• Architecture: x86_64

IDA8.3 released.

lumina:
Command "LuminaPullAllMds" failed

Describe the bug
I tried to replicate the steps on https://abda.nl/lumen/#ida--80 under Windows as best as possible, however, I can't get IDA to connect to the server.

To Reproduce
Download socat for windows and place the hexrays.crt into the same folder and run it with command given on the website.
Create a BAT file that does the same thing the instructions say but uses SET instead of export and runs ida64.exe after.
Have IDA try to connect to the Lumen server via localhost port 1234.
Bug appears.

Expected behavior
Should connect to the Lumen server.

Server (please complete the following information):
Whatever abda.nl is running.

Client (please complete the following information):

• OS: Windows 11 22H2
• Architecture: x64
• IDA Version: 8.3.230608

Additional context
Here are the errors socat and IDA generate:

lumina: recv: An existing connection was forcibly closed by the remote host.
2023/11/27 20:01:21 socat[41484] E SSL_connect(): error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed
2023/11/27 20:01:21 socat[41484] E no peer certificate
2023/11/27 20:02:10 socat[32636] E SSL_connect(): error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed
2023/11/27 20:02:10 socat[32636] E no peer certificate
2023/11/27 20:11:11 socat[32164] E SSL_connect(): error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed
2023/11/27 20:11:11 socat[32164] E no peer certificate
2023/11/27 20:11:21 socat[28128] E SSL_connect(): error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed
2023/11/27 20:11:21 socat[28128] E no peer certificate

When performing a push/pull i get this running a private lumina server

2022-11-29 12:47:29 nc: connect to db port 5432 (tcp) failed: Connection refused 2022-11-29 12:47:30 Connection to db 5432 port [tcp/*] succeeded! 2022-11-29 12:47:30 2022-11-29T01:47:30.552Z INFO lumen > starting private lumen server... 2022-11-29 12:47:30 2022-11-29T01:47:30.565Z INFO lumen > starting http api server on 0.0.0.0:8082 2022-11-29 12:47:30 2022-11-29T01:47:30.566Z INFO lumen > listening on 0.0.0.0:1234 secure=false 2022-11-29 12:47:44 2022-11-29 12:47:44 thread 'tokio-runtime-worker' has overflowed its stack 2022-11-29 12:47:44 fatal runtime error: stack overflow 2022-11-29 12:47:44 Aborted 2022-11-29 12:47:30 Detected custom config.toml 2022-11-29 12:47:30 Starting lumen without TLS. Make sure to set LUMINA_TLS = NO in ida.cfg 2022-11-29 12:47:44 Exiting due to error: Launching lumen

ida7.6 windows, cert updated[https://abda.nl/lumen/hexrays.crt]

Does not work for me anymore, after pulling he stuck with
"Retrieving metadata, 0 bytes out of 5941145"

After i wait ~ 5 MIn:

recv_data: recv: An existing connection was forcibly closed by the remote host.
lumina: connection closed (recv: An existing connection was forcibly closed by the remote host.). Trying to reconnect...
recv_data: recv: An existing connection was forcibly closed by the remote host.
lumina: No response
Command "LuminaPullAllMds" failed

Describe the bug
Pushing metadata for entire mshtml.dll from Windows 7 SP1 fails.

https://www.dropbox.com/s/wg5eufi9odmz3i3/mshtml.7z?dl=0

To Reproduce
1, Setup Windows Symbol path.
2, Open mshtml.dll.
3, Let it download the symbol file and analyze.
4, Push to Lumina server when finished analyzing.

lumina: lumen.abda.nl: db error; please try again later.
Command "LuminaPushAllMds" failed

Expected behavior
Push metadata without failing.

Server (please complete the following information):

• OS: Debian 10.
• Architecture: x86_64
• Postgresql (11+200+deb10u4)
• Rust: 1.48.0 (7eac88abb 2020-11-16)

Client (please complete the following information):

• OS: Windows 7
• Architecture: x86_64
• IDA Version: 7.3sp3

LUMINA_HOST = "192.168.1.100";
LUMINA_PORT = 1234
LUMINA_TLS = NO
LUMINA_MIN_FUNC_SIZE = 32

Additional context

(Not related, but you have a typo in the domain name, src/main.rs line 84.)

Describe the bug

#0 232.7    Compiling clap v4.1.9
#0 240.5    Compiling common v0.2.0 (/lumen/common)
#0 240.8 error: expected a single table name attribute
#0 240.8
#0 240.8          = note: remove this attribute
#0 240.8
#0 240.8  --> common/src/db/models.rs:8:10
#0 240.8   |
#0 240.8 8 | #[diesel(table_name = func_ranks)]
#0 240.8   |          ^^^^^^^^^^
#0 240.8
#0 242.3 error: could not compile `common` due to previous error
------
failed to solve: process "/bin/sh -c cd /lumen && cargo build --release" did not complete successfully: exit code: 101
**To Reproduce**

docker compose build

Expected behavior
Compile success

• OS: Debian
• Architecture: x86_64

Describe the bug
The public server fails with message:

lumina: InitializeSecurityContext[2]: The message received was unexpected or badly formatted.
Command "LuminaViewAllMds" failed

I suspect that this is a problem with SSL misconfiguration on the server side, specifically using a wrong TLS cipher (or in case multiple ciphers are configured, missing support) for the one used client-side in IDA 7.6, but I'm not certain.

This is the configuration:

LUMINA_HOST = "lumen.abda.nl";
LUMINA_PORT = 1235
LUMINA_TLS = YES

With LUMINA_TLS = NO , connection isn't established at all ("connection reset by peer")

To Reproduce
Lumina -> View All Metadata

Server (please complete the following information):

• lumen.abda.nl:1235 (public server)

Client (please complete the following information):

• OS: Windows 7 SP2
• Architecture: x86_64
• IDA Version: 7.6sp1

Describe the bug
IDA 8.1 added the option to use a private Lumina server in the UI, it however requires authentication when connecting to it, which unfortunately Lumen doesn't currently support. This is likely so Hex-Rays can shift to using their own private server in which they distribute to their users.

The environment variable, LUMINA_HOST does not work either without authentication being provided.

To Reproduce
Options -> Lumina -> Private Lumina Server

Expected behavior
Lumen server being able to auth

Server (please complete the following information):

• OS: N/A
• Architecture: N/A

Client (please complete the following information):

• OS: Windows 10
• Architecture: x86_64
• IDA Version: 8.1

Additional context
https://hex-rays.com/products/ida/news/8_1/

Describe the bug
Connection to the lumen server isn't possible with IDA Pro 8.1 for a unknown reason.

To Reproduce
Trying to connect to the lumen server, it fails with
2022-10-18T06:38:53.057Z INFO lumen > listening on 0.0.0.0:1234 secure=false 2022-10-18T06:38:59.754Z WARN lumen > err: IOError(Custom { kind: InvalidData, error: "request length exceeded maximum limit" })

Expected behavior
Lumen connection does work.

Server (please complete the following information):

• OS: Ubuntu 22.04
• Architecture: x86_64

Client (please complete the following information):

• OS: Ubuntu 22.04
• Architecture: x86_64
• IDA Version: 8.1

Additional context
More details for setting up a ida 8.1 server in the readme.md (such as: "Only use guest as username/password") might help.
IDA Pro 8.1 seems to ignore LUMEN_TLS = NO, giving a warning about being an invalid configuration parameter. A custom hexrays.crt isn't accepted.

I get this when I Push to your Lumen DB

But pulling from Lumen DB is no problem I get all the results..

I pushed alot of function prototypes to Lumen about 30,000 I have a bunch of pdb's for many MMO game servers I am trying to make it a complete lumen server.

Edit: now I can't even Pull from lumen server.

If I go on VPN connection it starts working for pulling but still doesn't push

Now it's working again thanks

Some of my submissions :D



Wow 7 MB submission.. also failed

lighthouse@VM-4-4-ubuntu:~/lumen$ cargo run -- -c config-example.toml
Finished dev [unoptimized + debuginfo] target(s) in 0.55s
Running target/debug/lumen -c config-example.toml
2022-10-23T13:43:27.638Z INFO lumen > starting private lumen server...
2022-10-23T13:43:27.672Z INFO lumen > starting http api server on 0.0.0.0:8082
2022-10-23T13:43:27.672Z INFO lumen > listening on 0.0.0.0:1234 secure=false

thread 'tokio-runtime-worker' has overflowed its stack
fatal runtime error: stack overflow
Aborted (core dumped)

When the release compiles everything is fine

Describe the bug
I can view all uploaded metadata via the "Lumina" -> "View all" entry. But the "Refresh" entry in the right-click menu in that window seems not working. When I push a function to lumen server, and then click the "Refresh" in the right-click menu in already opened lumina window, the newly pushed function does not appear. But if i close the lumina window, and repoen with "Lumina" -> "View all", the new function appears. So is the refresh function not implemented?

To Reproduce
Open the lumina window in IDA Pro with "Lumina" -> "View all", and keep it open. Push a new function with either in the right-click menu in IDA-View or in "Lumina" -> "Push function XXX". And then click "Refresh" in the right-click menu in the Lumina window.

Expected behavior
The newly pushed function appears in the lumina window.

Server (please complete the following information):

• OS: ArchLinux
• Architecture: x86_64

Client (please complete the following information):

• OS: ArchLinux
• Architecture: x86_64
• IDA Version: 8.4