columbia / egalito Goto Github PK

In the function

EgalitoInterface::EgalitoInterface(bool verboseLogging, bool useLoggingEnvVar) {
    if(!verboseLogging) muteOutput();

    if(!parseLoggingEnvVar()) {
        LOG(1, "Failed to parse EGALITO_DEBUG environment variable");
    }
}

it seems that the second argument is never used so that it always reads env vars

With a very small test shared lib and a small test binary that uses that lib, I get a segfault at runtime if I've run the library through etharden, even with the --nop flag. I'm getting the same segfault in my own tool without modifying the ELF file- just parse and then generate.

If necessary, I can supply the code, but there's nothing special about it. The library is built with various combinations of the following flags:

CPPFLAGS=-std=c++11 -shared -fPIC -rdynamic -fPIC -g -O0 -v -da -Q

gcc: g++ (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0

Requesting you to create and provide a docker image for experimenting with Egalito.

Hi there! Thank you for your work on Egalito.

I have noticed that in specific cases not all writes to memory seem to be captured by Egalito's UseDef analysis. When applying UseDef to a function like:

void foo(struct test *t, int n) {
  if (n < 5) {
    t->x = 10;  
  }
  else { 
    t->x = 12;
  }
}

both of the writes to t->x don't seem to occur in any of the states of the UDRegMemWorkingSet.

I have checked this with:

$ gcc --version
gcc (Debian 8.3.0-6) 8.3.0

$ clang --version
clang version 13.0.0

which generates assembly like:

0000000000001100 <foo>:
    1100:	55                   	push   %rbp
    1101:	48 89 e5             	mov    %rsp,%rbp
    1104:	48 89 7d f8          	mov    %rdi,-0x8(%rbp)
    1108:	89 75 f4             	mov    %esi,-0xc(%rbp)
    110b:	83 7d f4 05          	cmpl   $0x5,-0xc(%rbp)
    110f:	0f 8d 0f 00 00 00    	jge    1124 <foo+0x24>
    1115:	48 8b 45 f8          	mov    -0x8(%rbp),%rax
    1119:	c7 00 0a 00 00 00    	movl   $0xa,(%rax)
    111f:	e9 0a 00 00 00       	jmpq   112e <foo+0x2e>
    1124:	48 8b 45 f8          	mov    -0x8(%rbp),%rax
    1128:	c7 00 0c 00 00 00    	movl   $0xc,(%rax)
    112e:	5d                   	pop    %rbp
    112f:	c3                   	retq 

I hope to hear back from you. Thanks in advance!

Di (@SleepyMug) ran into a problem with parsing his libffi.so.6 (attached for reference), where it segfaults attempting to find function boundaries. The following lines from the debug output of etobjdump may be relevant:

Splitting code section into 29 fuzzy functions
disassembly error? fuzzyfunc-0x5fec 85 < 16e
WARNING: FallThrough: the last block was all NOPs
WARNING: FallThrough: the last block was all NOPs
etobjdump: pass/fallthrough.cpp:131: virtual void FallThroughFunctionPass::visit(Function*): Assertion `target' failed.

One big piece of support we're going to need from egalito before adding RISC-V support in (that's our goal for the next few months) is cross-platform analysis. Not cross-target compilation, but the ability to analyse RISC-V binaries on, say, an x86_64 host. We're running a RISC-V emulator and so doing everything on the same platform as the analysis is going to be very slow. Even when we do end up with real hardware AFAIK it's still going to be a soft-core with limited RAM, so the analysis may still be very slow.

Since I think you have cross-compilation support, I'm hoping this won't be too bad? We're going to be analysing 64-bit binaries from a 64-bit platform, so at least word/pointer sizes etc should be sized consistently.

Assume we have access to a copy of the root filesystem of the target, so we can still e.g. read its /etc/ld.so.conf and find appropriate shared libraries. I'm thinking about adding a FileAccessor or similar singleton to handle file accesses (rather than opening "/etc/ld.so.conf", you open FileAccessor::accessPath("/etc/ld.so.conf") but IIRC there's more tricks that Egalito pulls off that reference the current platform than just that (dep/rtld is a good example), and some feedback on what else we'd need to handle would be appreciated.

Thanks!

I was playing around with OffsetLink last night and noticed something: it doesn't appear to respect the offset and instead just seems to use the specified target's address. I'd appreciate some independent confirmation + maybe an explanation of where to go looking to try and fix it if someone has the time. :-)

Unfortunately I had some binaries which setup _libc_start_main parameter by mov instructions on x86_64. In below example 0x531e30 points to __libc_csu_init, however etelf failed to update 0x531e30 to the new address, and so crash happens...

0000000040000018 <_start>:
40000018: 31 ed xor %ebp,%ebp
4000001a: 49 89 d1 mov %rdx,%r9
4000001d: 5e pop %rsi
4000001e: 48 89 e2 mov %rsp,%rdx
40000021: 48 83 e4 f0 and $0xfffffffffffffff0,%rsp
40000025: 50 push %rax
40000026: 54 push %rsp
40000027: 49 c7 c0 20 1e 53 00 mov $0x531e20,%r8
4000002e: 48 c7 c1 30 1e 53 00 mov $0x531e30,%rcx
40000035: 48 c7 c7 70 39 42 00 mov $0x423970,%rdi
4000003c: e8 8f 02 60 c0 callq 6002d0 __libc_start_main@plt
40000041: f4 hlt
40000042: 90 nop
40000043: 90 nop

Maybe I will try to manually update it for now, and see if I have time to dig into this later...

Not sure why this dependency is not met. Let me know if I should install some library for this. Trying on a fresh install of ubuntu 20.04

make[2]: Entering directory '/root/egalito/dep/rtld' gcc -std=c99 -Wall -Wextra -Wno-format -Wno-int-conversion -Wno-unused-parameter -g -o build_x86_64/load_test load_test.c readlink -f /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/ld-linux-x86-64.so.2 > build_x86_64/interpreter readelf --dyn-syms /usr/lib/x86_64-linux-gnu/ld-2.31.so | grep rtld_global@ | awk '{ print $3 }' > build_x86_64/rtld_global_size readelf --dyn-syms /usr/lib/x86_64-linux-gnu/ld-2.31.so | grep rtld_global_ro@ | awk '{ print $3 }' > build_x86_64/rtld_global_ro_size gdb -n -q -x reconstruct.py -x rtld.gdb /usr/lib/x86_64-linux-gnu/ld-2.31.so | grep -v 'done\.' | grep -v 'Reading symbols' \ | sed 's/struct rtld/struct my_rtld/' > build_x86_64/rtld.h Python Exception <class 'gdb.error'> No struct type named rtld_global.: rtld.gdb:2: Error in sourced command file: Error occurred in Python: No struct type named rtld_global.

One feature I'd really love to see in Egalito is assembly manipulation, for handling cases when you want to e.g. modify which registers an instruction is referencing or change the opcode from add to or. Another feature that would be really nice is an API for generating assembly from an opcode + registers/memory addresses specification (similar to what XED provides, but integrated with the link/chunk/rewriting support etc in Egalito. Something like this would be lovely:

std::vector<Instruction *> instructions = {
    createInstruction("mov", RegisterOpcodeSpecification(X86PlatformRegister::RAX),
        DereferenceOpcodeSpecification(X86PlatformRegister::RDI)),
    createInstruction("lea", RegisterOpcodeSpecification(X86PlatformRegister::RBX),
        IPRelativeLinkReference(somelink))
};

One problem of course is making this somehow cross-platform... Integrating support for Keystone might be one option?

I am trying to install Egalito on Ubuntu server 22.04.3 with little success. Following the README, I can install make, g++, libreadline-dev, gdb, lsb-release, and libc6-dbg via apt without issue but libstdc++6-7-dbg can't be found.

The "git clone [email protected]:columbia/egalito.git --recursive" also didn't work for me due to "Permission denied (public key)" so I instead opted to run "git clone https://github.com/columbia/egalito.git" to clone over https.

I also encountered a number of errors when running the make -j 8 command which I captured using this script:

Script started on 2023-11-15 18:52:34+00:00 [TERM="linux" TTY="/dev/tty1" COLUMNS="100" LINES="37"]
almiles@egalito-server:/egalito$ make -j 8
make -C dep
make -C config
make[1]: Entering directory '/home/almiles/egalito/dep'
make[1]: Entering directory '/home/almiles/egalito/config'
make -C rtld
make -C ifunc
make -C distorm3/make/linux CC='gcc -Wno-missing-braces'
make[2]: Entering directory '/home/almiles/egalito/dep/rtld'
make[2]: Entering directory '/home/almiles/egalito/dep/ifunc'
make[2]: Entering directory '/home/almiles/egalito/dep'
make[2]: *** distorm3/make/linux: No such file or directory. Stop.
make[2]: Leaving directory '/home/almiles/egalito/dep'
make[1]: *** [Makefile:32: distorm3] Error 2
make[1]: *** Waiting for unfinished jobs....
make[1]: *** [Makefile:43: capstone] Error 1
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/home/almiles/egalito/dep/ifunc'
gcc -std=c99 -Wall -Wextra -Wno-format -Wno-int-conversion -Wno-unused-parameter -g -DRTLD_GLOBAL_SIZE= -DRTLD_GLOBAL_RO_SIZE= -c -o build_x86_64/test.o test.c
test.c: In function ‘info_mode’:
test.c:33:35: error: invalid application of ‘sizeof’ to incomplete type ‘char[]’
33 | print_raw(_rtld_global, sizeof(_rtld_global), "_rtld_global");
| ^
test.c:34:38: error: invalid application of ‘sizeof’ to incomplete type ‘char[]’
34 | print_raw(_rtld_global_ro, sizeof(_rtld_global_ro), "_rtld_global_ro");
| ^
test.c: In function ‘main’:
test.c:56:57: error: expected expression before ‘)’ token
56 | if(sizeof(struct my_rtld_global) != RTLD_GLOBAL_SIZE) {
| ^
test.c:59:63: error: expected expression before ‘)’ token
59 | if(sizeof(struct my_rtld_global_ro) != RTLD_GLOBAL_RO_SIZE) {
| ^
make[2]: *** [Makefile:82: build_x86_64/test.o] Error 1
make[2]: Leaving directory '/home/almiles/egalito/dep/rtld'
make[1]: *** [Makefile:25: rtld] Error 2
make[1]: Leaving directory '/home/almiles/egalito/dep'
make: *** [Makefile:30: dep/built] Error 2
make: *** Waiting for unfinished jobs....
make[1]: Leaving directory '/home/almiles/egalito/config'
almiles@egalito-server:/egalito$ exit
exit

Script done on 2023-11-15 18:52:43+00:00 [COMMAND_EXIT_CODE="2"]

If anyone could tell me what I'm doing wrong it would be greatly appreciated

As per your website, how do we get added to the slack channel?

OS: Ubuntu 18.04 (docker)
Host arch: x86_64

I am unable to recompile the binary /bin/ls (from Ubuntu18.04 ARM64), here's the command I used:

./etharden -m /mnt/aarch64/ls /tmp/hello

Here's the error I get:

Parsing ELF file...
etharden: analysis/usedef.cpp:1949: void UseDef::fillMovabs(UDState*, AssemblyPtr): Assertion `mode == AssemblyOperands::MODE_IMM_REG' failed.
Aborted (core dumped)

is there a special flag I should provide to support cross-recompilation?

Also: I tried to build egalito directly on ubuntu18.04 ARM64 but compilation fails because of a makefile error while building capstone:

make -C dep
make[1]: Entering directory '/home/luca/egalito/dep'
make -C rtld
make[2]: Entering directory '/home/luca/egalito/dep/rtld'
make[2]: Leaving directory '/home/luca/egalito/dep/rtld'
make -C ifunc
make[2]: Entering directory '/home/luca/egalito/dep/ifunc'
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/home/luca/egalito/dep/ifunc'
Makefile:43: recipe for target 'capstone' failed
make[1]: *** [capstone] Error 1
make[1]: Leaving directory '/home/luca/egalito/dep'
Makefile:30: recipe for target 'dep/built' failed
make: *** [dep/built] Error 2

I'm working on a tool that uses egalito to remove target functions from compiled libraries. How can I modify the symbol tables? I can't find any examples of this in the codebase.

I want to use egalito tool to remove some basic blocks from the binary as opposed to adding a few instructions. Is it possible to do so? I don't see any related documentation for this task.

I'm not sure whether this is due to changes over the past couple of years relating to glibc. However, I am having crashes and I figured I'd post a note to determine if its something simple, or difficult...

I first had a problem using the master in ubuntu 22.10, and I found that issue and swapped to the other newer branch. I saw in src/load/bootstrap-x86_64.s that it required __csu_fini, and init which are no longer easy for me to find or replace. I had commented out, and checked glibc source which showed an xor on r8, and rcx which used to contain those variables..

i am wondering if the route i should take is to update src/load/bootstrap-x86_64.s to the newer glibc of ubuntu, but this also makes me wonder if anything tested, or developed is stuck to the specific scenarios that compile/work. In other words, will it only work on the particular system I fix this issue with? and I cannot edit/write binaries related to older systems..

The alternate route I considered is getting deep into the API and possibly blocking use/rewriting of the loader/libc to allow using on a broader section (if this is the case and problem)... I've done some binary rewriting as well, and am trying to find an updated library (2023) that can be used. I've checked a lot of alternatives, and egalito seems pretty close to what I need for testing, so just wondering if I'm on the right track for ubuntu 22.10, and whether it is confined to writing binaries particular to a range of versions...

Oh the crash I was having was with the CURRENT ubuntu glibc sysdeps/x86_64/elf/start.S, rather than the modified one.. which meant that egalito resolved the dependencies and copied the current one however I am sure I must fix some of the parameters/addressing between them...

Thanks for your time.