dylandreimerink / edb Goto Github PK

Hi,

I am trying to give a shot for edb and this is my test eBPF program, which is a kprobe-type program to trace tcp connections.

I've loaded the eBPF program and set the breakpoint via edb, but i don't know how to make it run and hit the breakpoint. You can find the details below:

Could u pls offer more guide to show how to use edb?

This issue proposes to add commands with which the contents of registers and memory can be updated. The purpose of this feature is to explore conditions which normally don't occur or are difficult to recreate while in a debugging session. For example if you want to explore an error case, you can break just before evaluating something and change the value of R1 to trigger an artificial error.

The MVP version of the feature is to have it just work on registers, but being able to change local/global variables would be nice, completely or specific struct members. Additionally to also modify memory we don't have variables for like changing bytes as specific memory locations to change a packet for example.

Currently, the only way to know where breakpoints are placed is to look at breakpoint list and match the entries there to the code / instructions. It would be nice to have a breakpoint indicator like a red arrow or red arrow plus the breakpoint number besides/enfront of the code and instructions. That way it is obvious at which locations breakpoints are placed.

If a breakpoint is disabled we should color the arrow gray instead of red. If the breakpoint is conditional we should color it yellow or orange. For log breakpoints we should perhaps use a different symbol like a arrow like -> instead of a => arrow or =#.

In the course of developing EDB so far I have encounters a number of bugs, these are often only reproducible after following a very specific set of steps and a specific ELF file. It would be very hard for users to communicate a clear list of steps to reproduce the issue, therefor I want to add a "crash dump" system which will record all information that might be necessary to reproduce bugs.

The primary use case for such dumps would be to present them to the user when we recover from a panic with the request to upload them or send them privately. Additionally, calling the dump command should output the path to the dump file/folder to the user so they can send dumps of subtle bugs.

With this addition, I would also like to add a logging framework like you would use on a server application, its output would be part of the crash dump so we can add debug messages and data which can be used to see if we hit certain code paths.

These dumps should include the following elements:

• A history of all commands (together with the output in ASCIIcast format?)
• The output created by EDB
• The contents of ELF and CTX files loaded
• Source code referenced by the ELF file
• The contents of the debug log
• The version/commit information of the EDB version
• Any error message / panic
• ? Core dump (dump the processes memory so we can inspect it with delve)

This would have some serious implications since we will be capturing potentially sensitive information, which in itself isn't an issue, but we have to communicate this very clearly before asking people for the dumps. We should clearly outline what is in them (for example by including a readme in the dump dir) so it is an informed decision to send source code publicly or privately.

This issue proposes to add conditional breakpoints, which would allow a user to specify a expression(#16) to a breakpoint. Only when this expression evaluates to true would the breakpoint be triggered. This feature is very handy if you want to debug a specific condition which only occurs on for example the 200'th iteration of a loop, by being able to trigger on i == 200 you don't have to break and continue 200 times manually.

This issue depends on having the expression logic described in issue #16 in place, and is blocked until that issue is taken care of.

It can be difficult at times to look at highlighted code when using the list command while debugging. It would be nice to have rudimentary code highlighting for C code. During implementation, we should also take other languages that can generate eBPF program into account like Rust so a extensible system would be nice.

One handy feature other debuggers have is the ability to list and inspect variables of the current scope. This feature can be implemented in the following steps:

1. Index DW_TAG_subprogram and DW_TAG_inlined_subroutine DWARF tags in reverse index so we can find the most specific function scope from at any given instruction(should we include other scopes like loops? Do those have seperate DWARF tags). (Only one at load time)
2. Find the most specific scope for the current PC(Program counter) and get all DW_TAG_variable and DW_TAG_formal_parameter tags.
3. variables or parameters without DW_AT_location attribute should be displayed but marked as inlined
4. variables with the DW_AT_location class of loclistptr should pick a valid location expression from the referenced location-list. If non are valid of the current PC, we should still show the var/param but mark is as unavailable.
5. If a valid location expression can be found, execute it using a stack machine and current memory/registers, get the pointer to the data.
6. Copy the data from the register/memory as byte slice
7. Use BTF to lookup the size of the param/variable type, which can be gotten from the DW_AT_type attribute of the tag.
8. Use the BTF type plus the byte slice and attempt to 'inflate' the bytes into a C literal definition to show the user.

DWARF spec for reference: https://dwarfstd.org/doc/DWARF4.pdf

In some programs that deal with assembly like Ida pro or Radare2, jumps are annotates with arrows, this makes following what assembly does much easier. So I would also like to add this to the assembly view of EDB.

When a user uses the reset command, all of the current program state is reset(loaded context, stack, registers) but the maps stay changed. This is intentional, but in some cases a user might want to also reset the contents of one or more maps. So this issue proposes to add a map reset {map index|map name} command to clear the contents of a map.

Perhaps it is useful to also include a map reset-all command, in case one has a lot of maps to reset.

This issue proposes to add a map import {file path} and a map export {file path} command to import/export the contents of maps. At this moment, if you want to setup a map for a test you have to add the map contents one entry at a time via a macro. It would be nice if we could import the contents of a large map from a file.

Once we have the file format we can also export maps from the kernel by converting bpftool output or creating our own commands to capture the contents of maps.

Wanted to test edb and tried to install it using go install ... command line (as appears in the readme) but unfortunately I got the error below.

Currently, list shows us the source code at the current location. This issues proposes to extend its functionality by adding some optional parameters. It would, for example be handy to pick how many lines above and below the current line you want to see, so specifying a count like list 40 would show 20 lines above and 20 lines below the current point instead of the default.

The second idea is to allow users to specify a start and end line number within the current or another file like: list 100-120 for the current file and list somefile.h:100-120 to specify some specific file.

Thirdly, it would be handy to be able to list a function, for example by saying list @func123, or a function within a specific file like list somefile.h@func123. Perhaps this can extends to symbols in general.

The current context capture capabilities of EDB are somewhat lacking, as of writing this we only support capturing of XDP programs. Seeing as sk_buff based programs are more common generally, we should also add support for sk_buff contexts.

A huge improvement for EDB would be DAP support, this would allow users to use EDB from their editor. My personal goal is to get it to work in VSCode, this will also require us to write a extension.

DAP itself has a number of features which may or may not be implemented by the DAP server and/or client, so we will still have to make a selection of the initial features and increment in later updates.

Hi,

Great to see the missing debug tool for ebpf has finally come~

However, I can't install edb successfully, the error messages are listed below:

> go install github.com/dylandreimerink/edb@latest
go install: github.com/dylandreimerink/edb@latest (in github.com/dylandreimerink/[email protected]):
The go.mod file for the module providing named packages contains one or
more replace directives. It must not contain directives that would cause
it to be interpreted differently than if it were the main module.

Could u pls take a look?

BTW, I just built the code manually and got the right binary.

The step command we currently have always steps into functions, this is fine if we want to follow execution in detail, but it is very frustrating at times to have to step through every sub-function call there is. So it would be a nice feature to have a step-over command which will step to the next line within the same scope. A very similar, yet good command to have would be the step-out which will go the the first line outside of the current scope, which can be used to correct if you used step instead of step-over, or if you just want to leave the current function.

This issue proposes to add log points, which are a type of "breakpoint" that, instead of actually breaking the program will log a message and/or expression to some log buffer along with the location of the point. It is the fancy version of print statements in code, but in some situations it helps to be able to just see the order of logs instead of having to keep track of 30 times you were stopped by a breakpoint.

These log points should come in the conditional and non-conditional variants.

This issue is similar to #13, yet still requires some different steps. Global variables are DW_TAG_variable tags typically directly underneath the DW_TAG_compile_unit. Clang will output tags for these but the DW_AT_location attributes all point to address 0x00 because it doesn't know about the "global variables in array maps" trick eBPF/Clang employs. So we should should use the BTF to discover in which maps ".rodata/.bss/.data" each variable lives, at which offset and the memory address of the map values. With all of that we should be able to get the raw bytes at runtime and decode them using the BTF to a C literal declaration format.

A common feature is to be able to perform some basic evaluations on the current program state. This issue proposes to add a eval/exec/call command to the debugger which then will evaluate an expression like local_var_a == struct a{.a=123, .b=234}, global_var_b >= 20 && !global_var_c, or simply local_var_d (which will print its value if it exists).

This has the following purposes/goals:

• Together with the new command, this is an easy way to inspect the current program state.
• These same expressions can be used for conditional breakpoints and logging break points.
• Could be a stepping stone to more complex macros for more advanced automation.

The exact feature set is something to be determined and can be iterated over time, here are some initial suggestions:

• Reference local variables (see #13)
• Reference global variables (see #14)
• Reference registers, perhaps like #R1 or some other prefix which is illegal in C identifiers to avoid conflicts
• Have boolean operators ==, !=, >=, <=, <, >
• Use BTF to match int, string, bool, struct types.
• true and false keywords for boolean comparason
• Round brackets(( and )) to indicate precedence, otherwise left to right.

This issue proposes that we add the ability to use actual eBPF maps instead of the emulated maps. So after loading maps from an ELF file you would be able to map live pin {path to pinned map} or map live new to use an existing pinned map or create a new map in the kernel. After this, all reads/writes/deletes will use this actual map via the syscalls.

This would allow us to to a few things, for example in a use-case where a user has multiple eBPF programs coordinating via maps, this would allow you to replace one program with EDB while debugging it.

This also allows a user to very easily switch between running a program live and debugging it with little setup, to persist map state over multiple EDB sessions and it is a pre-cursor to run programs in "live" mode.

Before we switched to cilium/ebpf, we had Clang/LLVM style eBPF assembly, now we have the DSL of the asm package. This works but I personally still like the Clang/LLVM output since it uses actual operator chars like += or <<= instead of acronyms which reads better for me. So I would like to be able to configure it somehow.

Currently, the context capture feature support a lot of helpers, but not yet the most important onces, the map related calls. This is because most helper calls have a parameter with the max size of the output which we can use to capture the results. Map calls like bpf_map_lookup_elem, bpf_map_update_elem and bpf_map_delete_elem don't include size info for the keys and values because this information is included in the map spec. So to be able to instrument these helpers properly, we need to parse the map specs, track which one is referenced by the current helper function and use the appropriate sizes when copying the arguments or return data.

Hi, after I enter the command edb graph dist/tracee.bpf.core.o tracepoint__raw_syscalls__sys_enter -f dot -o res.dot. I got the following error:

Error: load collection: file dist/tracee.bpf.core.o: section "kprobe/security_file_open": string is not stack allocated: not supported

tracee.bpf.core.zip

Any help?

Edit:
I am trying to create a control flow graph for of the tracepoint__raw_syscalls__sys_enter program of the tracee project.

Steps to reproduce are:

1. Clone tracee: git clone https://github.com/aquasecurity/tracee.git
2. Build tracee cd tracee && make tracee-ebpf
3. Attempt to render CFG: edb graph dist/tracee.bpf.core.o tracepoint__raw_syscalls__sys_enter -f dot -o res.dot

./edb graph ./hello-debug.o hello -o aa.txt
Error: load collection: file ./hello-debug.o: section "tracepoint/syscalls/sys_enter_execve": string is not stack allocated: not supported
Usage:
  edb graph {ELF} {program name} [flags]

Flags:
  -f, --format string   The output format: dot, svg, pdf or png (default "svg")
  -h, --help            help for graph
  -o, --output string   output to given file path or - for stdout, instread of opening in browser

load collection: file ./hello-debug.o: section "tracepoint/syscalls/sys_enter_execve": string is not stack allocated: not supported

bpf code:

cat ../hello_bpf/hello.c
#include <linux/bpf.h>
#define SEC(NAME) __attribute__((section(NAME), used))

static int (*bpf_trace_printk)(const char *fmt, int fmt_size,
                               ...) = (void *)BPF_FUNC_trace_printk;

SEC("tracepoint/syscalls/sys_enter_execve")
int bpf_prog(void *ctx) {
  const char *msg = "Hello, BPF World!";
  bpf_trace_printk(msg, sizeof(msg));
  return 0;
}

char _license[] SEC("license") = "GPL";

Any idea? Error for bpf_trace_printk？

Since #13 and 484f163 we have the tools to determine in which function/scope we are and where we came from. The locals command uses the current scope to look for variables, but there is no reason why we can't also inspect variables in functions to which we have yet to return. To would be nice if we could for example do locals -1 and it will show the locals of our parents scope.

Another idea would be to assign indexes to each scope starting from 0 which is our main program and then increasing, this would require you to first execute callstack/cs to get a index.

Yet another approach might be to be able to "switch" scope. We wouldn't be able to continue execution without going back to the actual scope of the current instruction, but it could be some temporary state to browse around the callstack to inspect the locals for example.