Any attempt to create a custom memory mapping such as:
avatar.add_memory_range(0x0, 0x100, name='unmapped_memory', emulate=CustomMemoryPeripheral)
Will result in a timeout in when avatar tries to connect via GDB to a Qemu target. The same mapping works
if the start address is not 0. E.g.:
avatar.add_memory_range(0x8, 0x100, name='unmapped_memory', emulate=CustomMemoryPeripheral)

Regards,
Robert

PyPanda's Panda() top-level class by default creates an interactive monitor like -monitor unix:/tmp/pypanda_mbwril7yr,server,nowait. I dont believe monitor is not the same as QMP, but a text based protocol

There are couple of ideas:

• Always disable PyPanda's monitor by passing raw_monitor=True (https://github.com/panda-re/panda/blob/716b782f3f604d990f361cbab3186b4e0c73540f/panda/python/core/pandare/panda.py#L59)
• Provide an option to PyPanda target creation to disable this by default, but still enable it if needed

The main reason for disabling this is that running PyPanda many times can end up polluting /tmp

I get this error when running ./build_panda.sh:

/home/raywang/avatar2/targets/src/avatar-panda/hw/arm/configurable_machine.c: In function ‘load_configuration’:
/home/raywang/avatar2/targets/src/avatar-panda/hw/arm/configurable_machine.c:65:11: error: too many arguments to function ‘qobject_from_json’
     obj = qobject_from_json(filedata, &qerr);
           ^
In file included from /home/raywang/avatar2/targets/src/avatar-panda/hw/arm/configurable_machine.c:19:0:
/home/raywang/avatar2/targets/src/avatar-panda/include/qapi/qmp/qjson.h:20:10: note: declared here
 QObject *qobject_from_json(const char *string);
          ^
/home/raywang/avatar2/targets/src/avatar-panda/rules.mak:72: recipe for target 'hw/arm/configurable_machine.o' failed
make[1]: *** [hw/arm/configurable_machine.o] Error 1
make[1]: *** Waiting for unfinished jobs....
Makefile:320: recipe for target 'subdir-arm-softmmu' failed
make: *** [subdir-arm-softmmu] Error 2

I am running Ubuntu 16.04.2 LTS.

When I use avater to run a firmware, which address do I need in add_memory_range.

Getting an error when building avatar-qemu for a MIPS target:

avatar2/targets/src/avatar-qemu/hw/avatar/configurable_machine.c: In function ‘create_cpu’:
avatar2/targets/src/avatar-qemu/hw/avatar/configurable_machine.c:513:12: error: implicit declaration of function ‘cpu_mips_init’; did you mean ‘cpu_mmu_index’? [-Werror=implicit-function-declaration]
     cpuu = cpu_mips_init(cpu_model);
            ^~~~~~~~~~~~~
            cpu_mmu_index
avatar2/targets/src/avatar-qemu/hw/avatar/configurable_machine.c:513:12: error: nested extern declaration of ‘cpu_mips_init’ [-Werror=nested-externs]
avatar2/targets/src/avatar-qemu/hw/avatar/configurable_machine.c:513:10: error: assignment makes pointer from integer without a cast [-Werror=int-conversion]
     cpuu = cpu_mips_init(cpu_model);

Configured with:
../../src/avatar-qemu/configure --disable-sdl --target-list=mipsel-softmmu,mipsel-linux-user
Could not find cpu_mips_init definition for avatar-qemu (there is a prototype in cpu.h though). Avatar-panda has the definition on tranlate.c but avatar-qemu does not.

If I include:

MIPSCPU *cpu_mips_init(const char *cpu_model);
MIPSCPU *cpu_mips_init(const char *cpu_model)
{

    return NULL;
}

Inside configurable_machine.c it compiles with no errors.

Using Ubuntu 18.04.
Thanks

Please, release also on https://pypi.python.org

build_qemu.sh works fine. But build_panda.sh has an error:

fatal: reference is not a tree: bff62d20a88e700a6e5ec54afb457bd433e1d4ab
Unable to checkout 'bff62d20a88e700a6e5ec54afb457bd433e1d4ab' in submodule path 'avatar-panda'
error: pathspec 'dtc' did not match any file(s) known to git.
./build_panda.sh: line 27: ../../../src/avatar-panda/configure: No such file or directory
make: *** No targets specified and no makefile found.  Stop.

OS:

$ uname -a
Linux ubuntu 4.15.0-42-generic #45~16.04.1-Ubuntu SMP Mon Nov 19 13:02:27 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Avatar2 from git

[...]
  CC      util/aio-posix.o
  CC      util/compatfd.o
  CC      util/event_notifier-posix.o
  CC      util/mmap-alloc.o
  CC      util/oslib-posix.o
  CC      util/qemu-openpty.o
  CC      util/qemu-thread-posix.o
  CC      util/memfd.o
  CC      util/envlist.o
  CC      util/path.o
/home/benni/BA/avatar2/targets/src/avatar-qemu/util/memfd.c:40:12: error: static declaration of ‘memfd_create’ follows non-static declaration
 static int memfd_create(const char *name, unsigned int flags)
            ^~~~~~~~~~~~
In file included from /usr/include/x86_64-linux-gnu/bits/mman-linux.h:115:0,
                 from /usr/include/x86_64-linux-gnu/bits/mman.h:45,
                 from /usr/include/x86_64-linux-gnu/sys/mman.h:41,
                 from /home/benni/BA/avatar2/targets/src/avatar-qemu/include/sysemu/os-posix.h:29,
                 from /home/benni/BA/avatar2/targets/src/avatar-qemu/include/qemu/osdep.h:104,
                 from /home/benni/BA/avatar2/targets/src/avatar-qemu/util/memfd.c:28:
/usr/include/x86_64-linux-gnu/bits/mman-shared.h:46:5: note: previous declaration of ‘memfd_create’ was here
 int memfd_create (const char *__name, unsigned int __flags) __THROW;
     ^~~~~~~~~~~~
/home/benni/BA/avatar2/targets/src/avatar-qemu/rules.mak:66: recipe for target 'util/memfd.o' failed
make: *** [util/memfd.o] Error 1
make: *** Auf noch nicht beendete Prozesse wird gewartet …

There is a patch available, though.

Hi, thanks for avatar, it's highly interesting.

I was wondering if there would be interest in a feature that lets you generate a core dump in GDB.

My use case would look like this: Say I'm using angr's exploration technique "Symbion" (https://angr.io/blog/angr_symbion/) to dynamically analyze a binary. After angr has stepped through some of the code, at a certain point I might like to create a GDB core dump of a certain state that angr has produced, which would let me do further manual dynamic analysis in GDB.

If this seems like an OK idea, I'd take a crack at implementing it and then submit a PR.

Thanks again

when I test this example on my compute, I get the following error:

2023-03-24 16:22:16,651 | avatar.targets.GDBTarget0.INFO | State changed to TargetStates.STOPPED
2023-03-24 16:22:16,651 | avatar.INFO | Received state update of target GDBTarget0 to TargetStates.STOPPED
2023-03-24 16:22:16,671 | avatar.targets.GDBTarget0.INFO | Connected to Target
2023-03-24 16:22:16,681 | avatar.CRITICAL | No Memory range specified at 0x400008
Hello World!2023-03-24 16:22:16,701 | avatar.targets.GDBTarget0.INFO | State changed to TargetStates.RUNNING
2023-03-24 16:22:16,702 | avatar.INFO | Received state update of target GDBTarget0 to TargetStates.RUNNING
2023-03-24 16:22:16,701 | avatar.targets.GDBTarget0.INFO | State changed to TargetStates.EXITED
2023-03-24 16:22:16,702 | avatar.INFO | Received state update of target GDBTarget0 to TargetStates.EXITED

please tell me what should I do?

Try to execute examples from https://github.com/avatartwo/avatar2-examples/tree/master/qemu_uboot, but have a error:

$ python qemu_uboot_avatar2.py 
Traceback (most recent call last):
  File "qemu_uboot_avatar2.py", line 1, in <module>
    from avatar2 import *
  File "/home/user/.local/lib/python2.7/site-packages/avatar2/__init__.py", line 6, in <module>
    from .message import *
  File "/home/user/.local/lib/python2.7/site-packages/avatar2/message.py", line 45, in <module>
    from .targets.target import TargetStates
  File "/home/user/.local/lib/python2.7/site-packages/avatar2/targets/__init__.py", line 3, in <module>
    from .openocd_target import *
  File "/home/user/.local/lib/python2.7/site-packages/avatar2/targets/openocd_target.py", line 10, in <module>
    from avatar2.protocols.gdb import GDBProtocol
  File "/home/user/.local/lib/python2.7/site-packages/avatar2/protocols/gdb.py", line 17, in <module>
    from avatar2.archs.arm import ARM
  File "/home/user/.local/lib/python2.7/site-packages/avatar2/archs/__init__.py", line 2, in <module>
    from .arm import *
  File "/home/user/.local/lib/python2.7/site-packages/avatar2/archs/arm.py", line 12, in <module>
    class ARM(Architecture):
  File "/home/user/.local/lib/python2.7/site-packages/avatar2/archs/arm.py", line 38, in ARM
    keystone_arch = KS_ARCH_ARM
NameError: name 'KS_ARCH_ARM' is not defined

python -m avatar2.installer doesn't work too

avatar: pip install avatar2
OS: Ubuntu 18

Hi there,

I'm trying to build avatar-qemu. It fails with the following error:

~/.avatar2/avatar-qemu/hw/avatar/configurable_machine.c:67:20: error: unused
      function 'unset_feature' [-Werror,-Wunused-function]
static inline void unset_feature(CPUARMState *env, int feature)
                   ^
1 error generated.
/home/roskosch/.avatar2/avatar-qemu/rules.mak:69: recipe for target 'hw/avatar/configurable_machine.o' failed
make[1]: *** [hw/avatar/configurable_machine.o] Error 1
Makefile:483: recipe for target 'subdir-arm-softmmu' failed
make: *** [subdir-arm-softmmu] Error 2

lsb_release -a:

No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 18.04.2 LTS
Release:	18.04
Codename:	bionic

Is there anything i can do about it? Let me know when you need further information.

Thank!

Description

Building and installing Avatar2 out of the box and then running the nucleo_l152re example in avatartwo/avatar2-examples does not work on Ubuntu 16.04.

Reproduce

$ sudo apt-get install python-pip python-setuptools python-dev cmake
$ git clone https://github.com/avatartwo/avatar2.git
$ cd avatar2/
$ sudo python setup.py install
$ cd targets/
$ build_qemu.sh
$ build_panda.sh

$ git clone https://github.com/avatartwo/avatar2-examples.git
$ cd avatar2-examples/nucleo_l152re/
$ python nucleo_state_transfer.py

Error

$ python nucleo_state_transfer.py 
Traceback (most recent call last):
  File "nucleo_state_transfer.py", line 88, in <module>
    main()
  File "nucleo_state_transfer.py", line 38, in main
    avatar.init_targets()
  File "build/bdist.linux-x86_64/egg/avatar2/avatar2.py", line 207, in init_targets
  File "build/bdist.linux-x86_64/egg/avatar2/targets/qemu_target.py", line 166, in init
  File "build/bdist.linux-x86_64/egg/avatar2/targets/qemu_target.py", line 115, in generate_qemu_config
  File "build/bdist.linux-x86_64/egg/avatar2/avatar2.py", line 134, in generate_config
  File "build/bdist.linux-x86_64/egg/avatar2/targets/target.py", line 239, in dictify
Exception: Unsupported value for dictifying executable for target OpenOCDTarget0

Proposed Solution

Updating avatar2/avatar2/targets/target.py:228 from expected_types = (str, bool, int, list) to expected_types = (str, bool, int, list, unicode) fixes the error, however I am unsure if the change maintains correctness.

The QemuTarget allows overriding of the cpu_model, via a kwarg:
https://github.com/avatartwo/avatar2/blob/main/avatar2/targets/qemu_target.py#L21

However, by now, on init(), qemu target retrieves the cpu_model via avatar.generate_config(), which in turn takes the CPU model from the avatar object:

Hence, the cpu_model kwarg of the qemu-target is essentially useless at the moment.

Suggested fix:
Override cpu-model in the generate_qemu_config() function.
https://github.com/avatartwo/avatar2/blob/main/avatar2/targets/qemu_target.py#L162

I want to get the mmio recordings while the board is running (instead of doing the rehosting, the binary is actually running on the mcu chip). Is it possible to get the mmio (read/write) traffics by using avatar2? I modify some codes of pretender (e.g. make the memory map but set the forwarded argument of add_memory_range to false), however, there is no recordings in the output file, so I wonder whether it is possible to get the recordings by only using OpenOCD target.

Thanks!

avatar_gdb = AvatarGDBConcreteTarget(avatar2.archs.x86.X86,"127.0.0.1", 1234)
CRITICAL | 2021-01-12 07:11:52,893 | avatar.targets.GDBTarget0.GDBProtocol | GDBProtocol was unable to connect to remote target
2021-01-12 07:11:52,893 | avatar.targets.GDBTarget0.GDBProtocol.CRITICAL | GDBProtocol was unable to connect to remote target
2021-01-12 07:11:52,893 | avatar.targets.GDBTarget0.GDBProtocol.CRITICAL | GDBProtocol was unable to connect to remote target
2021-01-12 07:11:52,893 | avatar.targets.GDBTarget0.GDBProtocol.CRITICAL | GDBProtocol was unable to connect to remote target
2021-01-12 07:11:52,893 | avatar.targets.GDBTarget0.GDBProtocol.CRITICAL | GDBProtocol was unable to connect to remote target
2021-01-12 07:11:52,893 | avatar.targets.GDBTarget0.GDBProtocol.CRITICAL | GDBProtocol was unable to connect to remote target
2021-01-12 07:11:52,893 | avatar.targets.GDBTarget0.GDBProtocol.CRITICAL | GDBProtocol was unable to connect to remote target
2021-01-12 07:11:52,893 | avatar.targets.GDBTarget0.GDBProtocol.CRITICAL | GDBProtocol was unable to connect to remote target
2021-01-12 07:11:52,893 | avatar.targets.GDBTarget0.GDBProtocol.CRITICAL | GDBProtocol was unable to connect to remote target
2021-01-12 07:11:52,893 | avatar.targets.GDBTarget0.GDBProtocol.CRITICAL | GDBProtocol was unable to connect to remote target
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python3.9/dist-packages/angr_targets/targets/avatar_gdb.py", line 23, in init
self.avatar.init_targets()
File "/usr/local/lib/python3.9/dist-packages/avatar2/avatar2.py", line 222, in init_targets
t[1].init()
File "/usr/local/lib/python3.9/dist-packages/avatar2/targets/gdb_target.py", line 53, in init
if gdb.remote_connect(ip=self.gdb_ip, port=self.gdb_port):
File "/usr/local/lib/python3.9/dist-packages/avatar2/protocols/gdb.py", line 379, in remote_connect
raise Exception("GDBProtocol was unable to connect")
Exception: GDBProtocol was unable to connect

Attached; pid = 18032
Listening on port 1234
Remote debugging from host 127.0.0.1

gdb/mi -data-list-register-values has the potential to return at least three different types of results.

Normal

{'number': '183', 'value': '0x45a0'}

128bit uint

{'number': '54', 'value': '{v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 = 0x00000000000000000000000000000000}'}

128bit int

{'number': '205', 'value': '{v8_float = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_double = {0x0, 0x0, 0x0, 0x0}, v32_int8 = {0x2f <repeats 16 times>, 0x0 <repeats 16 times>}, v16_int16 = {0x2f2f, 0x2f2f, 0x2f2f, 0x2f2f, 0x2f2f, 0x2f2f, 0x2f2f, 0x2f2f, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int32 = {0x2f2f2f2f, 0x2f2f2f2f, 0x2f2f2f2f, 0x2f2f2f2f, 0x0, 0x0, 0x0, 0x0}, v4_int64 = {0x2f2f2f2f2f2f2f2f, 0x2f2f2f2f2f2f2f2f, 0x0, 0x0}, v2_int128 = {0x2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f2f, 0x00000000000000000000000000000000}}'}

When converting these string values to int(val, 16) and exception is thrown and cannot be returned from read_register_from_nr(...) in protocols/gdb.py.

I've written a regex that extracts the value from in the two latter cases as a quick workaround.

Patch

https://pastebin.com/hzgZ6KtW

This doesn't seem like a permanent solution especially as I'm ignoring half of the data in cases where a int128 number is utilised. It may be an idea to make a more generic way to read/write registers if supported by gdb.

I've only looked at this for the last few hours so I'm not sure.

Hi Guys,

I try to learn avatar2 in order to analyse IoT software.
My setup, which I set up manually:

• ARM qemu running Raspbian
• chroot inside the qemu with all required files from IoT device
• webserver-binary running
• gdbserver attached to running webserver

I managed to connect to the gdbserver with avatar2, but I wonder how this is the correct way to go.
Is it possible to create a QemuTarget for the already running Qemu process?
Or shall I do something completely different?

Every kind of advice or feedback is welcome :)

Cheers

I want to know rcc_dump mean what.

There is no code comment.

Thanks.

Hi, cloned avatar2 and installed it in a python3 virtual env. Then I use the installer (python -m avatar2.installer) in order to build avatar-panda.

I got the following error which makes sense because I use python3:

ERROR: Cannot use 'python', Python 2.6 or later is required.
       Note that Python 3 or later is not yet supported.
       Use --python=/path/to/python to specify a supported Python.

I wonder how to proceed. Shall I use python2 for avatar in general or only for the installer? I could also try to clone/build avatar-panda without the installer. Or i use the build_panda.sh provided in the targets directory of the avatar2 repo.

Hi there,

I tried to enable interrupt and inject interrupt, but the message sent to QMPProocol shows:

{"id": 3, "error": {"class": "CommandNotFound", "desc": "The command avatar-armv7m-enable-irq has not been found"}}
'{"id": 5, "error": {"class": "CommandNotFound", "desc": "The command avatar-armv7m-inject-irq has not been found"}}

Another error message is:

avatar.targets.PandaTarget0.ARMV7MInterruptProtocol.ERROR | Unable to create rx_queue: No queue exists with the specified name /avatar_v7m_irq_rx_queue

It seems even though PandaTarget inherits QEMUTarget, it doesn't support interrupts yet.

Hi,

May I ask whether avatar2 is able to rehost the OS-based firmware (e.g. Raspbian or Openwrt) or not? I notice that it is able to deal with armv7-M architecture (for microcontroller), but how about armv7-A/armv8-A (for application e.g. raspberry pi)? I have tried to use QEMU to emulate Raspbian, but I have not much idea for rehosting it.

If it is possible, how to set up the memory map? For the MCU, it is not difficult to figure out the base of rom (likely be 0x08000000), ram (0x20000000) and mmio (0x40000000). I know there is a difference between bare-metal program and OS-based kernel. However, when I search the memory mapping of raspberry pi, I got this picture and it is still confused for me on how to set up the mem map in avatar2.

Thankful for any answer/idea!

Dear Sir

I saw there is a angr target inside avatar2's publication. But I can't find it anywhere in the source code. How can I connect angr to get symbolic execution from avatar2?

Thanks

When I use pip to install avatar2, I got this error:
llvm/keystone/CMakeFiles/keystone.dir/build.make:1814: recipe for target 'llvm/keystone/CMakeFiles/keystone.dir/__/lib/Target/AArch64/AsmParser/AArch64AsmParser.cpp.o' failed make[2]: *** [llvm/keystone/CMakeFiles/keystone.dir/__/lib/Target/AArch64/AsmParser/AArch64AsmParser.cpp.o] Error 4 make[2]: *** 正在等待未完成的任务.... CMakeFiles/Makefile2:405: recipe for target 'llvm/keystone/CMakeFiles/keystone.dir/all' failed make[1]: *** [llvm/keystone/CMakeFiles/keystone.dir/all] Error 2 Makefile:151: recipe for target 'all' failed make: *** [all] Error 2 running install_lib running install_data error: can't copy 'src/build/llvm/lib/libkeystone.so': doesn't exist or not a regular file
Than , I try to build avatar2 manually, but when I use sudo python setup.py install, I got this error:
Please submit a full bug report, with preprocessed source if appropriate. See <file:///usr/share/doc/gcc-5/README.Bugs> for instructions. llvm/keystone/CMakeFiles/keystone.dir/build.make:1814: recipe for target 'llvm/keystone/CMakeFiles/keystone.dir/__/lib/Target/AArch64/AsmParser/AArch64AsmParser.cpp.o' failed make[2]: *** [llvm/keystone/CMakeFiles/keystone.dir/__/lib/Target/AArch64/AsmParser/AArch64AsmParser.cpp.o] Error 4 make[2]: *** 正在等待未完成的任务.... CMakeFiles/Makefile2:405: recipe for target 'llvm/keystone/CMakeFiles/keystone.dir/all' failed make[1]: *** [llvm/keystone/CMakeFiles/keystone.dir/all] Error 2 Makefile:151: recipe for target 'all' failed make: *** [all] Error 2 error: Setup script exited with error: can't copy 'src/build/llvm/lib/libkeystone.so': doesn't exist or not a regular file
I believe I have installed all dependencies. So, how can I solve these problems. Thanks

I am trying to use the mucleo L512RE example to test/learn avatar2. When I was runing the avatar2-example, I find it will get stuck in nucleo.wait() function which can be seen here.

• OS: Debian 11 (vmware workstation pro with VT-X and virtual IOMMU enabled)
• Environment: avatar2 (installed with docker-compose)

The code is here.

from os.path import abspath
from time import sleep

import os

from avatar2 import *

# set env var
os.environ['AVATAR2_QEMU_EXECUTABLE']="qemu-system-arm"

# Change to control whether the state transfer should be explicit or implicit
USE_ORCHESTRATION = 0

def obvious_print(s):
    print("=========" + s + "=========")


def main():

    # Configure the location of various files
    firmware = abspath('./firmware.bin')

    openocd_config = abspath('./nucleo-l152re.cfg')

    # Initiate the avatar-object
    avatar = Avatar(arch=ARM_CORTEX_M3, output_directory='/tmp/avatar')

    # Create the target-objects
    nucleo = avatar.add_target(OpenOCDTarget, openocd_script=openocd_config)

    qemu = avatar.add_target(QemuTarget, gdb_port=1236)

    # Define the various memory ranges and store references to them
    rom  = avatar.add_memory_range(0x08000000, 0x1000000, file=firmware)
    ram  = avatar.add_memory_range(0x20000000, 0x14000)
    mmio = avatar.add_memory_range(0x40000000, 0x1000000,
                                   forwarded=True, forwarded_to=nucleo)

    # Initialize the targets
    avatar.init_targets()
    obvious_print("Avatar Inited")

    if not USE_ORCHESTRATION:
        # This branch shows explicit state transferring using avatar

        # 1) Set the breakpoint on the physical device and execute up to there
        nucleo.set_breakpoint(0x8005104)
        nucleo.cont()
        nucleo.wait()

        # 2) Transfer the state from the physical device to the emulator
        obvious_print("Now the state is transfering")

        avatar.transfer_state(nucleo, qemu, synced_ranges=[ram])

        print("State transfer finished, emulator $pc is: 0x%x" % qemu.regs.pc)
    else:
        # This shows implicit state transferring using the orchestration plugin

        # 1) Load the plugin
        avatar.load_plugin('orchestrator')

        # 2) Specify the first target of the analysis
        avatar.start_target = nucleo

        # 3) Configure transitions
        #    Here, only one transition is defined. Note that 'stop=True' forces
        #    the orchestration to stop once the transition has occurred.
        avatar.add_transition(0x8005104, nucleo, qemu, synced_ranges=[ram], stop=True)

        # 4) Start the orchestration!
        obvious_print("Now we are trying to start orchestration")
        avatar.start_orchestration()
        obvious_print("Now start orchestration")

        print("State transfer finished, emulator $pc is: 0x%x" % qemu.regs.pc)

    # Continue execution in the emulator.
    # Due due to the forwarded mmio, output on the serial port of the physical
    # device (/dev/ttyACMx) can be observed, although solely the emulator
    # is executing.
    qemu.cont()

    # Further analysis could go here:
    # import IPython; IPython.embed()
    qemu.stop()
    obvious_print("Qemu Stoped")

    # Let this example run for a bit before shutting down avatar cleanly
    sleep(5)
    avatar.shutdown()


if __name__ == '__main__':
    main()

The QMPResponseListener seems to use the method parse_async_response which is only avaliable in GDBResponseListener.
https://github.com/avatartwo/avatar2/blob/master/avatar2/protocols/qmp.py#L58

Hi,

I have created a simple test case to test the custom peripheral feature of avatar:
Essentially I want to get a notification each time the emulated code accesses a certain memory area. Here's a simple example:

ldr r0,=0x40004c00
ldr r1,[r0]
str r1,[r0]

I use this avatar2 script (essentially I just added prints to the HelloWorld example):

from avatar2 import *
from avatar2.peripherals import AvatarPeripheral
from capstone import CS_MODE_THUMB

import os, struct
from time import sleep

from IPython import embed

class HelloWorldPeripheral(AvatarPeripheral):

    def hw_read(self, offset, size):         
        ret = self.hello_world[:size]
        self.hello_world = self.hello_world[size:] + self.hello_world[:size]
        print("READ")
        return ret

    def nop_write(self, offset, size, value):
        print("WRITE")
        return True    

    def __init__(self, name, address, size, **kwargs):
        AvatarPeripheral.__init__(self, name, address, size)
        
        self.hello_world='Hello World'
        
        self.read_handler[0:size] = self.hw_read 
        self.write_handler[0:size] = self.nop_write
        

avatar = Avatar(output_directory='/tmp/avatar123')
avatar.log.setLevel('DEBUG')

hw = avatar.add_memory_range(0x40004c00, 0x100, name='hello_world',
                             emulate=HelloWorldPeripheral, permissions='rw-')        

avatar.add_memory_range(0x1000, 0x1000, file='./test.raw')

qemu = avatar.add_target(QemuTarget, name='qemu1',entry_address=0x1000)
qemu.cpu_model = 'cortex-a8'

qemu.init()
qemu.cont()
qemu.wait()

The result I'd expect that both "READ" and "WRITE" is printed, as the code emulated both reads and writes to the emulated memory area. However, the python script simply hangs and must be killed.

I have attached the logs as well as the sources for my example.
example_script.tar.gz
logs.tar.gz

Regards,

Robert

Hey,

First, thanks for the really cool tool. :)
I'm currently trying my hand on it but I kind of struggle with few errors that might either be because I don't understand what I'm doing (insert here the dog-in-front-of-a-computer meme) or because of an issue with AvatarPeripheral.

Basically, here is a simplified example of what I'm doing:

from avatar2 import *
from avatar2.archs import X86
from avatar2.peripherals.avatar_peripheral import AvatarPeripheral

class HelloWorldPeripheral(AvatarPeripheral):

    def hw_read(self, offset, size):
        ret = self.hello_world[:size]
        self.hello_world = self.hello_world[size:] + self.hello_world[:size]

        # Convert the return value to an integer (py2/py3-compatible)
        # Python >3.2 could just call int.from_bytes(ret, byteorder='little')
        s2fmt = {1: 'B', 2: 'H', 4: 'I', 8: 'Q'}
        ret = struct.unpack('<' + s2fmt[size], ret)[0]
        return ret

    def nop_write(self, offset, size, value):
        return True

    def __init__(self, name, address, size, **kwargs):
        AvatarPeripheral.__init__(self, name, address, size)

        self.hello_world=b'Hello World'

        self.read_handler[0:size] = self.hw_read
        self.write_handler[0:size] = self.nop_write


avatar = Avatar(arch=X86, output_directory='/tmp/avatar_out')
qemu = avatar.add_target(QemuTarget, executable="/tmp/avatar-qemu/build/i386-softmmu/qemu-system-i386")

hw = avatar.add_memory_range(0x40004c00, 0x100, name='hello_world', emulate=HelloWorldPeripheral, permissions='rw-')

avatar.init_targets()

It does work perfectly fine if my target architecture is ARM but fails with the following error with x86:

Configurable: Adding peripheral[avatar-rmemory] region hello_world at address 0x40004c00
Bail out! ERROR:../qom/object.c:715:object_new_with_type: assertion failed: (type != NULL)

Do you have any idea about why I would obtain such behavior?

I'm having difficulty setting breakpoints over a remote GDB interaction. Specifically when I try to break on a hex address.

Here is my avatar python code.

gdb = ava.add_target(avatar2.GDBTarget, gdb_port=GDB_SERVER_PORT)
ava.init_targets()
gdb.bp(0x92b)
gdb.cont()

Debug log when attempting to set the breakpoint.

avatar.targets.GDBTarget0.GDBProtocol.DEBUG | gdb.py line:299 | Sending request: 4-break-insert *0x92b
avatar.targets.GDBTarget0.GDBProtocol.DEBUG | gdb.py line:504 | Attempted to set breakpoint. Received response: {'type': 'result', 'message': 'done', 'payload': {'bkpt': {'number':        '1', 'type': 'breakpoint', 'disp': 'keep', 'enabled': 'y', 'addr': '0x000000000000092b', 'thread-groups': ['i1'], 'times': '0', 'original-location': '*0x92b'}}, 'token': 4, 'stream':      'stdout'}

And GDBserver log.

Failed to read shadow memory of breakpoint at 0x92b (Input/output error).
Failed to insert breakpoint at 0x92b (-1).

Breakpoint functionality does however work when trying with a function name instead of adress. I've tried on multiple binaries and am sure I have the correct addresses. Can anyone try replicating the issue?

While working with some automation on top of Avatar2, capstone produced the name 'ip' for the register 'r12'. When querying Avatar2 for 'ip' it fails with the KeyError below.

As far as I can tell, this dictionary is populated (partially) by a request from gdb, which does not have the 'ip' naming. For now I will replace 'ip' with 'r12' in my code, but this may be worth supporting in Avatar2.

File "/home/valuta/PycharmProjects/d4a-beta/a2h/instructions.py", line 13, in load_reg_value
    return target.read_register(reg_name)
  File "/home/valuta/PycharmProjects/avatar2fork/avatar2/avatar2/watchmen.py", line 78, in watchtrigger
    ret = func(self, *args, **kwargs)
  File "/home/valuta/PycharmProjects/avatar2fork/avatar2/avatar2/targets/target.py", line 408, in read_register
    return self.protocols.registers.read_register(register)
  File "/home/valuta/PycharmProjects/avatar2fork/avatar2/avatar2/protocols/gdb.py", line 697, in read_register
    self._origin.regs._get_nr_from_name(reg)
  File "/home/valuta/PycharmProjects/avatar2fork/avatar2/avatar2/targets/target.py", line 127, in _get_nr_from_name
    return self.__dict__[reg_name]
KeyError: 'ip'

Avatar2 breaks with the new version of pygdbmi (0.10.0.0).
Installing this on ubuntu >= 19 seems to yield an error when running the "Hello world" example in the tests folder.

After installing a previous version of pygdbmi, everything runs smoothly
python3 -m pip pygdbmi==0.9.0.2

When I try to run qemu_uboot, I run into the following problem：

2022-04-19 02:25:29,050 | avatar.INFO | Initialized Avatar. Output directory is /tmp/myavatar
TargetLauncher is starting process "qemu-system-arm" "-M" "versatilepb" "-m" "256M" "-gdb" "tcp:127.0.0.1:1234" "-serial" "tcp:127.0.0.1:2000,server,nowait" "-kernel" "u-boot" "-S" "-nographic" "-monitor" "telnet:127.0.0.1:2001,server,nowait"
pulseaudio: set_sink_input_volume() failed
pulseaudio: Reason: Invalid argument
pulseaudio: set_sink_input_mute() failed
pulseaudio: Reason: Invalid argument
2022-04-19 02:25:29,326 | avatar.targets.GDBTarget0.INFO | State changed to TargetStates.STOPPED
2022-04-19 02:25:29,327 | avatar.INFO | Received state update of target GDBTarget0 to TargetStates.STOPPED
2022-04-19 02:25:29,338 | avatar.targets.GDBTarget0.INFO | Connected to Target
2022-04-19 02:25:29,339 | avatar.targets.QemuTarget0.WARNING | No cpu_model specified - are you sure?
2022-04-19 02:25:29,341 | avatar.targets.QemuTarget0.INFO | QEMU process running
2022-04-19 02:25:29,419 | avatar.targets.QemuTarget0.INFO | State changed to TargetStates.STOPPED
2022-04-19 02:25:29,419 | avatar.INFO | Received state update of target QemuTarget0 to TargetStates.STOPPED
2022-04-19 02:25:29,432 | avatar.targets.QemuTarget0.QMPProtocol.INFO | b'{"return": {}, "id": 0}\r\n'
2022-04-19 02:25:29,432 | avatar.targets.QemuTarget0.INFO | Connected to remote target
2022-04-19 02:25:29,432 | avatar.targets.QemuTarget0.RemoteMemoryProtocol.INFO | Successfully connected rmp
2022-04-19 02:25:29,451 | avatar.targets.QemuTarget0.INFO | State changed to TargetStates.RUNNING
2022-04-19 02:25:29,452 | avatar.INFO | Received state update of target QemuTarget0 to TargetStates.RUNNING
2022-04-19 02:25:29,451 | avatar.targets.QemuTarget0.INFO | State changed to TargetStates.BREAKPOINT
2022-04-19 02:25:29,452 | avatar.INFO | Breakpoint hit for Target: QemuTarget0
2022-04-19 02:25:29,452 | avatar.INFO | Received state update of target QemuTarget0 to TargetStates.BREAKPOINT
2022-04-19 02:25:29,452 | avatar.targets.QemuTarget0.INFO | State changed to TargetStates.STOPPED
2022-04-19 02:25:29,452 | avatar.INFO | Received state update of target QemuTarget0 to TargetStates.STOPPED
================== Arrived at clear_bss =========================
2022-04-19 02:25:29,470 | avatar.ERROR | RemoteMemoryRead failed: Failed to read memory!
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/avatar2-1.4.7-py3.8.egg/avatar2/avatar2.py", line 453, in _handle_remote_memory_read_message
mem = range.forwarded_to.read_memory(
File "/usr/local/lib/python3.8/dist-packages/avatar2-1.4.7-py3.8.egg/avatar2/watchmen.py", line 78, in watchtrigger
ret = func(self, *args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/avatar2-1.4.7-py3.8.egg/avatar2/targets/target.py", line 35, in check
return func(self, *args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/avatar2-1.4.7-py3.8.egg/avatar2/targets/target.py", line 386, in read_memory
return self.protocols.memory.read_memory(address, size, num_words, raw)
File "/usr/local/lib/python3.8/dist-packages/avatar2-1.4.7-py3.8.egg/avatar2/protocols/gdb.py", line 664, in read_memory
raise Exception("Failed to read memory!")
Exception: Failed to read memory!
2022-04-19 02:25:29,473 | avatar.targets.QemuTarget0.INFO | State changed to TargetStates.RUNNING
2022-04-19 02:25:29,474 | avatar.INFO | Received state update of target QemuTarget0 to TargetStates.RUNNING
2022-04-19 02:25:29,484 | avatar.targets.QemuTarget0.INFO | State changed to TargetStates.EXITED
2022-04-19 02:25:29,484 | avatar.INFO | Received state update of target QemuTarget0 to TargetStates.EXITED
Arrived at main loop, demo is over
2022-04-19 02:25:29,965 | avatar.targets.QemuTarget0.RemoteMemoryProtocol.WARNING | Tried to close/unlink non existent rx_queue
2022-04-19 02:25:29,965 | avatar.targets.QemuTarget0.RemoteMemoryProtocol.WARNING | Tried to close/unlink non existent tx_queue
None

Hi,

setup.py ('python setup.py install --user --prefix=') fails on debian 9 with the message:

[ 95%] Building CXX object llvm/keystone/CMakeFiles/keystone.dir/__/lib/Target/X86/MCTargetDesc/X86AsmBackend.cpp.o
[ 95%] Building CXX object llvm/keystone/CMakeFiles/keystone.dir/__/lib/Target/X86/MCTargetDesc/X86ELFObjectWriter.cpp.o
[ 96%] Building CXX object llvm/keystone/CMakeFiles/keystone.dir/__/lib/Target/X86/MCTargetDesc/X86MCAsmInfo.cpp.o
[ 97%] Building CXX object llvm/keystone/CMakeFiles/keystone.dir/__/lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp.o
[ 97%] Building CXX object llvm/keystone/CMakeFiles/keystone.dir/__/lib/Target/X86/MCTargetDesc/X86MCTargetDesc.cpp.o
[ 98%] Building CXX object llvm/keystone/CMakeFiles/keystone.dir/__/lib/Target/X86/TargetInfo/X86TargetInfo.cpp.o
[ 99%] Building CXX object llvm/keystone/CMakeFiles/keystone.dir/ks.cpp.o
[100%] Linking CXX shared library ../lib/libkeystone.so
[100%] Built target keystone
error: Setup script exited with error: SandboxViolation: mkdir('/usr/lib/python2.7/dist-packages/keystone', 511) {}

Please find a patch solving this issue below.

Best,
mwenzl

setup.py.patch.txt

diff --git a/setup.py b/setup.py
index 29be93b..0a27039 100644
--- a/setup.py
+++ b/setup.py
@@ -1,5 +1,19 @@
 from setuptools import setup
 from sys import version_info
+from setuptools.command.install import install
+
+class CustomInstallCommand(install):
+    def run(self):
+        try:
+            from setuptools.sandbox import DirectorySandbox
+            def violation(operation, *args, **_):
+                print "SandboxViolation: %s" % (args,)
+            
+            DirectorySandbox._violation = violation
+        except ImportError:
+            pass
+        
+        return install.run(self)
 
 
 setup(
@@ -31,5 +45,6 @@ setup(
     url='https://github.com/avatartwo/avatar2',
     description='A Dynamic Multi-Target Orchestration Framework',
     maintainer='Marius Muench',
-    maintainer_email='[email protected]'
+    maintainer_email='[email protected]',
+    cmdclass={'install': CustomInstallCommand}
 )

Please, also support installing and running for Python 3.

Hey!

I have found several invalid references to GitHub repos while I was trying to install avatar2.

When I try to run python -m avatar2.installer, most targets, that are available from the package manager, install without a problem, however, others exited when the installer had tried to clone the given repos.

After debugging a little, I found out that branches for the targets' repos have been changed!

Hi, can i get a complete example?

from avatar2 import *

class HelloWorldPeripheral(AvatarPeripheral):

    def hw_read(self, offset, size):         
        ret = self.hello_world[:size]
        self.hello_world = self.hello_world[size:] + self.hello_world[:size]
        
        # Convert the return value to an integer (py2/py3-compatible)
        # Python >3.2 could just call int.from_bytes(ret, byteorder='little')
        s2fmt = {1: 'B', 2: 'H', 4: 'I', 8: 'Q')
        ret = struct.unpack('<' + s2fmt[size], ret)[0]

        return ret

    def nop_write(self, offset, size, value):
        return True    

    def __init__(self, name, address, size, **kwargs):
        AvatarPeripheral.__init__(self, name, address, size)
        
        self.hello_world=b'Hello World'
        
        self.read_handler[0:size] = self.hw_read 
        self.write_handler[0:size] = self.nop_write
        
[...]        

hw = avatar.add_memory_range(0x40004c00, 0x100, name='hello_world',
                             emulate=HelloWorldPeripheral, permissions='rw-')       

I want a working example, because I don't understand some functions very well, I hope I can refer to it.
Thank you so much!

I want to hook a function by using breakpoint and watchmen in avatar2. Then, I find that I can cont() the breakpoint in main function, but in watchmen callback, it has some issues. When I debug avatar2, I find cont() from watchmen callback will reach the code /avatar2/avatar2/targets/target.py line 77: state_reached.wait() and suspended with the variable state_reached._Event__flag = false. But, cont() in main at this code has the variable state_reached._Event__flag = true. Then, I don't debug in the threading.pyi. I find if blocking is True, the avatar2 will not in this wait() function. But, I find target.cont() will not pass the blocking information to this part. Than, I change the code in /avatar2/avatar2/targets/gdb_target.py line 82: from super(GDBTarget, self).cont() to super(GDBTarget, self).cont(blocking=blocking). And fix this problem. Hope it's useful for you. Have a nice day,

Hi there.

Avatar2 worked very well when I made a software-defined peripheral and accessed it with MMIO (Memory Mapped I/O). The run trace is the same as it runs on a real board. P.S. This firmware contains no interrupt handling routine.

Nowadays, I have decided to try firmware with interrupt handling procedures. I am using the old method, a software-defined peripheral for the lack of real board. But quickly, I got a problem. NVIC, the ARM's nested vector interrupt controller, is the core of the problem.

The Definitive Guide to ARM Cortex-M3 and Cortex-M4 Processors, 3rd Edition said that NVIC-related registers could be found in the system block, which is located from 0xE000E000 to 0xE000EFFF memory range. However, when I tried to modify nvic related registers mapped to this memory range, I found the memory write operation could not realize. Is this a problem for Qemu writing not in a privileged mode?

I have been stuck on this problem for several days. Could you give me an example of how to use avatar2 to trigger an external interrupt and jump to the interrupt service routine? Thanks a lot.

When I try to run python3 test_pyperipheral.py, I run into the following problem：

2022-04-19 06:33:03,450 | avatar.INFO | Initialized Avatar. Output directory is /tmp/avatar
2022-04-19 06:33:03,456 | avatar.targets.QemuTarget0.INFO | QEMU process running
2022-04-19 06:33:08,531 | avatar.targets.QemuTarget0.GDBProtocol.CRITICAL | GDBProtocol was unable to connect to remote target
Traceback (most recent call last):
File "test_pyperipheral.py", line 121, in
setup_func()
File "/home/desh0ng/.local/lib/python3.8/site-packages/nose/tools/nontrivial.py", line 97, in newfunc
result = func(*arg, **kw)
File "test_pyperipheral.py", line 47, in setup_func
avatar.init_targets()
File "/usr/local/lib/python3.8/dist-packages/avatar2-1.4.7-py3.8.egg/avatar2/avatar2.py", line 233, in init_targets
t[1].init()
File "/usr/local/lib/python3.8/dist-packages/avatar2-1.4.7-py3.8.egg/avatar2/watchmen.py", line 78, in watchtrigger
ret = func(self, *args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/avatar2-1.4.7-py3.8.egg/avatar2/targets/qemu_target.py", line 274, in init
self._connect_protocols()
File "/usr/local/lib/python3.8/dist-packages/avatar2-1.4.7-py3.8.egg/avatar2/targets/qemu_target.py", line 314, in _connect_protocols
connect_success = connect_success and gdb.remote_connect(port=self.gdb_port)
File "/usr/local/lib/python3.8/dist-packages/avatar2-1.4.7-py3.8.egg/avatar2/protocols/gdb.py", line 399, in remote_connect
return self._remote_connect_common('%s:%d' % (ip, int(port)))
File "/usr/local/lib/python3.8/dist-packages/avatar2-1.4.7-py3.8.egg/avatar2/protocols/gdb.py", line 384, in _remote_connect_common
raise Exception("GDBProtocol was unable to connect")
Exception: GDBProtocol was unable to connect

In container avatar2, I ran those python scripts:

from avatar2 import *
avatar = Avatar()
qemu = avatar.add_target(QemuTarget, name='qemu1')

and I got those error outputs (python trace has been ignored).

Have you tried running the avatar2-installer (python -m avatar2.installer) or setting the AVATAR2_QEMU_EXECUTABLE environment variable?

When a synchronous watchman hook is called after a qemu.cont, the QemuTarget's state is still TargetStates.STOPPED.

Sample script output:

$ python3 avatar_state_test.py
State in watchman before memory access: TargetStates.STOPPED
State in watchman after memory access: TargetStates.STOPPED
State in watchman before memory access: TargetStates.STOPPED
State in watchman after memory access: TargetStates.STOPPED
State in watchman before memory access: TargetStates.STOPPED
State in watchman after memory access: TargetStates.STOPPED
State in watchman before memory access: TargetStates.STOPPED
State in watchman after memory access: TargetStates.STOPPED
State in watchman before memory access: TargetStates.STOPPED
State in watchman after memory access: TargetStates.RUNNING
State in main thread after cont(): TargetStates.RUNNING
State in watchman before memory access: TargetStates.RUNNING
State in watchman after memory access: TargetStates.RUNNING
State in watchman before memory access: TargetStates.RUNNING
State in watchman after memory access: TargetStates.RUNNING
State in watchman before memory access: TargetStates.RUNNING
State in watchman after memory access: TargetStates.RUNNING
[...]

Sample script:

from avatar2 import *
from avatar2.peripherals import AvatarPeripheral

import os

class CustomMemoryPeripheral(AvatarPeripheral):

    def hw_read(self, offset, size):
        ret = self.hello_world[:size]
        self.hello_world = self.hello_world[size:] + self.hello_world[:size]
        return int.from_bytes(ret, 'little')

    def nop_write(self, offset, size, value):
        return True

    def __init__(self, name, address, size, **kwargs):
        AvatarPeripheral.__init__(self, name, address, size)

        self.hello_world = 'Hello World'

        self.read_handler[0:size] = self.hw_read
        self.write_handler[0:size] = self.nop_write

def before_remote_memory_access(avatar, remote_memory_msg, **kwargs):
    print('State in watchman before memory access: %s' % qemu.state)

def after_remote_memory_access(avatar, remote_memory_msg, **kwargs):
    print('State in watchman after memory access: %s' % qemu.state)

def main():
    global qemu

    dir_path = os.path.dirname(os.path.realpath(__file__))

    avatar = Avatar()

    # Text section
    avatar.add_memory_range(0x00000000, 0x00013000, file=dir_path+'/sample.bin')

    # Watchmen
    avatar.watchmen.add_watchman('RemoteMemoryRead', 'before', before_remote_memory_access, is_async=False)
    avatar.watchmen.add_watchman('RemoteMemoryWrite', 'before', before_remote_memory_access, is_async=False)

    avatar.watchmen.add_watchman('RemoteMemoryRead', 'after', after_remote_memory_access, is_async=False)
    avatar.watchmen.add_watchman('RemoteMemoryWrite', 'after', after_remote_memory_access, is_async=False)

    # Memory Peripheral
    avatar.add_memory_range(0x00013000, 0xfffecfff, name='unmapped_memory', emulate=CustomMemoryPeripheral, permission='rw-')

    # QEMU CONFIG
    qemu = avatar.add_target(QemuTarget, name='qemu1')
    qemu.entry_address = 0x100
    qemu.cpu_model = 'cortex-a8'

    qemu.init()
    qemu.cont()
    print('State in main thread after cont(): %s' % qemu.state)
    qemu.wait()
    qemu.shutdown()

if __name__ == '__main__':
    main()

Circumstances

I am using the GDBTarget target to connect to QEMU. I can't use the built-in QEMU target because QEMU gets spawned from another process. There seems to be an issue with synchronous hooks being async. I have the following testcase to reproduce the issue:

import avatar2
import os
import logging
import sys
import time

from nose.tools import *


def test_race():

    def hook_callback(avatar, *args, **kwargs):
        gdb = avatar.targets['gdbtest']
        pc = gdb.read_register("pc")
        assert pc is not None, f"ILLEGAL STATE {gdb.get_status()}"
        print(pc)

        if pc == 0x1000:
            sys.exit()

    avatar = avatar2.Avatar(arch=avatar2.ARM)
    gdb = avatar.add_target(avatar2.GDBTarget,
                              name='gdbtest',
                              gdb_port=1234,
                              gdb_verbose_mi=True,
                              gdb_executable='/usr/bin/gdb-multiarch'
                            )

    # add breakpoint callback
    avatar.watchmen.add('BreakpointHit', when='after', callback=hook_callback, is_async=False)


    print("Init avatar targets...")
    avatar.init_targets()

    gdb.set_breakpoint(0x4)

    gdb.write_register('pc', 0)
    # Start running
    gdb.cont()

    # wait until we hit a breakpoint, once we hit the breakpoint, continue this python script
    print("waiting until we hit a breakpoint")
    gdb.wait()
    # add two breakpoints
    gdb.set_breakpoint(0x8)
    gdb.set_breakpoint(0xc)

    gdb.set_breakpoint(0x1000)

    # Continue executing from main
    gdb.cont()
    time.sleep(1)
    while True:
        # Wait until we hit a breakpoint
        gdb.wait()
        gdb.cont()


    avatar.shutdown()

if __name__ == '__main__':
    test_race()

This results in this crash:

redacted@5d755f437052:~/avatar_bug$ python3 target_wait.py 
Init avatar targets...
waiting until we hit a breakpoint
4
8
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.8/dist-packages/avatar2/avatar2.py", line 400, in run
    handler(message)
  File "/usr/local/lib/python3.8/dist-packages/avatar2/watchmen.py", line 80, in watchtrigger
    cb_ret = avatar.watchmen.t(watched_type, AFTER, *args, **cb_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/avatar2/watchmen.py", line 185, in trigger
    watchman.react(self._avatar, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/avatar2/watchmen.py", line 126, in react
    ret = self._callback(avatar, *args, **kwargs)
  File "target_wait.py", line 18, in hook_callback
    assert pc is not None, f"ILLEGAL STATE {gdb.get_status()}"
AssertionError: ILLEGAL STATE {'state': <TargetStates.RUNNING: 8>}

See also the following trace with gdb_verbose_mi enabled (around message 15-16):

writing: 13-data-list-register-values x 15
{'message': 'done',
 'payload': {'register-values': [{'number': '15', 'value': '0x8'}]},
 'stream': 'stdout',
 'token': 13,
 'type': 'result'}
8
writing: 14-exec-continue
{'message': 'running',
 'payload': None,
 'stream': 'stdout',
 'token': 14,
 'type': 'result'}
{'message': 'running',
 'payload': {'thread-id': 'all'},
 'stream': 'stdout',
 'token': None,
 'type': 'notify'}
{'message': 'breakpoint-modified',
 'payload': {'bkpt': {'addr': '0x0000000c',
                      'disp': 'keep',
                      'enabled': 'y',
                      'number': '3',
                      'original-location': '*0xc',
                      'thread-groups': ['i1'],
                      'times': '1',
                      'type': 'breakpoint'}},
 'stream': 'stdout',
 'token': None,
 'type': 'notify'}
{'message': None, 'payload': '\\n', 'stream': 'stdout', 'type': 'console'}
{'message': None,
 'payload': 'Breakpoint 3, 0x0000000c in ?? ()\\n',
 'stream': 'stdout',
 'type': 'console'}
{'message': 'stopped',
 'payload': {'bkptno': '3',
             'disp': 'keep',
             'frame': {'addr': '0x0000000c',
                       'arch': 'arm',
                       'args': [],
                       'func': '??'},
             'reason': 'breakpoint-hit',
             'stopped-threads': 'all',
             'thread-id': '1'},
 'stream': 'stdout',
 'token': None,
 'type': 'notify'}
writing: 15-exec-continue
writing: 16-data-list-register-values x 15
{'message': 'running',
 'payload': None,
 'stream': 'stdout',
 'token': 15,
 'type': 'result'}
{'message': 'running',
 'payload': {'thread-id': 'all'},
 'stream': 'stdout',
 'token': None,
 'type': 'notify'}
{'message': 'error',
 'payload': {'msg': 'Selected thread is running.'},
 'stream': 'stdout',
 'token': 16,
 'type': 'result'}
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
{'message': 'breakpoint-modified',
 'payload': {'bkpt': {'addr': '0x00000008',
                      'disp': 'keep',
                      'enabled': 'y',
                      'number': '2',
                      'original-location': '*0x8',
                      'thread-groups': ['i1'],
                      'times': '2',
                      'type': 'breakpoint'}},
 'stream': 'stdout',
 'token': None,
 'type': 'notify'}
    self.run()
  File "/usr/local/lib/python3.8/dist-packages/avatar2/avatar2.py", line 400, in run
{'message': None, 'payload': '\\n', 'stream': 'stdout', 'type': 'console'}
{'message': None,
 'payload': 'Breakpoint 2, 0x00000008 in ?? ()\\n',
 'stream': 'stdout',
 'type': 'console'}
{'message': 'stopped',
 'payload': {'bkptno': '2',
             'disp': 'keep',
             'frame': {'addr': '0x00000008',
                       'arch': 'arm',
                       'args': [],
                       'func': '??'},
             'reason': 'breakpoint-hit',
             'stopped-threads': 'all',
             'thread-id': '1'},
 'stream': 'stdout',
 'token': None,
 'type': 'notify'}
    handler(message)
  File "/usr/local/lib/python3.8/dist-packages/avatar2/watchmen.py", line 80, in watchtrigger
    cb_ret = avatar.watchmen.t(watched_type, AFTER, *args, **cb_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/avatar2/watchmen.py", line 185, in trigger
    watchman.react(self._avatar, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/avatar2/watchmen.py", line 126, in react
    ret = self._callback(avatar, *args, **kwargs)
  File "target_wait.py", line 15, in hook_callback
    assert pc is not None, f"ILLEGAL STATE {gdb.get_status()}"
AssertionError: ILLEGAL STATE {'state': <TargetStates.RUNNING: 8>}

Steps to reproduce this

1. Run qemu-system-arm -machine virt -gdb tcp::1234 -S -nographic -bios qemu_arm_test in a separate terminal. The qemu_arm_test binary is in avatar2/tests/binaries of this repository.
2. Run the Python script, for me it crashes every time (I have 4 CPU cores)

Expected behaviour

Machine state is STOPPED in synchronous hooks

Actual behaviour

Synchronous hooks act like async hooks?

when I want to connect QemuTarget I face to this error

Hi,
I run the script 'nucleo_state_transfer.py' in avatar2-examples, but using a different firmware which enables a button interrupt. I want avatar2 to monitor and automatically transfer to execution to qemu when a button interrupt comes. I have tried to set the breakpoint in the interrupt handler, and then transfering the execution to qemu. However, when I step the execution it crashs, when jumping out of the interrupt handler(pc goes 0xfffffff8).
Then I find some instructions in armv7m_interrupt.py: 'extracting interrupt exits and putting them into the avatar queue'. However, I don't know how to use it.
Could you give me some advice on how to achieve ideas above?
Thanks!

Hi,

I am trying to make a dummy flash controller for an ARM cortex-m3 based MCU that I emulate with QEMU. The goal is to retrieve the different parameters (address, value, size) and then modify the corresponding memory region that I allocated using avatar.add_memory_range(). I took inspiration from the example in the repo avatar2/avatar2_example so I made read and write handlers in which I perform the different actions depending on the offset and the values.

My problem is that when I use qemu.write_memory() in my controller read/write handlers, nothing happens. I can read the memory afterwards and I notice no changes. However, when I modify the memory using the same API after I reach a breakpoint, the memory is modified as expected.

Any idea how I could achieve these memory modifications in my controller?

Here is an extract of my code:

    def do_flash_op(self):
        if self.opcode == FlashOpcode.NONE:
            print("Warning: FlashOp: wrong op code, doing nothing")

        elif self.opcode == FlashOpcode.WRITE or self.opcode == FlashOpcode.ERASE:
            print("Write flash at " + hex(self.flashAddr) + " (" + hex(self.nwords) + " words) in bank " + hex(self.bank))
            for x in range(0, self.nwords):
                print(qemu.write_memory(self.flashAddr + x, 4, self.wordsToWrite[x]))
        else:
            print("Read bank " + str(self.bank) + " (info=" + str(self.isInfo) +") op " + str(self.opcode) + " words " + str(self.nwords) + " at " + hex(self.flashAddr))


    def handle_read(self, offset, size):
        if (offset == 0xc):
            return self.offsetValue
        return 0x00 # TODO: read info bank

    def handle_write(self, offset, size, value):
        if offset == 0x4 or offset == 0x8: # Control register
            if offset == 0x4:
                self.bank = 0
            else:
                self.bank = 1

            if value == 0x31415927: # Erase
                self.opcode = FlashOpcode.ERASE
            elif value == 0x27182818: # Write
                self.opcode = FlashOpcode.WRITE
            elif value == 0x16021765: # Read (only for info bank)
                self.opcode = FlashOpcode.READ
            self.parse_offset()
            self.do_flash_op()
            self.reset_regs()

Thanks

Hi
I simulated lm3s6965 that write with UART in output (hello world)
1-in QemuTarget mode ,anything write in output why?
2-after I transfer_state from QemuTarget to gdbTarget and I want go foeward one command with gdb.step() its get into halt without any error and i dont know what i do?

I get an error when I use the command：
qemu-arm -singlestep -g 1235 ./vuln3_debug

Code：

import os
import subprocess

from avatar2 import *


filename = 'vuln3_debug'
GDB_PORT = 1235   

shellcode = (b'\x68\x72\x6c\x64\x21\x48\xb8\x48\x65\x6c\x6c\x6f\x20\x57\x6f\x50'
             b'\x48\x89\xef\x48\x89\xe6\x6a\x0c\x5a\x6a\x01\x58\x0f\x05')
 

# Create the avatar instance and specify the architecture for this analysis
avatar = Avatar(arch=archs.arm.ARM)

# Create the endpoint: a gdbserver connected to our tiny ELF file
gdbserver = subprocess.Popen('qemu-arm -g %d vuln3_debug' % GDB_PORT, shell=True)

# Create the corresponding target, using the GDBTarget backend
target = avatar.add_target(GDBTarget, gdb_port=GDB_PORT)

target = avatar.add_target(QemuTarget, 
    gdb_port=GDB_PORT,executable = "/usr/bin/qemu-arm",
    gdb_executable = "gdb-multiarch")

# Initialize the target. 
# This usually connects the target to the endpoint
target.init()
bkpt = qemu.set_breakpoint(0x103fe)
# Now it is possible to interact with the target.
# For example, we can insert our shellcode at the current point of execution
target.write_memory(target.read_register('pc'), len(shellcode),
                    shellcode, raw=True)

# We can now resume the execution in our target
# You should see hello world printed on your screen! :)
target.cont()

# Clean up!
#os.remove(filename)
avatar.shutdown()

error：

➜  my_code python3 x86_fuzz.py
Traceback (most recent call last):
  File "x86_fuzz.py", line 29, in <module>
    target.init()
  File "/usr/local/lib/python3.6/dist-packages/avatar2/watchmen.py", line 78, in watchtrigger
    ret = func(self, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/avatar2/targets/qemu_target.py", line 206, in init
    if gdb.remote_connect(port=self.gdb_port) and qmp.connect():
  File "/usr/local/lib/python3.6/dist-packages/avatar2/protocols/qmp.py", line 31, in connect
    self._telnet = telnetlib.Telnet('127.0.0.1', self.port)
  File "/usr/lib/python3.6/telnetlib.py", line 218, in __init__
    self.open(host, port, timeout)
  File "/usr/lib/python3.6/telnetlib.py", line 234, in open
    self.sock = socket.create_connection((host, port), timeout)
  File "/usr/lib/python3.6/socket.py", line 724, in create_connection
    raise err
  File "/usr/lib/python3.6/socket.py", line 713, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
qemu-arm: QEMU: Terminated via GDBstub

Can someone tell me how to solve this error, thank you very much

Hey!

After tinkering with avatar2, I still cannot really debug a ESP32.:(

I have a ESP32 with a very simple firmware flashed to it, I am using a ES-Prog as a JTAG adapter. Following the demonstration video from ACM WiSec 2021, I copied the followint code:

from avatar2 import Avatar
from avatar2.archs import #architeture
from avatar2.targets import OpenOCDTarget

avatar = Avatar(arch= , output_directory='/tmp/avatar2')

#Specify target
esp = avatar.add_target(OpenOCDTarget, openocd_script='')

#Add memory
#Skip

avatar.init_targets()

#esp.set_breakpoint()
esp.cont()
esp.wait()

import IPython; IPython.embed()

However, I do not know what architecture should I define fot the ESP32 and what OpenOCD target script should I use. Any help is appreciated!

I want to use python -m avatar2.installer to build avatar-panda,but I got those error(Ubuntu 14.04 64bit):
Cloning into '/home/XXXX/.avatar2/avatar-panda'...
remote: Enumerating objects: 1938, done.
remote: Counting objects: 100% (1938/1938), done.
remote: Compressing objects: 100% (659/659), done.
remote: Total 311315 (delta 1355), reused 1707 (delta 1262), pack-reused 309377
Receiving objects: 100% (311315/311315), 129.64 MiB | 3.72 MiB/s, done.
Resolving deltas: 100% (248868/248868), done.
Checking connectivity... done.
Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc'
Cloning into 'dtc'...
remote: Counting objects: 4807, done.
remote: Compressing objects: 100% (2194/2194), done.
remote: Total 4807 (delta 3561), reused 3434 (delta 2571)
Receiving objects: 100% (4807/4807), 1.04 MiB | 241.00 KiB/s, done.
Resolving deltas: 100% (3561/3561), done.
Checking connectivity... done.
Submodule path 'dtc': checked out 'ec02b34c05be04f249ffaaca4b666f5246877dea'

ERROR: sizeof(size_t) doesn't match GLIB_SIZEOF_SIZE_T.
You probably need to set PKG_CONFIG_LIBDIR
to point to the right pkg-config files for your
build target

I check the configure file of avatar-panda and find the check:
if ! compile_prog "$CFLAGS" "$LIBS" ; then error_exit "sizeof(size_t) doesn't match GLIB_SIZEOF_SIZE_T."\ "You probably need to set PKG_CONFIG_LIBDIR"\ "to point to the right pkg-config files for your"\ "build target" fi
Howerver, I don not know how to solve it :( .
Thanks for your help.