keystone-engine / keystone Goto Github PK

Keystone assembler framework: Core (Arm, Arm64, Hexagon, Mips, PowerPC, Sparc, SystemZ & X86) + bindings

Home Page: http://www.keystone-engine.org

License: GNU General Public License v2.0

CMake 0.75% Makefile 0.03% Python 0.74% C 0.69% C++ 84.43% Shell 0.05% Batchfile 0.01% JavaScript 0.04% Ruby 0.03% Go 0.04% Rust 0.05% Haskell 0.07% OCaml 0.98% PowerShell 0.03% C# 0.12% Assembly 0.15% Java 0.25% VBA 0.03% NASL 11.49% Visual Basic 6.0 0.03%

assembler framework security reverse-engineering arm arm64 x86 sparc mips powerpc systemz hexagon x86-64

keystone's Introduction

Keystone Engine

Keystone is a lightweight multi-platform, multi-architecture assembler framework. It offers some unparalleled features:

Multi-architecture, with support for Arm, Arm64 (AArch64/Armv8), Ethereum Virtual Machine, Hexagon, Mips, PowerPC, RISC-V, Sparc, SystemZ & X86 (include 16/32/64bit).
Clean/simple/lightweight/intuitive architecture-neutral API.
Implemented in C/C++ languages, with bindings for Java, Masm, C#, PowerShell, Perl, Python, NodeJS, Ruby, Go, Rust, Haskell, VB6 & OCaml available.
Native support for Windows & *nix (with Mac OSX, Linux, *BSD & Solaris confirmed).
Thread-safe by design.
Open source - with a dual license.

Keystone is based on LLVM, but it goes much further with a lot more to offer.

Further information is available at http://www.keystone-engine.org

License

Keystone is available under a dual license:

Version 2 of the GNU General Public License (GPLv2). (I.e. Without the "any later version" clause.). License information can be found in the COPYING file and the EXCEPTIONS-CLIENT file.

This combination allows almost all of open source projects to use Keystone without conflicts.
For commercial usage in production environments, contact the authors of Keystone to buy a royalty-free license.

See LICENSE-COM.TXT for more information.

Compilation & Docs

See COMPILE.md file for how to compile and install Keystone.

More documentation is available in docs/README.md.

Contact

Contribute

Keystone is impossible without generous support from our sponsors. We cannot thank them enough!

CREDITS.TXT records other important contributors of our project.

If you want to contribute, please pick up something from our Github issues.

We also maintain a list of more challenged problems in a TODO list.

keystone's People

Contributors

Stargazers

Watchers

Forkers

aziem mewbak kz9 datasweeper matthewxie lic91 zhftao tempbottle breagitt ta01st ezhangle ieswxia sigma-random f8left logan-lu mclovi9 feomao lunixbochs polelf cynecx brainiarc7 fvrmatteo nnamon bigendiansmalls sarikayamehmet brendabrandy redteamcaliber zachlungu layerfsd bjzz quangnh89 caineqt nsxz aisling-kells hkingz avgirl seth1002 zchee redd0glee mrmugiwara piaoyunsoft ognz 8l xxoxx unxi0101 lyhlyhlyhlyh2008 nightoftwelve jack51706 echotyh houweifeng firewolf-ljw mrexodia girtsf 0x66eng evoliptic dmxcsnsbh xn0px90 j123123 techlord-rce shakamd sbhamad spnow neoz evil-e smartvirus-alex computerskys venkatarajasekhar noword krystalgamer luciouskami ldyeax rflaperuta wsxarcher jatinkataria sinsixx aagallag c8h10n4o2ed shuixi2013 srinathv h3ph4est7s primekun ghetto2199 dwarfcrank suramya hayate891 xiaohuajiao yws nevermoe hoangvantu celestialwy idkwim codingsf zjnixiang rajkosto maelstrompli joseph-giron houcy jameslinus nicholas1126 pvsaas

keystone's Issues

Make archs selectable

loop 0x7ff6bf912aad

0x00007FF6BF912A84 "loop 0x7ff6bf912aad" failed to assemble (ks_asm() failed: count = 0, error = 512):
    Expected (02) E2 27

RIP-relative addressing using nasm syntax

The syntax for RIP-relative addressing works different in nasm. F.e.: lea rax, [rel label1].

There's also a directive to switch between rip relative addressing by default (DEFAULT REL) and back to absolute addressing (DEFAULT ABS).

I'll send a PR for a regress test later today.

Add pkg-config support

Crash in llvm::MCAssembler::registerSymbol()

The following input crashes kstool

kstool ppc64 (echo "bN3XeGVadMzMzMzMdAd3YyYmJiYmJgYmJv///4AmJiYmBCYmcv9///8xDS50EHdVbVgNLszMzMzMzMzMc2IDbVgfbDENk3QQeDENRw0udBB3Y20+DS50ZXh0AMzMzMx0B3djJiYmJiYmBiYm////gCYmJiYmJiYmJiYm////gCYmJiYmJiYmJiYmJiYmAXExDS50B3djbQ==" | base64 -d)

input content:

00000000  6c dd d7 78 65 5a 74 cc  cc cc cc cc 74 07 77 63  |l..xeZt.....t.wc|
00000010  26 26 26 26 26 26 06 26  26 ff ff ff 80 26 26 26  |&&&&&&.&&....&&&|
00000020  26 04 26 26 72 ff 7f ff  ff 31 0d 2e 74 10 77 55  |&.&&r....1..t.wU|
00000030  6d 58 0d 2e cc cc cc cc  cc cc cc cc 73 62 03 6d  |mX..........sb.m|
00000040  58 1f 6c 31 0d 93 74 10  78 31 0d 47 0d 2e 74 10  |X.l1..t.x1.G..t.|
00000050  77 63 6d 3e 0d 2e 74 65  78 74 00 cc cc cc cc 74  |wcm>..text.....t|
00000060  07 77 63 26 26 26 26 26  26 06 26 26 ff ff ff 80  |.wc&&&&&&.&&....|
00000070  26 26 26 26 26 26 26 26  26 26 26 ff ff ff 80 26  |&&&&&&&&&&&....&|
00000080  26 26 26 26 26 26 26 26  26 26 26 26 26 01 71 31  |&&&&&&&&&&&&&.q1|
00000090  0d 2e 74 07 77 63 6d                              |..t.wcm|

This crash is related to ELF section parsing and goes away when applying #81. The address of Symbol is set to 0x80000000 which seems to be a default value when looking at the backtrace. The program crashes when trying to call isRegistered() for that symbol.

360 void MCAssembler::registerSymbol(const MCSymbol &Symbol, bool *Created) {
361   bool New = !Symbol.isRegistered();

gdb-peda$ p Symbol
$1 = (const llvm::MCSymbol &) @0x80000000: <error reading variable>

backtrace

#0  llvm::MCSymbol::isRegistered (this=0x80000000) at ../llvm/include/llvm/MC/MCSymbol.h:206
#1  0x000000000047b225 in llvm::MCAssembler::registerSymbol (this=0xe7e5a0, Symbol=..., Created=0x0) at ../llvm/lib/MC/MCAssembler.cpp:361
#2  0x0000000000471f1b in llvm::MCELFStreamer::ChangeSection (this=0xe7e410, Section=0xe83030, Subsection=0x0) at ../llvm/lib/MC/MCELFStreamer.cpp:149
#3  0x00000000004ed2f1 in llvm::MCStreamer::SwitchSection (this=0xe7e410, Section=0xe83030, Subsection=0x0) at ../llvm/lib/MC/MCStreamer.cpp:729
#4  0x00000000004df963 in (anonymous namespace)::DarwinAsmParser::parseSectionSwitch (this=0xe7f640, Segment=0x801178 "__TEXT", Section=0x80118a "__text", TAA=0x80000000, Align=0x0, StubSize=0x0)
    at ../llvm/lib/MC/MCParser/DarwinAsmParser.cpp:393
#5  0x00000000004e2937 in (anonymous namespace)::DarwinAsmParser::parseSectionDirectiveText (this=0xe7f640) at ../llvm/lib/MC/MCParser/DarwinAsmParser.cpp:361
#6  0x00000000004e29e9 in llvm::MCAsmParserExtension::HandleDirective<(anonymous namespace)::DarwinAsmParser, &(anonymous namespace)::DarwinAsmParser::parseSectionDirectiveText> (Target=0xe7f640, Directive=..., DirectiveLoc=...)
    at ../llvm/include/llvm/MC/MCParser/MCAsmParserExtension.h:38
#7  0x00000000004ae9a5 in (anonymous namespace)::AsmParser::parseStatement (this=0xe7f1a0, Info=..., SI=0x0, Address=@0x7fffffffcdb0: 0x0) at ../llvm/lib/MC/MCParser/AsmParser.cpp:1619
#8  0x00000000004a3c9c in (anonymous namespace)::AsmParser::Run (this=0xe7f1a0, NoInitialTextSection=0x0, Address=0x0, NoFinalize=0x0) at ../llvm/lib/MC/MCParser/AsmParser.cpp:705
#9  0x000000000046e003 in ks_asm (ks=0xe775e0, 
    assembly=0x7fffffffdaf0 "l\335\327xeZt\314\314\314\314\314t\awc&&&&&&\006&&\377\377\377\200&&&&\004&&r\377\177\377\377\061\r.t\020wUmX\r.\314\314\314\314\314\314\314\314sb\003mX\037l1\r\223t\020x1\rG\r.t\020wcm>\r.text", 
    address=0x0, insn=0x7fffffffdc30, insn_size=0x7fffffffdc28, stat_count=0x7fffffffdc38) at ../llvm/keystone/ks.cpp:551
#10 0x000000000046b740 in main (argc=0x2, argv=0x7fffffffdd58) at ../kstool/kstool-stdin.cpp:214
#11 0x00007ffff718d830 in __libc_start_main (main=0x46b020 <main(int, char**)>, argc=0x2, argv=0x7fffffffdd58, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffdd48) at ../csu/libc-start.c:291
#12 0x000000000046af49 in _start ()

Segmentation Fault on Ubuntu 14.04 64bit

I just followed the COMPILE-NIX.md and did a quick test kstool x32 "add eax, ebx". But it crashed on my machine.

The crash:
`gdb-peda$ r x32 "add eax, ebx"
Starting program: /usr/local/bin/kstool x32 "add eax, ebx"

Program received signal SIGSEGV, Segmentation fault.
Stopped reason: SIGSEGV
std::string::swap (this=this@entry=0x6039f0, __s="i386") at /build/gcc-4.8-_Vitw3/gcc-4.8-4.8.4/build/x86_64-linux-gnu/libstdc++-v3/include/bits/basic_string.tcc:521
521 /build/gcc-4.8-_Vitw3/gcc-4.8-4.8.4/build/x86_64-linux-gnu/libstdc++-v3/include/bits/basic_string.tcc: No such file or directory.

gdb-peda$ bt
#0 std::string::swap (this=this@entry=0x6039f0, __s="i386") at /build/gcc-4.8-_Vitw3/gcc-4.8-4.8.4/build/x86_64-linux-gnu/libstdc++-v3/include/bits/basic_string.tcc:521
#1 0x00007ffff6c45829 in std::string::operator=(std::string&&) (this=0x6039f0, __str=) at /build/gcc-4.8-_Vitw3/gcc-4.8-4.8.4/build/x86_64-linux-gnu/libstdc++-v3/include/bits/basic_string.h:583
#2 0x00007ffff777e561 in InitKs (ks=0x6039d0, TripleName=...) at /home/vps/git/beta/llvm/keystone/ks.cpp:131
#3 0x00007ffff777e344 in ks_open (arch=KS_ARCH_X86, mode=KS_MODE_MIPS32, result=0x7fffffffe8e8) at /home/vps/git/beta/llvm/keystone/ks.cpp:366
#4 0x0000000000400c46 in main (argc=0x3, argv=0x7fffffffea18) at /home/vps/git/beta/kstool/kstool.c:62
#5 0x00007ffff6eb1ec5 in __libc_start_main (main=0x400b9f

, argc=0x3, argv=0x7fffffffea18, init=, fini=, rtld_fini=, stack_end=0x7fffffffea08) at libc-start.c:287
#6 0x00000000004009a9 in _start ()

gdb-peda$ frame 2
#2 0x00007ffff777e561 in InitKs (ks=0x6039d0, TripleName=...) at /home/vps/git/beta/llvm/keystone/ks.cpp:131

131 ks->TripleName = Triple::normalize(TripleName);`

Missing instruction `int1`

Which assembles to F1

segfault in X86AsmParser::InfixCalculator::execute

A simple way to crash kstool:

kstool x32 "add rax, 1*"

The crashing line in X86AsmParser.cpp:

return OperandStack.pop_back_val().second;

From what I understand llvm tries to parse '1*' as an infix operator but it is missing a value after the *. I need to go to bed now but I can investigate more tomorrow.

backtrace:

gdb-peda$ bt
#0  (anonymous namespace)::X86AsmParser::InfixCalculator::execute (this=this@entry=0x7fffffffc1a0)
    at /home/ekse/git/keystone/llvm/lib/Target/X86/AsmParser/X86AsmParser.cpp:242
#1  0x00007ffff77ee288 in (anonymous namespace)::X86AsmParser::IntelExprStateMachine::getImm (this=0x7fffffffc160)
    at /home/ekse/git/keystone/llvm/lib/Target/X86/AsmParser/X86AsmParser.cpp:288
#2  (anonymous namespace)::X86AsmParser::ParseIntelOperand (this=this@entry=0x6227a0, Mnem=...)
    at /home/ekse/git/keystone/llvm/lib/Target/X86/AsmParser/X86AsmParser.cpp:1836
#3  0x00007ffff77ef152 in (anonymous namespace)::X86AsmParser::ParseOperand (this=this@entry=0x6227a0, Mnem=...)
    at /home/ekse/git/keystone/llvm/lib/Target/X86/AsmParser/X86AsmParser.cpp:1149
#4  0x00007ffff77f143b in (anonymous namespace)::X86AsmParser::ParseInstruction (this=this@entry=0x6227a0, Info=..., Name=..., NameLoc=..., 
    Operands=..., ErrorCode=@0x7fffffffd020: 0x0) at /home/ekse/git/keystone/llvm/lib/Target/X86/AsmParser/X86AsmParser.cpp:2354
#5  0x00007ffff7243efc in llvm::MCTargetAsmParser::ParseInstruction (ErrorCode=@0x7fffffffd020: 0x0, Operands=..., 
    Token=<error reading variable: access outside bounds of object referenced via synthetic pointer>, Name=..., Info=..., this=0x6227a0)
    at /home/ekse/git/keystone/llvm/include/llvm/MC/MCParser/MCTargetAsmParser.h:162
#6  (anonymous namespace)::AsmParser::parseStatement (this=this@entry=0x6201b0, Info=..., SI=SI@entry=0x0, Address=@0x7fffffffcf98: 0x0)
    at /home/ekse/git/keystone/llvm/lib/MC/MCParser/AsmParser.cpp:1863
#7  0x00007ffff724ae46 in (anonymous namespace)::AsmParser::Run (this=0x6201b0, NoInitialTextSection=<optimized out>, Address=0x0, 
    NoFinalize=<optimized out>) at /home/ekse/git/keystone/llvm/lib/MC/MCParser/AsmParser.cpp:705
#8  0x00007ffff77fdca5 in ks_asm (ks=0x6185e0, assembly=assembly@entry=0x7fffffffe0f5 "add rax, 1*", address=address@entry=0x0, 
    insn=insn@entry=0x7fffffffdc68, insn_size=insn_size@entry=0x7fffffffdc70, stat_count=stat_count@entry=0x7fffffffdc60)
    at /home/ekse/git/keystone/llvm/keystone/ks.cpp:547
#9  0x0000000000401da2 in main (argc=argc@entry=0x3, argv=argv@entry=0x7fffffffdd88) at /home/ekse/git/keystone/kstool/kstool.cpp:211
#10 0x00007ffff6bc1830 in __libc_start_main (main=0x400d00 <main(int, char**)>, argc=0x3, argv=0x7fffffffdd88, init=<optimized out>, 
    fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffdd78) at ../csu/libc-start.c:291
#11 0x0000000000402c79 in _start ()

Debug:

(anonymous namespace)::X86AsmParser::InfixCalculator::execute (this=this@entry=0x7fffffffc1a0) at /home/ekse/git/keystone/llvm/lib/Target/X86/AsmParser/X86AsmParser.cpp:165
165         InfixCalculatorTok StackOp = InfixOperatorStack.pop_back_val();
gdb-peda$ next
gdb-peda$ p StackOp 
$9 = (anonymous namespace)::X86AsmParser::IC_MULTIPLY

236             OperandStack.push_back(std::make_pair(IC_IMM, Val))

Invalid argument for branch/call on x86

This list of instructions cant be assembled right now:

call 5
jg 3
jae 10
jb 4
...

$ kstool x32 'call 5'
ERROR: failed on ks_asm() with count = 0, error = 'Invalid operand (KS_ERR_ASM_INVALIDOPERAND)' (code = 512)

comiss xmm1, xmmword ptr [rip + 0x1ca5681]

0x00007FF6BF90CACF "comiss xmm0, xmmword ptr [rip + 0x1ca568a]" failed to assemble (ks_asm() failed: count = 0, error = 512):
    Expected (07) 0F 2F 05 8A 56 CA 01 
0x00007FF6BF90CAE8 "comiss xmm1, xmmword ptr [rip + 0x1ca5681]" failed to assemble (ks_asm() failed: count = 0, error = 512):
    Expected (07) 0F 2F 0D 81 56 CA 01

this is how capstone disassembles this instruction

No opcode emitted when using intel/nasm syntax arithmetic

When using an arithmetic expression that references a symbol, no opcode is emitted (strangely enough, no error or warning either):

$ kstool x32 "nop; sub eax, bar+5; bar:; nop"
Kstool v1.0 for Keystone Engine (www.keystone-engine.org)

nop; sub eax, bar+5; bar:; nop = [ 90 90 ]

operand size override (push 0xd)

push 0xd outputs 66 6A 0D which is push word 0xd the correct output is 6A 0D (push a dword)

Testing notes

I did some quick testing tonight on Ubuntu 15.10 x64. Keystone built and installed correctly, however /usr/lib/local is not in the ld path by default on Ubuntu. I had to add it to /etc/ld.so.conf and then run sudo ldconfig for kstool to run. You might want to add a note about this in the readme.

The python bindings are also working, both the python2 and python3 version. If you want sample.py to also work for python3 the asm strings need to be declared as bytes literals (eg. b"add eax, ecx"), else a type error is thrown. The b' ' prefix is ignored in python2 so it's not a problem to add it.

Traceback (most recent call last):
  File "sample.py", line 32, in <module>
    test_ks(KS_ARCH_ARM, KS_MODE_ARM, "sub r1, r2, r5")
  File "sample.py", line 15, in test_ks
    encoding, count = ks.asm(code)
  File "/home/ekse/git/keystone/bindings/python/keystone/keystone.py", line 189, in asm
    status = _ks.ks_asm(self._ksh, string, addr, byref(encode), byref(encode_size), byref(stat_count))
ctypes.ArgumentError: argument 2: <class 'TypeError'>: wrong type

fstp st(1), st(0)

0x00007FF6BF90DA6C "fstp st(1), st(0)" failed to assemble (ks_asm() failed: count = 0, error = 512):
    Expected (02) DF D9

ljmp ptr [rcx + 1]

0x00007FF6BF916AA4 "ljmp ptr [rcx + 1]" failed to assemble (ks_asm() failed: count = 0, error = 512):
    Expected (03) FF 69 01 
0x00007FF6BF91AB2C "ljmp ptr [rdx - 0x54faffff]" failed to assemble (ks_asm() failed: count = 0, error = 512):
    Expected (06) FF AA 01 00 05 AB

Use 2 char spacing to be in sync with llvm

Mode x86-64 does not handle integers > 2^31

Hi,

First of, I love the library. Awesome job!
When working with it, I noticed that when in mode x86-64 (ie mode=KS_MODE_X86 arch=KS_MODE_64+KS_MODE_LITTLE_ENDIAN) keystone fails to encode integers higher that 2^31-1.

Quick repro:

~/tools/keystone/build $ kstool/kstool x64 "mov rdi, `python -c 'print 2**31-1'`"
mov rdi, 2147483647 = [ 48 c7 c7 ff ff ff 7f ]
~/tools/keystone/build $ kstool/kstool x64 "mov rdi, `python -c 'print 2**31'`"
ERROR: failed on ks_asm() with count = 0, error = 'Invalid operand KS_ERR_ASM_INVALIDOPERAND)' (code = 512)

I'll check tomorrow if I can submit a PR.

Thanks again for your work.

Possible heap-buffer-overflows detected by libfuzzer

Hi Quynh,

I've been running a few tests using LibFuzzer. Even though no crash was detected, LibFuzz found a number of heap overflows so I thought I would let you know.

How to reproduce:

$ clang-3.8 -fsanitize=address -o keystone-replay keystone-replay.c -lkeystone
$ echo -n > empty
$ ./keystone-replay ./empty
=================================================================
==24129==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60200000eff1 at pc 0x000000438cb3 bp 0x7ffd524173c0 sp 0x7ffd52416b70
READ of size 2 at 0x60200000eff1 thread T0
    #0 0x438cb2  (/home/chris/labs/libFuzz/keystone-replay+0x438cb2)
    #1 0x7f683be94618  (/usr/local/lib/libkeystone.so.1+0x2b0618)
    #2 0x4ec9a7  (/home/chris/labs/libFuzz/keystone-replay+0x4ec9a7)
    #3 0x7f683ad2360f  (/lib/x86_64-linux-gnu/libc.so.6+0x2060f)
    #4 0x41b0e8  (/home/chris/labs/libFuzz/keystone-replay+0x41b0e8)

0x60200000eff1 is located 0 bytes to the right of 1-byte region [0x60200000eff0,0x60200000eff1)
allocated by thread T0 here:
    #0 0x4bb208  (/home/chris/labs/libFuzz/keystone-replay+0x4bb208)
    #1 0x4ec8df  (/home/chris/labs/libFuzz/keystone-replay+0x4ec8df)
    #2 0x7f683ad2360f  (/lib/x86_64-linux-gnu/libc.so.6+0x2060f)

SUMMARY: AddressSanitizer: heap-buffer-overflow (/home/chris/labs/libFuzz/keystone-replay+0x438cb2)
[...]

# another test
$ ./keystone-replay ./crash-87a95a75e133a709ee625e42ef42815dfb6f84ff.txt
=================================================================
==24151==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60600000f000 at pc 0x000000438cb3 bp 0x7ffff0c5b1a0 sp 0x7ffff0c5a950
READ of size 65 at 0x60600000f000 thread T0
    #0 0x438cb2  (/home/chris/labs/libFuzz/keystone-replay+0x438cb2)
    #1 0x7ffa0fb13618  (/usr/local/lib/libkeystone.so.1+0x2b0618)
    #2 0x4ec9a7  (/home/chris/labs/libFuzz/keystone-replay+0x4ec9a7)
    #3 0x7ffa0e9a260f  (/lib/x86_64-linux-gnu/libc.so.6+0x2060f)
    #4 0x41b0e8  (/home/chris/labs/libFuzz/keystone-replay+0x41b0e8)
[...]

All the files used are attached. I'll add more info as I go through the reports.

Thanks for your awesome work.

keystone-replay.c.txt

crash-87a95a75e133a709ee625e42ef42815dfb6f84ff.txt

`ldr Rd, =imm` pseudo-instruction missing on ARM

$ kstool arm "ldr r0, =0x41414141"
error: too few operands for instruction
ldr r0, =0x41414141
^
Kstool v1.0 for Keystone Engine (www.keystone-engine.org)

ldr r0, =0x41414141 = [ ]

The code to handle the argument of this pseudo-instruction is #ifdef 0'ed in llvm/lib/Target/ARM/AsmParser/ARMAsmParser.cpp. The pseudo-instruction works fine on AArch64 though.

Use only .md (do not .TXT this is not MSDOS)

Crash reusing deinitialized keystone instance

i am having some crashes that fall in functions like :

frame #2: 0x000000010280f60d libkeystone.1.dylib`llvm::MCContext::getELFSection(llvm::StringRef, unsigned int, unsigned int, unsigned int, llvm::MCSymbolELF const*, unsigned int, char const*, llvm::MCSectionELF const*) + 493

i dont think keystone should parse file formats at all

kstool arch list doesn't matches the supported ones

Absolute label resolution not working on x64 lea / mov instructions

When using a label to reference a memory address for lea or mov instructions on x64 the address always resolves as 0. This only happens for absolute addressing. RIP relative addressing works as expected, call does use the correct address, on x86 32bit this also works as expected.

Unit test to demonstrate this is on its way.

Crash/assert with large value of LocalLabelVal in llvm::MCContext::createDirectionalLocalSymbol()

The following input crashes kstool.

kstool arm64 "nop;b.ne{;;nop;0:ffff{;X;{;nop;{;;nop;0xfffffffffffffff:"

The value 0xfffffffffffffff is parsed as a directional label in AsmParser::parseStatement() and LLVM tries to It then tries to read outside the bounds of a DenseMap using this value as an index.

1406 LocalLabelVal = getTok().getIntVal();

  if (LocalLabelVal == -1) {
      if (ParsingInlineAsm && SI) {
        StringRef RewrittenLabel =
            SI->LookupInlineAsmLabel(IDVal, getSourceManager(), IDLoc, true);
        assert(RewrittenLabel.size() &&
               "We should have an internal name here.");
        Info.AsmRewrites->emplace_back(AOK_Label, IDLoc, IDVal.size(),
                                       RewrittenLabel);
        IDVal = RewrittenLabel;
      }
      Sym = getContext().getOrCreateSymbol(IDVal);
    } else
      Sym = Ctx.createDirectionalLocalSymbol(LocalLabelVal);

It then tries to read outside read outside the bounds of Instances based on the value of LocalLabelVal.

  226 MCLabel *&Label = Instances[LocalLabelVal];

The code should probably be changed to check that LocalLabelVal is in bounds.

It generates the following assert when compiling in debug mode.

kstool: ../llvm/include/llvm/ADT/DenseMap.h:484: bool llvm::DenseMapBase<llvm::DenseMap<unsigned int, llvm::MCLabel *, llvm::DenseMapInfo<unsigned int>, llvm::detail::DenseMapPair<unsigned int, llvm::MCLabel *> >, unsigned int, llvm::MCLabel *, llvm::DenseMapInfo<unsigned int>, llvm::detail::DenseMapPair<unsigned int, llvm::MCLabel *> >::LookupBucketFor(const LookupKeyT &, const BucketT *&) const [DerivedT = llvm::DenseMap<unsigned int, llvm::MCLabel *, llvm::DenseMapInfo<unsigned int>, llvm::detail::DenseMapPair<unsigned int, llvm::MCLabel *> >, KeyT = unsigned int, ValueT = llvm::MCLabel *, KeyInfoT = llvm::DenseMapInfo<unsigned int>, BucketT = llvm::detail::DenseMapPair<unsigned int, llvm::MCLabel *>, LookupKeyT = unsigned int]: Assertion `!KeyInfoT::isEqual(Val, EmptyKey) && !KeyInfoT::isEqual(Val, TombstoneKey) && "Empty/Tombstone value shouldn't be inserted into map!"' failed.

Program received signal SIGABRT, Aborted.

Backtrace:

#0  0x00007ffff71a2418 in __GI_raise (sig=sig@entry=0x6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1  0x00007ffff71a401a in __GI_abort () at abort.c:89
#2  0x00007ffff719abd7 in __assert_fail_base (fmt=<optimized out>, 
    assertion=assertion@entry=0x7fec5b "!KeyInfoT::isEqual(Val, EmptyKey) && !KeyInfoT::isEqual(Val, TombstoneKey) && \"Empty/Tombstone value shouldn't be inserted into map!\"", file=file@entry=0x7fece1 "../llvm/include/llvm/ADT/DenseMap.h", line=line@entry=0x1e4, 
    function=function@entry=0x803fba "bool llvm::DenseMapBase<llvm::DenseMap<unsigned int, llvm::MCLabel *, llvm::DenseMapInfo<unsigned int>, llvm::detail::DenseMapPair<unsigned int, llvm::MCLabel *> >, unsigned int, llvm::MCLabel *, llvm"...) at assert.c:92
#3  0x00007ffff719ac82 in __GI___assert_fail (
    assertion=0x7fec5b "!KeyInfoT::isEqual(Val, EmptyKey) && !KeyInfoT::isEqual(Val, TombstoneKey) && \"Empty/Tombstone value shouldn't be inserted into map!\"", file=0x7fece1 "../llvm/include/llvm/ADT/DenseMap.h", line=0x1e4, 
    function=0x803fba "bool llvm::DenseMapBase<llvm::DenseMap<unsigned int, llvm::MCLabel *, llvm::DenseMapInfo<unsigned int>, llvm::detail::DenseMapPair<unsigned int, llvm::MCLabel *> >, unsigned int, llvm::MCLabel *, llvm"...) at assert.c:101
#4  0x000000000049d2f8 in llvm::DenseMapBase<llvm::DenseMap<unsigned int, llvm::MCLabel*, llvm::DenseMapInfo<unsigned int>, llvm::detail::DenseMapPair<unsigned int, llvm::MCLabel*> >, unsigned int, llvm::MCLabel*, llvm::DenseMapInfo<unsigned int>, llvm::detail::DenseMapPair<unsigned int, llvm::MCLabel*> >::LookupBucketFor<unsigned int> (this=0x7fffffffd3f0, Val=@0x7fffffffbd24: 0xffffffff, FoundBucket=@0x7fffffffbc30: 0x7fffffffbc60)
    at ../llvm/include/llvm/ADT/DenseMap.h:482
#5  0x000000000049d198 in llvm::DenseMapBase<llvm::DenseMap<unsigned int, llvm::MCLabel*, llvm::DenseMapInfo<unsigned int>, llvm::detail::DenseMapPair<unsigned int, llvm::MCLabel*> >, unsigned int, llvm::MCLabel*, llvm::DenseMapInfo<unsigned int>, llvm::detail::DenseMapPair<unsigned int, llvm::MCLabel*> >::LookupBucketFor<unsigned int> (this=0x7fffffffd3f0, Val=@0x7fffffffbd24: 0xffffffff, FoundBucket=@0x7fffffffbc80: 0x7fffffffbca8)
    at ../llvm/include/llvm/ADT/DenseMap.h:521
#6  0x000000000049d123 in llvm::DenseMapBase<llvm::DenseMap<unsigned int, llvm::MCLabel*, llvm::DenseMapInfo<unsigned int>, llvm::detail::DenseMapPair<unsigned int, llvm::MCLabel*> >, unsigned int, llvm::MCLabel*, llvm::DenseMapInfo<unsigned int>, llvm::detail::DenseMapPcreateDirectionalLocalSymbolair<unsigned int, llvm::MCLabel*> >::FindAndConstruct (this=0x7fffffffd3f0, Key=@0x7fffffffbd24: 0xffffffff) at ../llvm/include/llvm/ADT/DenseMap.h:227
#7  0x000000000049308d in llvm::DenseMapBase<llvm::DenseMap<unsigned int, llvm::MCLabel*, llvm::DenseMapInfo<unsigned int>, llvm::detail::DenseMapPair<unsigned int, llvm::MCLabel*> >, unsigned int, llvm::MCLabel*, llvm::DenseMapInfo<unsigned int>, llvm::detail::DenseMapPair<unsigned int, llvm::MCLabel*> >::operator[] (this=0x7fffffffd3f0, Key=@0x7fffffffbd24: 0xffffffff) at ../llvm/include/llvm/ADT/DenseMap.h:234
#8  0x000000000048e230 in llvm::MCContext::NextInstance (this=0x7fffffffd110, LocalLabelVal=0xffffffff) at ../llvm/lib/MC/MCContext.cpp:226
#9  0x000000000048e49f in llvm::MCContext::createDirectionalLocalSymbol (this=0x7fffffffd110, LocalLabelVal=0xffffffff)
    at ../llvm/lib/MC/MCContext.cpp:248
#10 0x00000000004ae436 in (anonymous namespace)::AsmParser::parseStatement (this=0xe7d170, Info=..., SI=0x0, Address=@0x7fffffffcee0: 0x0)
    at ../llvm/lib/MC/MCParser/AsmParser.cpp:1551
#11 0x00000000004a3c8c in (anonymous namespace)::AsmParser::Run (this=0xe7d170, NoInitialTextSection=0x0, Address=0x0, NoFinalize=0x0)
    at ../llvm/lib/MC/MCParser/AsmParser.cpp:705
#12 0x000000000046ded3 in ks_asm (ks=0xe765e0, assembly=0x7fffffffe0b1 "nop;b.ne{;;nop;0:ffff{;X;{;nop;{;;nop;0x", 'f' <repeats 15 times>, ":", 
    address=0x0, insn=0x7fffffffdc20, insn_size=0x7fffffffdc18, stat_count=0x7fffffffdc28) at ../llvm/keystone/ks.cpp:547
#13 0x000000000046b67c in main (argc=0x3, argv=0x7fffffffdd48) at ../kstool/kstool.cpp:211
#14 0x00007ffff718d830 in __libc_start_main (main=0x46af90 <main(int, char**)>, argc=0x3, argv=0x7fffffffdd48, init=<optimized out>, 
    fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffdd38) at ../csu/libc-start.c:291
#15 0x000000000046aeb9 in _start ()

ldr Rd, =label not working properly on ARM

It looks like ldr Rd, =label is wrongly interpreted as the load of the address of a constant pool allocated string label instead of the address of the actual label. Will send PR with regress test tomorrow.

invalid output (jmp [5])

Assembling jmp [5] gives me: E9 00 00 00 00 (which disassembles to jmp rip+5). Expected result: FF 25 05 00 00 00

Seems like this is a syntax error that slips by unnoticed. It my opinion it should detect that a dword ptr ds:[5] was meant.

mov eax, eflags

I tried to assemble the instruction "mov eax, eflags". An invalid instruction as far as I know, because the "eflags" register cannot be used directly.

The output (x32) is "a1 00 00 00 00" and disassemblying it I obtain "mov eax,ds:0x0".
The output (x64) is "8b 04 25 00 00 00 00" and disassemblying it I obtain "mov eax, dword ptr ds:0x0".

In my opinion the assembled bytes are wrong, how do we interpret this? :)

P.s. obviously the right instructions to use with eflags are popf/popfd/pushf/pushfd, but I tried the wrong MOV instruction to see the output.

Cleanup code

At the moment there are still quite a lot of redundant code that we can trim away. Feel free to comment on this issue when you find things that can be deleted.

fstp st(1), st(0)

0x00007FF6BF90DA6C "fstp st(1), st(0)" failed to assemble (ks_asm() failed: count = 0, error = 512):
    Expected (02) DF D9

ffreep st(5)

0x00007FF6BFBCC6FC "ffreep st(5)" failed to assemble (ks_asm() failed: count = 0, error = 514):
    Expected (02) DF C5

call foo assembled as call [foo] in intel and nasm syntax

$ kstool x32nasm "call foo; foo:"
Kstool v1.0 for Keystone Engine (www.keystone-engine.org)

call [foo]; foo: = [ ff 15 00 00 00 00 ]

For comparison, using AT&T syntax yields the correct result:

$ kstool x32att "call foo; foo:"
Kstool v1.0 for Keystone Engine (www.keystone-engine.org)

call foo; foo: = [ e8 00 00 00 00 ]

Division by zero in MCExpr.cpp

A floating point exception is triggered when executing the following:

kstool x32att "add $0x1,%rbx;add $0x8,%rsp;callq 4008f0 <_init>;callq *(%r12,%rbx,8);cmp %rbp,%rbx;je 401486 <__t+0x56>;jne 4014%0 <__rbx;"

gdb output:

Stopped reason: SIGFPE
0x00000000007935b5 in llvm::MCExpr::evaluateAsRelocatableImpl (this=0xe79c30, Res=..., Asm=0x0, Layout=0x0, Fixup=0x0, Addrs=0x0, InSet=0x0) at ../llvm/lib/MC/MCExpr.cpp:750
750     case MCBinaryExpr::Mod:  Result = LHS % RHS; break;


[----------------------------------registers-----------------------------------]
RAX: 0xfae 
RBX: 0x0 
RCX: 0xa ('\n')
RDX: 0x0 
RSI: 0x7fffffff5078 --> 0x0 
RDI: 0xa ('\n')
RBP: 0x7fffffff51f0 --> 0x7fffffff5500 --> 0x7fffffff55a0 --> 0x7fffffff55e0 --> 0x7fffffff5610 --> 0x7fffffff56a0 --> 0x7fffffff5c60 --> 0x7fffffff5e20 --> 0x7fffffff5e70 --> 0x7fffffffb4a0 --> 0x7fffffffb520 --> 0x7fffffffc470 --> 0x7fffffffc6d0 --> 0x7fffffffd370 --> 0x7fffffffd420 --> 0x7fa920 (<__libc_csu_init>:  push   r15)
RSP: 0x7fffffff4ef0 --> 0x0 
RIP: 0x7935b5 (<llvm::MCExpr::evaluateAsRelocatableImpl(llvm::MCValue&, llvm::MCAssembler const*, llvm::MCAsmLayout const*, llvm::MCFixup const*, llvm::DenseMap<llvm::MCSection const*, unsigned long, llvm::DenseMapInfo<llvm::MCSection const*>, llvm::detail::DenseMapPair<llvm::MCSection const*, unsigned long> > const*, bool) const+2709>:  idiv   QWORD PTR [rbp-0x188])


backtrace
#0  0x00000000007935b5 in llvm::MCExpr::evaluateAsRelocatableImpl (this=0xe79c30, Res=..., Asm=0x0, Layout=0x0, Fixup=0x0, Addrs=0x0, InSet=0x0) at ../llvm/lib/MC/MCExpr.cpp:750
#1  0x0000000000793107 in llvm::MCExpr::evaluateAsRelocatableImpl (this=0xe79ca8, Res=..., Asm=0x0, Layout=0x0, Fixup=0x0, Addrs=0x0, InSet=0x0) at ../llvm/lib/MC/MCExpr.cpp:697
#2  0x0000000000792ac8 in llvm::MCExpr::evaluateAsAbsolute (this=0xe79ca8, Res=@0x7fffffff5658: 0xe7e200, Asm=0x0, Layout=0x0, Addrs=0x0, InSet=0x0) at ../llvm/lib/MC/MCExpr.cpp:452
#3  0x00000000007928c3 in llvm::MCExpr::evaluateAsAbsolute (this=0xe79ca8, Res=@0x7fffffff5658: 0xe7e200, Asm=0x0, Layout=0x0, Addrs=0x0) at ../llvm/lib/MC/MCExpr.cpp:437
#4  0x000000000079286c in llvm::MCExpr::evaluateAsAbsolute (this=0xe79ca8, Res=@0x7fffffff5658: 0xe7e200) at ../llvm/lib/MC/MCExpr.cpp:407
#5  0x00000000004a71c2 in (anonymous namespace)::AsmParser::parseExpression (this=0xe7e1e0, Res=@0x7fffffff5c10: 0xe79ca8, EndLoc=...) at ../llvm/lib/MC/MCParser/AsmParser.cpp:1122
#6  0x000000000076cd62 in (anonymous namespace)::X86AsmParser::ParseMemOperand (this=0xe807d0, SegReg=0x0, MemStart=...) at ../llvm/lib/Target/X86/AsmParser/X86AsmParser.cpp:2037
#7  0x0000000000763d7f in (anonymous namespace)::X86AsmParser::ParseATTOperand (this=0xe807d0) at ../llvm/lib/Target/X86/AsmParser/X86AsmParser.cpp:1910
#8  0x0000000000760953 in (anonymous namespace)::X86AsmParser::ParseOperand (this=0xe807d0, Mnem=...) at ../llvm/lib/Target/X86/AsmParser/X86AsmParser.cpp:1150
#9  0x0000000000751ef4 in (anonymous namespace)::X86AsmParser::ParseInstruction (this=0xe807d0, Info=..., Name=..., NameLoc=..., Operands=..., ErrorCode=@0x7fffffffc5d0: 0x0) at ../llvm/lib/Target/X86/AsmParser/X86AsmParser.cpp:2354
#10 0x00000000005b0f21 in llvm::MCTargetAsmParser::ParseInstruction (this=0xe807d0, Info=..., Name=..., Token=..., Operands=..., ErrorCode=@0x7fffffffc5d0: 0x0) at ../llvm/include/llvm/MC/MCParser/MCTargetAsmParser.h:162
#11 0x00000000004b0a2c in (anonymous namespace)::AsmParser::parseStatement (this=0xe7e1e0, Info=..., SI=0x0, Address=@0x7fffffffc6a0: 0x0) at ../llvm/lib/MC/MCParser/AsmParser.cpp:1852
#12 0x00000000004a3a5c in (anonymous namespace)::AsmParser::Run (this=0xe7e1e0, NoInitialTextSection=0x0, Address=0x0, NoFinalize=0x0) at ../llvm/lib/MC/MCParser/AsmParser.cpp:701
#13 0x000000000046ddf3 in ks_asm (ks=0xe765e0, assembly=0x7fffffffd9c3 "add bashx1,%rbx;add bashx8,%rsp;callq 4008f0 <_init>;callq *(%r12,%rbx,8);cmp %rbp,%rbx;je 401486 <__t+0x56>;jne 4014%0 <__rbx;", address=0x0, insn=0x7fffffffd3e0, 
    insn_size=0x7fffffffd3d8, stat_count=0x7fffffffd3e8) at ../llvm/keystone/ks.cpp:543
#14 0x000000000046b5fc in main (argc=0x3, argv=0x7fffffffd508) at ../kstool/kstool.cpp:211
#15 0x00007ffff718d830 in __libc_start_main (main=0x46af10 <main(int, char**)>, argc=0x3, argv=0x7fffffffd508, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffd4f8) at ../csu/libc-start.c:291
#16 0x000000000046ae39 in _start ()

Reduce binary size

Right now every single plugin is 6MB in size

invalid operand (jmp 0)

When I run kstool x64 "jmp 0" it gives me an invalid operand error. The correct output should be EB FE

option for default hex number parsing

It would be great if there was an option/syntax to override number parsing. With this option all numbers are interpreted as hex (mov eax, 01334000 assembles to mov eax, 0x1334000 instead of mov eax, 0x5B800).

keystone.dll does not have any exports

Sorry if this is a dumb question but after successfully building keystone.dll with nmake-dll.bat, the resulting keystone.dll does not have any exports. Is it possible that KEYSTONE_SHARED macro is not getting set somewhere it should be?

Address passed to ks_asm() not applied to labels

When you pass an address to ks_asm() and reference a label the address is not added to the encoded address (at least on x86).

movsxd rbx, edx: x64 (useless 0x48 prefix)

0x00007FF6BF90DA78 "movsxd rbx, edx" size mismatch (expected 2, actual 3):
    Expected (02) 63 DA 
    Actual   (03) 48 63 DA

Missing instructions

please list all the missing instructions here, so i can fix all of them in 1 go.

salc (X86, #54)
int1 (X86, #53)
fsetpm (X86)

memory leak?

Not very familiar with the code, but isn't this a memory leak?

https://github.com/keystone-engine/beta/blob/master/llvm/keystone/ks.cpp#L448

Assembling Memory Instructions

I was playing with Keystone today and noticed the following: Keystone expects the memory size to be specified to generate the assembled bytes. If the segment is specified the output correctly adapts to it. But, if I try the following: "./kstool x32 "add ptr ss:[eax + ebx], 0x777" the output is not generated: "add ptr ss:[eax + ebx], 0x777 = [ ]". This because the memory size is not specified and I guess this is a bug.

useless 3E (ds segment override) prefix

When assembling JMP DWORD PTR DS:[100] (x32 mode) it will output: 3E FF 25 64 00 00 00 (DS segment override prefix) this is also the implicit segment so the correct output should be FF 25 64 00 00 00

Missing instruction: salc

Which assembles to D6

Unable to build in Windows

I'm unable to build the Keystone DLL, unfortunately. For reference, I have the following installed:

CMake 3.5.2
Visual Studio Community 2015 - Version 14.0.25123.00 Update 2

I followed the listed instructions including running from the VS command prompt and ensured that the cmake bin dir was included in %PATH%. What version of VS have you been using to build with? Pardon my ignorant question if this is one but I'm just not familiar with cmake.

C:\Users\TestUser\Desktop\beta\build>..\nmake-dll.bat

C:\Users\TestUser\Desktop\beta\build>rmdir /S /Q .
The process cannot access the file because it is being used by another process.

C:\Users\TestUser\Desktop\beta\build>cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=ON -DLLVM_TARGETS_TO_BUILD="all" -G "NMake Makefiles" ..
-- The C compiler identification is MSVC 19.0.23918.0
-- The CXX compiler identification is MSVC 19.0.23918.0
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/cl.exe -- broken
CMake Error at C:/Program Files (x86)/CMake/share/cmake-3.5/Modules/CMakeTestCCompiler.cmake:61 (message):
  The C compiler "C:/Program Files (x86)/Microsoft Visual Studio
  14.0/VC/bin/cl.exe" is not able to compile a simple test program.

  It fails with the following output:

   Change Dir: C:/Users/TestUser/Desktop/beta/build/CMakeFiles/CMakeTmp



  Run Build Command:"nmake" "/NOLOGO" "cmTC_beb53\fast"

        "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\nmake.exe" -f
  CMakeFiles\cmTC_beb53.dir\build.make /nologo -L
  CMakeFiles\cmTC_beb53.dir\build

  Building C object CMakeFiles/cmTC_beb53.dir/testCCompiler.c.obj

        C:\PROGRA~2\MICROS~1.0\VC\bin\cl.exe
  @C:\Users\TestUser\AppData\Local\Temp\nm875A.tmp

  testCCompiler.c

  Linking C executable cmTC_beb53.exe

        "C:\Program Files (x86)\CMake\bin\cmake.exe" -E vs_link_exe
  --intdir=CMakeFiles\cmTC_beb53.dir --manifests --
  C:\PROGRA~2\MICROS~1.0\VC\bin\link.exe /nologo
  @CMakeFiles\cmTC_beb53.dir\objects1.rsp
  @C:\Users\TestUser\AppData\Local\Temp\nm88B3.tmp

  LINK : fatal error LNK1104: cannot open file 'ucrtd.lib'

  LINK Pass 1 failed.  with 1104

  NMAKE : fatal error U1077: '"C:\Program Files (x86)\CMake\bin\cmake.exe"' :
  return code '0xffffffff'

  Stop.

  NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio
  14.0\VC\BIN\nmake.exe"' : return code '0x2'

  Stop.





  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt


-- Configuring incomplete, errors occurred!
See also "C:/Users/TestUser/Desktop/beta/build/CMakeFiles/CMakeOutput.log".
See also "C:/Users/TestUser/Desktop/beta/build/CMakeFiles/CMakeError.log".

C:\Users\TestUser\Desktop\beta\build>nmake

Microsoft (R) Program Maintenance Utility Version 14.00.23918.0
Copyright (C) Microsoft Corporation.  All rights reserved.

NMAKE : fatal error U1064: MAKEFILE not found and no target specified
Stop.

ks = Ks(KS_ARCH_ARM, KS_MODE_THUMB)
encoding, count = ks.asm(b"blx 0x86535200",addr=0x865351d4)

Keystone returns ['0x35', '0xf1', '0x0', '0xe1'] 1

However, if I use Capstone to disassembly the returned bytes

cs = Cs(CS_ARCH_ARM,CS_MODE_THUMB)
for insn in cs.disasm(b"\x35\xf1\x00\xe1",0x865351d4):
    print "0x%x:\t%s\t%s" % (insn.address, insn.mnemonic, insn.op_str)

I get 0x865351d4: blx #0x86a6a3d8
instead of 0x865351d4: blx #0x86835200

Why?

0x00007FF6BF908614 "test eax, ecx" data mismatch:
    Expected (02) 85 C1 
    Actual   (02) 85 C8 disassembles to "test ecx, eax"
0x00007FF6BF90861D "test eax, edx" data mismatch:
    Expected (02) 85 C2 
    Actual   (02) 85 D0 disassembles to "test edx, eax"
0x00007FF6BF90B090 "test al, bpl" data mismatch:
    Expected (03) 40 84 C5 
    Actual   (03) 40 84 E8  disassembles to "test bpl, al"