Code in master should also build successfully as well as pass tests.

Right now we need to copy the gen.go output and insert that into the rest cases.
It should be automatic.
Maybe even as a part of running a test.

Initially submitted code is quite messy as it was written as a proof of concept.
It should be cleaned up.

Instead of loading classes one-by-one it's desired to load packages and all their dependencies atomically, so we can have all relocation targets we need during the code generation.

For deferred constants, maps are used.

It can be more efficient to use slices with key-value pairs since we don't ever do a lookup or delete, only loop over the entire map.

It would also be nice to profile/measure performance differences of that transition on some large class files.

type deferredConst struct {
  dst *string
  index int
}

Instead of Reg we need Reg8, Reg16, Reg32, and Reg64.
Or we need to handle function args properly.
If an instruction does not explicitly specify operand size, we don't know which code to generate.

Reproducer:

go test -v -count 10 ./javatest

goroutine 31 [GC assist marking (scan)]:
runtime: unexpected return pc for github.com/quasilyte/go-jdk/javatest.golibPrintInt called from 0x7f4413295010
stack: frame={sp:0xc00008dcc0, fp:0xc00008dd30} stack=[0xc00008d000,0xc00008e000)
000000c00008dbc0:  000000c000001680  0000000000458690 <runtime.gcAssistAlloc.func1+0> 
000000c00008dbd0:  000000c000001680  0000000000010000 
000000c00008dbe0:  000000c00008dc80  000000000040c337 <runtime.mallocgc+2311> 

(+ more similar lines)

Right now not all info is printed.

We can start from arrays of primitive types.

Short version

This code generates incorrect IR:

return (x < 0) ? -x : x;

The problem is that we don't generate proper load (move) instructions for it.

Long version

Java sources:

public class Simple {
    //  0: lload_0
    //  1: lconst_0
    //  2: lcmp
    //  3: ifge 11
    //  6: lload_0
    //  7: lneg
    //  8: goto 12
    // 11: lload_0
    // 12: lreturn
    //
    // b0   0: flags = Lcmp r0 0
    // b0   1: JumpGtEq @4 flags
    // b1   2: r2 = Lneg r0
    // b1   3: Jump @4
    // b2   4: Lret r0
    public static long abs1(long x) {
        return (x < 0) ? -x : x;
    }

    //  0: lload_0
    //  1: lconst_0
    //  2: lcmp
    //  3: ifge 9
    //  6: lload_0
    //  7: lneg
    //  8: lreturn
    //  9: lload_0
    // 10: lreturn
    //
    // b0   0: flags = Lcmp r0 0
    // b0   1: JumpGtEq @4 flags
    // b1   2: r2 = Lneg r0
    // b1   3: Lret r2
    // b2   4: Lret r0
    public static long abs2(long x) {
        if (x < 0) {
            return -x;
        }
        return x;
    }
}

The IR generated for abs1 is incorrect.

Potential fixes:

1. Lneg should place results in r0 (good).
2. There should be a r0 = Lload r2 before Lret r0, and JumpGtEq should jump there (generates a load that needs to be optimized away).

Here is the most interesting bytecode chunk:

    //  7: lneg
    //  8: goto 12
    // 11: lload_0
    // 12: lreturn

The reason why we fail to generate a correct IR is goto. It jumps over lload and uses the result of the lneg for lreturn, but irgen does not handle it correctly and emits code that always returns lload_0 result.

We currently don't produce a real Load for VM load instructions, they only do affect the operands stack simulations (do value push). If we will generate a Load for them, the problem should disappear, but that could produce a lot of redundant load instructions.

https://docs.oracle.com/javase/8/docs/technotes/tools/unix/jdeps.html

Implementation:
https://github.com/FauxFaux/jdk9-langtools/blob/087d91fb0a972c533c075e9f1cbdcbb3a65fa1fc/src/jdk.jdeps/share/classes/com/sun/tools/classfile/Dependencies.java#L564

• Add test Java classes
• A test compiles them with javac (if there is no such tool on a system, the test is skipped)
• Run Java code with local Java implementation, save results
• Run Java code with GopherJRE, save results
• Compare results; test passes if they are identical

https://golang.org/doc/go1.14#compiler

https://stackoverflow.com/questions/41067095/what-is-a-context-register-in-golang

Looks like DX is already reserved as Go func context register.
It might not be a problem in practice, but I'm not 100% sure right now.

Let's track important optimizations list, so they don't become impossible due to our design decisions

• Method inlining.
• Escape analysis to allocate fewer objects on the heap.
• Batched heap allocations for loops.
• Mapping at least some frame "slots" to hardware registers.

References:

• Reducing OpenJDK Java Garbage Collection times with stack allocation

The current approach is straightforward: we store jclass descriptor strings "as is". This is simple, but there can be a better solution.

Right now we don't try to choose a correct instruction for most immediate (const) operands:

	case ir.InstLcmp:
		switch {
		case a1.Kind == ir.ArgReg && a2.Kind == ir.ArgIntConst:
			asm.CmpqConst8Mem(int8(a1.Value), x64.RSI, int32(a1.Value*8))

Here we need to check if a1.Value actually fits into 8 bits. If it does not, different machine code should be generated instead.

Our algorithm works in the simplest cases, but it does fail badly in cases where we have gotos (see #37).

We could use the approach outlined in Virtual Machine Showdown: Stack Versus Registers, but it would require doing more optimizations for the generated IR:

1. More unoptimized IR nodes allocations (that we want to remove later).
2. It would need more time to make code optimized afterward.
3. Less optimized code with -opt=0 level.

The current schema, of course, does not generate valid code.
If we can fix it without the above-mentioned drawbacks, that would be an optimal solution.

http://tutorials.jenkov.com/java/modules.html

Instead of printing sym{x,y,z} it should print a symbol name instead.

Right now we don't do any version checks.
This may lead to incorrect class file loading.

Missing features:

• Strings
• Modulo operator

Currently, I see these candidates:

• File.ThisClass can be replaced with a ThisClassName.
• Method.NameIndex can be replaced with a MethodName.

• It's good to avoid excessive []byte -> []byte copying due to the fact that assembler produces a non-executable code first and then it's copied to the destination mapped memory.
• It can be easier to produce proper relocations. (Needs verification.)

We can just add a io.Writer argument to Link() method because of the API that codegen.Compiler provides (it asks to returns a [][]byte for compiler methods).

It's possible to speed symbol lookup even further.

Some notes about maps:

• We're not using maps to make it possible to get fast indexing (by int index).
• Another reason not to use maps is memory consumption and GC pressure.

Search performance:

• Linear search is faster when len(symbols)<=6.
• Binary search is efficient with higher lengths.
• Instead of using sort.Search we can inline that loop and avoid a function call for every comparison.

This would be useful for IR generation debugging.

At least during the first phase of the GopherJRE development while we don't have better options.

Currently, code just assumes that we're running on x86-64 linux.
Some proper environment detection is required.

We don't want to use maps to avoid extra GC pressure and memory overhead.

It's possible to improve method lookup by sorting a methods slice by name, so we can use a binary search there.

https://groups.google.com/forum/#!topic/golang-nuts/SxWxUG0uezY
https://go-review.googlesource.com/c/go/+/202337/

When the code calls gofunc1, it may find that there is not enough
stack space, which will cause the runtime to copy the stack to a new
location. When copying the stack the runtime will examine all
pointers on the stack, and, if they point to locations elsewhere on
the stack, adjust them to point to the new stack location. So if the
pointers passed to your function happen to point to locations on the
stack, and you use NO_LOCAL_POINTERS, then they will not be adjusted,
and your program will have dangling pointers leading to memory
corruption or other problems. (C) Ian Lance Taylor

It seems like as long as our pointers do not point to stack locations, we're safe.
We need to keep that in mind.

I propose 3 levels:

• Opt=0, no optimizations. Fastest codegen, pretty slow code. Mostly for things that are intended to be executed once.
• Opt=1, basic optimizations. Very cheap optimizations that make code significantly faster. This is the default level. Fast codegen, good performance of the produced code.
• Opt=2 all optimizations. Potentially slow codegen, but the generated code is as good as we can make it.

https://docs.oracle.com/javase/specs/jvms/se11/html/jvms-4.html#jvms-4.10

Required to make #38 possible.

Steps:

1. Add a new InstKind to ir package.
2. Handle new JVM opcode (bytecode.I2l) in irgen package.
3. Add appropriate support for jit/x64 package (if a new instruction is needed).
4. Handle new InstKind in jit/compiler/x64 package.

How to run tests:

go test -v ./javatest

How to extend jit/x64: 8c329e4
How to extend jit/compiler/x64: 40c5493

Most likely, movlqsx needs to be added to jit/x64.
https://godbolt.org/z/5FxRxn

Should print generated machine code for a given class file.

When code is executed inside VM context, we don't yield to Go scheduler.

While Go scheduler is cooperative, it's required to call to runtime at least once in a while. Normal Go code does it during a function call.

There are multiple solutions to this problem:

• Try to use Go calling convention for the VM calls, so every VM call is preemptible as well. This might require using SP register and CALL instructions to work properly.
• Call runtime.Gosched during the VM calls. If done every time, it'll have noticeable overhead, but we can avoid inserting that into the leaf functions (that don't have loops?).
• Let it be and see what happens. Maybe it's not a big deal if VM goroutine is not preemptible during a code evaluation.

Currently, there are no tests for jclass package. We should have at least some basic coverage for it.