examples: demonstrate build tags

Generate x86 Assembly with Go

avo makes high-performance Go assembly easier to write, review and maintain. The avo package presents a familiar assembly-like interface that simplifies development without sacrificing performance:

Use Go control structures for assembly generation; avo programs are Go programs
Register allocation: write functions with virtual registers and avo assigns physical registers for you
Automatically load arguments and store return values: ensure memory offsets are correct for complex structures
Generation of stub files to interface with your Go package

For more about avo:

Introductory talk "Better x86 Assembly Generation with Go" at dotGo 2019 (slides)
Longer tutorial at Gophercon 2019 showing a highly-optimized dot product (slides)
Watch Filippo Valsorda live code the rewrite of filippo.io/edwards25519 assembly with avo
Explore projects using avo
Discuss avo and general Go assembly topics in the #assembly channel of Gophers Slack

Note: APIs subject to change while avo is still in an experimental phase. You can use it to build real things but we suggest you pin a version with your package manager of choice.

Quick Start

Install avo with go get:

$ go get -u github.com/mmcloughlin/avo

avo assembly generators are pure Go programs. Here's a function that adds two uint64 values:

//go:build ignore

package main

import . "github.com/mmcloughlin/avo/build"

func main() {
	TEXT("Add", NOSPLIT, "func(x, y uint64) uint64")
	Doc("Add adds x and y.")
	x := Load(Param("x"), GP64())
	y := Load(Param("y"), GP64())
	ADDQ(x, y)
	Store(y, ReturnIndex(0))
	RET()
	Generate()
}

go run this code to see the assembly output. To integrate this into the rest of your Go package we recommend a go:generate line to produce the assembly and the corresponding Go stub file.

//go:generate go run asm.go -out add.s -stubs stub.go

After running go generate the add.s file will contain the Go assembly.

// Code generated by command: go run asm.go -out add.s -stubs stub.go. DO NOT EDIT.

#include "textflag.h"

// func Add(x uint64, y uint64) uint64
TEXT ·Add(SB), NOSPLIT, $0-24
	MOVQ x+0(FP), AX
	MOVQ y+8(FP), CX
	ADDQ AX, CX
	MOVQ CX, ret+16(FP)
	RET

The same call will produce the stub file stub.go which will enable the function to be called from your Go code.

// Code generated by command: go run asm.go -out add.s -stubs stub.go. DO NOT EDIT.

package add

// Add adds x and y.
func Add(x uint64, y uint64) uint64

See the examples/add directory for the complete working example.

Examples

See examples for the full suite of examples.

Slice Sum

Sum a slice of uint64s:

func main() {
	TEXT("Sum", NOSPLIT, "func(xs []uint64) uint64")
	Doc("Sum returns the sum of the elements in xs.")
	ptr := Load(Param("xs").Base(), GP64())
	n := Load(Param("xs").Len(), GP64())

	Comment("Initialize sum register to zero.")
	s := GP64()
	XORQ(s, s)

	Label("loop")
	Comment("Loop until zero bytes remain.")
	CMPQ(n, Imm(0))
	JE(LabelRef("done"))

	Comment("Load from pointer and add to running sum.")
	ADDQ(Mem{Base: ptr}, s)

	Comment("Advance pointer, decrement byte count.")
	ADDQ(Imm(8), ptr)
	DECQ(n)
	JMP(LabelRef("loop"))

	Label("done")
	Comment("Store sum to return value.")
	Store(s, ReturnIndex(0))
	RET()
	Generate()
}

The result from this code generator is:

// Code generated by command: go run asm.go -out sum.s -stubs stub.go. DO NOT EDIT.

#include "textflag.h"

// func Sum(xs []uint64) uint64
TEXT ·Sum(SB), NOSPLIT, $0-32
	MOVQ xs_base+0(FP), AX
	MOVQ xs_len+8(FP), CX

	// Initialize sum register to zero.
	XORQ DX, DX

loop:
	// Loop until zero bytes remain.
	CMPQ CX, $0x00
	JE   done

	// Load from pointer and add to running sum.
	ADDQ (AX), DX

	// Advance pointer, decrement byte count.
	ADDQ $0x08, AX
	DECQ CX
	JMP  loop

done:
	// Store sum to return value.
	MOVQ DX, ret+24(FP)
	RET

Full example at examples/sum.

Features

For demonstrations of avo features:

args: Loading function arguments.
returns: Building return values.
complex: Working with complex{64,128} types.
data: Defining DATA sections.
ext: Interacting with types from external packages.
pragma: Apply compiler directives to generated functions.

Real Examples

Implementations of full algorithms:

sha1: SHA-1 cryptographic hash.
fnv1a: FNV-1a hash function.
dot: Vector dot product.
md5x16: AVX-512 accelerated MD5.
geohash: Integer geohash encoding.
stadtx: StadtX hash port from dgryski/go-stadtx.

Adopters

Popular projects¹ using avo:

golang / go ⭐ 121.7k

The Go programming language

klauspost / compress ⭐ 4.6k

Optimized Go Compression Packages

golang / crypto ⭐ 3k

[mirror] Go supplementary cryptography libraries

klauspost / reedsolomon ⭐ 1.8k

Reed-Solomon Erasure Coding in Go

bytedance / gopkg ⭐ 1.6k

Universal Utilities for Go

cloudflare / circl ⭐ 1.2k

CIRCL: Cloudflare Interoperable Reusable Cryptographic Library

segmentio / asm ⭐ 863

Go library providing algorithms optimized to leverage the characteristics of modern CPUs

zeebo / xxh3 ⭐ 393

XXH3 algorithm in Go

zeebo / blake3 ⭐ 380

Pure Go implementation of BLAKE3 with AVX2 and SSE4.1 acceleration

lukechampine / blake3 ⭐ 349

An AVX-512 accelerated implementation of the BLAKE3 cryptographic hash function

See the full list of projects using avo.

Contributing

Contributions to avo are welcome:

Feedback from using avo in a real project is incredibly valuable. Consider porting an existing project to avo.
Submit bug reports to the issues page.
Pull requests accepted. Take a look at outstanding issues for ideas (especially the "good first issue" label).
Join us in the #assembly channel of Gophers Slack.

Credits

Inspired by the PeachPy and asmjit projects. Thanks to Damian Gryski for advice, and his extensive library of PeachPy Go projects.

License

avo is available under the BSD 3-Clause License.

Projects drawn from the avo third-party test suite. Popularity estimated from Github star count collected on Aug 1, 2024. ↩

	// NewAsmTest prints one massive assembly function containing a line for every
	// instruction form in the database. The intention is to pass this to the Go
	// assembler and confirm there are no errors, thus helping to ensure our
	// database is compatible.
	func NewAsmTest(cfg printer.Config) Interface {
	return &asmtest{cfg: cfg}
	}

	func TestAllocatorSimple(t *testing.T) {
	c := reg.NewCollection()
	x, y := c.XMM(), c.YMM()

	a, err := NewAllocatorForKind(reg.KindVector)
	if err != nil {
	t.Fatal(err)
	}

	a.Add(x)
	a.Add(y)
	a.AddInterference(x, y)

	alloc, err := a.Allocate()
	if err != nil {
	t.Fatal(err)
	}

	t.Log(alloc)

	if alloc[x] != reg.X0 \|\| alloc[y] != reg.Y1 {
	t.Fatalf("unexpected allocation")
	}
	}

	func TestAllocatorImpossible(t *testing.T) {
	a, err := NewAllocatorForKind(reg.KindVector)
	if err != nil {
	t.Fatal(err)
	}

	a.AddInterference(reg.X7, reg.Z7)

	_, err = a.Allocate()
	if err == nil {
	t.Fatal("expected allocation error")
	}
	}

	LABEL(labels[6]) // LABEL(labels[6])
	MOVBQZX(Mem{Base: ptr, Disp: 5}, ch) // MOVZX(reg_ch, byte[reg_ptr+5])
	SHLQ(U8(48), ch) // SHL(reg_ch, 48)
	ADDQ(ch, v1) // ADD(reg_v1, reg_ch)
	//
	LABEL(labels[5]) // LABEL(labels[5])
	MOVBQZX(Mem{Base: ptr, Disp: 4}, ch) // MOVZX(reg_ch, byte[reg_ptr+4])
	SHLQ(U8(16), ch) // SHL(reg_ch, 16)
	ADDQ(ch, v0) // ADD(reg_v0, reg_ch)

	func (c *Context) ADDSD(mx, x operand.Op) {
	if inst, err := x86.ADDSD(mx, x); err == nil {
	c.Instruction(inst)
	} else {
	c.AddError(err)
	}
	}
	func ADDSD(mx, x operand.Op) { ctx.ADDSD(mx, x) }

	func GLOBL(name string, a avo.Attribute) operand.Mem {
	g := ctx.StaticGlobal(name)
	ctx.DataAttributes(a)
	return g
	}

	import (
	. "github.com/mmcloughlin/avo/build"
	"github.com/mmcloughlin/avo/operand"
	)

	h := Mem{Base: Load(Param("h"), GP64())}
	m := Mem{Base: Load(Param("m").Base(), GP64())}

	ptr := Load(Param("xs").Base(), GP64())
	n := Load(Param("xs").Len(), GP64())

	// Loop until zero bytes remain.
	Label("loop")
	CMPQ(n, Imm(0))
	JE(LabelRef("done"))

	// Load from pointer and add to running sum.
	ADDQ(Mem{Base: ptr}, s)

	// Advance pointer, decrement byte count.
	ADDQ(Imm(8), ptr)
	DECQ(n)
	JMP(LabelRef("loop"))

	// Store sum to return value.
	Label("done")

	func (c Context) Result() (avo.File, []error) {
	return c.file, c.errs
	}

	// TODO(mbm): support AVX512
	if strings.HasPrefix(isa.ID, "AVX512") {
	return false
	}

Repository	Description	Stars
`Yawning/chacha20`	ChaCha20 cryptographic cipher.	33
`Yawning/aez`	AEZ authenticated-encryption scheme.	6
`robskie/bp128`	SIMD-BP128 integer encoding and decoding.	22
`dgryski/go-marvin32`	Microsoft's Marvin32 hash function.	7
`dgryski/go-highway`	Google's Highway hash function.	55
`dgryski/go-metro`	MetroHash function.	66
`dgryski/go-stadtx`	Stadtx hash function. See `examples/stadtx`	7
`dgryski/go-sip13`	SipHash 1-3 function.	17
`dgryski/go-chaskey`	Chaskey MAC.	5
`dgryski/go-speck`	SPECK cipher.	7
`dgryski/go-bloomindex`	Bloom-filter based search index.	79
`dgryski/go-groupvariant`	SSE-optimized group varint integer encoding.	25
`bwesterb/go-sha256x8`	Eight-way SHA256	0
`gtank/ed25519`	`radix51` sub-package originally generated with PeachPy	8

	// Imm returns an unsigned integer constant with size guessed from x.
	func Imm(x uint64) Constant {
	// TODO(mbm): remove this function
	switch {
	case uint64(uint8(x)) == x:
	return U8(x)
	case uint64(uint16(x)) == x:
	return U16(x)
	case uint64(uint32(x)) == x:
	return U32(x)
	}
	return U64(x)
	}

	// Process instructions in reverse: poor approximation to topological sort.
	// TODO(mbm): process instructions in topological sort order

	// Load initial hash.
	h0, h1, h2, h3, h4 := GP32(), GP32(), GP32(), GP32(), GP32()

	MOVL(h.Offset(0), h0)
	MOVL(h.Offset(4), h1)
	MOVL(h.Offset(8), h2)
	MOVL(h.Offset(12), h3)
	MOVL(h.Offset(16), h4)

	func (p *goasm) header() {
	p.Comment(p.cfg.GeneratedWarning())
	p.NL()
	p.include("textflag.h")
	}

	// ContainsTextFlags() returns whether the Asm() representation requires macros in "textflags.h".
	func (a Attribute) ContainsTextFlags() bool {

mmcloughlin / avo Goto Github PK

avo's Introduction

Quick Start

Examples

Slice Sum

Features

Real Examples

Adopters

Contributing

Credits

License

Footnotes

avo's People

Contributors

Stargazers

Watchers

Forkers

avo's Issues

Loading Pointers

Loading Slices

Working with Labels

Block Register Allocations

Summary

Examples

Static TEXT Symbols

Package-Qualified TEXT Symbols

Missing a dot

Processing Code

Recommend Projects

Recommend Topics

Recommend Org

Static `TEXT` Symbols