November 25, 2012

llgo update #10: "hello, world!" redux

It’s about time for another progress update on llgo. I’ve made decent progress recently, so let’s go through what’s new.

Highlights

I’ve been refactoring bits of code and fixing bugs aplenty, so there is a mass of noise in the git commits. In terms of new function, the news is that we now have:

  • Type switches.
  • Type assertions.
  • Labeled statements; goto, labeled break and continue.
  • The llgo-dist command; more on this below.
  • String conversions: to/from byte slices; from rune/int.
  • String range. I’m sure the implementation could be improved.
  • Implemented sync/atomic using LLVM atomic operations intrinsics.
  • Various changes to enable linking multiple packages (e.g. exported symbols are now prefixed with their package path).
  • Additional support for floats (thanks to spate); partial support for complex numbers.
  • ”…args” calls to variadic functions (including slice append).
  • A self-contained runtime package. I have cloned (and slightly modified in some cases) the Go portion of the runtime package from gc, and combined it with the runtime code I had already written for llgo.
  • Bridge code for the math package, which mostly just redirects the exported functions to the internal, pure-Go implementations.
  • System calls (Linux/AMD64 only so far).
  • Closures; more below.

llgo-dist

I have begun implementing a command that takes care of building llgo, its runtime, and in the future any other tools that might be considered part of llgo (e.g. an in-development linker). This tool will set up the cgo flags given the path to an “llvm-config” program, and build gollvm.

reflect, fmt, oh my!


Last week, I mentioned on Google+ that I managed to get the reflect package working. At least enough of it to get the fmt package to work. At least enough of the fmt package to get fmt.Println(“Hello, world!”) to work… Yep, the holy grail of programming examples now compiles, links, and runs, using llgo. This demonstrates the following things work:

  1. Compilation of the following packages: errors, io, math, os, reflect, runtime, strconv, sync, sync/atomic, syscall, time, unicode/utf8, unsafe.
  2. Package imports (still using the gcimporter from exp/types.)
  3. Linking multiple compiled packages using llvm-link.
  4. Interfaces and reflection (fmt.Println uses reflection to determine the underlying type).
  5. System calls (fmt.Println will eventually issue a system call to write to the stdout file).

Closures

Yes indeed, we now have closures. The code is pretty hackish, so I expect it’s not very solid. I have implemented them using LLVM’s trampoline intrinsics. Essentially you provide LLVM with a function that takes N parameters, give it a block of (executable) memory and an argument to bind, and it fills in the block with function code for a function with N-1 parameters (the Nth one being bound).

Unfortunately I have found that the closures are not playing nicely with lli/JIT, which means the closure unit test I have written fails. If I compile it with llc/gcc, though, it works just fine. So either I’ve done something subtly stupid, or the JIT is clobbering something it shouldn’t. As far as I got with debugging was finding that the bound argument value is wrong when the function is entered.

I expect I’ll probably replace this implementation for a couple of reasons:

  • Portability: I’d rather avoid platform-specific code like this. For one thing, the PNaCl ABI calls out trampoline intrinsics as being unsupported.
  • Testability: I should investigate the problems I observed with lli/JIT further, and I’m loath to change implementation to support tests, it is a real problem. I rely heavily on tests to make sure I haven’t broken anything.
Until I find out that using trampolines has a marked benefit to performance in real programs, I intend to replace the current implementation with one that uses a pair of pointers for functions. The bound argument will stored in one pointer, and the function pointer in another. This has implications for all function calls, though it should be simple to achieve good performance in most cases.

What’s next?

Haven’t figured this one out yet. I have been meaning to play more with PNaCl, so I might take some time now to do that. I expect I’ll be slowing down development considerably early 2013, as (a) we’re knocking down our place and rebuilding, and (b) my second child is on the way. I hope to have llgo in a better state for contributions by then, so others can pick up the slack.

I expect in the near future I’ll start playing with clang/cgo integration, as I start playing with PNaCl. I’ll write back when I have something to demonstrate.

Until then.