Overview of Jai

Wiki
1

History

  • Jonathan Blow started this compiler project in 2014 as a C++ replacement for game development
  • Jonathan Blow is creating a 3D Sokoban commercial video game (still in development) to test the Jai Compiler
  • Sokoban is a 3D game around 250,000 lines of Jai code
  • The closed beta started around January 2020
  • During the 1st year, the beta grew to over 100 members
  • During the 3rd year, the beta is over 500 members
  • The compiler was around 10,000 lines of code in the initial stages of 2014
  • The compiler expanded to around 50,000 lines of code after adding the LLVM backend, macros, operator overloading, multiple return values, unions, etc.
  • The compiler jumped to around 75,000 lines of code after adding inline assembly support

Licensing

  • Compiler is currently proprietary, and the compiler should not be distributed to anyone outside of beta testers
  • When compiler source code is released, it will have a permissive software license (e.g. BSD or MIT license)
  • Licensing to prevent embrace, extend, extinguish tactics of larger companies

Current State

  • Possible language syntax rewrite in the future
  • Most of language features nailed down, adding minor improvements
  • Focused on language semantics, code optimization not implemented yet
  • Language will be ready when Jonathan Blow releases Sokoban

Performance

  • The goal is to compile 1M lines of code in <1s (from scratch, without delta builds).
  • Official compilation speed of the compiler is 250,000 lines of code per second using the x64 backend, according to the CHANGELOG of beta 0.0.045 (using Sokoban as a benchmark)
  • Optimized Code generated is just as fast as C. Code that does arithmetic on scalar values is just as fast as C code. However, SIMD operations must be hand optimized rather than relying on the compiler to do magic SIMD intrinsics for you (see Benchmarks for more info).

Compiler Internals

  • Compiler code is proprietary, but Jonathan intends to open-source later
  • Compiler was programmed in 87,000 lines of C++ code
  • Jonathan Blow will not be self-hosting the compiler in the near future
  • Multi-threaded Job System Compiler, Lexing and Parsing Parallelized
  • Hand-written recursive descent top-down parser
  • Compiler front-end is primarily written by Jonathan Blow
  • Compiler has its own internal byte-code
  • Two compiler backends: x64 and LLVM backend
  • x64 backend was developed by Jonathan Blow and his compiler team
  • x64 backend does fast but naive code generation, without any code optimization
  • LLVM backend does better code optimization, but has slower compilation speed
  • LLVM backend produces significantly better code generation than x64 backend
  • Out of order compilation model

Features

  • Imperative, general purpose programming language
  • Statically, strongly typed
  • Supports x86 architecture only (limited Nintendo Switch support)
  • Platforms: Windows, Ubuntu Linux 18.04, MacOS, at least one gaming console
  • Basic WASM support, some Android support, iOS support
  • Compiled language as fast and optimized as C or C++ code
  • Arbitrary compile-time execution, with powerful metaprogramming features to modify your code in whatever way you want
  • Compiler gives you the compiler AST (Abstract Syntax Tree) for you to modify
  • User can access compiler through a compiler message loop
  • Parameterized structs (similar to C++ templates or Java generics, except cleaner)
  • Function polymorphism
  • Multiple return values
  • Operator Overloading
  • Hygienic macros
  • Stack Traces
  • Syntax is not final
  • Struct of Arrays supported through #insert directives.
  • Context based allocation and Temporary Storage
  • Basic multi-threading support (threads, mutexes)
  • Inline Assembly with SSE, AVX, AVX2, AVX512 SIMD instructions support
  • Lambda Expressions (no closures)
  • Notes (similar to Java annotations)
  • Pointer auto-dereferencing
  • Array Bounds Checking
  • Cast Bounds Checking
  • Math Bounds Checking
  • Null Pointer Checking
  • Cross Compiling (somewhat working. not fully working yet.)

What will NOT be in Jai

  • No Makefiles
  • No header files
  • No preprocessor
  • No constructors or destructors
  • No RAII (Resource acquisition is initialization)
  • No exceptions
  • No virtual functions/design patterns/object oriented inheritance hierarchies
  • No incremental rebuilds
  • No garbage collection
  • No coroutines (e.g. no Go-style channels)
  • No package managers
  • No sum types/tagged unions (this functionality can be covered by parameterized structs and unions)
  • No reference types
  • No array programming
  • No UFCS (Uniform Function Call Syntax, dot calls, object member function)
  • No relative pointers
  • No gotos