The Core Language

A compiled language that assumes C as base reality but it also has lots of ideas taken from Jai, Go like type system, order indepent declarations using Ion algorithm. Syntax currently is whitespace significant. Made to practice language development, it's not supposed to be used. It has lot's of ideas from modern programming languages that you would not find in any compiler book. It supports modules combined with ordered independent declarations and lazy typechecking. Also runtime reflection, slices and other standard features you would find in C.

The language is currently very debuggable. It can produce readable C code with line directives. This allows you to debug the programs with Visual Studio with full source mapping, exactly like you would debug C programs.

Language examples

• More examples can be found in /examples and /modules

//
// Virtual memory abstraction using Windows API
//

#import "kernel32.kl"
#import "base.kl"

PAGE_SIZE :: 4096
Memory :: struct
  commit : SizeU
  reserve: SizeU
  data   : *U8

ProcessHeap: HANDLE
Allocate :: (size: U64): *void
  if ProcessHeap == 0
    ProcessHeap = GetProcessHeap()
  return HeapAlloc(ProcessHeap, 0, size)

Reserve :: (size: SizeU): Memory
  result := Memory{reserve=AlignUp(size, PAGE_SIZE)}
  result.data = VirtualAlloc(
    flProtect        = PAGE_READWRITE,
    dwSize           = result.reserve,
    flAllocationType = MEM_RESERVE,
    lpAddress        = 0)->*U8
  return result

Commit :: (m: *Memory, size: SizeU): Bool
  commit_size := AlignUp(size, PAGE_SIZE)
  total_commit := m.commit + commit_size
  clamped_commit := ClampTopSizeU(total_commit, m.reserve)
  adjusted_commit := clamped_commit - m.commit
  if adjusted_commit != 0
    result := VirtualAlloc(
      lpAddress        = (m.data + m.commit)->*void,
      dwSize           = adjusted_commit,
      flAllocationType = MEM_COMMIT,
      flProtect        = PAGE_READWRITE,
    )
    Assert(result != 0)
    m.commit += adjusted_commit
    return true
  return false

Release :: (m: *Memory)
  result := VirtualFree(m.data->*void, 0, MEM_RELEASE)
  if result != 0
    m.data    = 0
    m.commit  = 0
    m.reserve = 0

What's missing ?

• Constant evaluation and constant folding - Folding and precomputing every expression that can be calculated at compile time. Which allows to check if a given constant expression is actually something that can be computed at compile time

  • Constant expressions with concrete types

• Untyped types - Context dependent type assignment of constant expressions, this is a feature I really loved in Go, it makes constants work very well with a very strict type system and it makes errors like overflow of constants in C due to bad size specifier impossible

  • Infinite precision integers in constants
  • Resolution of all untyped types in the typechecking stage
  • Special case of booleans and their type propagation

• Module system

  • Lazy evaluation of modules (unused code is not compiled or typechecked)
  • Import module, load project file distinction

• Order independent declarations - The ordering of functions in code files or modules does not matter, compiler figures all that stuff out for you. "main" can be wherever you want it to be and all functions should be available without problems

• Synchronize generated C code with original source using line directives so that debuggers work

• Expressions

  • Compounds with named fields and numbered fields
  • Functions calls with named arguments
  • All the standard binary, unary expressions
  • Pointer arithmetic and pointer as array
  • Dot access expression needs a redesign because it doesn't handle expressions after the dot it requires names instead
  • Casting might need a redesign not sure

• Runtime reflection

  • Dumping info
  • Is the design of this correct? That's a lot of data.

• Builtin data structures

  • Arrays
  • Slices
  • Dynamic arrays
  • Hash tables
  • Tuples
    • Multiple return values from a function
    • Some kind of tuple expressions
    • Using tuples as single values without unpacking

• Generics / Parametric polymorphism

• Function overloading

• Operator overloading

  • Binary operators
  • Unary operators
  • Assignment expressions?
  • Bulletproof

• Platforms

  • Windows
  • Unix based

• Language constructs

  • If
  • For loops
  • Functions
  • Structures
  • Switch (but maybe needs redesign?)
  • Jai like using statement
  • Defer statement
  • Unions (or something like unions)
  • Unnamed blocks

Building

1. Install Visual Studio and Clang
2. Run build.bat

Resources

Stuff that helped me a lot programming the compiler. Hopefully they also will be helpful to you!

• https://bitwise.handmade.network/ - series by Per Vognsen where he actually creates a C like language, very helpful, very hands on!
• https://hero.handmade.network/episode/code/day206/ - this episode of handmade hero started me on the entire compiler journey a long, long time ago.
• https://www.youtube.com/watch?v=TH9VCN6UkyQ&list=PLmV5I2fxaiCKfxMBrNsU1kgKJXD3PkyxO - I have rewatched this playlist many this, searching for keywords and ideas. Jonathan Blow's compiler was a big inspiration of mine when learning programming languages.
• A Retargetable C Compiler: Design and Implementation by Christopher W. Fraser and David R. Hanson - sometimes looked at this as a reference to figure stuff out. Very helpful resource on compiler construction, the way it's written reads more like documentation but you use what you have.
• https://github.com/JoshuaManton/sif - looked at this as a reference from time to time, author seems like a Jonathan Blow fan so it was a good resource informed by similar resources as I used.
• https://github.com/c3lang/c3c - I sometimes looked at C3 compiler as a reference, the author also let me use his big int library he wrote sometime in the past! Thanks a lot!
• https://go.dev/blog/constants - article on golang type system, untyped types, constants that kind of stuff.