Writing a Programming Language & Native x86-64 Compiler from Scratch in C: The Story of Yolish

Question

Every programmer has, at some point, looked at their favorite programming language and thought:

“Why was it designed this way? How hard can it actually be to build one myself?”

Most developers who explore language design stop at simple arithmetic evaluators or toy interpreters. But when I was building Exploidus OS—a custom x86-64 reactive capability operating system—I ran into a fundamental architectural barrier:

Standard systems languages (like C) have zero concept of operating-system capability tokens. In C, if you have a pointer, you can dereference it. If you have an address space, any line of code can call any syscall unless restricted by complex external sandboxes.

I wanted a modern, expressive language that felt as ergonomic as Python or Rust, but was deeply capability-aware at the syntax level—able to compile directly to bare-metal x86-64 binaries without pulling in gigabytes of LLVM dependencies.

That ambition gave birth to Yolish—a fast, expressive, capability-aware programming language with a tree-walking interpreter, a bytecode virtual machine, and a direct x86-64 native compiler.

Here is how I designed Yolish, wrote its compiler from scratch in pure C, and what I learned building a complete language ecosystem.

---

1. Language Design: Ergonomics Meets Systems Control

When designing Yolish's syntax (.y), I wanted code to read cleanly without syntactic noise, while enforcing defensive systems programming invariants.

Immutability by Default

Variables in Yolish default to immutable constants using let, while mutable state must be explicitly declared using var:

let name  = "Yolish"   -- immutable constant
var count = 0          -- mutable state
count = count + 1

Pattern Matching with Ranges & Guards

Instead of archaic C-style switch statements with fallthrough bugs, Yolish features expressive pattern matching:

let grade = match score {
    90..100      => "A"
    80..90       => "B"
    n if n >= 60 => "C"
    _            => "F"
}

Structs and impl Blocks

Yolish rejects complex class hierarchies in favor of data-first structs and explicit method implementations:

struct Circle { radius }

impl Circle {
    fn area(self) {
        return y.math.pi * self.radius * self.radius
    }
}

let c = Circle { radius: 5 }
y.println(c.area())

---

2. Bridging Code to the Kernel: First-Class Capabilities

Because Yolish is the official userspace language for Exploidus OS, security cannot be an afterthought.

In Yolish, dangerous operations (network access, raw disk writes, peripheral I/O) are gated by declarative capability annotations:

@cap(net.read, fs.write)
fn sync_data(url, filepath) {
    let payload = y.net.get(url)
    y.fs.write(filepath, payload)
}

When compiled for Exploidus OS, the Yolish compiler inspects these annotations and validates them against the process's cryptographic capability table. If an unprivileged function attempts to make an unauthorized syscall, the operation is blocked before execution even reaches kernel space.

---

3. The Tri-Engine Architecture: Interpreter, Bytecode VM, and Native Compiler

Building a language engine involves tough tradeoffs. Tree-walking interpreters are easy to write but slow. Bytecode VMs are fast and portable. Native compilers deliver maximum speed but are deeply architecture-specific.

Instead of choosing one, Yolish implements all three:

Source Code (.y)
       |
    [Lexer] -> Tokens
       |
   [Parser] -> Abstract Syntax Tree (AST)
       |
       +---> Tree-Walking Interpreter  (`ys file.y` - Instant startup)
       |
       +---> Bytecode Compiler & VM    (`ys vm file.y` - High speed execution)
       |
       +---> Direct x86-64 Machine Code (`ys -c file.y` - Standalone binaries)

1. The Interactive Interpreter (ys file.y): Evaluates the AST directly. Perfect for quick scripts, one-liners, and the interactive REPL.
2. The Bytecode VM (ys vm file.y): Compiles the AST into an optimized instruction stream executed by a custom register/stack-based virtual machine.
3. The Native x86-64 Compiler (ys -c file.y): Generates real machine code instructions (MOV, ADD, CALL, RET, SSE2 float operations) directly into executable binary containers.

---

4. No LLVM, No Bloat: Direct Multi-Platform Binary Emission

Most modern compiler projects delegate code generation to LLVM. While LLVM is powerful, it is also a massive, multi-gigabyte dependency with slow compilation times.

I wanted the Yolish toolchain (ys) to be a single, self-contained C binary with zero external dependencies. You can clone the repo and compile the entire language toolchain with a single make command using standard gcc or clang in seconds.

To achieve this without LLVM, Yolish writes binary headers and machine code directly:

• Linux ELF64: Directly formats ELF headers, program headers (PT_LOAD), .text sections, and Linux syscall dispatchers.
• Windows PE32+: Generates valid Portable Executable (PE32+) headers, image import directories, and embeds Windows application icons.
• macOS Mach-O: Generates 64-bit Mach-O load commands (LC_SEGMENT_64).

You can cross-compile for any major operating system with a single CLI flag:

ys -c app.y --target linux    # Emits static Linux ELF64 binary
ys -c app.y --target windows  # Emits Windows PE32+ .exe
ys -c app.y --target macos    # Emits macOS Mach-O binary

---

5. Runtime Architecture: Mark-and-Sweep Garbage Collection

A systems language with dynamic strings and closures needs safe, automatic memory reclamation.

Yolish implements a custom Mark-and-Sweep Garbage Collector (GC) written from scratch in C:

1. Root Scanning: Traverses the active call frame stack, global environment variables, and the VM operand stack to identify all actively reachable pointers.
2. Mark Phase: Recursively traverses heap-allocated objects (strings, dynamic arrays, closures, struct instances), setting the GC mark bit.
3. Sweep Phase: Scans the global heap allocation pool. Any block whose mark bit is zero was unreachable; it is freed back to the system memory pool, while surviving objects have their mark bits cleared for the next cycle.

Developers have full programmatic introspection into memory usage via gc.stats() and can trigger explicit collection via gc.collect().

---

6. A Tooling-First Philosophy

A great programming language is more than just a syntax parser; it is an entire developer experience.

Too many new languages fail because they require third-party formatters, linter plugins, and package wrappers before you can write a serious project.

From day one, Yolish was designed with a tooling-first philosophy:

• Built-in Test Runner (ys test file.y): Write native test blocks alongside code and execute them without third-party frameworks.
• Static Checker (ys check file.y): Performs static semantic analysis, flags undefined variables, and detects type mismatches before runtime.
• Code Formatter (ys fmt file.y): Enforces an opinionated, canonical code style across codebases.
• Interactive REPL (ys): Instant feedback loop with color-highlighted evaluation.

---

7. What Writing a Compiler in C Teaches You

Writing a programming language and compiler from scratch in pure C is one of the most humbling and rewarding journeys in software engineering:

1. Memory Discipline: Managing AST nodes, symbol tables, and lexical scopes in C forces you to understand every single byte you allocate. Memory leaks in a compiler slow down compilation; memory corruption in code generation crashes the output binary.
2. The Magic of Abstraction: When you write the code that translates a simple if (x > 10) into a CMP and JLE assembly jump, you demystify how computers actually think. You realize that high-level abstractions are not magic; they are just structured transformations of intent.
3. Simplicity Over Complexity: Features are easy to add to a grammar; keeping the language coherent, orthogonal, and bug-free across interpretation and native compilation is where the real engineering happens.

---

Conclusion & Open Source

Yolish started as a specialized language project for Exploidus OS, but has evolved into a versatile, cross-platform programming environment with its own native compiler, VM, and standard library.

The entire project is 100% open-source under the MIT license:

• GitHub Repository: https://github.com/rahadbhuiya/Yolish
• Supported Platforms: Linux, Windows, macOS, and Exploidus OS.

Download the binary, try out the REPL, write a script, and inspect the C compiler source!

Have you ever thought about designing a programming language? What is the one feature you wish your daily language had? Let's discuss in the comments below!