Skip to content

stdlib.md

Latest commit

 

History

History
483 lines (363 loc) · 10.6 KB

stdlib.md

File metadata and controls

483 lines (363 loc) · 10.6 KB
Raw
title standard library
status α1
implemented
basic-io
result-maybe
pending
io-capability
file-system
math
text
codec
runtime-architecture
deferred
network (α2)
concurrency (β)
simd (β)

standard library

the standard library is organized in layers. lower layers are always available, higher layers need capabilities or a runtime.

layer 0: core (no effects, no regions, always available)

pure functions and types. no imports needed for prelude types.

core.types (prelude, auto-imported)

type Bool;
type Char;
type String;
type Unit;
type Never;

type i8, i16, i32, i64, i128;
type u8, u16, u32, u64, u128, usize;
type f32, f64;

type Result<T, E>;
type Maybe<T>;
type Array<T, const N: usize>;
type View<T>;
type View<mut T>;

core.math

import core.math { sin, cos, sqrt, min, max, abs, pow, floor, ceil, round, clamp, PI, E };

math.sin(x)
math.cos(x)
math.sqrt(x)
math.abs(x)
math.min(a, b)
math.max(a, b)
math.clamp(x, min, max)
math.pow(base, exp)
math.floor(x)
math.ceil(x)
math.round(x)
math.PI
math.E

core.text

import core.text { split, join, trim, contains, starts_with, ends_with, to_upper, to_lower };

text.trim(s)
text.split(s, delim)
text.join(parts, sep)
text.contains(s, substr)
text.starts_with(s, prefix)
text.ends_with(s, suffix)
text.to_upper(s)
text.to_lower(s)

core.mem

import core.mem { copy, compare, zero, secure_zero };

mem.copy(dst, src)
mem.compare(a, b)
mem.zero(view)
mem.secure_zero(view)   // not optimized away

intrinsics

intrinsics are built into the compiler. they're accessed with @ prefix, no import needed.

@sizeOf<T>()           // size in bytes
@alignOf<T>()          // alignment
@typeInfo<T>()         // type reflection
@intToPtr<*T>(addr)    // address to pointer
@ptrToInt(ptr)         // pointer to address
@bitCast<T>(val)       // reinterpret bits
@compileError(msg)     // compile-time error

intrinsics are implemented directly in the compiler, generating llvm ir.

layer 1: alloc (needs alloc effect, needs a region)

growable data structures that allocate into a region.

alloc.vec

import alloc.vec { Vec };

Vec.new<T>()
vec.push(item)
vec.pop()
vec.len()
vec.get(index)

alloc.hashmap

import alloc.hashmap { HashMap };

HashMap.new<K, V>()
map.insert(key, value)
map.get(key)
map.remove(key)

alloc.string

import alloc.string { StringBuilder };

StringBuilder.new()
sb.append(s)
sb.toString()

alloc.buffer

import alloc.buffer { Buffer };

Buffer.new()
buf.write(data)
buf.toView()

layer 2: codec (needs alloc, pure otherwise)

encoding and decoding. pure computation over allocated buffers.

codec.json

import codec.json { Json };

Json.parse(data)        -> Result<JsonValue, ParseError>
Json.stringify(value)   -> String
Json.encode<T>(value)   -> Result<String, EncodeError>
Json.decode<T>(data)    -> Result<T, DecodeError>

codec.msgpack

import codec.msgpack { MsgPack };

MsgPack.encode<T>(value) -> Result<View<u8>, EncodeError>
MsgPack.decode<T>(data)  -> Result<T, DecodeError>

codec.toml

import codec.toml { Toml };

Toml.parse(data)        -> Result<TomlValue, ParseError>
Toml.encode<T>(value)   -> Result<String, EncodeError>
Toml.decode<T>(data)    -> Result<T, DecodeError>

debug builtins (temporary, no capability needed)

these are bootstrap builtins for development. they bypass the capability system and write directly to stdout/stderr. they will be replaced by Io capability methods when the runtime model is implemented.

// available anywhere, no import, no capability
println(message)          // print line to stdout
print(message)            // print without newline
print_i32(value)          // print integer
print_i64(value)          // print i64
print_f64(value)          // print float
print_bool(value)         // print boolean
print_char(value)         // print character from i32 code point
print_newline()           // print newline
dbg(label, value)         // debug print to stdout: "label = value"

these are not part of the final language. they exist because the Io capability and Show trait aren't implemented yet. use them for testing and bootstrapping only.

layer 3: io (needs capabilities)

capability-gated operations. methods on capability values, not imported functions.

io (needs Io): stdin, stdout, stderr

the Io capability provides access to standard streams. writing is cheap (no allocation, no failure). reading may allocate and can fail.

stdout (no alloc, no fail, just io effect):

io.print(s: String) -> Unit effects [io]
io.println(s: String) -> Unit effects [io]
io.write(data: View<u8>) -> Unit effects [io]
io.flush() -> Unit effects [io]

stderr (same):

io.eprint(s: String) -> Unit effects [io]
io.eprintln(s: String) -> Unit effects [io]

stdin, buffer-based (no alloc, caller provides buffer):

io.read(buf: View<mut u8>) -> usize effects [io, fail<IoError>]
io.readChar() -> Char effects [io, fail<IoError>]

stdin, allocating (needs region for returned data):

io.readLine(region: Region) -> String effects [io, alloc, fail<IoError>]
io.readAll(region: Region) -> View<u8> effects [io, alloc, fail<IoError>]

formatted output (needs Show trait, α2):

io.show<T>(value: T) -> Unit effects [io] where T impl Show
io.showln<T>(value: T) -> Unit effects [io] where T impl Show

io.show replaces the typed print builtins (print_i32, print_f64, etc.). any type that implements Show can be printed.

io.fs (needs Fs)

fs.open(path)            -> Result<File, IoError>
fs.create(path)          -> Result<File, IoError>
fs.readAll(path)         -> Result<View<u8>, IoError>
fs.readAllText(path)     -> Result<String, IoError>
fs.writeAll(path, data)  -> Result<Unit, IoError>
fs.exists(path)          -> Bool
fs.remove(path)          -> Result<Unit, IoError>
fs.mkdir(path)           -> Result<Unit, IoError>
fs.readDir(path)         -> Result<Vec<DirEntry>, IoError>

io.net (needs Net)

net.connect(host, port)  -> Result<Socket, IoError>
net.listen(addr)         -> Result<Listener, IoError>
sock.read(buf)           -> Result<usize, IoError>
sock.write(data)         -> Result<usize, IoError>

io.net.http

http.get(url)            -> Result<Response, IoError>
http.post(url, body)     -> Result<Response, IoError>

io.time (needs Clock)

clock.now()              -> Instant
clock.sleep(duration)    -> Unit
Duration.seconds(n)
Duration.ms(n)

io.rng (needs Rng)

rng.u32()                -> u32
rng.range(min, max)      -> i64
rng.bytes(view)          -> Unit

layer 4: framework (needs runtime)

higher-level frameworks that take a full runtime.

framework.http

import framework.http { Server, Router };

Server.new(rt)
router.get(path, handler)
router.post(path, handler)
server.listen(addr)

framework.cli

import framework.cli { Cli, Arg };

Cli.new("myapp")
cli.arg(Arg.positional("input"))
cli.flag("verbose", "v")
cli.parse(args)

framework.test

test "description" {
    assert_eq(result, expected);
}

how it fits together

@sizeOf<T>()           -> compiler intrinsic, emits LLVM IR directly
core.math.sin(x)       -> compiled ferrule, pure computation
Vec.new<T>()           -> compiled ferrule, calls region allocator
fs.readAll(path)        -> ferrule method on Fs capability
                           -> calls rt_read() in runtime (zig)
                           -> zig calls read() syscall

the stdlib is written in ferrule (once bootstrapped). the runtime core is zig: thin syscall wrappers, region allocators, panic handler, async executor.

runtime architecture

three layers:

  1. platform: raw syscalls / wasi / bare metal
  2. runtime core (zig): region allocators, syscall wrappers, panic handler, startup. ~2000-5000 lines.
  3. ferrule stdlib (written in ferrule): Vec, HashMap, codecs, etc.

no libc by default. raw syscalls. libc available as opt-in (--link-libc) for interop.

embedded support

for embedded/bare metal, skip the runtime:

#![no_std]
#![no_runtime]

// no stdlib imports available
// must use intrinsics directly

const UART: *volatile u32 = @intToPtr(*volatile u32, 0x4000_0000);

function uart_write(byte: u8) -> Unit {
    unsafe {
        UART.* = @as(u32, byte);
    }
}

#[entry]
function main() -> Never {
    uart_write('H');
    loop {}
}

for bare metal: #[entry] attribute, no runtime, no capabilities.

statics

for global state:

static BUFFER: Array<u8, 1024> = [0; 1024];
static CONFIG: Config = Config { baud: 9600 };

// mutable statics require unsafe
static mut COUNTER: u32 = 0;

unsafe {
    COUNTER = COUNTER + 1;
}

extern structs

for c-compatible layout:

type CHeader = extern {
    magic: u32,
    version: u16,
    flags: u16,
};

packed structs

for bit-level layout:

type NetworkHeader = packed {
    version: u4,
    ihl: u4,
    dscp: u6,
    ecn: u2,
};

volatile

for memory-mapped io:

type UartRegisters = extern {
    data: volatile u32,
    status: volatile u32,
    control: volatile u32,
};

layout

layout.sizeof<T>()
layout.alignof<T>()
layout.page_size()
layout.cache_line_size()

stdlib structure

path layer purpose
stdlib/core/types.fe 0 prelude types (auto-imported)
stdlib/core/math.fe 0 pure math functions
stdlib/core/text.fe 0 pure text operations
stdlib/core/mem.fe 0 pure memory operations
stdlib/alloc/vec.fe 1 growable vector
stdlib/alloc/hashmap.fe 1 hash map
stdlib/alloc/string.fe 1 string builder
stdlib/alloc/buffer.fe 1 growable byte buffer
stdlib/codec/json.fe 2 json codec
stdlib/codec/msgpack.fe 2 messagepack codec
stdlib/codec/toml.fe 2 toml codec
stdlib/io/io.fe 3 stdin, stdout, stderr
stdlib/io/fs.fe 3 file system
stdlib/io/net.fe 3 networking
stdlib/io/time.fe 3 clock/sleep
stdlib/io/rng.fe 3 randomness
stdlib/framework/http.fe 4 http server framework
stdlib/framework/cli.fe 4 cli argument parsing
stdlib/framework/test.fe 4 test framework
stdlib/runtime/runtime.zig -- zig runtime support

what's planned

simd (β):

simd.add(a, b)
simd.mul(a, b)
simd.reduce_add(v)