My favorite way to safely handle C structs in Zig November 10, 2024 source/commit

#What’s FFI and/or C interoperability

Before the real blogpost starts, we should get to know what FFI is and why it is important, and what better example to use than (bear with me for a second) machine learning Python libraries.

Have you ever wondered why majority AI applications are written in Python? It is definitely easy to use, but the world’s slowest mainstream language might seem like a poor fit for resource intensive mathematical calculations. That’s why most popular libraries geared towards data analysis use a little trick called “not actually being written in Python, but some other, more performant language”.

FFI or Foreign Function Interface allows calling/using code written in language A from language B. In practice language A is always something that can compile to C-compatible shared library, for example C, C++, Rust or Zig.

Whether it’s for performance gains or because you simply wish to use a library, that doesn’t have an alternative in your language of choice you may find yourself figuring out things like “how the fuck do I load this DLL into my Lua script”.

#Zig and C

Although most other languages also have some form of interop with C it usually involves writing glue code, re-declaring functions in the other language and making it match the declarations in the foreign library, and then of course somehow linking it together. Zig on the other hand can compile, link and also include C code and headers. Using imported C functions is as simple as using any other Zig function.

It is as easy as1:

1
2
3
4
5
6
7

const c = @cImport({
    @cInclude("stdio.h");
});

pub fn main() void {
    _ = c.printf("hello\n");
}

Armed with this newfound knowledge you rush out, get yourself a copy of your favorite library, and begin to hack on some new cool project, no longer constraint by pesky language boundaries and… oh no…

#void pointers, type conversions and callbacks

Zig understands C, but void pointers and callbacks are not fun to deal with. Let’s say you want to use libuv↗, it may look something like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56

const std = @import("std");
const c = @cImport({
    @cInclude("uv.h");
});

const zeroes = std.mem.zeroes;

const Context = struct {
    count: u64 = 0,
};

// convert libuv result into error set
fn check(res: isize) !void {
    if (res >= 0) return;

    return switch (res) {
        c.UV_EOF => error.EndOfFile,
        // ---SNIP---
        else => error.UnknownError,
    };
}

fn counter(timer: ?*c.uv_timer_t) callconv(.C) void {
    // the `data` field is void pointer so we need to cast it
    const ctx: *Context = @ptrCast(@alignCast(timer.?.data));

    std.debug.print("counter: {d}\n", .{ctx.count});

    ctx.count += 1;
    if (ctx.count >= 10) {
        _ = c.uv_timer_stop(timer.?);
        c.uv_close(@ptrCast(timer.?), null);
    }
}

pub fn main() !void {
    // initialize the libuv event loop
    const loop = c.uv_default_loop();
    defer _ = c.uv_loop_close(loop);

    // equivalent to C's `uv_timer_t timer = {0};`
    var timer = zeroes(c.uv_timer_t);

    // initialize the timer
    try check(c.uv_timer_init(loop, &timer));

    // create context
    var ctx = Context{};
    timer.data = &ctx;

    // start the timer and pass our callback to it
    try check(c.uv_timer_start(&timer, counter, 0, 100));

    // run the loop
    try check(c.uv_run(loop, c.UV_RUN_DEFAULT));
}

The example isn’t representative of real world use of libuv, but it gives us something simple with a callback and nullable pointers. Inside the code we create a libuv event loop, initialize a timer and let it run a callback with a given context every 0.1 seconds. As far as FFI goes this is amazing, you get to call C functions as if they were native and it just works

However, if you look closer at the counter method there are a few things I dislike there.

  1. Any pointer passed from C will be optional, even tho the library promises us to always pass a valid pointer to our callback, the type system cannot know that.

  2. Callbacks usually require context, this context is in the form of void pointers, in this example it is the data field on uv_timer_t. Using @ptrCast(@alignCast(...)) throws away any type safety, that Zig offers us.

In simple example like this it hardly matters, but the more complex a codebase becomes the easier it is to introduce undefined behaviour as a result of misusing a void pointer, or unwrapping a null value whilst assuming it cannot be null.

#Making a type-safe wrapper

What if instead of the previous example we had something similar, but with Zig types and Zig callbacks.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

const std = @import("std");
const uv = @import("uv.zig");

const Context = struct {
    count: u64 = 0,
};

const Timer = uv.TimerTyped(Context);

fn counter(timer: *Timer) void {
    const ctx = timer.getData().?;

    std.debug.print("counter: {d}\n", .{ctx.count});

    ctx.count += 1;
    if (ctx.count >= 10) {
        timer.stop();
        timer.close(null);
    }
}

pub fn main() !void {
    const loop = uv.Loop.default();
    defer loop.deinit();

    var timer = Timer{};
    try timer.init(loop);

    var ctx = Context{};
    timer.setData(&ctx);

    try timer.start(counter, 0, 100);

    try loop.run(.default);
}

This might come as a surprise, but the types here are bit for bit identical with their C counterparts and even the compiled binary is very similar to the one created from the first example.

The definition of uv.TimerTyped starts like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

pub fn TimerTyped(DataType: ?type) type {
    return extern struct {
        const Self = @This();

        inner: c.uv_timer_t = zeroes(c.uv_timer_t);

        pub usingnamespace Cast(Self, DataType);

        // ---SNIP---
    };
}

All structures representing libuv handles look similar, they contain only a single field of the type they are meant to represent and are extern. What this gives us is a structure with exact same size and alignment as its C counterpart. usingnamespace↗ is used here to allow structs to access common functions, such as close for all handles.

#Casting

Now for the Cast function.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60

fn Cast(Self: type, DataType: ?type) type {
    const info = @typeInfo(Self).Struct;

    // if DataType is null, then use `anyopaque` to still allow void pointers
    // if user specifically opts in
    const Data = if (DataType) |Data| Data else anyopaque;

    // some comptime asserts to make sure the type is actually what we want
    comptime {
        assert(info.layout == .@"extern");
        assert(info.fields.len == 1);
    }

    const Inner = info.fields[0].type;

    // the size and alignment must always match
    comptime {
        assert(@sizeOf(Self) == @sizeOf(Inner));
        assert(@alignOf(Self) == @alignOf(Inner));
    }

    return struct {
        pub inline fn toUv(self: *Self) *Inner {
            return @ptrCast(self);
        }

        pub inline fn fromUv(ptr: *Inner) *Self {
            return @ptrCast(ptr);
        }

        pub inline fn getLoop(self: *Self) *Loop {
            return Loop.fromUv(self.toUv().loop);
        }

        pub inline fn setData(self: *Self, data: ?*Data) void {
            self.toUv().data = data;
        }

        pub inline fn getData(self: *Self) ?*Data {
            return @ptrCast(@alignCast(self.toUv().data));
        }

        pub fn close(
            self: *Self,
            comptime callback: ?*const fn (*Self) void,
        ) void {
            c.uv_close(
                @ptrCast(self),
                if (callback) |cb|
                    struct {
                        fn func(handle: ?*c.uv_handle_t) callconv(.C) void {
                            cb(Self.fromUv(@ptrCast(handle.?)));
                        }
                    }.func
                else
                    null,
            );
        }
    };
}

Each of these calls is in essence used to eliminate pointer casts at the side of the user, although the functions are identical they only serve as a barrier that prevents wrong types from being passed or returned by casts. Since they are inlined they also don’t add any size or performance hit to the program.2

The notable thing here is the close method which takes a pointer of type Self and a callback to pass it to after closing, but uv_close actually takes a generic *uv_handle_t and not a specific handle type. That’s why we use @ptrCast(self) to turn our pointer into a generic handle and then as the second argument we create an anonymous function3, that takes a handle and then inside we cast it to our Zig type and pass it to the callback.

#“Implementing” the functions

I use the word implement very lightly here as most of the functions are very simple wrappers, which are fairly easy to write. Let’s go ahead and define all the functions used in the example with the timer.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54

const RunMode = enum(c_uint) {
    default = c.UV_RUN_DEFAULT,
    once = c.UV_RUN_ONCE,
    nowait = c.UV_RUN_NOWAIT,
};

pub const Loop = extern struct {
    // the Cast stuff
    // ---SNIP---

    pub fn default() *Self {
        return Self.fromUv(c.uv_default_loop());
    }

    pub fn run(self: *Self, mode: RunMode) !void {
        try check(c.uv_run(self.toUv(), @intFromEnum(mode)));
    }

    pub fn deinit(self: *Self) void {
        _ = c.uv_loop_close(self.toUv());
    }
};

pub fn TimerTyped(DataType: ?type) type {
    return extern struct {
        // ---SNIP---

        pub fn init(self: *Self, loop: *Loop) !void {
            try check(c.uv_timer_init(loop.toUv(), self.toUv()));
        }

        pub fn start(
            self: *Self,
            comptime callback: *const fn (*Self) void,
            timeout: u64,
            repeat: u64,
        ) !void {
            try check(c.uv_timer_start(
                self.toUv(),
                struct {
                    fn func(timer: ?*c.uv_timer_t) callconv(.C) void {
                        callback(Self.fromUv(timer.?));
                    }
                }.func,
                timeout,
                repeat,
            ));
        }

        pub fn stop(self: *Self) void {
            _ = c.uv_timer_stop(self.toUv());
        }
    };
}

All the methods are very simple, probably no-op due to compiler optimizations, since the only thing they perform are mostly casts and most importantly all pointer shenanigans happen inside them, directly inside the struct, which leaves very little room for error.

#Conclusion

I find this the most elegant solution to dealing with C types in a foreign language. No complex wrappers, no allocations and most importantly no void pointers. All at the small cost of writing a few very straightforward wrappers.

You can find the code used in this post here↗. The two examples are in counter1.zig and counter2.zig respectively. Both of them can be compiled via zig build-exe -lc -luv <FILE>.

  1. You also need to link the libc↗. ↩︎

  2. The real implementation is a bit more complex and uses some comptime type resolution to work for both const and non-const pointers at the same time. ↩︎

  3. Zig doesn’t truly have anonymous functions, but does have anonymous structs which can have functions, see this↗. ↩︎

Have something to say? Feel free to leave your thoughts in my public inbox↗, comment on an associated social media post (Ziggit↗) or contact me personally.