Foreign Function Interface

Introduction

This guide will use the snappycompression/decompression library as an introduction to writing bindings forforeign code. Rust is currently unable to call directly into a C++ library, butsnappy includes a C interface (documented insnappy-c.h).

A note about libc

Many of these examples use the libc crate, which provides varioustype definitions for C types, among other things. If you’re trying theseexamples yourself, you’ll need to add libc to your Cargo.toml:

  1. [dependencies]
  2. libc = "0.2.0"

and add extern crate libc; to your crate root.

Calling foreign functions

The following is a minimal example of calling a foreign function which willcompile if snappy is installed:

  1. extern crate libc;
  2. use libc::size_t;
  4. #[link(name = "snappy")]
  5. extern {
  6.     fn snappy_max_compressed_length(source_length: size_t) -> size_t;
  7. }
  9. fn main() {
  10.     let x = unsafe { snappy_max_compressed_length(100) };
  11.     println!("max compressed length of a 100 byte buffer: {}", x);
  12. }

The extern block is a list of function signatures in a foreign library, inthis case with the platform’s C ABI. The #[link(...)] attribute is used toinstruct the linker to link against the snappy library so the symbols areresolved.

Foreign functions are assumed to be unsafe so calls to them need to be wrappedwith unsafe {} as a promise to the compiler that everything contained withintruly is safe. C libraries often expose interfaces that aren’t thread-safe, andalmost any function that takes a pointer argument isn’t valid for all possibleinputs since the pointer could be dangling, and raw pointers fall outside ofRust’s safe memory model.

When declaring the argument types to a foreign function, the Rust compilercannot check if the declaration is correct, so specifying it correctly is partof keeping the binding correct at runtime.

The extern block can be extended to cover the entire snappy API:

  1. extern crate libc;
  2. use libc::{c_int, size_t};
  4. #[link(name = "snappy")]
  5. extern {
  6.     fn snappy_compress(input: *const u8,
  7.                        input_length: size_t,
  8.                        compressed: *mut u8,
  9.                        compressed_length: *mut size_t) -> c_int;
  10.     fn snappy_uncompress(compressed: *const u8,
  11.                          compressed_length: size_t,
  12.                          uncompressed: *mut u8,
  13.                          uncompressed_length: *mut size_t) -> c_int;
  14.     fn snappy_max_compressed_length(source_length: size_t) -> size_t;
  15.     fn snappy_uncompressed_length(compressed: *const u8,
  16.                                   compressed_length: size_t,
  17.                                   result: *mut size_t) -> c_int;
  18.     fn snappy_validate_compressed_buffer(compressed: *const u8,
  19.                                          compressed_length: size_t) -> c_int;
  20. }
  21. # fn main() {}

Creating a safe interface

The raw C API needs to be wrapped to provide memory safety and make use of higher-level conceptslike vectors. A library can choose to expose only the safe, high-level interface and hide the unsafeinternal details.

Wrapping the functions which expect buffers involves using the slice::raw module to manipulate Rustvectors as pointers to memory. Rust’s vectors are guaranteed to be a contiguous block of memory. Thelength is the number of elements currently contained, and the capacity is the total size in elements ofthe allocated memory. The length is less than or equal to the capacity.

  1. # extern crate libc;
  2. # use libc::{c_int, size_t};
  3. # unsafe fn snappy_validate_compressed_buffer(_: *const u8, _: size_t) -> c_int { 0 }
  4. # fn main() {}
  5. pub fn validate_compressed_buffer(src: &[u8]) -> bool {
  6.     unsafe {
  7.         snappy_validate_compressed_buffer(src.as_ptr(), src.len() as size_t) == 0
  8.     }
  9. }

The validate_compressed_buffer wrapper above makes use of an unsafe block, but it makes theguarantee that calling it is safe for all inputs by leaving off unsafe from the functionsignature.

The snappy_compress and snappy_uncompress functions are more complex, since a buffer has to beallocated to hold the output too.

The snappy_max_compressed_length function can be used to allocate a vector with the maximumrequired capacity to hold the compressed output. The vector can then be passed to thesnappy_compress function as an output parameter. An output parameter is also passed to retrievethe true length after compression for setting the length.

  1. # extern crate libc;
  2. # use libc::{size_t, c_int};
  3. # unsafe fn snappy_compress(a: *const u8, b: size_t, c: *mut u8,
  4. #                           d: *mut size_t) -> c_int { 0 }
  5. # unsafe fn snappy_max_compressed_length(a: size_t) -> size_t { a }
  6. # fn main() {}
  7. pub fn compress(src: &[u8]) -> Vec<u8> {
  8.     unsafe {
  9.         let srclen = src.len() as size_t;
  10.         let psrc = src.as_ptr();
  12.         let mut dstlen = snappy_max_compressed_length(srclen);
  13.         let mut dst = Vec::with_capacity(dstlen as usize);
  14.         let pdst = dst.as_mut_ptr();
  16.         snappy_compress(psrc, srclen, pdst, &mut dstlen);
  17.         dst.set_len(dstlen as usize);
  18.         dst
  19.     }
  20. }

Decompression is similar, because snappy stores the uncompressed size as part of the compressionformat and snappy_uncompressed_length will retrieve the exact buffer size required.

  1. # extern crate libc;
  2. # use libc::{size_t, c_int};
  3. # unsafe fn snappy_uncompress(compressed: *const u8,
  4. #                             compressed_length: size_t,
  5. #                             uncompressed: *mut u8,
  6. #                             uncompressed_length: *mut size_t) -> c_int { 0 }
  7. # unsafe fn snappy_uncompressed_length(compressed: *const u8,
  8. #                                      compressed_length: size_t,
  9. #                                      result: *mut size_t) -> c_int { 0 }
  10. # fn main() {}
  11. pub fn uncompress(src: &[u8]) -> Option<Vec<u8>> {
  12.     unsafe {
  13.         let srclen = src.len() as size_t;
  14.         let psrc = src.as_ptr();
  16.         let mut dstlen: size_t = 0;
  17.         snappy_uncompressed_length(psrc, srclen, &mut dstlen);
  19.         let mut dst = Vec::with_capacity(dstlen as usize);
  20.         let pdst = dst.as_mut_ptr();
  22.         if snappy_uncompress(psrc, srclen, pdst, &mut dstlen) == 0 {
  23.             dst.set_len(dstlen as usize);
  24.             Some(dst)
  25.         } else {
  26.             None // SNAPPY_INVALID_INPUT
  27.         }
  28.     }
  29. }

Then, we can add some tests to show how to use them.

  1. # extern crate libc;
  2. # use libc::{c_int, size_t};
  3. # unsafe fn snappy_compress(input: *const u8,
  4. #                           input_length: size_t,
  5. #                           compressed: *mut u8,
  6. #                           compressed_length: *mut size_t)
  7. #                           -> c_int { 0 }
  8. # unsafe fn snappy_uncompress(compressed: *const u8,
  9. #                             compressed_length: size_t,
  10. #                             uncompressed: *mut u8,
  11. #                             uncompressed_length: *mut size_t)
  12. #                             -> c_int { 0 }
  13. # unsafe fn snappy_max_compressed_length(source_length: size_t) -> size_t { 0 }
  14. # unsafe fn snappy_uncompressed_length(compressed: *const u8,
  15. #                                      compressed_length: size_t,
  16. #                                      result: *mut size_t)
  17. #                                      -> c_int { 0 }
  18. # unsafe fn snappy_validate_compressed_buffer(compressed: *const u8,
  19. #                                             compressed_length: size_t)
  20. #                                             -> c_int { 0 }
  21. # fn main() { }
  23. #[cfg(test)]
  24. mod tests {
  25.     use super::*;
  27.     #[test]
  28.     fn valid() {
  29.         let d = vec![0xde, 0xad, 0xd0, 0x0d];
  30.         let c: &[u8] = &compress(&d);
  31.         assert!(validate_compressed_buffer(c));
  32.         assert!(uncompress(c) == Some(d));
  33.     }
  35.     #[test]
  36.     fn invalid() {
  37.         let d = vec![0, 0, 0, 0];
  38.         assert!(!validate_compressed_buffer(&d));
  39.         assert!(uncompress(&d).is_none());
  40.     }
  42.     #[test]
  43.     fn empty() {
  44.         let d = vec![];
  45.         assert!(!validate_compressed_buffer(&d));
  46.         assert!(uncompress(&d).is_none());
  47.         let c = compress(&d);
  48.         assert!(validate_compressed_buffer(&c));
  49.         assert!(uncompress(&c) == Some(d));
  50.     }
  51. }

Destructors

Foreign libraries often hand off ownership of resources to the calling code.When this occurs, we must use Rust’s destructors to provide safety and guaranteethe release of these resources (especially in the case of panic).

For more about destructors, see the Drop trait.

Callbacks from C code to Rust functions

Some external libraries require the usage of callbacks to report back theircurrent state or intermediate data to the caller.It is possible to pass functions defined in Rust to an external library.The requirement for this is that the callback function is marked as externwith the correct calling convention to make it callable from C code.

The callback function can then be sent through a registration callto the C library and afterwards be invoked from there.

A basic example is:

Rust code:

  1. extern fn callback(a: i32) {
  2.     println!("I'm called from C with value {0}", a);
  3. }
  5. #[link(name = "extlib")]
  6. extern {
  7.    fn register_callback(cb: extern fn(i32)) -> i32;
  8.    fn trigger_callback();
  9. }
  11. fn main() {
  12.     unsafe {
  13.         register_callback(callback);
  14.         trigger_callback(); // Triggers the callback.
  15.     }
  16. }

C code:

  1. typedef void (*rust_callback)(int32_t);
  2. rust_callback cb;
  4. int32_t register_callback(rust_callback callback) {
  5.     cb = callback;
  6.     return 1;
  7. }
  9. void trigger_callback() {
  10.   cb(7); // Will call callback(7) in Rust.
  11. }

In this example Rust’s main() will call trigger_callback() in C,which would, in turn, call back to callback() in Rust.

Targeting callbacks to Rust objects

The former example showed how a global function can be called from C code.However it is often desired that the callback is targeted to a specialRust object. This could be the object that represents the wrapper for therespective C object.

This can be achieved by passing a raw pointer to the object down to theC library. The C library can then include the pointer to the Rust object inthe notification. This will allow the callback to unsafely access thereferenced Rust object.

Rust code:

  1. #[repr(C)]
  2. struct RustObject {
  3.     a: i32,
  4.     // Other members...
  5. }
  7. extern "C" fn callback(target: *mut RustObject, a: i32) {
  8.     println!("I'm called from C with value {0}", a);
  9.     unsafe {
  10.         // Update the value in RustObject with the value received from the callback:
  11.         (*target).a = a;
  12.     }
  13. }
  15. #[link(name = "extlib")]
  16. extern {
  17.    fn register_callback(target: *mut RustObject,
  18.                         cb: extern fn(*mut RustObject, i32)) -> i32;
  19.    fn trigger_callback();
  20. }
  22. fn main() {
  23.     // Create the object that will be referenced in the callback:
  24.     let mut rust_object = Box::new(RustObject { a: 5 });
  26.     unsafe {
  27.         register_callback(&mut *rust_object, callback);
  28.         trigger_callback();
  29.     }
  30. }

C code:

  1. typedef void (*rust_callback)(void*, int32_t);
  2. void* cb_target;
  3. rust_callback cb;
  5. int32_t register_callback(void* callback_target, rust_callback callback) {
  6.     cb_target = callback_target;
  7.     cb = callback;
  8.     return 1;
  9. }
  11. void trigger_callback() {
  12.   cb(cb_target, 7); // Will call callback(&rustObject, 7) in Rust.
  13. }

Asynchronous callbacks

In the previously given examples the callbacks are invoked as a direct reactionto a function call to the external C library.The control over the current thread is switched from Rust to C to Rust for theexecution of the callback, but in the end the callback is executed on thesame thread that called the function which triggered the callback.

Things get more complicated when the external library spawns its own threadsand invokes callbacks from there.In these cases access to Rust data structures inside the callbacks isespecially unsafe and proper synchronization mechanisms must be used.Besides classical synchronization mechanisms like mutexes, one possibility inRust is to use channels (in std::sync::mpsc) to forward data from the Cthread that invoked the callback into a Rust thread.

If an asynchronous callback targets a special object in the Rust address spaceit is also absolutely necessary that no more callbacks are performed by theC library after the respective Rust object gets destroyed.This can be achieved by unregistering the callback in the object’sdestructor and designing the library in a way that guarantees that nocallback will be performed after deregistration.

Linking

The link attribute on extern blocks provides the basic building block forinstructing rustc how it will link to native libraries. There are two acceptedforms of the link attribute today:

  • #[link(name = "foo")]
  • #[link(name = "foo", kind = "bar")]

In both of these cases, foo is the name of the native library that we’relinking to, and in the second case bar is the type of native library that thecompiler is linking to. There are currently three known types of nativelibraries:

  • Dynamic - #[link(name = "readline")]
  • Static - #[link(name = "my_build_dependency", kind = "static")]
  • Frameworks - #[link(name = "CoreFoundation", kind = "framework")]

Note that frameworks are only available on macOS targets.

The different kind values are meant to differentiate how the native libraryparticipates in linkage. From a linkage perspective, the Rust compiler createstwo flavors of artifacts: partial (rlib/staticlib) and final (dylib/binary).Native dynamic library and framework dependencies are propagated to the finalartifact boundary, while static library dependencies are not propagated atall, because the static libraries are integrated directly into the subsequentartifact.

A few examples of how this model can be used are:

  • A native build dependency. Sometimes some C/C++ glue is needed when writingsome Rust code, but distribution of the C/C++ code in a library format isa burden. In this case, the code will be archived into libfoo.a and then theRust crate would declare a dependency via #[link(name = "foo", kind = "static")].

    Regardless of the flavor of output for the crate, the native static librarywill be included in the output, meaning that distribution of the native staticlibrary is not necessary.

  • A normal dynamic dependency. Common system libraries (like readline) areavailable on a large number of systems, and often a static copy of theselibraries cannot be found. When this dependency is included in a Rust crate,partial targets (like rlibs) will not link to the library, but when the rlibis included in a final target (like a binary), the native library will belinked in.

On macOS, frameworks behave with the same semantics as a dynamic library.

Unsafe blocks

Some operations, like dereferencing raw pointers or calling functions that have been markedunsafe are only allowed inside unsafe blocks. Unsafe blocks isolate unsafety and are a promise tothe compiler that the unsafety does not leak out of the block.

Unsafe functions, on the other hand, advertise it to the world. An unsafe function is written likethis:

  1. unsafe fn kaboom(ptr: *const i32) -> i32 { *ptr }

This function can only be called from an unsafe block or another unsafe function.

Accessing foreign globals

Foreign APIs often export a global variable which could do something like trackglobal state. In order to access these variables, you declare them in externblocks with the static keyword:

  1. extern crate libc;
  3. #[link(name = "readline")]
  4. extern {
  5.     static rl_readline_version: libc::c_int;
  6. }
  8. fn main() {
  9.     println!("You have readline version {} installed.",
  10.              unsafe { rl_readline_version as i32 });
  11. }

Alternatively, you may need to alter global state provided by a foreigninterface. To do this, statics can be declared with mut so we can mutatethem.

  1. extern crate libc;
  3. use std::ffi::CString;
  4. use std::ptr;
  6. #[link(name = "readline")]
  7. extern {
  8.     static mut rl_prompt: *const libc::c_char;
  9. }
  11. fn main() {
  12.     let prompt = CString::new("[my-awesome-shell] $").unwrap();
  13.     unsafe {
  14.         rl_prompt = prompt.as_ptr();
  16.         println!("{:?}", rl_prompt);
  18.         rl_prompt = ptr::null();
  19.     }
  20. }

Note that all interaction with a static mut is unsafe, both reading andwriting. Dealing with global mutable state requires a great deal of care.

Foreign calling conventions

Most foreign code exposes a C ABI, and Rust uses the platform’s C calling convention by default whencalling foreign functions. Some foreign functions, most notably the Windows API, use other callingconventions. Rust provides a way to tell the compiler which convention to use:

  1. extern crate libc;
  3. #[cfg(all(target_os = "win32", target_arch = "x86"))]
  4. #[link(name = "kernel32")]
  5. #[allow(non_snake_case)]
  6. extern "stdcall" {
  7.     fn SetEnvironmentVariableA(n: *const u8, v: *const u8) -> libc::c_int;
  8. }
  9. # fn main() { }

This applies to the entire extern block. The list of supported ABI constraintsare:

  • stdcall
  • aapcs
  • cdecl
  • fastcall
  • vectorcallThis is currently hidden behind the abi_vectorcall gate and is subject to change.
  • Rust
  • rust-intrinsic
  • system
  • C
  • win64
  • sysv64

Most of the abis in this list are self-explanatory, but the system abi mayseem a little odd. This constraint selects whatever the appropriate ABI is forinteroperating with the target’s libraries. For example, on win32 with a x86architecture, this means that the abi used would be stdcall. On x86_64,however, windows uses the C calling convention, so C would be used. Thismeans that in our previous example, we could have used extern "system" { ... }to define a block for all windows systems, not only x86 ones.

Interoperability with foreign code

Rust guarantees that the layout of a struct is compatible with the platform’srepresentation in C only if the #[repr(C)] attribute is applied to it.#[repr(C, packed)] can be used to lay out struct members without padding.#[repr(C)] can also be applied to an enum.

Rust’s owned boxes (Box<T>) use non-nullable pointers as handles which pointto the contained object. However, they should not be manually created becausethey are managed by internal allocators. References can safely be assumed to benon-nullable pointers directly to the type. However, breaking the borrowchecking or mutability rules is not guaranteed to be safe, so prefer using rawpointers (*) if that’s needed because the compiler can’t make as manyassumptions about them.

Vectors and strings share the same basic memory layout, and utilities areavailable in the vec and str modules for working with C APIs. However,strings are not terminated with \0. If you need a NUL-terminated string forinteroperability with C, you should use the CString type in the std::ffimodule.

The libc crate on crates.io includes type aliases and functiondefinitions for the C standard library in the libc module, and Rust linksagainst libc and libm by default.

Variadic functions

In C, functions can be ‘variadic’, meaning they accept a variable number of arguments. This canbe achieved in Rust by specifying ... within the argument list of a foreign function declaration:

  1. extern {
  2.     fn foo(x: i32, ...);
  3. }
  5. fn main() {
  6.     unsafe {
  7.         foo(10, 20, 30, 40, 50);
  8.     }
  9. }

Normal Rust functions can not be variadic:

  1. // This will not compile
  3. fn foo(x: i32, ...) { }

The “nullable pointer optimization”

Certain Rust types are defined to never be null. This includes references (&T,&mut T), boxes (Box<T>), and function pointers (extern "abi" fn()). Wheninterfacing with C, pointers that might be null are often used, which would seem torequire some messy transmutes and/or unsafe code to handle conversions to/from Rust types.However, the language provides a workaround.

As a special case, an enum is eligible for the “nullable pointer optimization” if it containsexactly two variants, one of which contains no data and the other contains a field of one of thenon-nullable types listed above. This means no extra space is required for a discriminant; rather,the empty variant is represented by putting a null value into the non-nullable field. This iscalled an “optimization”, but unlike other optimizations it is guaranteed to apply to eligibletypes.

The most common type that takes advantage of the nullable pointer optimization is Option<T>,where None corresponds to null. So Option<extern "C" fn(c_int) -> c_int> is a correct wayto represent a nullable function pointer using the C ABI (corresponding to the C typeint (*)(int)).

Here is a contrived example. Let’s say some C library has a facility for registering acallback, which gets called in certain situations. The callback is passed a function pointerand an integer and it is supposed to run the function with the integer as a parameter. Sowe have function pointers flying across the FFI boundary in both directions.

  1. extern crate libc;
  2. use libc::c_int;
  4. # #[cfg(hidden)]
  5. extern "C" {
  6.     /// Registers the callback.
  7.     fn register(cb: Option<extern "C" fn(Option<extern "C" fn(c_int) -> c_int>, c_int) -> c_int>);
  8. }
  9. # unsafe fn register(_: Option<extern "C" fn(Option<extern "C" fn(c_int) -> c_int>,
  10. #                                            c_int) -> c_int>)
  11. # {}
  13. /// This fairly useless function receives a function pointer and an integer
  14. /// from C, and returns the result of calling the function with the integer.
  15. /// In case no function is provided, it squares the integer by default.
  16. extern "C" fn apply(process: Option<extern "C" fn(c_int) -> c_int>, int: c_int) -> c_int {
  17.     match process {
  18.         Some(f) => f(int),
  19.         None    => int * int
  20.     }
  21. }
  23. fn main() {
  24.     unsafe {
  25.         register(Some(apply));
  26.     }
  27. }

And the code on the C side looks like this:

  1. void register(void (*f)(void (*)(int), int)) {
  2.     ...
  3. }

No transmute required!

Calling Rust code from C

You may wish to compile Rust code in a way so that it can be called from C. This isfairly easy, but requires a few things:

  1. #[no_mangle]
  2. pub extern fn hello_rust() -> *const u8 {
  3.     "Hello, world!\0".as_ptr()
  4. }
  5. # fn main() {}

The extern makes this function adhere to the C calling convention, asdiscussed above in “Foreign CallingConventions“. The no_mangleattribute turns off Rust’s name mangling, so that it is easier to link to.

FFI and panics

It’s important to be mindful of panic!s when working with FFI. A panic!across an FFI boundary is undefined behavior. If you’re writing code that maypanic, you should run it in a closure with catch_unwind:

  1. use std::panic::catch_unwind;
  3. #[no_mangle]
  4. pub extern fn oh_no() -> i32 {
  5.     let result = catch_unwind(|| {
  6.         panic!("Oops!");
  7.     });
  8.     match result {
  9.         Ok(_) => 0,
  10.         Err(_) => 1,
  11.     }
  12. }
  14. fn main() {}

Please note that catch_unwind will only catch unwinding panics, notthose who abort the process. See the documentation of catch_unwindfor more information.

Representing opaque structs

Sometimes, a C library wants to provide a pointer to something, but not let youknow the internal details of the thing it wants. The simplest way is to use avoid * argument:

  1. void foo(void *arg);
  2. void bar(void *arg);

We can represent this in Rust with the c_void type:

  1. extern crate libc;
  3. extern "C" {
  4.     pub fn foo(arg: *mut libc::c_void);
  5.     pub fn bar(arg: *mut libc::c_void);
  6. }
  7. # fn main() {}

This is a perfectly valid way of handling the situation. However, we can do a bitbetter. To solve this, some C libraries will instead create a struct, wherethe details and memory layout of the struct are private. This gives some amountof type safety. These structures are called ‘opaque’. Here’s an example, in C:

  1. struct Foo; /* Foo is a structure, but its contents are not part of the public interface */
  2. struct Bar;
  3. void foo(struct Foo *arg);
  4. void bar(struct Bar *arg);

To do this in Rust, let’s create our own opaque types:

  1. #[repr(C)] pub struct Foo { _private: [u8; 0] }
  2. #[repr(C)] pub struct Bar { _private: [u8; 0] }
  4. extern "C" {
  5.     pub fn foo(arg: *mut Foo);
  6.     pub fn bar(arg: *mut Bar);
  7. }
  8. # fn main() {}

By including a private field and no constructor,we create an opaque type that we can’t instantiate outside of this module.(A struct with no field could be instantiated by anyone.)We also want to use this type in FFI, so we have to add #[repr(C)].And to avoid warning around using () in FFI, we instead use an empty array,which works just as well as an empty type but is FFI-compatible.

But because our Foo and Bar types aredifferent, we’ll get type safety between the two of them, so we cannotaccidentally pass a pointer to Foo to bar().

Notice that it is a really bad idea to use an empty enum as FFI type.The compiler relies on empty enums being uninhabited, so handling values of type&Empty is a huge footgun and can lead to buggy program behavior (by triggeringundefined behavior).