milen.me
milen.me

Swift, Module Maps & VFS Overlays

Permalink | RSS

When compiling code in large non-trivial codebases, the filesystem directory layout might not be matching the structure that's encoded in the source code. There are multiple technical reasons why it would not be possible to make them match, so compilers usually provide a way to accommodate such use cases.

Clang provides two mechanisms: header maps and virtual file system (VFS) overlays.

In this article, we will explore the usage of VFS overlays for the purposes of exposing C/Obj-C code to Swift using VFS overlays.

Header Maps

Header maps are just hash maps from strings to paths. When the compiler encounters an #include <X> directive, it will look up X in any provided header maps. According to the docs in clang/Lex/HeaderMap.h:

To the #include file resolution process, it [a header map] basically acts like a directory of symlinks to files. Its advantages are that it is dense and more efficient to create and process than a directory of symlinks.

The key part is efficiency at scale: it's much cheaper to create a header map file which contains a mapping for thousands of include paths than to create the same number of symlinks on the filesystem.

Usage

First, you need to create a header map file (it's a binary format). Usually, Xcode does it for you as long as USE_HEADERMAP build setting is turned on. External build systems, like Buck, include their own code to create such files (HeaderMap.java).

I've also written a command line tool in Swift, called hmap, that can be used to read and write such header map files. If you have Homebrew, you can install it easily using brew install milend/taps/hmap.

Once you have a header map file, pass it to Clang: clang -I/path/to/file.hmap ... more params.

Modules

If you're writing Swift code and need to expose C/Obj-C code to it, you need to use Clang Modules. At a high level, this works by creating module map files which describe how to map headers to modules. Here's an example of a module.modulemap file:

module Rocket {
  header "Fuel.h"
  export *
}

Crucially, header paths are resolved relative to the module map file, so all those files must be co-located. But that might not be practical (e.g., you're generating module maps outside the source repo; you have multiple module mappings, etc). So, how do you make it work?

You might try to use header maps and create relavant entries. Unfortunately, lookups in module maps do not use mappings from header maps. Fortunately, we can use another mechanism: VFS overlays!

VFS Overlays

VFS overlays are YAML files which provide a mechanism to simulate a filesystem structure which is overlaid on top of the real filesystem. For example, you can create an entry which maps /Imaginary/File.txt to /Users/user/Desktop/File.txt.

The VFS mechanism works for module maps, so we can use it to expose any module structure we would like without touching the filesystem structure (e.g., imagine source repo mounted as read-only).

Example

You can find the example code on GitHub: just clone the repository and execute the command in README.md.

The command has three important parts:

  • Usage of -Xcc to pass arguments to the C/C++/Objective-C compiler.
  • Usage -ivfsoverlay to pass the VFS overlay to the C/C++/Objective-C compiler.
  • Usage of -I /Rocket which will make Clang find the virtual /Rocket/module.modulemap and create a Rocket module. That module can be seen by the Swift code, as demonstrated.

← Back to Writings

References

Any opinions and viewpoints expressed, explicitly or implicitly, are not endorsed by and do not represent any of my previous, current or future employers.