This appendix describes the differences in the implementation of Wrapfs from the initial port (Solaris) to the ports that followed: Linux and FreeBSD.
When we began the Solaris work we referred to the implementation of other file systems such as lofs. Linux 2.0 did not have a loopback file system as part of standard distributions, but we were able to locate a prototype2 and use it in our port.
The Linux Vnode and VFS interfaces contains a different set of functions and data structures than Solaris, but it operates in a similar fashion. In Linux, much of the common file system code was extracted and moved to a generic (higher) level. Many generic file system functions exist that can be used by default if the file system does not define its own version thereof. This leaves the file system developer to deal with only the core issues of the file system. For example, Solaris User I/O (uio) structures contain various fields that must be updated carefully and consistently. Linux simplifies data movement by passing vnode functions such as read and write a simple allocated (char *) buffer and an integer describing how many bytes to read into or write out of the buffer passed.
Memory mapped operations are also easier in Linux. The vnode interface in Solaris includes functions that must be able to manipulate one or more pages. In Linux, a file system handles one page at a time, leaving page clustering and multiple-page operations to the higher and more generic code.
Directory reading was much simpler in Linux. In Solaris, we had to read a number of raw bytes from the lower level file system, and parse them into chunks of sizeof(struct dirent), set the proper fields in this structure, and append the file name bytes to the end of the structure. In Linux, we provided the kernel with a callback function for iterating over the entries in a directory. This function was called by higher level code and asked us to simply process one file name at a time.
There was only one caveat to the portability of the Linux code. Most of the structures used in the file system (inode, super_block, and file) include a private field into which file system specific opaque data could be placed. We used this field to store information pertinent for stacking. We had to add a private field to only one structure which was missing it, the vm_area_struct. This structure represents custom per-process virtual memory manager page-fault handlers. Since Wrapfs is the first fully stackable file system for Linux, we feel that these changes are small and acceptable, given that more stackable file systems are likely to be developed.3
FreeBSD 3.0 is based on BSD-4.4Lite. We chose it as the third port because it represents another major section of Unix operating systems--the BSD ones. FreeBSD's vnode interface is very similar to Solaris's and the port was straightforward. FreeBSD's version of the loopback file system is called nullfs[15], a useful template for writing stackable file systems. Unfortunately, ever since the merging of the VM and Buffer Cache in FreeBSD 3.0, stackable file systems stopped working because of the inability of the VFS to correctly map data pages of stackable file systems to their on-disk locations. We had to work around these deficiencies in nullfs by forcing all writes to be synchronous and by implementing getpages and putpages using read and write, respectively.