Portal de Periódicos da CAPES

Design of the server for the Spiralog file system

1996; Volume: 8; Issue: 2 Linguagem: Inglês

0898-901X

Christopher Whitaker, J. Stuart Bayley, Rod D. W. Widdowson,

Parallel Computing and Optimization Techniques

ion. The B-tree mapping mechanism uses write-ahead logging to give stability and recoverability guarantees. By combining writeahead logging with a log-structured, on-disk format, the Spiralog server merges file system data and recovery log records into a single, sequential write stream. and bounded mechanism for reconstructing the file system after a failure. Researchers at the University of California, Berkeley, took this process one stage further and treated the whole disk as a single, sequential log where all file system modifications are appended to the tail of the log. Log-structured file system technology is particularly appropriate to the Spiralog file system, because it is designed as a clusterwide file system. The server must support a large number of file system clerks, each of which may be reading and writing data to the disk. The clerks use large write-back caches to reduce the need to read data from the server. The caches also allow the clerks to buffer write requests destined for the server. A log-structured design allows multiple concurrent writes to be grouped together into large, sequential I/Os to the disk. This I/O pattern reduces disk head movement during writing and allows the size of the writes to be matched to characteristics of the underlying disk. This is particularly beneficial for storage devices with redundant arrays of inexpensive disks (RAID). The use of a log-structured, on-disk format greatly simplifies the implementation of an on-line backup capability. Here, the challenge is to provide a consistent snapshot of the file system that can be copied to the backup media while normal operations continue to modify the file system. Because an LFS appends all data to the tail of a log, all data writes within the log are temporally ordered. A complete snapshot of the file system corresponds to the contents of the sequential log up to the point in time that the snapshot was created. By extension, an incremental backup corresponds to the section of the sequential log created since the last backup was taken. The Spiralog backup utility uses these features to provide a fast, on-line, full and incremental backup scheme. We have taken a number of features from the existing log-structured file system implementations, in particular, the idea of dividing the log into fixed-sized segments as the basis for space allocation and cleaning. Fundamentally, however, existing log-structured file systems have been built by using the main body of an existing file system and layering on top of an underlying, log-structured container. This design has been taken to the logical extreme with the implementation of a log-structured disk. For the Spiralog file system, we have chosen to use the sequential log capability provided by the log-structured, on-disk format throughout the file system. The Spiralog server combines logstructured technology with more traditional B-tree technology to provide a general server abstraction. The B-tree mapping mechanism uses write-ahead logging to give stability and recoverability guarantees. By combining write-ahead logging with a log-structured on-disk format, the Spiralog server merges file system data and recovery log records into a single, sequential write stream. The Spiralog file system differs from existing logstructured implementations in a number of other important ways, in particular, the mechanisms that we have chosen to use for the cleaner. In subsequent sections of this paper, we compare these differences with existing implementations where appropriate. Spiralog File System Server Architecture The Spiralog file system employs a client-server architecture. Each node in the cluster that mounts a Spiralog volume runs a file system clerk. The term clerk is used in this paper to distinguish the client component of the file system from clients of the file system as a whole. Clerks implement all the file functions associated with maintaining the file system state with the exception of persistent storage of file system and user data. This latter responsibility falls on the Spiralog server. There is exactly one server for each volume, which must run on a node that has a direct connection to the disk containing the volume. This distribution of function, where the majority of file system processing takes place on the clerk, is similar to that of the Echo file system. The reasons for choosing this architecture are described in more detail in the paper “Overview of the Spiralog File System,” elsewhere in this issue. Spiralog clerks build files and directories in a structured address space called the file address space. This address space is internal to the file system and is only loosely related to that perceived by clients of the file system. The server provides an interface that allows the clerks to persistently map to file space addresses. Internally, the server uses a logically infinite log structure, built on top of a physical disk, to store the file system data and the structures necessary to locate the data. Figure 1 shows the relationship between the clerks and the server and the relationships among the major components within the server. 16 Digital Technical Journal Vol. 8 No. 2 1996 MAPPING LAYER LOG DRIVER CLEANER SERVER CLERK CLERK CLERK