Clusterfile

Parallel Distributed File System

Summary

Clusterfile is a parallel file system for clusters of computers. In 2002, we focused on broadening the applicapable areas. The goal of our earlier design was to efficiently utilize internal parallelism of applications. Internal parallelism is created by I/O access of multiple processes of the same application. External parallelism, on the other side, arises from simultaneous access of different applications. Our extensions address external parallelism by introducing not only application-specific, but also system-wide optimizations.

We implemented the file system partially in user level, partially in the Linux kernel. Based on a kernel module supporting the VFS (Virtual File System) interface, Clusterfile may be mounted in the local directory tree of any cluster node. Meta data is managed by cooperation of the kernel module with a central manager instance. We introduced collective I/O operations for optimizing simultaneous access from many nodes to the same file. Furthermore, we implemented an MPI I/O interface for Clusterfile, which we currently compare with other MPI I/O implementations.

For the future, we plan to increase application performance and scalability by introducing cooperatice cacheing. This extension will be done in cooperation with subproject Scalable Servers on Cluster Computers. Furthermore, we will improve scalability of meta data management by decentralization. Currently, we are studying various policies, such as distributing or replicating meta data on the cluster nodes.

Software
Related to competence
Title
Publications
Title Kind of Publication Author (random order) Year

Konferenzartikel 

Florin Isaila, Walter F. Tichy

2001

Konferenzartikel 

Norbert Eicker, Florin Isailă, Thomas Lippert, Thomas Moschny, Walter F. Tichy 

2003 

Technischer Artikel 

Florin Isailă

2003 

Technischer Artikel 

Mike Sibler, Tobias Kluge, Lars Bauer, Philipp Mergenthaler

2003 

Zeitschriftenartikel 

Florin Isaila, Walter F. Tichy 

2003 

Dissertation 

Florin Isailă 

2004 

Thesis subjects
News About The Project