Skip to content

snua12/zlomekfs

Repository files navigation

Preparing a single-node ZlomekFS configuration
==============================================
You can find detailed documentation of the configuration parameters in
Josef Zlomek's thesis (doc/Zlomek-SharedFileSystem.pdf).

To run ZlomekFS on a single node, you need to create the following directories:
* a local configuration (TOP/local-config)
* a configuration volume cache (TOP/shared-config)
* a data volume cache (TOP/data)

The configuration volume and data volume will be visible in the mounted
ZlomekFS file system.

Contents of the TOP/local-config directory:
-------------------------------------------
* zfsd.conf
	# Example ZlomekFS configuration file
	# configuration version
	version = "1.0";

	# configuration of local node
	local_node:
	{
		name = "the_only_node";
		id = 1;
	};

	# volume list
	volumes:
	(
	{ 
	#	name = "shared-config";
		id = 1;
		cache_size = 0;
		local_path = "/var/zfs/config";
	},
	{
	#	name = "data";
		id = 2;
		local_path = "/var/zfs/data";
		cache_size = 0;
	}
	);

	# system specific configuratiom
	system:
	{
		mlock = false;
		metadata_tree_depth = 1;
		local_config = "/var/zfs/config";
	};

	threads:
	{
	#max_total = 10;
	#min_spare = 10;
	#max_spare = 10;
	};

	users:
	{
		default_user = "nobody";
	#	default_uid = 65534;
	};


	groups:
	{
		default_group = "nogroup";
	#	default_gid = 65534;
	};

Contents of the TOP/shared-config directory:
--------------------------------------------
* subdirectories "group", "user", "volume"
* node_list
	node:
	{
		list=
		(
		{
			id=1;
			name="the_only_node";
			address="127.0.0.1";
		}
		);
	};
* volume_list
	volume:
	{
		list =
		(
			{
				id = 1;
				name = "config";
				mountpoint="/config";
			},
			{
				id = 2;
				name = "data";
				mountpoint="/data";
			}
		);
		layout =
		(
			{
				volume = "config";
				tree =
				{
					node="the_only_node";
					# list of subtrees
					children = ();
				};
			},
			{
				volume = "data";
				{
					node="the_only_node";
					# list of subtrees
					children = ();
				};

			}
		);

	};
* user_list:
	user:
	{
		list = 
		(
			{
				id = 65534;
				name = "nobody";
			}
		);
	};
* group_list:
	group:
	{
		# group id and group name list
		list = 
		(
			{
				# group id
				id = 65534;
				# group name
				name = "nogroup";
			}
		);
	};
* group/default:
	# group configuration
	group:
	{
		# mapping between local and remote node
		mapping = 
		(
			{
				# default is reserved name for default configuration, don't use default for node name
				node="default";
				pairs=
				(
					{
						local="nogroup";
						remote="nogroup";
					}
				);
			}
		);
	};
* user/default:
	user:
	{
		mapping = 
		(
			{
				node = "default";
				pairs =
				(
					{
						local = "nobody";
						remote = "nobody";
					}
				);
			}
		);
	};


Linux
=====
(Tested on Fedora 7, kernel 2.6.22)
(Tested on Ubuntu 10.10 kernel 2.6.35)

Compilation
-----------
* unpack zfs.tar.bz2 (to SRC/zfs)
* cd ../zfsd
* cmake .. -DTESTS=all -DCONFIGURATION=Debug
* make
* Optionally, run (make install) to install zfsd to /usr/sbin.

Running
-------
* as an unprivileged user:
  - SRC/zfs/zfsd/zfsd -o config=TOP/local-config/config MOUNT_POINT
    zfsd will stay in the foreground.  To finish and unmount the file system,
    terminate zfsd by pressing Ctrl-C.


Configuring ZlomekFS for multiple-node operation
================================================
To configure nodes A and B to support disconnected operation, start with the
single-node configuration, and modify it as follows on node A:
* change local_node name in zfsd.conf
	local_node:(name="node_a"; id=1;);
* change TOP/shared-config/node_list
  	node:
	{
		list=
		(
		{
			id=1;
			name="node_a";
			address="HOST_NAME_OF_NODE_A"
		},
		{
			id=2;
			name="HOST_NAME_OF_NODE_B";
		}
		);
	}
  (or use IP addresses instead of host names)
* change volume_list to
	volume:
	{
		list =
		(
			{
				id = 1;
				name = "config";
				mountpoint="/config";
			},
			{
				id = 2;
				name = "data";
				mountpoint="/data";
			}
		);
		layout =
		(
			{
				volume = "config";
				tree =
				{
					node="node_a";
					# list of subtrees
					children = (
					{
						node="node_b";
						children = ();
					});
				};
			},
			{
				volume = "data";
				tree =
				{
					node="node_a";
					# list of subtrees
					children = (
					{
						node="node_b";
						children = ();
					});
				};

			}
		);

	};

On node B, create the TOP/{local-config,shared-config,data} directories, and
set up the following contents of TOP/local-config:
* zfsd.conf
	# Example ZlomekFS configuration file
	# configuration version
	version = "1.0";

	# configuration of local node
	local_node:
	{
		name = "node_b";
		id = 2;
	};

	# volume list
	volumes:
	(
	{ 
	#	name = "shared-config";
		id = 1;
		cache_size = 0;
		local_path = "/var/zfs/config";
	},
	{
	#	name = "data";
		id = 2;
		local_path = "/var/zfs/data";
		cache_size = 0;
	}
	);

	# system specific configuratiom
	system:
	{
		mlock = false;
		metadata_tree_depth = 1;
		local_config = "/var/zfs/config";
	};
	users:
	{
		default_user = "nobody";
	};
	groups:
	{
		default_group = "nogroup";
	};

Make sure port 12323 is open on both nodes.

On node A, run
	SRC/zfs/zfsd/zfsd -o config=TOP/local-config/config MOUNT_POINT
On node B, run
	SRC/zfs/zfsd/zfsd -o \
	  config=TOP/local-config/config,node=1:node_a:HOST_NAME_OF_NODE_A \
	  MOUNT_POINT

To test conflict handling, modify FILE on node A when node B is not connected,
and modify the same FILE on node B when node A is not connected.  Then start
both nodes and on node B (not on the master node A - conflicts are always
handled at the "lowest level") look at FILE - it should be a directory
containing files "node_a" and "node_b".  Remove one of the files, and FILE will
be replaced by the other version.