Views
VHACS Virtual Machine (VHACS-VM) Images
VHACS is an storage cloud based on at least 8 (eight) open source technologies running on Linux v2.6.25. There is now an alpha release of the software stack running in a virtualized x86_64 environment. The VHACS-VM images are available now, and oriented towards kernel developers, storage and network systems researchers, C programmers, Python and Shell programmers, and anyone who wants to see a real life open storage cloud on their virtualized x86 PC. Using the VHACS-Vm is the fastest method for users to get involved with the project. This is a very developer oriented release, so you should feel comfortable using a modern CLI Linux shell. A complete development environment is included in the images.
What is VHACS..?
Please have a look at VHACS for more information.
What do the system images look like..?
There are two (vhacs64-west and vhacs64-east) x86_64 VMware v6 compatible images, and there is one BUILD .vmdk containing the source for the VHACS Alpha Preview. The BUILD disk image is distributed seperately in order to reduce the size for obtaining a two (2) node VHACS-VM cloud of your own.
-rw-r--r-- 1 root root 830805854 Jul 16 19:57 VHACS64-BUILD-0.8.15.zip -rw-r--r-- 1 root root 314996216 Jul 16 19:56 VHACS64-East-0.8.15-2.zip -rw-r--r-- 1 root root 299372440 Jul 16 19:56 VHACS64-West-0.8.15-2.zip
# cat md5sum.txt e1923fbea84b2e91739ca1199e4695f4 VHACS64-BUILD-0.8.15.zip 32bc1e16c1961c3fb3e66ea52976060a VHACS64-East-0.8.15-2.zip 160e3cdee8344faedf148c5eb845ed2c VHACS64-West-0.8.15-2.zip
What is included in the VHACS-VM Alpha images..?
Each image has an Debian Etch v4.0 r3 x86_64 installation. The kernel is kernel.org v2.6.25.9 with KDB patches. The VHACS-VM kernel is also configured with ParaVirt-Ops for the least amount of virtualized execution overhead on modern PC hardware.
The .vmx images are configured with BRIDGED networking mode, and are configured to grab a DHCP address.
The .vmx images are configured with 256 MB of system memory.
The .vmx images are configured with three (3) Virtual Disks:
- ) 4 GB for rootfs (sda)
- ) 20 GB for /mnt/build (sdb)
- ) 40 GB for the VHACS Volume Group (sdc)
What are the current hardware requirements for VHACS-VM..?
Currently, a x86_64 CPU *WITH* hardware x86 Virtualization. This means the microprocessor has a Virtualized Memory Management Unit capable of handling multiple page tables from multiple *DIFFERENT* unmodified operating systems.
This by no means that VHACS itself requires hardware x86 virtualization, it simply means for the purpose of the Alpha Preview this is the easiest way to get a high performance open source storage cloud running on recent x86_64 microprocessors containing this particular feature.
How do I configure the system images..?
- ) /etc/hosts
Based on your DHCP configured IP address (from ifconfig eth0), set the vhacs64-west.linux-iscsi.org and vhacs64-east.linux-iscsi.org hostnames to your locally allocated IP addresses on both vhacs64-west and vhacs64-east VM images.
- ) /etc/vhacs.conf
Configure the storage network and mask:
STORAGE_NETWORK = "192.168.0.0" STORAGE_NETMASK = "24"
Configure the range of IP addresses that are assigned to VHACS cloud from the resource pool.
- WARNING**: Make sure this range is either explictly not in use by your DHCP server, or will not have an affect on your running network.
FIRST_STORAGE_IP = "192.168.0.50" LAST_STORAGE_IP = "192.168.0.99"
Can I run different STORAGE_IFNAME and HEARTBEAT_IFNAME interfaces in the current release of VHACS-VM..?
Most certainly! In most simple example, this consists of:
- ) Having two (2) physical hosts running vmware and VHACS-VM. This means any PC with hardware x86_64 virtualization that can run vmware with the current 64-bit VHACS-VM images.
- ) Having two (2) bridged network interfaces on the host machines running vhacs64-[west,east] demo images. They should be running on a different local subnet or network range from each other.
The current setup using two (2) network bridges looks something like:
- ) Having 192.168.0.0/eth0 for STORAGE_IFNAME:
HOST0: 192.168.0.*/ethX via DHCP bridged into vhacs64-east VM
HOST1: 192.168.0.*/ethX via DHCP bridged into vhacs64-west VM
vhacs64-east: 192.168.0.*/eth0 via DHCP
vhacs64-west: 192.168.0.*/eth0 via DHCP
- ) Having 10.10.0.0/eth1 for HEARTBEAT_IFNAME:
HOST0: 10.10.0.5/ethY bridged into vhacs64-east VM
HOST1: 10.10.0.10/ethY bridged into vhacs64-west VM
vhacs64-east: 10.10.0.15/eth1 via static IP
vhacs64-west: 10.10.0.20/eth1 via static IP
Also, you need to change your /etc/ais/openais.conf to the 10.10.0.0 network for broadcast packets. Have a look at VHACS#How_are_the_network_interfaces_within_the_VHACS_design_allocated...3F for an example.
How can I check that cluster communication is working on my VHACS-VM system..?
After booting, you should see:
vhacs64-west:~# crm_mon -1
============
Last updated: Thu Jul 3 00:38:29 2008
Current DC: vhacs64-east (vhacs64-east)
2 Nodes configured.
0 Resources configured.
============
Node: vhacs64-west (vhacs64-west): online
Node: vhacs64-east (vhacs64-east): online
What do I do now that both vhacs64-west and vhacs64-east have active cluster communication..?
- ) Now, you want to assign roles to the nodes in the VHACS cluster. Run the following:
vhacs64-west:~# vhacs node -s ALL vhacs64-west,vhacs64-east
role vhost now set on node vhacs64-west
role storage now set on node vhacs64-west
role vhost now set on node vhacs64-east
role storage now set on node vhacs64-east
- ) You can now allocate first your first VHACS storage cloud!
vhacs64-west:~# vhacs storage -c vmcloud0 -s 200M
storage vmcloud0 has been assigned IP 192.168.0.50
LVM volume vmcloud0 created on vhacs64-west
LVM volume vmcloud0 created on vhacs64-east
Promoting node vhacs64-west
Waiting until sync is finished (be patient) ...
drbd devices configured and formated as ext3 on vhacs64-west and vhacs64-east
master/slave drbd resource created and constrained for storage vmcloud0
iSCSI target iqn.2003-01.org.linux-iscsi:192.168.0.50 created and constrained for storage vmcloud0
iSCSI filesystem for iqn.2003-01.org.linux-iscsi:192.168.0.50 created and constrained for storage vmcloud0
- ) From there, open up another ssh connection to one of the VHACS-VM nodes, and run to start the VHACS monitor
vhacs64-east:~# vhacs cluster -m
[00:56:00] [vhacs64-east] [VHACS cluster storage resources status]
____________________________________________________________________________________________________________________________________________________________________
| | | | | |
| NODE | HA STATUS | FREE STORAGE | STORAGE ROLE | VHOST ROLE |
|________________________________|________________________________|________________________________|________________________________|________________________________|
| | | | | |
| (A)vhacs64-west | online | 39.80G/39.99G | 1 exported | 0 mounted |
| (A)vhacs64-east | online | 39.80G/39.99G | 0 exported | 1 mounted |
|________________________________|________________________________|________________________________|________________________________|________________________________|
____________________________________________________________________________________________________________________________________________________________________
| | | | | |
| STORAGE | PRI/SEC DRBD | DRBD TARGET | ISCSI MOUNT | FREE SPACE |
|________________________________|________________________________|________________________________|________________________________|________________________________|
| | | | | |
| (A)vmcloud0 | (P)vhacs64-west/(S)vhacs64-east| (S)vhacs64-west | (S)vhacs64-east | 179M/194M (96%) |
|________________________________|________________________________|________________________________|________________________________|________________________________|
What if I can't get cluster communication between nodes..?
Try calling the VHACS cluster initialization logic to restart services on both vhacs64-west and vhacs64-east images.
vhacs cluster -I vhacs64-west,vhacd64-east
Again, make double check that the hostnames 'vhacs64-west' and vhacs64-east' in *BOTH* VHACS-VM images /etc/hosts file matchs each VHACS-VM images individual ifconfig output.
What if I only see the local node with 'crm_mon -1' on each VHACS-VM image..?
This seems to be something related to VHACS-VM using UDP broadcast packets for cluster communciation is not working.
The VHACS-VM prototype is a 2.2 Ghz Core 2 Duo w/ 1 GB of DDR2 800 running Debian Etch i386. This local bridged ethernet interface is an on-board Realtek RTL8111/8168B PCI-Express 1 Gb/sec MAC.
FIXME: Note that not all ethernet interfaces support bridging sufficently to run VHACS-VM. Is there a way the user can determine which networking hardware can be used with the various virtualization hypervisors with VHACS-VM images..?
I am running into a problem with VHACS-VM..
As this is a very developer oriented Alpha preview, it is up to YOU to help us find bugs! Here is the 10,000 ft view:
- ) VHACS kernel/systems developer
The included v2.6.25.9 x86_64 kernel is patched with SGI Kernel Debugger. At any time, you can break into the kernel debugger with the good old' PC key 'Pause/Break'.
Note that not all laptops have this key, so you will have to use a USB keyboard to access KDB. :-)
- ) Cluster infrastructure developer
Ok, the cluster stack (please see VHACS for projects that make up the stack) is enabled with verbose debug. You can view the complete logs with 'tail -f /var/log/syslog'
- ) VHACS systems administator
'vhacs cluster -m' is the primary status monitor. Exceptions related to cluster RAs will appear as they occur.
How many VHACS clouds can I create with the Alpha Preview
4-8 should work as long as your network bridge setup is working as expected (see above). Try to keep them under a few GB as well. :-)
More vhacs cluster monitor status
vhacs64-east:~# vhacs cluster -M
_________________________________________________________________________________________________________________________________________________________________________
| | | | | |
| NODE | HA STATUS | FREE STORAGE | STORAGE ROLE | VHOST ROLE |
|_________________________________|_________________________________|_________________________________|_________________________________|_________________________________|
| | | | | |
| | | | | |
| (A)vhacs64-west | online | 39.21G/39.99G | 3 exported | 2 mounted |
| (A)vhacs64-east | online | 39.21G/39.99G | 1 exported | 2 mounted |
|_________________________________|_________________________________|_________________________________|_________________________________|_________________________________|
_________________________________________________________________________________________________________________________________________________________________________
| | | | | |
| NODE | WHEN | OPERATION TYPE | RESOURCE | RETURN CODE |
|_________________________________|_________________________________|_________________________________|_________________________________|_________________________________|
| | | | | |
| vhacs64-west | 2h30m28s ago | initial start | ms_drbd_vmcloud1_CLONE:0 | ERROR |
|_________________________________|_________________________________|_________________________________|_________________________________|_________________________________|
_________________________________________________________________________________________________________________________________________________________________________
| | | | | |
| STORAGE | PRI/SEC DRBD | DRBD TARGET | ISCSI MOUNT | FREE SPACE |
|_________________________________|_________________________________|_________________________________|_________________________________|_________________________________|
| | | | | |
| (A)vmcloud0 | (P)vhacs64-west/(S)vhacs64-east | (S)vhacs64-west | (S)vhacs64-east | 179M/194M (96%) |
| (A)vmcloud1 | stopped/(P)vhacs64-east | (S)vhacs64-east | (S)vhacs64-west | 179M/194M (96%) |
| (A)vmcloud2 | (P)vhacs64-west/(S)vhacs64-east | (S)vhacs64-west | (S)vhacs64-west | 179M/194M (96%) |
| (A)vmcloud3 | (S)vhacs64-east/(P)vhacs64-west | (S)vhacs64-west | (S)vhacs64-east | 179M/194M (96%) |
|_________________________________|_________________________________|_________________________________|_________________________________|_________________________________|