Linux : Oracle unbreakable kernel and OCFS2 on CentOS/RHEL6

By | August 8, 2014
0 Comment

Oracle Cluster File System 2 (well known as OCFS2) is a pretty decent choice when come to chose a filesystem that need to be accessed and written using several server nodes. Unfortunately, RedHat do not support by default OCFS2 in their kernel since they are using GFS instead.

I am not aware of any unofficial kmod-ocfs2 module available for RHEL6. It seem that Oracle is making life more difficult to implement it in the kernel, forcing people to use their “UBLK” kernel instead. Compiling from source might work, however since my current use case will be for database server, Oracle kernel database optimized will be just fine!

Here is the procedure how to install Oracle UBK and OCFS2 on CentOS / RedHat Enterprise Linux 6.

1. First add the Oracle Yum repository for RHEL 6 :

/etc/yum.repos.d/public-yum-ol6.repo

And append the following :

[ol6_ga_base]
name=Oracle Linux 6 GA - $basearch - base
baseurl=http://public-yum.oracle.com/repo/OracleLinux/OL6/0/base/$basearch/
gpgkey=http://public-yum.oracle.com/RPM-GPG-KEY-oracle-ol6
gpgcheck=1
name=Oracle Linux 6 GA - $basearch - base
enabled=1

2. Install the kernel and OCFS2 tools :

yum install kernel-uek ocfs2-tools

3. Restart your system and boot on your new kernel.

5. Make folder /etc/ocfs2 and create cluster.conf :

mkdir /etc/ocfs2
vi /etc/ocfs2/cluster.conf

Then append the following :

cluster:
       node_count=<maximum number of node that will access the filesystem>
       name=<label of your ocfs2 volume>
node:
        ip_port = 7777
        ip_address = <server1 ip address>
        number = <node numerical id>
        name = <node1 fqdn>
        cluster = <label of your ocfs2 volume>

node:
        ip_port = 7777
        ip_address = <server2 ip address>
        number = <node numerical id>
        name = <node2 fqdn>
        cluster = <label of your ocfs2 volume>

Example :

cluster:
       node_count=64
       name=san01vd02v001

node:
        ip_port = 7777
        ip_address = 10.10.30.1
        number = 1
        name = node1.itechlounge.net
        cluster = san01vd02v001

node:
        ip_port = 7777
        ip_address = 10.10.30.2
        number = 2
        name = node2.itechlounge.net
        cluster = san01vd02v001

[...add as many nodes you need]

6. Configure the Oracle Cluster Stack for OCFS2 (which is called o2cb) :

service o2cb configure

Then answer the questions that suite well your setup and match your “cluster.conf” settings (values assume the example mentioned above) :

Load O2CB driver on boot (y/n) [y]: y
Cluster stack backing O2CB [o2cb]: 
Cluster to start on boot (Enter "none" to clear) [san01vd02v001]: 
Specify heartbeat dead threshold (>=7) [31]: 
Specify network idle timeout in ms (>=5000) [30000]: 
Specify network keepalive delay in ms (>=1000) [2000]: 
Specify network reconnect delay in ms (>=2000) [2000]:

This will actually write a file under the following path :

/etc/sysconfig/o2cb

And should look like :

# O2CB_ENABLED: 'true' means to load the driver on boot.
O2CB_ENABLED=true

# O2CB_STACK: The name of the cluster stack backing O2CB.
O2CB_STACK=o2cb

# O2CB_BOOTCLUSTER: If not empty, the name of a cluster to start.
O2CB_BOOTCLUSTER=san01vd02v001

# O2CB_HEARTBEAT_THRESHOLD: Iterations before a node is considered dead.
O2CB_HEARTBEAT_THRESHOLD=

# O2CB_IDLE_TIMEOUT_MS: Time in ms before a network connection is considered dead.
O2CB_IDLE_TIMEOUT_MS=

# O2CB_KEEPALIVE_DELAY_MS: Max time in ms before a keepalive packet is sent
O2CB_KEEPALIVE_DELAY_MS=

# O2CB_RECONNECT_DELAY_MS: Min time in ms between connection attempts
O2CB_RECONNECT_DELAY_MS=

NOTE : Both configuration above (cluster.conf and o2cb) have to be copied on all nodes.

7. Format your OCFS2 filesystem :

mkfs.ocfs2 /dev/<device_name> -N <number_of_nodes> -b <block_size: 512 | 1K | 2K | 4K> -C <cluster_size: 4K | 8K | 16K | 32K | 64K | 128K | 256K | 512K | 1M> -T <filesystem_type: mail | datafiles> --fs-features=<optional_features> -L "<label_name>"

(See ocfs2 man page for more details and options, you may want to go through the whole list to tune it according your needs.)

Here is an example of what I used :

mkfs.ocfs2 /dev/mapper/mpatha -N 64 -b 4K -C 256K -T mail --fs-features=extended-slotmap --fs-feature-level=max-features -L "san01vd02v001"

Then just mount your volume :

mkdir /mnt/san01_vd02_v001
mount -t ocfs2 /dev/mapper/mpatha /mnt/san01_vd02_v001