Friday, September 11, 2009

Configuring ASM Clustered File Systems (ACFS) in Oracle 11g Release 2

After having finished the installation of Oracle 11g Release 2 Grid Infrastructure, the next step is to configure your storage.
ASM has always been a layer between the hardware and the database files.
Starting with Oracle 11g Release 2, Oracle has introduced a second cluster file system (next to OCFS2) called ACFS (ASM Cluster File System), to be hosted within ASM This is a major new feature of Oracle 11g Release 2.
This cluster file system not only supports binaries, like the ORACLE_HOME. In fact, what we'll see further on in this post, there is a differentiation between a file system for Oracle Home purposes and for General Purposes.
In order to setup an ASM Cluster File System, you simply start the ASM Configuration Assistant. Source the ORACLE HOME of Grid Infrastructure, make the bin subdir accessible from the PATH variable and simply type: asmca
What you will get is something similar to this (click on the images to get a larger view):

You can see the four tabs, ASM Instances, Disk Groups, Volumes and ASM Cluster File System. We'll cover each of them later on. For now, only the first two tabs are shown.

What we'll have to do first is to create a diskgroup. I happened to be prepared for this, so I created an ASM disk prior to running this Configuration Assistant. Just hit the "Create" button and you'll get:

I have called my first disk group DATA. By clicking on the button "Show Advanced Options" you will see the compatibility parameters that can be set per diskgroup (ASM, Database and ADVM (ASM Dynamic Volume Manager) compatibility). Select the disk(s) and click on OK to create the Disk Group:


Well, I think that was a success:-)
Next, go to the Volumes tab. We will have to create a volume first. By the way, doesn't this sound like an old-fashioned Volume Manager setup? It does, because it actually is. The difference with all the OS's proprietary volume managers is that this one is optimized for Oracle, and that should be enough to prefer this one over anything else.

Hit Create and this is what you will get:

Now, be a little careful here. Notwithstanding the fact that the CA tells you that (in this case) you have 14,91 GB of space, you cannot use it all. I have tried 14,91, 14,9, they all failed. The example above shows 14.5, which worked for me. Don't take all you can get, don't get too greedy:-). You have to reserve some space for management, apparently.

Another success!
Now we're almost there, and the most interesting part has to come yet: The ASM Cluster File System tab:

Again, just hit the Create button to get started:
Interesting choice here: You can choose between a Database Home File System or a General Purpose File System. When you choose your file system to be a database home file system, it will be registered under Grid Infrastructure as a managed resource, so Grid Infrastructure can maintain dependencies (- hey, we can all read, you know...). Anyway, I chose the Database Home File System for DATAVOL01. Just enter the mountpoint (/ora11g in the example), who should own it and to which group it should belong:

There is a button there at the bottom: "Show Command", which is worthy to be investigated. It shows you the commands that will be executed to create the ACFS and the registration command:

And the bottom half of the commands:

After the file system has been created, it needs to be registered, by running a script. This needs to be done by a privileged user (e.g. root):

Screenshot of running the script, and Oh my! It's there! I just created a clustered file system under ASM and it is registered in Grid Infrastructure:

OK. Now we'll configure another file system, this time a General Purpose File System. Starting with the second tab in the Configuration Assistant (Disk Groups), define the name of the disk group (DATABASE). You can see that suddenly I have got another set of disks available. Well I created those earlier, but didn't register yet them with ASMlib, while going through the previous cycle.
This time I have got two disks, and I will both use them for my DATABASE disk group. Still with External Redundancy, because the diskspace is too costly for me.

After the Disk Group has been created, I create a volume on it:

After that, I come to the ASM Cluster File System tab again and now I choose a General Purpose File System. I will register this one as well, hopefully resulting in an automount feature at boot time.

The file system has been created and registered successfully:


Now you have two clustered file systems, one designated for Oracle Homes, the other for anything you would like (like databases?).
Next step is to install Oracle 11g Release 2 Database software. That will be covered in the next post.
Happy testing!

9 comments:

  1. Very informative and very helpful.

    Thanks for sharing it.

    ReplyDelete
  2. I'm implementing an 11.2 Linux RAC cluster within a month and I'll be taking advantage of this feature. Thank you for the screenshots. I picture truly is worth a 1000 words,.

    Good work, thanks.

    ReplyDelete
  3. Thanks for your kind words. It's truly one of the reasons why I am doing this. Have to get started again, however. I haven't had much time to blog lately...

    ReplyDelete
  4. I'm trying to create volumes using asmca. But "Volumes" and "ASM Cluster File Systems" tab are grayed out. How to enable them?

    ReplyDelete
    Replies
    1. I also hit into the same issues. Both tabs were grayed out and disabled. I am on Oracle Linux 64 bit 6 Update 5.

      I am suspecting some asm libraries are not there.

      Delete
  5. ACFS is not for the Database files

    ReplyDelete
  6. You are right, ACFS is not for any filetype that can be stored directly in ASM (see http://oraclever.blogspot.nl/2010/01/acfs-not-for-database-files.html for a complete list of file types that cannot be stored in ACFS)

    ReplyDelete