March 16, 2018

ODA X6-2M – Expanding Storage can be unsuccessful when impatient

In 'A Brief History of Time' (RIP Mr. Hawking), Chapter 2 is named 'Space and Time' – and this title fits perfect with a current problem, I was faced with, when expanding disk capacity in an ODA X6-2M.

Long story, told short

An ODA X6-2M, equipped with the standard disk configuration, ran out of space and my customer decided to add two NVMe's. So, he ordered two NVMe's – and, by the way - we were waiting for two weeks (!), until delivery. 

Next step - mounting the disks. According documentation, expanding disk storage is a quite easy job:
  • Put the disks in
  • Set the disks in online state (odaadmcli power disk on pd_02 and odaadmcli power disk on pd_03)
  • Expand storage (odaadmcli expand storage)

This results in my environment in a complete disaster:

  • Disks are already online when trying a 'power disk on',
  • 'expand storage' meant, the disks have already an ASM signature and aborted,
  • 'odaadmcli show disk' shows the disks, but als UNKNOWN and with the /dev/nmv* names of the existing NMVes, whilst the old ones had, all of a sudden, new names … (!)
  • The command to power off said that the disks are not online, but a subsequent remove of the disks resulted in a system crash …

In short: horror!! Opened a Service Request, tried this, tried that – and finally, after three weeks fighting with Oracle Support, I started a last try: Put the disks in, went to the coffee bar, returned about an hour later. Started to set the disks in online state (result: the disks are already online), and issued an 'odaadmcli expand storage' – expecting the same result as with every try before … but, surprisingly, the command returned no error, ASM started rebalancing and everything went fine J

Solution / Chapter 2 ... 'Space and Time'

You'll get the disk space, when You take Your time ...

Put the disks in, wait at least 15 minutes, power on the disks and issue the 'expand storage' command. Oracle Supports explaination of this behavior was (in my words): The oakd process needs some time to prepare the disks in a way that expand storage works properly …



Post a Comment

Copyright © Robert Crames' Oracle Database and Middleware Blog | Powered by Blogger
Design by SimpleWpThemes | Blogger Theme by