Web Analytics

Exadata onecommand fails at cell disk creation

I was installing another Exadata last month when I got an error on create cell disks step. I've seen the same error before when I was extending two to three rack Exadata configuration but thought it was one-off.

The cell disk creation failed as below:

[root@exa01db01 linux-x64]# ./install.sh -cf Customer-exa01.xml -s 8

 Executing Create Cell Disks
 Checking physical disks for errors before creating celldisks.........................
 Restarting cell services....................................................

 Stopping the RS, CELLSRV, and MS services...
 The SHUTDOWN of services was successful.
 Starting the RS, CELLSRV, and MS services...
 Getting the state of RS services...  running
 Starting CELLSRV services...
 The STARTUP of CELLSRV services was not successful.
 CELL-01533: Unable to validate the IP addresses from the cellinit.ora file because the IP addresses may be down or misconfigured.
 Starting MS services...
 The STARTUP of MS services was successful.

Going through the cell configuration is obvious why the process failed. The cell still had the default name and the IP addresses that the cell services should use are still the default ones:

CellCLI> list cell detail
         name:                   ru02
         cellsrvStatus:          stopped
         msStatus:               running
         rsStatus:               running

In short when you see an error like the one below then your ipaddress1 and/or ipaddress2 fields are most probably wrong:

         2       2015-12-15T17:57:03+00:00       critical        "ORA-00700: soft internal error, arguments: [main_6a], [3], [IP addresses in cellinit.ora not operational], [], [], [], [], [], [], [], [], []"

The solution to that is simple. You need to alter the cell name and IP addresses manually:

CellCLI> alter cell name=exa01cel02a,ipaddress1='',ipaddress2=''
Network configuration altered. Please issue the following commands as root to restart the network and open IB stack:
service openibd restart
service network restart
A restart of all services is required to put new network configuration into effect. MS-CELLSRV communication may be hampered until restart.
Cell exa01cel02a successfully altered

CellCLI> alter cell restart services all

Make sure all cells are fixed and re-run the onecommand step, this time it will succeed:

 Successfully completed execution of step Create Cell Disks [elapsed Time [Elapsed = 128338 mS [2.0 minutes] Thu Dec 17 14:26:59 GMT 2015]]

I've checked some older deployments and it's the same step which should change the cell name and restart the cell services. For some reason this didn't happened for me. For both deployments I used OEDA v15.300 (Oct 2015) so this might be a bug in this version.