Database 11.2 bug causes huge number of alert log entries

Svetoslav Gyurov

Dec 22, 2011 • 1 min read

Few days ago I received a call from customer about problem with their EM console and messages about file system full. They run DB 11.2.0.2 on OEL 5.7 and had only binaries installation at that file system and the database itself was using ASM. I quickly logged on to find out the file system was really full and after looking around I figure out that all the free space was eaten by alert and trace diagnostic directories. The trace directory was full of 10MB files and the alertlog file was quick growing with following messages:

WARNING: failed to read mirror side 1 of virtual extent 2917 logical extent 0 of file 271 in group [1.2242406296] from disk DATA_0000 allocation unit 24394 reason error; if possible,will try another mirror side
Errors in file /oracle/app/oracle/diag/rdbms/baandb/baandb/trace/baandb_ora_17785.trc:
WARNING: Read Failed. group:1 disk:0 AU:24394 offset:1007616 size:8192
WARNING: failed to read mirror side 1 of virtual extent 2917 logical extent 0 of file 271 in group [1.2242406296] from disk DATA_0000 allocation unit 24394 reason error; if possible,will try another mirror side
Errors in file /oracle/app/oracle/diag/rdbms/baandb/baandb/trace/baandb_ora_17785.trc:

At first I though there is a storage problem, but looking at the ASM views everything seemed to be all right and these seemed to be false messages. I deleted all the trace files, but then few minutes later the file system became again full. It turned out that generated log per minute were more than 60MBor around 7GB for two hours, because of this huge number of messages the machine was already loaded.

Then after quick MOS search I found that this is a Bug 10422126: FAILED TO READ MORROR SIDE 1 and there is a 70KB patch for 11.2.0.2.

The following MOS notes are also useful:
WARNING: 'Failed To Read Mirror Side 1' continuously reported in the alert log [ID 1289905.1]
Huge number of alert log entries: 'WARNING: IO Failed...' 'WARNING: failed to read mirror side 1 of virtual extent ...' [ID 1274852.1]

After applying the patch everything became normal and no more false messages appeared in the logs. The bug is fixed in 11.2.0.3.

Regards,
Sve

Sign up for more like this.