EMDIAG Repvfy 12c kit - basics
Second post in the series of emdiag repvfy kit about the basics of the tool. Having the kit already installed in earlier post it is time now to get some basics before we start troubleshooting.
There are three main commands with repvfy:
verify - repository-wide verification
analyze - objects specific verification/analysis
dump - dump specific repository object
As you can tell from the description, the verify is run against the repository and doesn't require any arguments by default while analyze and dump commands require specific object to be given. To get a list of all available commands of the kit run repvfy -h1.
Verify command
Let's make something clear in the begging, the verify command will run repository-wide verification against many tests which are FIRST grouped into modules and SECOND categorized in several levels. To get a list of available modules run repvfy -h4, there are more than 30 modules and I won't go into detail for each, but most useful are - Agents, Plugins, Exadata, Repository, Targets. The list of levels can be found at the end of the post, it's important to say that levels are cumulative and by default tests are run in level 2!
When investigate or debug a problem with the repository always start with verify command. It's a good starting point to run verify without any arguments, it will go through all modules and give you summary if certain problems (violations) are present and also get an initial look on the health of the repository and then start debugging specific problem.
So here is how verify output looked for my OEM repository:
[oracle@oem bin]$ ./repvfy verify
Please enter the SYSMAN password:
-- --------------------------------------------------------------------- --
-- REPVFY: 2013.1008 Repository: 12.1.0.3.0 29-Oct-2013 11:30:37 --
---------------------------------------------------------------------------
-- Module: Test: 0, Level: 2 --
-- --------------------------------------------------------------------- --
verifyAGENTS
verifyASLM
verifyAVAILABILITY
1002. Disabled response metrics (16570376): 2
verifyBLACKOUTS
verifyCAT
verifyCORE
verifyECM
1002. Unregistered ECM metadata tables: 2
verifyEMDIAG
verifyEVENTS
verifyEXADATA
2001. Exadata plugin version mismatches: 5
verifyJOBS
verifyJVMD
verifyLOADERS
verifyMETRICS
verifyNOTIFICATIONS
verifyOMS
verifyPLUGINS
1003. Plugin metadata versions out of sync: 13
verifyREPOSITORY
verifyTARGETS
1021. Composite metric calculation with inconsistent dependant metadata versions: 3
2004. Targets without an ORACLE_HOME association: 2
2007. Targets with unpromoted ORACLE_HOME target: 2
verifyUSERS
The verify command can also be run with -detail argument to get more details for the problem found. It will also show which test found the problem and what actions can be taken to correct the problem. That's useful for another reason - it will print the target name and guid which can be used for further analysis using analyze and dump commands.
The command can also be run with -level argument, starting with zero for a fatal errors and increasing to nine for more minor errors and best practices, list of levels can be found at the end of the post.
Analyze command
Analyze command is run against specific target which can be specific either by its name or its unique identifier (guid). To get a list of supported targets run repvfy -h5. The analyze command is very similar to the verify command except it is run against specific target. Again it can be run with -level and -detail arguments, like this:
[oracle@oem bin]$ ./repvfy analyze exadata -guid 6744EED794F4CCCDBA79EC00332F65D3 -level 9
Please enter the SYSMAN password:
-- --------------------------------------------------------------------- --
-- REPVFY: 2013.1008 Repository: 12.1.0.3.0 29-Oct-2013 12:00:09 --
---------------------------------------------------------------------------
-- Module: EXADATA Test: 0, Level: 9 --
-- --------------------------------------------------------------------- --
analyzeEXADATA
2001. Exadata plugin version mismatches: 1
6002. Exadata components without a backup Agent: 4
6006. Check for DB_LOST_WRITE_PROTECT: 1
6008. Check for redundant control files: 5
For that Exadata target we can see there are few more problems found with level 9 except the one found earlier about plugin version mismatch with level 2.
One of the next posts will be dedicated to troubleshooting and fixing problems in Exadata module.
Dump command
Dump command is used to dump all the information about specific repository object, as analyze command it expects either target name or target guid. For a list of supported targets run repvfy -h6.
I won't show any example because it will dump all the details about that target - more than 2000 lines. If you run the dump command against the same target used in analyze you will get a ton of information like - associated targets with this Exadata (hosts, iloms, databases, instances), list of monitoring agents, plugin version, some address details, long list of targets alerts/warnings.
Seems rather useless because it just dumps a lot of information but actually it helped me identifying the problem I had about plugin version mismatch within the Exadata module.
Repository verification and object analysis levels:
0 - Fatal issues (functional breakdown)
These test highlight fatal errors found in the repository. These errors will prevent EM from functioning
normally and should get addressed straight away.
1 - Critical issues (functionality blocked)
2 - Severe issues (restricted functionality)
3 - Warning issues (potential functional issue)
These tests are meant as 'warning', to highlight issues which could lead to potential problems.
4 - Informational issues (potential functional issue)
These tests are informational only. They represent best practices, potential issues, or just areas to verify.
5 - Currently not used
6 - Best practice violations
These test highlight discrepancies between the known best practices, and the actual implementation
of the EM environment.
7 - Purging issues (obsolete data)
These test highlight failures to clean up (all the) historical data, or problems with orphan data
left behind in the repository.
8 - Failure Reports (historical failures)
These test highlight historical errors that have occurred.
9 - Tests and internal verifications
These tests are internal tests, or temporary and diagnostics tests added to resolve specific problems.
They are not part of the 'regular' kit, and are usually added while debugging or testing specific issues.
In the next post I'll troubleshoot and fix the errors I had within the Availability module - Disabled response metrics.
For more information and examples refer to following notes:
EMDIAG Repvfy 12c Kit - How to Use the Repvfy 12c kit (Doc ID 1427365.1)
EMDIAG REPVFY Kit - Overview (Doc ID 421638.1)
EMDIAG repvfy blog series:
EMDIAG Repvfy 12c kit – installation
EMDIAG Repvfy 12c kit – basics
EMDIAG Repvfy 12c kit – troubleshoot Availability module
EMDIAG Repvfy 12c kit – troubleshoot Exadata module
EMDIAG Repvfy 12c kit – troubleshoot Plugins module
EMDIAG Repvfy 12c kit – troubleshoot Targets module