In addition, the file should end with the following line:
END OF SYSTEM STATE 数据挖掘研究院
Actually, because the system state information has been dumped twice into the same trace file, the above patterns should appear twice.
Still not a rocket science... - individual system state dumps using OS debugger
You are experiencing a severe database hang and can′t even login to the database on a particular node. Then you have to use Linux ′gdb′ debugger to generate a system state dump on every node where you can′t connect via SQL*Plus.
Following is the how-to steps. 数据挖掘研究院
Pick a user (a.k.a. shadow) process. Do not attach with gdb to a background process - you can crash the entire instance if the background process dumps the core during the procedure. The example assumes that the database name is ′myrac′: 数据挖掘研究院
dbhost|/home/oracle > ps -ef | grep oraclemyrac | grep -v grep
oracle 8136 1 0 Sep22 ? 00:00:01 oraclemyrac1 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))
oracle 8425 1 0 Sep22 ? 00:00:00 oraclemyrac1 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))
oracle 8433 1 0 Sep22 ? 00:00:00 oraclemyrac1 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))
oracle 8439 1 0 Sep22 ? 00:00:00 oraclemyrac1 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))
oracle 15965 15964 0 09:20 ? 00:00:00 oraclemyrac1 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))
数据挖掘研究院
Let′s choose the shadow process with PID=15965 as a victim for gdb. Invoke the debugger and attach to the above process. Do not forget you have to tell gdb which executable file the process is running:
Dbhost:/home/oracle > which gdb
/usr/bin/gdb
Dbhost://home/oracle > gdb $ORACLE_HOME/bin/oracle 15965
GNU gdb Red Hat Linux (6.1post-1.20040607.52rh)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux-gnu"...(no debugging symbols found)...Using host libthread_db library "/lib/tls/libthread_db.so.1".
Attaching to program: /usr/app/oracle/product/10.2.0/db_1/bin/oracle, process 15965
Reading symbols from /usr/app/oracle/product/10.2.0/db_1/lib/libskgxp10.so...(no debugging symbols found)...done.
Loaded symbols for /usr/app/oracle/product/10.2.0/db_1/lib/libskgxp10.so
Reading symbols from /usr/app/oracle/product/10.2.0/db_1/lib/libhasgen10.so...done.
Loaded symbols for /usr/app/oracle/product/10.2.0/db_1/lib/libhasgen10.so
Reading symbols from /usr/app/oracle/product/10.2.0/db_1/lib/libskgxn2.so...done.
Loaded symbols for /usr/app/oracle/product/10.2.0/db_1/lib/libskgxn2.so
Reading symbols from /usr/app/oracle/product/10.2.0/db_1/lib/libocr10.so...done.
Loaded symbols for /usr/app/oracle/product/10.2.0/db_1/lib/libocr10.so
Reading symbols from /usr/app/oracle/product/10.2.0/db_1/lib/libocrb10.so...done.
Loaded symbols for /usr/app/oracle/product/10.2.0/db_1/lib/libocrb10.so
Reading symbols from /usr/app/oracle/product/10.2.0/db_1/lib/libocrutl10.so...done.
Loaded symbols for /usr/app/oracle/product/10.2.0/db_1/lib/libocrutl10.so
Reading symbols from /usr/app/oracle/product/10.2.0/db_1/lib/libjox10.so...done.
Loaded symbols for /usr/app/oracle/product/10.2.0/db_1/lib/libjox10.so
Reading symbols from /usr/app/oracle/product/10.2.0/db_1/lib/libclsra10.so...done.
Loaded symbols for /usr/app/oracle/product/10.2.0/db_1/lib/libclsra10.so
Reading symbols from /usr/app/oracle/product/10.2.0/db_1/lib/libdbcfg10.so...done.
Loaded symbols for /usr/app/oracle/product/10.2.0/db_1/lib/libdbcfg10.so
Reading symbols from /usr/app/oracle/product/10.2.0/db_1/lib/libnnz10.so...done.
Loaded symbols for /usr/app/oracle/product/10.2.0/db_1/lib/libnnz10.so
Reading symbols from /usr/lib/libaio.so.1...done.
Loaded symbols for /usr/lib/libaio.so.1
Reading symbols from /lib/libdl.so.2...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /lib/tls/libm.so.6...done.
Loaded symbols for /lib/tls/libm.so.6
Reading symbols from /lib/tls/libpthread.so.0...done.
[Thread debugging using libthread_db enabled]
[New Thread -1218553600 (LWP 15965)]
Loaded symbols for /lib/tls/libpthread.so.0
Reading symbols from /lib/libnsl.so.1...done.
Loaded symbols for /lib/libnsl.so.1
Reading symbols from /lib/tls/libc.so.6...done.
Loaded symbols for /lib/tls/libc.so.6
Reading symbols from /lib/ld-linux.so.2...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /lib/libnss_files.so.2...done.
Loaded symbols for /lib/libnss_files.so.2
0x0011ca4e in __read_nocancel () from /lib/tls/libpthread.so.0
(gdb)
