Sunday, March 21, 2010

Analyze kernel dump of hanged windows server

Programmer Question

I have a production server that is hanging for around five minutes once or twice a week. During this time it responds to pings but nothing else. After shooting in the dark for a while ( upgrading drivers and firmwares, changing network parameters etc ) we took a kernel dump during such hang. We have send it to Microsoft support. But in the mean time I am trying to look at it. The output of the !locks command is as follow



5: kd> !locks
**** DUMP OF ALL RESOURCE OBJECTS ****
KD: Scanning for held locks.......

Resource @ Ntfs!NtfsData (0xfffffade5ad9bfa0) Shared 1 owning threads
Contention Count = 16
Threads: fffffade70af8bf0-01<*>
KD: Scanning for held locks.......

Resource @ 0xfffffade6a6e7800 Shared 2 owning threads
Contention Count = 4114
NumberOfSharedWaiters = 71
NumberOfExclusiveWaiters = 1
Threads: fffffade69e98530-01<*> fffffade76007040-01<*> fffffade6a076840-01 fffffade6a045bf0-01
fffffade72317040-01 fffffade70af77a0-01 fffffade6a05c040-01 fffffade7a3d2040-01
<lines removed>
fffffade6936e4a0-01 fffffade6b530bf0-01 fffffade6ed36bf0-01 fffffade6b95e040-01
fffffade6fbf6040-01
Threads Waiting On Exclusive Access:
fffffade6a9ddbf0

KD: Scanning for held locks........................................

Resource @ 0xfffffade6abab640 Shared 1 owning threads
Contention Count = 115580
NumberOfSharedWaiters = 2
NumberOfExclusiveWaiters = 2
Threads: fffffadfdb3fb040-01<*> fffffade70af8040-01 fffffade70af8bf0-01
Threads Waiting On Exclusive Access:
fffffade69e98530 fffffade76007040

KD: Scanning for held locks........................

Resource @ 0xfffffade69e0acf0 Exclusively owned
Contention Count = 4
NumberOfExclusiveWaiters = 4
Threads: fffffade70afa040-01<*>
Threads Waiting On Exclusive Access:
fffffade6ecc0bf0 fffffade6bbb9040 fffffade6ecafbf0 fffffade6fb9ebf0

KD: Scanning for held locks.......................

Resource @ 0xfffffade755b0ec0 Exclusively owned
Threads: fffffade76007040-01<*>
KD: Scanning for held locks.

Resource @ 0xfffffade6fdd63c0 Exclusively owned
Threads: fffffade72317040-01<*>

Resource @ 0xfffffade7788c2e0 Exclusively owned
Threads: fffffade7a3d2040-01<*>

Resource @ 0xfffffade71c3f030 Exclusively owned
Threads: fffffade7a920a30-01<*>
KD: Scanning for held locks.

Resource @ 0xfffffade75ebef80 Exclusively owned
Threads: fffffade6ecc5040-01<*>

Resource @ 0xfffffade74529030 Exclusively owned
Threads: fffffade7a978770-01<*>
KD: Scanning for held locks.

Resource @ 0xfffffade7cdfae70 Exclusively owned
Threads: fffffade727e69f0-01<*>

Resource @ 0xfffffade8e69e030 Exclusively owned
Threads: fffffade765230c0-01<*>

Resource @ 0xfffffade7dfa6030 Exclusively owned
Threads: fffffade82d248c0-01<*>

Resource @ 0xfffffade6aa52540 Exclusively owned
Threads: fffffade75428760-01<*>

Resource @ 0xfffffade5be9b930 Exclusively owned
Threads: fffffade70af8bf0-01<*>
KD: Scanning for held locks.

Resource @ 0xfffffade6eeaf3f0 Exclusively owned
Threads: fffffade6b6615a0-01<*>
KD: Scanning for held locks.

Resource @ 0xfffffade7db22030 Exclusively owned
Threads: fffffade6fbf6040-01<*>
118264 total locks, 17 locks currently held


What does this tell me? What should I do next?
Thanks.



Find the answer here

No comments:

Post a Comment

LinkWithin

Related Posts with Thumbnails