How to find out exactly which process was killed by OOM killer?

Question

OOM killer just killed some process. I can know about it when the value of grep oom_kill /proc/vmstat increases.

How would I know which process exactly was killed (name, pid, owner and so on) and when exactly (timestamp)?

Answer 1

Look for entries from OOM killer events in the logs.

sudo grep -i "oom-killer" /var/log/messages

Result looks something like this:

kernel: [timestamp] Out of memory: Kill process [process_name] ([process_pid]), UID [user_id]/[username], VmSize:[memory_size] kB, VmRSS:[resident_memory_size] kB, MemLimit:[memory_limit] kB

… or /var/log/syslog. And some related events may be described with OOM killer or so, thus grep -i 'oom.kill' may be a good idea.

Answer 2

The crude, but easy way to know details on recent OOM-kill is to grep everywhere (proper log path may differ from distrib to distrib):

sudo dmesg -T | grep -Ei 'killed process|oom.killer' sudo grep -Ei 'killed process|oom.killer' /var/log/messages /var/log/syslog 2>/dev/null

. means “any character” in regexp for grep, so it will find lines containing both “oom-killer” and “oom killer”.

2>/dev/null on the end is needed to get rid of the “No such file or directory” error, if there is no such logfiles in your system.

Answer 3

So you may have found an out of memory event in your kernel log. It will start with*'[process] invoked oom-killer’* followed by a stack trace and and a list of running processes and child processes.

As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.