[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [tracecompass-dev] Kernel Lock Analysis
- From: Matthew Khouzam <matthew.khouzam@xxxxxxxxxxxx>
- Date: Tue, 10 Apr 2018 16:05:54 -0400
- Delivered-to: firstname.lastname@example.org
- User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0
Wow, this is really nice, would you be interested in contributing this
to the incubator, it looks like a prime candidate for the "LTTng Extras".
On 18-04-10 12:38 PM, aleix wrote:
> Hello List!
> Just as I have done with the LTTng people I would like to share my
> experience playing with Kernel lock instrumentation, LTTng and Trace
> Compass which might be useful for others when analyzing applications
> Currently, the Linux kernel features a set of lock tracepoints which
> track lock acquisition and contention :
> - lock_acquire
> - lock_acquired
> - lock_contended
> - lock_release
> However, the current version of LTTng does not instrument this
> tracepoints (the code is in the source code but commented ). The
> guys on LTTng IRC helped me enabling them again by just uncommenting
> the code. I have currently reported a bug about this here .
> Thanks to these tracepoints, I could create a couple of Trace Compass
> views (attached in this mail), to easily track the kernel lock
> contention, which clearly showed me what was going on. One view shows
> locking status per each kernel lock and the other shows per thread
> contention for all locks). Please, see the attached screenshot for an
> example of per thread view.
> The application under analysis is a parallel cholesky benchmark run on
> a server with 56 CPUs. I was trying to figure out why almost all
> application threads became blocked at some point as seen in the
> screenshot. The lock view showed that there was a huge contention on
> the mm->mmap_sem lock when all threads tried to allocate memory by
> calling mmap(), mmprotect() and triggered page faults when data is
> written on the recently mmapped memory.
> Hence, I would like to point out how useful it has been for me to
> enable the LTTng lock tracepoints and use the custom lock views. If
> the lock tracepoints are enabled again, I think that it might useful
> for others to add similar views. However, It is worth noting that this
> tracepoints are only enabled if compiling the kernel with
> CONFIG_LOCK_STAT, which is a kernel debugging feature not set by
> Thanks for your work!
>  See Documentation/locking/lockstat.txt on the Linux Kernel source
> for more information.
>  As Compudj pointed out, the reason is found in this conversation:
>  LTTng lock bug: https://bugs.lttng.org/issues/1157
> tracecompass-dev mailing list
> To change your delivery options, retrieve your password, or unsubscribe from this list, visit