I started subscribing linux kernel mailing list with my gmail account in 2013. Dealing with the large amount of mails was not that easy, probably due to my poor setup and being lazy at trying more tools.
In 2019, while returning from Linux Plumbers Conference, I read an LWN article saying Linus Torvalds is now considering unsubscribing the linux kernel mailing list, owing to the evolvement of the public-inbox and lore.
After about 8 years of progress on my little hobby project[1], the Korean translation of Paul E. McKenney ‘s book, I’m marking it as completed a humble milestone[2]. It was a very fun and informative journey.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/paulmck/perfbook.git/commit/?id=edbfcdee0460 [2] https://lore.kernel.org/perfbook/20240211175355.4986-1-sj38.park@gmail.com/
Below is a list of news around DAMON project.
This list is not exhaustive but just a DAMON maintainer’s collection of news. If you find a news that should be added to this list, please let us know at sj@kernel.org and/or damon@lists.linux.dev.
2024 2024-04-03: Oracle released a tool that helps finding distros that enabled DAMON
2024-04-01: DAMO v2.2.8 is out. This version supports recording memory footprint of monitoring target processes together with their access pattern.
This document helps you estimating the amount of benefit that you could get from DAMON-based system optimizations, and describes how you could achieve it.
Check The Signs No optimization can provide same extent of benefit to every case. Therefore you should first guess how much improvements you could get using DAMON. If some of below conditions match your situation, you could consider using DAMON.
Low IPC and High Cache Miss Ratios.
I stopped translation of perfbook since 2019-09-27. Because the 2nd edition of perfbook is released[0], I’m starting the translation again[1], from the scratch. The old versions are still available, of course[2].
[0] https://mirrors.edge.kernel.org/.../perfbook/perfbook.html
[1] https://github.com/sjp38/perfbook-ko_KR/commit/1c44ef30179b
[2] https://github.com/sjp38/perfbook-ko_KR-pdf
Read-copy update (RCU) can provide ideal scalability for read-mostly workloads, but some believe that it provides only poor performance for updates. This belief is due to the lack of RCU-centric update synchronization mechanisms. RCU instead works with a range of update-side mechanisms, such as locking. In fact, many developers embrace simplicity by using global locking. Logging, hardware transactional memory, or fine-grained locking can provide better scalability, but each of these approaches has limitations, such as imposing overhead on readers or poor scalability on non-uniform memory access (NUMA) systems, mainly due to their lack of NUMA-aware design principles.
With increasingly data-intensive workloads and limited DRAM capacity, optimal memory management based on dynamic access patterns is becoming increasingly important. Such mechanisms are only possible if accurate and efficient dynamic access pattern monitoring is available.
DAMON is a Data Access MONitoring framework subsystem for the Linux kernel developed for such memory management. It is designed with some key mechanism (refer to Design for the detail) that make it
accurate (the monitoring output is useful enough for DRAM level memory management; It might not be appropriate for CPU Cache levels, though), light-weight (the monitoring overhead is low enough to be applied online), and scalable (the upper-bound of the overhead is in constant range regardless of the size of target workloads).
Memory pressure is inevitable as the size of working sets is rapidly growing while the capacity of dynamic random access memory (DRAM) is not. Meanwhile, storage devices have evolved so that their speed is comparable to the speed of DRAM while their capacity scales are comparable to that of hard disk drives (HDD). Thus, hierarchial memory systems configuring DRAM as the main memory and high-end storages as swap devices will be common.
idle_page_tracking[1] is a simple, stupid toolbox for idle pages tracking. It can be used to get real working set size of a process.
Tools This section describes two tools in the box though more tools exists. You can get more description about each tool from the README in the repository[1].
userprog userprog is a sample synthetic workload for test of other tools. It interactively allocates and access specified pages in the allocated pages.
I wrote a simple, stupid program[1] that showing LKML[2] mails in terminal briefly like twitter livestream in Go language. It is just an early version and so many things to be complemented exists, though. Nevertheless, now it just works as I wanted at first.
References [1] https://github.com/sjp38/lkml
[2] https://en.wikipedia.org/wiki/Linux_kernel_mailing_list