Container testing reveals ‘memory pressure’ on apps linux kernel v2

Container testing reveals ‘memory pressure’ on apps linux kernel v2 With early adopters of application container technology completing early testing in multi-tenant settings, potential performance issues are beginning to surface. Kernel linux android Among them, according to hyper-scaler LinkedIn, is a Linux kernel feature called “control groups” used with most containers to assign resources. In an analysis based on several months of “pressure testing,” Zhenyun Zhuang, a software engineer at LinkedIn (NYSE: LNKD), notes that many container projects running on Docker or rival CoreOS platforms rely on “cgroups” to limit resources like computing and memory. Linux network kernel “Ensuring the high performance of the applications running in cgroups is very


important for business-critical computing environments.

” Zhuang stressed in a paper released Thursday (Aug. Linux kernel lts 18). The social media company has been using cgroups for an internal container project called LinkedIn Platform as a Service. Kernel upgrade in linux One goal was to determine how resource-limiting policies would affect application performance.

Linux kernel development “We have found that cgroups do not totally isolate resources, but rather limit resource usage so that applications running in memory-limited cgroups do not starve other cgroups,” the paper concludes. One result is “memory pressure,” which the author warns could raise issues affecting the performance of applications running in cgroups, including: • Unlike virtual machines, memory is not reserved for cgroups. • Page cache used by containerized applications is counted against a cgroup’s memory limits, “therefore anonymous memory usage can steal page cache usage for the same cgroup.” • The operating system also can “steal” memory from cgroups.

What is linux kernel The researcher attributes this to the fact that the root cgroup is “unbounded.” Among the performance shortfalls posed by memory pressure on either root or “regular” cgroups is much slower application startup, especially if the OS has to free up memory to meet an application memory request, Zhuang noted. Rather than reserving memory as with virtual machines, cgroups impose only an upper limit on memory usage for applications in the control group. Vanilla linux kernel Hence, the LinkedIn engineer found that memory is allocated on demand, and applications deployed in cgroups must compete for free memory from the OS. One implication is that the OS must reclaim memory from the page cache or from “anonymous” memory if it does not have enough free memory to meet the cgroup request.

Linux kernel changes “Memory reclamation by the OS could be a performance killer, affecting the performance of other cgroups,” Zhuang warned. LinkedIn also found that page cache used by applications is counted against a cgroup’s memory limit, and anonymous memory usage can “steal” page cache for the same cgroup. Kernel linux version Further, the paper notes, the OS also can if needed steal page cache from cgroups.

The Linux kernel also allows for swapping computing and memory resources between the unbounded root and bounded regular cgroups. Linux kernel browser The LinkedIn analysis notes that each cgroup can have its own “swappiness” setting, but all cgroups rely on the same swap space configured by the OS. Learn linux kernel Hence, memory pressure on the root or regular cgroups could have a cascade effect on other cgroups in the container. Among the proposed remedies to the memory pressure issue arising from early application container stress testing is “pre-touching” memory limits for cgroups while avoiding “use-as-you-go” memory requests. Vanilla linux kernel Hence, LinkedIn recommended allocating memory for application containers beforehand.

Moreover, when “onboarding an application to cgroups, the memory footprint of the application needs to be sized,” Zhuang stressed. Linux kernel community “Since a cgroup’s memory limit counts both anonymous memory and page cache used by the cgroup, sizing memory footprint should consider both memory types.” LinkedIn also recommends isolating memory usage when running containers, including moving as many “house-keeping” processes from the root cgroup to “properly-sized user cgroups.” In sum, Zhuang called cgroups “a decent mechanism to limit memory usage of applications” so long as steps are taken ahead of time to avoid memory pressure.

Check linux kernel With that in mind, Control Group v2 containing performance improvements was released earlier this year. How the linux kernel works LinkedIn reported this week it expects to test the latest version as part of its container platform service.

Representatives from application container leaders Docker and CoreOS were not immediately available to comment on the LinkedIn analysis. Intel Launches Silicon Photonics Chip, Previews Next-Gen Phi for AI At the Intel Developer Forum, held in San Francisco this week, Intel Senior Vice President and General Manager Diane Bryant announced the launch of Intel’s Silicon Photonics product line and teased a brand-new Phi product, codenamed “Knights Mill,” aimed at machine learning workloads. Kernel linux mint Read more… High performance computing (HPC) systems are comprised of multiple compute nodes interconnected by a network.

Linux kernel latest Read more… • Think Fast – Is Neuromorphic Computing Set to Leap Forward? Steadily advancing neuromorphic computing technology has created high expectations for this fundamentally different approach to computing.

Linux kernel jobs Read more… Here at HPCwire, we aim to keep the HPC community apprised of the most relevant and interesting news items that get tweeted throughout the week. Compile kernel linux Read more…

With early adopters of application container technology completing early testing in multi-tenant settings, potential performance issues are beginning to surface. Linux kernel learning Read more… The flooding now occurring in Louisiana and Mississippi is a stark reminder of how disruptive and potentially dangerous floods can be. Newest linux kernel Read more…

The Convergence of HPC and Big Data for Enterprise 7/14/16 | Bright Computing | The combined power of clustered servers, advanced networking and massive datasets are merging, and a new Big Data reality is on the rise. Linux kernel introduction Read more… Manufacturing is on the upswing, but with more competition than ever, the pressure to trim product lifecycles and increase productivity is greater than ever.

Linux kernel gpl Read more… Do-It-Yourself HPC: How the Cloud Democratizes HPC and Public Data for Researchers Creating the right technology environment is a time-consuming task for researchers who want to focus on science (not servers or long wait times at supercomputing centers). How to know linux kernel version Read more… Silicon Mechanics and Van Andel Institute partner to deliver an OpenStack HPC solution The Van Andel Institute (VAI) worked with Silicon Mechanics to not only provide its users a more powerful platform, but a hybrid OpenStack HPC solution with the flexibility to support VAI’s commitment to improve the health and change the lives of current and future generations.

Linux kernel ebook Read more… HPCwire is a registered trademark of Tabor Communications, Inc. Upgrade linux kernel Use of this site is governed by our Terms of Use and Privacy Policy. Reproduction in whole or in part in any form or medium without express written permission of Tabor Communications Inc. Learn linux kernel is prohibited. Site: https://www.hpcwire.com/2016/08/18/container-testing-reveals-memory-pressure-apps/

banner