From: eLinux.org

Runtime Memory Measurement

This page has a collection of ideas and resources having to do with
measuring runtime memory of a Linux system.

Unfortunately, the existing memory measurement techniques do not give a
100% accurate accounting of memory pages (since some pages are counted
more than once by some measures). See Accurate Memory
Measurement

  • that page describes techniques (and patches) which can be used to
    measure the runtime memory more accurately.

Contents

Measuring memory in Linux (the basics)

Here are some basic techniques for measuring memory usage in Linux.

‘free’ and /proc

The ‘free’ command shows the memory on a machine, in certain categories.

[need explanation of categories here…’man free’ doesn’t explain the
numbers]

  1. $ free
  2.              total       used       free     shared    buffers     cached
  3. Mem:        507564     481560      26004          0      68888     185220
  4. -/+ buffers/cache:     227452     280112
  5. Swap:      2136604     105168    2031436

This information is obtained from /proc/meminfo, which has additional
details not shown by the ‘free’ command.

The following is on my machine with 512 Mb RAM, running Linux 2.6.3:

  1. $ cat /proc/meminfo
  2. MemTotal:       507564 kB
  3. MemFree:         26004 kB
  4. Buffers:         68888 kB
  5. Cached:         185220 kB
  6. SwapCached:      29348 kB
  7. Active:         342488 kB
  8. Inactive:        32092 kB
  9. HighTotal:           0 kB
  10. HighFree:            0 kB
  11. LowTotal:       507564 kB
  12. LowFree:         26004 kB
  13. SwapTotal:     2136604 kB
  14. SwapFree:      2031436 kB
  15. Dirty:              88 kB
  16. Writeback:           0 kB
  17. Mapped:         165648 kB
  18. Slab:            73212 kB
  19. Committed_AS:   343172 kB
  20. PageTables:       2644 kB
  21. VmallocTotal:   524212 kB
  22. VmallocUsed:      7692 kB
  23. VmallocChunk:   516328 kB

See http://lwn.net/Articles/28345/ for
a description of these fields

Measuring user process memory use

‘ps’ fields for memory information

The ‘ps’ command provides information about the memory usage of
processes on a Linux system. However, it is not well documented. Here
are some notes on using ‘ps’ and /proc to view memory usage information
on a running Linux system:

meaning of ps fields:

  • %Mem - percent of memory
  • VSZ - Virtual Size
  • RSS - Resident Set Size
  • SIZE - Equivalent to VSZ
  • others??

‘top’ fields for memory information

See ‘man top’:

  • %MEM — Memory usage (RES)

      • A task’s currently used share of available physical memory.

  • VIRT — Virtual Image (kb)

      • The total amount of virtual memory used by the task. It
        includes all code, data and shared libraries plus pages that
        have been swapped out.
      • VIRT = SWAP + RES

  • SWAP — Swapped size (kb)

      • The swapped out portion of a task’s total virtual memory
        image.

  • RES — Resident size (kb)

        • The non-swapped physical memory a task has used.
        • RES = CODE + DATA.

  • CODE — Code size (kb)

        • The amount of physical memory devoted to executable
          code, also known as the ‘text resident set’ size or TRS

  • DATA — Data+Stack size (kb)

        • The amount of physical memory devoted to other than
          executable code, also known as the ‘data resident set’
          size or DRS.

  • SHR — Shared Mem size (kb)

        • The amount of shared memory used by a task. It simply
          reflects memory that could be potentially shared with
          other processes.

  • nDRT — Dirty Pages count

        • The number of pages that have been modified since they
          were last written to disk. Dirty pages must be written
          to disk before the corresponding physical memory
          location can be used for some other virtual page.

Are the following assertions true:??

  • virtual memory usage of a process, excluding shared libs = VIRT -
    SHR
  • physical memory usage of a process excluding shared libraries = RES
    • SHR

/proc info

see ‘man proc’ for detailed information about the files and fields in
the /proc filesystem.

/proc/\/statm

/proc/\/statm fields: columns are (in pages):

























total program size|
resident set size|
shared pages|
text (code) |
data/stack |
library |
dirty pages |

Here an example: 693 406 586 158 0 535 0

/proc/\/status

/proc/\/status fields:

  • Vm Size: 2772 kB
  • Vm Lck: 0 kB - ???
  • Vm RSS: 1624 kB
  • Vm Data: 404 kB
  • Vm Stk: 24 kB
  • Vm Exe: 608 kB
  • Vm Lib: 1440 kB

/proc/\/maps

The process maps shows the actual memory areas that have been mapped
into a process’ address space, and their permissions.

Example:

  1. $ cat /proc/25042/maps
  2. 08048000-080e0000 r-xp 00000000 03:05 196610     /bin/bash
  3. 080e0000-080e6000 rw-p 00097000 03:05 196610     /bin/bash
  4. 080e6000-08148000 rwxp 00000000 00:00 0
  5. 40000000-40016000 r-xp 00000000 03:05 147471     /lib/ld-2.3.3.so
  6. 40016000-40017000 rw-p 00015000 03:05 147471     /lib/ld-2.3.3.so
  7. 40017000-40018000 rw-p 00000000 00:00 0
  8. 40018000-40019000 r--p 00000000 03:05 184090     /usr/share/locale/en_US/LC_IDENTIFICATION
  9. 40019000-4001a000 r--p 00000000 03:05 184089     /usr/share/locale/en_US/LC_MEASUREMENT
  10. 4001a000-4001b000 r--p 00000000 03:05 184083     /usr/share/locale/en_US/LC_TELEPHONE
  11. 4001b000-4001c000 r--p 00000000 03:05 184091     /usr/share/locale/en_US/LC_ADDRESS
  12. 4001c000-4001d000 r--p 00000000 03:05 184086     /usr/share/locale/en_US/LC_NAME
  13. 4001d000-4001e000 r--p 00000000 03:05 184084     /usr/share/locale/en_US/LC_PAPER
  14. 4001e000-4001f000 r--p 00000000 03:05 184088     /usr/share/locale/en_US/LC_MESSAGES/SYS_LC_MESSAGES
  15. 4001f000-40020000 r--p 00000000 03:05 184087     /usr/share/locale/en_US/LC_MONETARY
  16. 40020000-40026000 r--p 00000000 03:05 183689     /usr/share/locale/ISO-8859-1/LC_COLLATE
  17. 40026000-40027000 r--p 00000000 03:05 184082     /usr/share/locale/en_US/LC_TIME
  18. 40027000-4002a000 r-xp 00000000 03:05 147459     /lib/libtermcap.so.2.0.8
  19. 4002a000-4002b000 rw-p 00002000 03:05 147459     /lib/libtermcap.so.2.0.8
  20. 4002b000-4002c000 rw-p 00000000 00:00 0
  21. 4002c000-4002e000 r-xp 00000000 03:05 147482     /lib/libdl-2.3.3.so
  22. 4002e000-4002f000 rw-p 00001000 03:05 147482     /lib/libdl-2.3.3.so
  23. 4002f000-40171000 r-xp 00000000 03:05 147511     /lib/tls/libc-2.3.3.so
  24. 40171000-40174000 rw-p 00142000 03:05 147511     /lib/tls/libc-2.3.3.so
  25. 40174000-40177000 rw-p 00000000 00:00 0
  26. 40177000-40178000 r--p 00000000 03:05 184085     /usr/share/locale/en_US/LC_NUMERIC
  27. 40178000-401a4000 r--p 00000000 03:05 183688     /usr/share/locale/ISO-8859-1/LC_CTYPE
  28. 401a4000-401a5000 r-xp 00000000 03:05 180462     /usr/lib/gconv/ISO8859-1.so
  29. 401a5000-401a6000 rw-p 00001000 03:05 180462     /usr/lib/gconv/ISO8859-1.so
  30. 401b3000-401bd000 r-xp 00000000 03:05 147492     /lib/libnss_files-2.3.3.so
  31. 401bd000-401be000 rw-p 00009000 03:05 147492     /lib/libnss_files-2.3.3.so
  32. bfffa000-c0000000 rwxp ffffb000 00:00 0
  33. ffffe000-fffff000 ---p 00000000 00:00 0
mem_usage command to consolidate data

David Schleef wrote a program to consolidate the information from
/proc/\/maps, and total up each kind of memory for a process.

Here it is: Media:mem_usage (It
was obtained from
http://www.schleef.org/~ds/mem_usage)

Here is the result of running mem_usage on the process used in the
previous example:

  1. $ ./mem_usage 25042
  2. Backed by file:
  3.   Executable                r-x  2048
  4.   Write/Exec (jump tables)  rwx  0
  5.   RO data                   r--  240
  6.   Data                      rw-  56
  7.   Unreadable                ---  0
  8.   Unknown                        0
  9. Anonymous:
  10.   Writable code (stack)     rwx  416
  11.   Data (malloc, mmap)       rw-  20
  12.   RO data                   r--  0
  13.   Unreadable                ---  4
  14.   Unknown                        0

Inaccuracies of kernel reporting mechanisms

Many of the memory reporting mechanisms for the kernel are inaccurate,
due to not recording sufficient information about the true state of the
system. Here are some random notes on these inaccuracies. To see
information on different methods of getting more accurate memory
information, see Accurate Memory
Measurement

  • “copy-on-write” pages - an mmap’ed file may be very large in the
    process address space, but empty until written to.

From Ratboy on Slashdot:

  1. The mmap() call can map a file (backing store) and allow data to be shared. Memory does
  2. not need to be used until the data is read (or written). And this time, the backing
  3. store doesn't even need swap (because the file is the backing store).
  5. ...
  7. A page of code that is shared - may become a page of code that is private. A page of
  8. data that is unwritten doesn't have to exist. Even if it is read! A page of data that
  9. is written may STILL be shared.

From others on Slashdot:

Top will show you the same as ps does, ps reads /proc/\/statm and
asks what’s going on. The problem on linux is the copy on write
principle which saves heaps of memory, but makes it virtually impossible
to figure out what belongs to what. The thing is, when you fork it maps
the memory and marks everything as copy on write, when something needs
to write to part of the memory, then it will make the copy for each
process.

However asking the process how much memory it has allocated will show
all memory including stuff that is marked copy on write - that is, I
could have 100 processes showing they each use 1.4MB of memory, because
they all share the same library, but in fact, its the same copy they are
all using so I’m only using 1.4 MB instead of 140MB (+PCB et. al)

Each thread in a process shows up as consuming the same amount of memory
(either this only happens under Linuxthreads or I don’t have any
threaded applications running on my system).

Device mappings show up as consumed memory (which generates plenty of
complaints about the X server). If you want to find out how much memory
X is actually using (bytes in cached pixmaps on behalf of each process
and sans device mappings), try the program here:
http://69.142.116.122/dist/pixmap_mem-1.0.tgz

This contains a tiny program that lists how much memory X is using for
other programs by caching pixmaps and a perl script that lists how much
memory X is using sans device mappings. }}}

  • pmap is a utility which shows the memory usage of a process (it
    looks like it just reads and interprets /proc/\/maps).

Someone on Slashdot said:

  1. pmap *also* overestimates memory usage, because some portion of the mapped address
  2. space isn't actually in use. RSS, on the other hand, only measures memory that is
  3. actually in use, but doesn't distinguish between memory that is shared and memory
  4. that is not. VSZ is the most pessimistic measure, since it includes all mapped
  5. memory, shared and unshared.

Heap memory usage

Heap is the dynamically allocated memory inside each process’ address
space that is managed by the application itself. The structure of this
memory is actually managed by the C library, with the application
calling malloc() and free().

glibc has the capability to collect statistics information of heap
functions like malloc() and other functions like memory leak checking or
double free.

Memory Debuggers

Several tools are available to analyze memory allocations, watch for
reading and writing beyond the end of allocated memory, and do other
tasks which help with debugging and tuning memory operations of a
program. See Memory Debuggers
for a list of different tools and their features.

Measuring kernel memory use

Kernel Stack Usage

  • Tim is adding a stack checking function to KFT (See Kernel Function
    Trace    )

      • This new feature has not yet been published

  • Recent -mm tree added stack-corruption-detector.patch (8th March,
    2006)

  • The scripts/checkstack.pl script in the kernel tree will show the
    functions with the largest static stack footprint.
  • Enabling CONFIG_DEBUG_STACKOVERFLOW in the kernel will enable
    checking for low-stack situations in the irq handler.
  • Enabling CONFIG_4KSTACKS will cause stack overflows to occur more
    frequently, particularly with certain kernel code such as XFS
    enabled.

General kernel memory use

  • http://www.halobates.de/memorywaste.pdf
    • Great paper by Andi Kleen, of SUSE Labs, about dynamic memory
      usage of Linux systems
  • Check /proc/slabinfo to find how much memory is being used by the
    kernel SLAB allocator (or SLUB or SLOB, depending on what is
    enabled).

Kernel memory analysis tools and project

The Linux Foundation CE Workgroup has a project to analyze the kernel’s
dynamic memory utilization. See: Kernel dynamic memory
analysis

Category: