Exploring Linux procfs via shell scripts

Very often, while working on a Linux project, we’d like information about the system we’re working on: both at a global scope and a local (process) scope.

Have we not wondered: is there a quick way to query which kernel version am using, what interrupts are enabled & hit, what my processor(s) are, details about kernel subsystems, memory usage, file, network, IPC usage, etc etc. Linux’s proc filesystem makes this easy.

So what exactly is the proc filesystem all about?

Essentially, some quick salient points about the proc filesystem:

  • it’s a RAM-based filesystem (think ramdisk; yup, it’s volatile)
  • it’s a kernel feature, not userspace – proc is a filesystem supported by the Linux kernel VFS
  • it serves two primary purposes
    • proc serves as a “view” deep into the kernel internals; we can see details about hardware and software subsystems that userspace otherwise would have no access to (no syscalls)
    • certain “files” under proc, typically anchored under /proc/sys, can be written into: these basically are the “tuning knobs” of the Linux kernel. Sysads, developers, apps, etc exploit this feature
  • proc is mounted on start-up under /proc
  • a quick peek under /proc will show you several “files” and “folders”. These are pseudo-entries in the sense that they exist only in RAM while power is applied. The “folders” that are numbers are in fact the PID of each process that’s alive when you typed ‘ls’! it’s a snapshot of the system at that moment in time..
  • in fact, the name “proc” suggests “process”

At this point, and if you’re not really familiar with this stuff, I’d urge you to peek around /proc on your Linux box, cat-ting stuff as you go. (Also, lest i forget, it’s better to run as root (sudo /bin/bash) so that we don’t get annoying ‘permission denied’ messages). Of course, be careful when you run as root!!!

For example, to get one started off:

# cat /proc/version 
Linux version 3.0.0-16-generic-pae (buildd@palmer) (gcc version 4.6.1 (Ubuntu/Linaro 4.6.1-9ubuntu3) ) #28-Ubuntu SMP Fri Jan 27 19:24:01 UTC 2012
# cat /proc/meminfo
 MemTotal: 4045072 kB
 MemFree: 1840648 kB
 Buffers: 163552 kB
 Cached: 990596 kB
 ...
# ls /proc/1
 attr/ clear_refs cpuset fd/ limits mountinfo ns/ pagemap schedstat stack syscall
 autogroup cmdline cwd@ fdinfo/ loginuid mounts oom_adj personality seccomp_filter stat task/
 auxv comm environ io maps mountstats oom_score root@ sessionid statm wchan
 cgroup coredump_filter exe@ latency mem net/ oom_score_adj sched smaps status
#

The output above is but a sampling of what one can see! (See the SIDEBAR lower down for more).
Anyway, to cut to the chase: after some experimenting (and experience), I’ve written a couple of shell scripts that could be quite useful (to me at least :).

The first of them, proc_show.sh, is kinda like a “global proc” information script. It provides several macro-level details about the system you’re on.
The second script (pidshow.sh) is geared towards provinding info about a particular process (given it’s PID).

THE SCRIPTS

Click here to download a zip file containing the procfs shell scripts used in this article.

Please try them out and give me feedback. Finding and fixing bugs is a bonus!

Note that these can be tuned for & used on an embedded Linux system just as well.

IMP: The detailed kernel documentation for procfs is here.

SIDEBAR: Sample Output


1. Output of the “global info”  proc_show.sh script on a QEMU-emulated ARM/Linux platform:

Note:
a. Realize that on the embedded system the shell process ‘/bin/sh’ is really ‘/bin/busybox’
b. The script below first shows the command being run, and then it’s output..
c. The output below is clipped for readability… 

 
ARM /myprj $ ./procshow.sh 
 ----------------------------------------------------------------------------
 cd /proc ; pwd ; ls --color=auto -F
 ----------------------------------------------------------------------------
 /proc
 1/ 4/ devices loadavg swaps
 10/ 5/ diskstats locks sys/
 11/ 53/ driver/ meminfo sysrq-trigger
 12/ 55/ execdomains misc sysvipc/
 13/ 56/ fb modules timer_list
 14/ 6/ filesystems mounts@ tty/
 15/ 7/ fs/ mtd uptime
 16/ 8/ interrupts net@ version
 17/ 9/ iomem pagetypeinfo vmallocinfo
 18/ buddyinfo ioports partitions vmstat
 2/ bus/ irq/ sched_debug zoneinfo
 25/ cmdline kallsyms self@
 26/ consoles kmsg slabinfo
 3/ cpu/ kpagecount softirqs
 31/ cpuinfo kpageflags stat
 ----------------------------------------------------------------------------
============================================================================
 procfs as a Viewport to Hardware & Kernel Information .................
 ============================================================================
 ----------------------------------------------------------------------------
 cd /proc/ ; ls -lF --color=auto |grep '^[^d]'
 ----------------------------------------------------------------------------
 total 0
 -r--r--r-- 1 0 0 0 Jan 1 00:00 buddyinfo
 -r--r--r-- 1 0 0 0 Jan 1 00:00 cmdline
 -r--r--r-- 1 0 0 0 Jan 1 00:00 consoles
 -r--r--r-- 1 0 0 0 Jan 1 00:00 cpuinfo
 -r--r--r-- 1 0 0 0 Jan 1 00:00 devices
 -r--r--r-- 1 0 0 0 Jan 1 00:00 diskstats
 -r--r--r-- 1 0 0 0 Jan 1 00:00 execdomains
 -r--r--r-- 1 0 0 0 Jan 1 00:00 fb
 -r--r--r-- 1 0 0 0 Jan 1 00:00 filesystems
 -r--r--r-- 1 0 0 0 Jan 1 00:00 interrupts
 -r--r--r-- 1 0 0 0 Jan 1 00:00 iomem
--snip--
----------------------------------------------------------------------------
 ----------------------------------------------------------------------------
 cat /proc/cpuinfo
 ----------------------------------------------------------------------------
 Processor : ARM926EJ-S rev 5 (v5l)
 BogoMIPS : 360.03
 Features : swp half thumb fastmult vfp edsp java
 CPU implementer : 0x41
 CPU architecture: 5TEJ
 CPU variant : 0x0
 CPU part : 0x926
 CPU revision : 5
Hardware : ARM-Versatile PB
 Revision : 0000
 Serial : 0000000000000000
 ----------------------------------------------------------------------------
 Number of processor cores:
 ----------------------------------------------------------------------------
 find /sys/devices/system/cpu/ -type d |grep 'cpu[0-9]' |wc -l
 ----------------------------------------------------------------------------
 1
 ----------------------------------------------------------------------------
 ----------------------------------------------------------------------------
 cat /proc/interrupts
 ----------------------------------------------------------------------------
 CPU0
 4: 5333 VIC timer
 12: 1895 VIC uart-pl011
 25: 0 VIC eth0
 35: 7 SIC kmi-pl050
 36: 89 SIC kmi-pl050
 Err: 0
 ----------------------------------------------------------------------------
 ============================================================================
 procfs as a Viewport to Software Information .................
 ============================================================================
 ----------------------------------------------------------------------------
 uname -a
 ----------------------------------------------------------------------------
 Linux (none) 3.1.5 #9 Wed Feb 22 13:26:00 IST 2012 armv5tejl GNU/Linux
 ----------------------------------------------------------------------------
 ----------------------------------------------------------------------------
 cat /proc/version
 ----------------------------------------------------------------------------
 Linux version 3.1.5 (root@kaiwan-N53Jq) (gcc version 4.4.1 (Sourcery G++ Lite 2009q3-67) ) #9 Wed Feb 22 13:26:00 IST 2012
 ----------------------------------------------------------------------------
 ----------------------------------------------------------------------------
 cat /proc/buddyinfo
 ----------------------------------------------------------------------------
 Node 0, zone Normal 6 5 5 1 2 3 2 3 4 4 24
 ----------------------------------------------------------------------------
 ----------------------------------------------------------------------------
 cat /proc/cmdline
 ----------------------------------------------------------------------------
 console=ttyAMA0 root=/dev/ram rdinit=/sbin/init
 ----------------------------------------------------------------------------
 ----------------------------------------------------------------------------
 grep register_chrdev /proc/kallsyms
 ----------------------------------------------------------------------------
 c008fc94 t __unregister_chrdev_region
 c008fd34 T __unregister_chrdev
 c008fd60 T unregister_chrdev_region
 c008fdac t __register_chrdev_region
 c008ff44 T __register_chrdev
 c0090020 T register_chrdev_region
 c032ecd8 r __ksymtab___register_chrdev
--snip--

 ----------------------------------------------------------------------------
 grep sys_call_table /proc/kallsyms
 ----------------------------------------------------------------------------
 c0014e28 T sys_call_table
 ----------------------------------------------------------------------------
 ============================================================================
 procfs for kernel Tuning : sysctl .................
 ============================================================================
 ----------------------------------------------------------------------------
 cd /proc/sys ; ls --color=auto -F
 ----------------------------------------------------------------------------
 debug/ dev/ fs/ kernel/ net/ sunrpc/ vm/
 ----------------------------------------------------------------------------
 ----------------------------------------------------------------------------
 cd /proc/sys/kernel ; ls --color=auto -F
 ----------------------------------------------------------------------------
 auto_msgmni printk_delay
 blk_iopoll printk_ratelimit
 cad_pid printk_ratelimit_burst
 core_pattern pty/
 core_pipe_limit random/
 core_uses_pid randomize_va_space
--snip--
 ----------------------------------------------------------------------------
 ----------------------------------------------------------------------------
 cd /proc/sys/kernel ; cat msgmni msgmnb msgmax
 ----------------------------------------------------------------------------
 246
 16384
 8192
 ----------------------------------------------------------------------------

–snip–


2. Output of the “local process” script  pidshow.sh on the same QEMU-emulated ARM/Linux platform:

Note:
a. Realize that on the embedded system the shell process ‘/bin/sh’ is really ‘/bin/busybox’
b. I’ve modified a shell variable in the script BBOX_LS setting it to 1, for this run (see the comments in the script for details on why exactly this needs to be done)
c. The output below is highlighted in places and clipped for readability…  

ARM /myprj $ ps
PID USER TIME COMMAND
 1 0 0:00 init
 2 0 0:00 [kthreadd]
--snip--
 26 0 0:00 [kworker/0:2]
 31 0 0:00 /bin/sh
 93 0 0:00 ps
ARM /myprj $ 
ARM /myprj $ ./pidshow.sh 
Usage: pidshow.sh PID V=[0|1]
 PID: pid of process whose details will be displayed
Verbosity: 0 = non-verbose mode (default)
 1 = verbose mode
ARM /myprj $ ./pidshow.sh 31 1 << Run the script on 'sh' (really 'busybox') in verbose mode >> -------------------------------------------------------------------------------
System Minimal Info
-------------------------------------------------------------------------------
cat: can't open '/etc/issue': No such file or directory
Kernel: 3.1.5
Linux version 3.1.5 (root@kaiwan-N53Jq) (gcc version 4.4.1 (Sourcery G++ Lite 2009q3-67) ) #9 Wed Feb 22 13:26:00 IST 2012

-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Info for process sh TGID 31
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Name: sh

Is NOT a Kernel Thread
Cmd Line: /bin/sh
EXE: /bin/busybox
-rwxr-xr-x 1 0 0 917240 Feb 19 2012 /bin/busybox
CWD: /myprj
Root Dir: /
## Err: file /proc/31/cgroup non-existant ##
-------------------------------------------------------------------------------
Task Details:
-------------------------------------------------------------------------------
Name: sh
State: S (sleeping)
Tgid: 31
Pid: 31
PPid: 1
TracerPid: 0
Uid: 0 0 0 0
Gid: 0 0 0 0
FDSize: 32
Groups:
VmPeak: 3280 kB
VmSize: 3280 kB
VmLck: 0 kB
VmHWM: 732 kB
VmRSS: 732 kB
VmData: 168 kB
VmStk: 136 kB
VmExe: 896 kB
VmLib: 2008 kB
VmPTE: 8 kB
VmSwap: 0 kB
Threads: 1
SigQ: 0/955
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: 0000000000004004
SigCgt: 0000000000000002
CapInh: 0000000000000000
CapPrm: ffffffffffffffff
CapEff: ffffffffffffffff
CapBnd: ffffffffffffffff
Cpus_allowed: 1
Cpus_allowed_list: 0
voluntary_ctxt_switches: 248
nonvoluntary_ctxt_switches: 23

Wait channel: do_wait
## Err: file /proc/31/stack non-existant ##
-------------------------------------------------------------------------------
Open Files:
-------------------------------------------------------------------------------
total 0
lrwx------ 1 0 0 64 Jan 1 00:12 0 -> /dev/console
lrwx------ 1 0 0 64 Jan 1 00:12 1 -> /dev/console
lrwx------ 1 0 0 64 Jan 1 00:12 2 -> /dev/console
-------------------------------------------------------------------------------
Threads:
# threads: 1  
-------------------------------------------------------------------------------
Verbose Info for process sh TGID 31
-------------------------------------------------------------------------------
## Err: file /proc/31/cpuset non-existant ##
-------------------------------------------------------------------------------
VM maps:
-------------------------------------------------------------------------------
00008000-000e8000 r-xp 00000000 00:01 353 /bin/busybox
000ef000-000f0000 rwxp 000df000 00:01 353 /bin/busybox
000f0000-00113000 rwxp 00000000 00:00 0 [heap]
40026000-40027000 rwxp 00000000 00:00 0
40075000-40076000 rwxp 00000000 00:00 0
400a8000-400c6000 r-xp 00000000 00:01 59 /lib/ld-2.10.1.so
400cd000-400ce000 r-xp 0001d000 00:01 59 /lib/ld-2.10.1.so
400ce000-400cf000 rwxp 0001e000 00:01 59 /lib/ld-2.10.1.so
400cf000-40168000 r-xp 00000000 00:01 61 /lib/libm-2.10.1.so
40168000-4016f000 ---p 00099000 00:01 61 /lib/libm-2.10.1.so
4016f000-40170000 r-xp 00098000 00:01 61 /lib/libm-2.10.1.so
40170000-40171000 rwxp 00099000 00:01 61 /lib/libm-2.10.1.so
40171000-402ac000 r-xp 00000000 00:01 56 /lib/libc-2.10.1.so
402ac000-402b3000 ---p 0013b000 00:01 56 /lib/libc-2.10.1.so
402b3000-402b5000 r-xp 0013a000 00:01 56 /lib/libc-2.10.1.so
402b5000-402b6000 rwxp 0013c000 00:01 56 /lib/libc-2.10.1.so
402b6000-402ba000 rwxp 00000000 00:00 0
be7f3000-be814000 rw-p 00000000 00:00 0 [stack]
ffff0000-ffff1000 r-xp 00000000 00:00 0 [vectors]

-------------------------------------------------------------------------------
VM smaps:
-------------------------------------------------------------------------------
00008000-000e8000 r-xp 00000000 00:01 353 /bin/busybox
Size: 896 kB
Rss: 196 kB
Pss: 97 kB
Shared_Clean: 0 kB
Shared_Dirty: 168 kB
Private_Clean: 0 kB
Private_Dirty: 28 kB
Referenced: 196 kB
Anonymous: 0 kB
AnonHugePages: 0 kB
Swap: 0 kB
KernelPageSize: 4 kB
MMUPageSize: 4 kB
Locked: 0 kB
000ef000-000f0000 rwxp 000df000 00:01 353 /bin/busybox
Size: 4 kB
Rss: 4 kB
Pss: 4 kB
Shared_Clean: 0 kB
Shared_Dirty: 0 kB
Private_Clean: 0 kB
Private_Dirty: 4 kB
Referenced: 4 kB
Anonymous: 4 kB
AnonHugePages: 0 kB
Swap: 0 kB
KernelPageSize: 4 kB
MMUPageSize: 4 kB
Locked: 0 kB
000f0000-00113000 rwxp 00000000 00:00 0 [heap]
Size: 140 kB
Rss: 28 kB
Pss: 28 kB

--snip--
ffff0000-ffff1000 r-xp 00000000 00:00 0 [vectors]
Size: 4 kB
Rss: 4 kB
Pss: 4 kB
Shared_Clean: 0 kB
Shared_Dirty: 0 kB
Private_Clean: 0 kB
Private_Dirty: 4 kB
Referenced: 4 kB
Anonymous: 0 kB
AnonHugePages: 0 kB
Swap: 0 kB
KernelPageSize: 4 kB
MMUPageSize: 4 kB
Locked: 0 kB

stat: 31 (sh) S 1 31 31 0 -1 4194304 777 44492 0 0 17 29 438 384 20 0 1 0 122 3358720 183 4294967295 32768 946564 3196141232 3196139424 1075910784 0 0 16388 2 3221405000 0 0 17 0 0 0 0 0 0

statm: 820 183 160 224 0 76 0

## Err: file /proc/31/syscall non-existant ##
Coredump filter: 00000033

## Err: file /proc/31/io non-existant ##
-------------------------------------------------------------------------------
Limits:
-------------------------------------------------------------------------------
Limit       Soft Limit  Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 955 955 processes
Max open files 1024 4096 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 955 955 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us

oom_adj: 0

oom_score: 1

oom_score_adj: 0

-------------------------------------------------------------------------------
sched:
-------------------------------------------------------------------------------
sh (31, #threads: 1)
---------------------------------------------------------
se.exec_start : 1005258.718614
se.vruntime : 9839.731606
se.sum_exec_runtime : 462.068385
nr_switches : 271
nr_voluntary_switches : 248
nr_involuntary_switches : 23
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 2750

## Err: file /proc/31/schedstat non-existant ##
-------------------------------------------------------------------------------
Environment:
-------------------------------------------------------------------------------
USER=rootHOME=/TERM=vt102PATH=/sbin:/usr/sbin:/bin:/usr/binSHELL=/bin/shPWD=/PS1=ARM \w $
Personality: 00c00000

## Err: file /proc/31/seccomp_filter non-existant ##
-------------------------------------------------------------------------------
Per Thread Stack
-------------------------------------------------------------------------------
## Err: file /proc/31/task/31/stack non-existant ##

-------------------------------------------------------------------------------
Done.
ARM /myprj $

Okay.
Hey, another very interesting thing to realize: procfs can be tremendously leveraged to help developers debug applications and/or kernel-space.
But that’s a topic for another day :)

Have fun with proc!  And write in..

3 thoughts on “Exploring Linux procfs via shell scripts”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s