Search This Blog


Linux Performance Monitoring Using Base Tools

Monitoring Performance

r: total number of processes waiting for CPU time
b: blocked processes waiting for disk or network IO
Buff: used to store file allocation tables in buffers (directories, permissions etc)
Cache: Used to load files into memory (contents of files)

vmstat -S M - show memory usage in MB
vmstat -a - show active and inactive memory (memory in use and no longer in use)

Manually clean up memory

free -m; sync
sudo bash -c "echo 3 > /proc/sys/vm/drop_caches"
free -m

Sync will write buffers through to disk

Real Time Monitoring

# Which ports are listening on tcp
netstat -alt

# Transmit and rx data
netstat -i

# Show counters hitting iptables rules
iptables -nvL

# Watch changes of iptables
watch -d iptables -nvL

Historical Performance

Historical performance metrics are generally achieved in a basic sense through the use of the sysstat package.

There are two scripts sa1 and sa2
sa1 - collects data every 10 minutes
sa2 - collects summary daily information
sar - displays historical data

CentOS File locations:


Some tools within sysstat:
iostat - disk info
mpstat - cpu info
pidstat - displays process id info
cifsiostat - details about samba shares
nfsiostat - details about nfs export io
sar - collects and displays system activities

After sysstat is installed there will be a /etc/cron.d/sysstat file such that every 10 minutes sa1 will be run once every 10 minutes. The sa2 script will be running daily at around 23:53

Messages will all be stored in /var/log/sa/ directory

sar01 sar23 sar24 sar25 sar26 sar27 sar28 sar29 sar30

Using Sar

# Display the version number
sar -V

# Display CPU information
sar -u

# Show load information
sar -q

# Use historical data file
sar -q -f /var/log/sa/sa15

# Context switches per second and number of processes per second
sar -w

# Interface Statistics
sar -n DEV

# Overall IO activity
sar -b

# Display load averages from 10 to 11 on the current day
sar -q -s 10:00:00 -e 11:00:00

# Grabbing from a file in a time period
sar -q -s 10:00:00 -e 11:00:00 -f /var/log/sa/sa02
Linux 2.6.32-279.23.1.el6.x86_64 () 05/02/2015 _x86_64_ (2 CPU)

10:00:01 AM runq-sz plist-sz ldavg-1 ldavg-5 ldavg-15
10:10:01 AM 1 219 1.00 1.01 1.00
10:20:01 AM 1 219 1.00 1.00 1.00
10:30:01 AM 1 220 1.00 1.00 1.00
10:40:01 AM 1 218 1.00 1.00 1.00
10:50:01 AM 1 217 1.00 1.00 1.00

# runq-sz is the number of running processes waiting to be run
# plist-sz number of active processes being handled

Graphical Historical Data

Requires the EPEL repo, collectd, and web server

yum install -y collectd collect-rrdtool rrdtool collecd-web httpd

Configuration is stored in: /etc/collectd.conf

BaseDir /var/lib/collectd
LoadPlugin rrdtool

DataDir "/var/lib/collectd/rrd"
CreateFilesAsync false
CacheTimeout 120
CacheFlush 900
WritesPerSecond 50

By default apache only listens on IPv6 so we have to change /etc/httpd/conf/httpd.conf to allow IPv4 listening


Then you need to set web permissions

Require ip 192.168.0

Where the IP is the subnet you wish to allow

Ensure the system hostname is resolvable using DNS

Enable the collectd service

systemctl enable collectd
systemctl enable httpd
systemctl start httpd
systemctl start collectd

Go to the web address: http://$ipAddress/collectd/

No comments:

Post a Comment