Understanding Linux CPU Load - when should you be worried?

([C^z;M tI,ec0翻译：http://www.gracecode.com/archives/2973/51Testing软件测试网g;m6h&H O!jKdk

51Testing软件测试网A8OPY b0[PL:p p%k

1v-h.FeT-P0You might be familiar with Linux load averages already. Load averages are the three numbers shown with theuptimeandtopcommands - they look like this:51Testing软件测试网Q B8w2O]7gY%L0_o

Most people have an inkling of what the load averages mean: the three numbers represent averages over progressively longer periods of time (one, five, and fifteen minute averages), and that lower numbers are better. Higher numbers represent a problem or an overloaded machine. But, what's the the threshold? What constitutes "good" and "bad" load average values? When should you be concerned over a load average value, and when should you scramble to fix it ASAP?51Testing软件测试网GRu@9O-Y
x

w-u$Cg]Y)j3L0First, a little background on what the load average values mean. We'll start out with the simplest case: a machine with one single-core processor.

The traffic analogy

A single-core CPU is like a single lane of traffic. Imagine you are a bridge operator ... sometimes your bridge is so busy there are cars lined up to cross. You want to let folks know how traffic is moving on your bridge. A decent metric would behow many cars are waitingat a particular time. If no cars are waiting, incoming drivers know they can drive across right away. If cars are backed up, drivers know they're in for delays.

Sw!C#H:f}/O/}0So, Bridge Operator, what numbering system are you going to use? How about:

• 0.00 means there's no traffic on the bridge at all. In fact, between 0.00 and 1.00 means there's no backup, and an arriving car will just go right on.
• 1.00 means the bridge isexactlyat capacity.All is still good, but if traffic gets a little heavier, things are going to slow down.
• over 1.00 means there's backup.How much? Well, 2.00 means that there are two lanes worth of cars total -- one lane's worth on the bridge, and one lane's worth waiting. 3.00 means there are three lane's worth total -- one lane's worth on the bridge, and two lanes' worth waiting. Etc.

O`c(ZM}(i.y3C0= load of 1.0051Testing软件测试网PS(i.]r3W-CM
D

= load of 0.50

= load of 1.7051Testing软件测试网:`y uZ#sIP3N0j;H

51Testing软件测试网_g|!q+n^Xye
51Testing软件测试网k'Y|!_.l
ZB+H-kRf
51Testing软件测试网4I#bj"T
oB|

"Rk7r!U8E,h0This is basically what CPU load is. "Cars" are processes using a slice of CPU time ("crossing the bridge") or queued up to use the CPU. Unix refers to this as therun-queue length: the sum of the number of processes that are currently running plus the number that are waiting (queued) to run.

}Dj7[
V$Z0Like the bridge operator, you'd like your cars/processes to never be waiting. So, your CPU load should ideally stay below 1.00. Also like the bridge operator, you are still ok if you get some temporary spikes above 1.00 ... but when you're consistently above 1.00, you need to worry.51Testing软件测试网5n-b0r\ _!R#g,]

So you're saying the ideal load is 1.00?

t2nU;y,h
X0Well, not exactly. The problem with a load of 1.00 is that you have no headroom. In practice, many sysadmins will draw a line at 0.70:51Testing软件测试网1yGZ_-Aj-P)t

• 51Testing软件测试网 I C6{-L3_U/zI`-`

  The"Need to Look into it"Rule of Thumb:0.70If your load average is staying above > 0.70, it's time to investigate before things get worse.51Testing软件测试网kU0U I0`+s,i"^

• i NroxB#r w0The"Fix this now"Rule of Thumb:1.00. If your load average stays above 1.00, find the problem and fix it now. Otherwise, you're going to get woken up in the middle of the night, and it's not going to be fun.51Testing软件测试网?Q(VA*AI1ac

• 51Testing软件测试网p3Y*O+Y~gX(H

  The"Arrgh, it's 3AM WTF?"Rule of Thumb:5.0. If your load average is above 5.00, you could be in serious trouble, your box is either hanging or slowing way down, and this will (inexplicably) happen in the worst possible time like in the middle of the night or when you're presenting at a conference. Don't let it get there.51Testing软件测试网 a1o?|o'{*cu7l%\

What about Multi-processors? My load says 3.00, but things are running fine!

Got a quad-processor system? It's still healthy with a load of 3.00.51Testing软件测试网U1[qN#j4Nt"Ix7r

)dx$[FC0On multi-processor system, the load is relative to the number of processor cores available. The "100% utilization" mark is 1.00 on a single-core system, 2.00, on a dual-core, 4.00 on a quad-core, etc.51Testing软件测试网6e1c5W0H7SZ

m0\
l Ov+Z0If we go back to the bridge analogy, the "1.00" really means "one lane's worth of traffic". On a one-lane bridge, that means it's filled up. On a two-late bridge, a load of 1.00 means its at 50% capacity -- only one lane is full, so there's another whole lane that can be filled.

= load of 2.00 on two-lane road

Same with CPUs: a load of 1.00 is 100% CPU utilization on single-core box. On a dual-core box, a load of 2.00 is 100% CPU utilization.51Testing软件测试网1GtQa)a3M|

Multicore vs. multiprocessor

P`6y_i%m6YX0While we're on the topic, let's talk about multicore vs. multiprocessor. For performance purposes, is a machine with a single dual-core processor basically equivalent to a machine with two processors with one core each? Yes. Roughly. There are lots of subtleties here concerning amount of cache, frequency of process hand-offs between processors, etc. Despite those finer points, for the purposes of sizing up the CPU load value, thetotal number of coresis what matters, regardless of how many physical processors those cores are spread across.51Testing软件测试网2c:d'n5x&Hsf8E yk

Which leads us to a two new Rules of Thumb:51Testing软件测试网Jzx3A1L3y7vo l0R5}.N

• jR)L6}z
  N}ga0The "number of cores = max load"Rule of Thumb: on a multicore system, your load should not exceed the number of cores available.

  K
  d!dK.j;c%N0
• 51Testing软件测试网F}'~K;@.u,W

  The"cores is cores"Rule of Thumb: How the cores are spread out over CPUs doesn't matter. Two quad-cores == four dual-cores == eight single-cores. It's all eight cores for these purposes.

  ,oY#RIja*v0

Bringing It Home

Let's take a look at the load averages output fromuptime:51Testing软件测试网3{ d2u{8{W

z;Ir:UA0This is on a dual-core CPU, so we've got lots of headroom. I won't even think about it until load gets and stays above 1.7 or so.51Testing软件测试网%G%UlHaf6~Jh

Now, what about those three numbers? 0.65 is the average over the last minute, 0.42 is the average over the last five minutes, and 0.36 is the average over the last 15 minutes. Which brings us to the question:51Testing软件测试网/Ik_p&CD;`DV

-?Rp ~7k0Which average should I be observing? One, five, or 15 minute?51Testing软件测试网.{8i[,TT!^(rt~

For the numbers we've talked about (1.00 = fix it now, etc), you should be looking at the five or 15-minute averages. Frankly, if your box spikes above 1.0 on the one-minute average, you're still fine. It's when the 15-minute average goes north of 1.0 and stays there that you need to snap to. (obviously, as we've learned, adjust these numbers to the number of processor cores your system has).51Testing软件测试网Tu*s}W*P$dYx*}&~'Z

So # of cores is important to interpreting load averages ... how do I know how many cores my system has?51Testing软件测试网!]~yop,LX3W*o

aa\!Wb,Q,c`M0cat /proc/cpuinfoto get info on each processor in your system.Note: not available on OSX, Google for alternatives. To get just a count, run it throughgrepand word count:grep 'model name' /proc/cpuinfo | wc -l