发布新日志

  • Linux中几个不常用但是很有用的命令

    2007-02-01 23:14:53

    Linux中有非常多很有用的命令,比如ps, top, vmstat等等,这些是应用比较广泛的,广为人知的。还有一些命令,非常有用,但是应用并不是十分广泛,如果充分利用这些命令,有时可以完成一些复杂的任务,起到很奇妙的效果。今天就来介绍几个这样的命令:

    Could the command-line tools you've forgotten or never knew save time and some frustration?


    One incarnation of the so called 80/20 rule has been associated with software systems. It has been observed that 80% of a user population regularly uses only 20% of a system's features. Without backing this up with hard statistics, my 20+ years of building and using software systems tells me that this hypothesis is probably true. The collection of linux command-line programs is no exception to this generalization. Of the dozens of shell-level commands offered by Linux, perhaps only ten commands are commonly understood and utilized, and the remaining majority are virtually ignored.

    Which of these dogs of the linux shell have the most value to offer? I'll briefly describe ten of the less popular but useful Linux shell commands, those which I have gotten some mileage from over the years. Specifically, I've chosen to focus on commands that parse and format textual content.

    The working examples presented here assume a basic familiarity with command-line syntax, simple shell constructs and some of the not-so-uncommon linux commands. Even so, the command-line examples are fairly well commented and straightforward. Whenever practical, the output of usage examples is presented under each command-line execution.

    The following eight commands parse, format and display textual content. Although not all provided examples demonstrate this, be aware that the following commands will read from standard input if file arguments are not presented.

    • Head/Tail
    显示一个文件指定的前面或者后面多少行,也可以用来实时显示文件的动态情况。比较适合用来跟踪日至文件的变化情况。

    As their names imply, head and tail are used to display some amount of the top or bottom of a text block. head presents beginning of a file to standard output while tail does the same with the end of a file. Review the following commented examples:

    ## (1) displays the first 6 lines of a file
    head -6 readme.txt
    ## (2) displays the last 25 lines of a file
    tail -25 mail.txt

    Here's an example of using head and tail in concert to display the 11th through 20th line of a file.


    # (3)
    head -20 file | tail -10

    Manual pages show that the tail command has more command-line options than head. One of the more useful tail option is -f. When it is used, tail does not return when end-of-file is detected, unless it is explicitly interrupted. Instead, tail sleeps for a period and checks for new lines of data that may have been appended since the last read.


    ## (4) display ongoing updates to the given
    ## log file

    tail -f /usr/tmp/logs/daemon_log.txt

    Imagine that a dæmon process was continually appending activity logs to the /usr/adm/logs/daemon_log.txt file. Using tail -f at a console window, for example, will more or less track all updates to the file in real time. (The -f option is applicable only when tail's input is a file).

    If you give multiple arguments to tail, you can track several log files in the same window.

    ## track the mail log and the server error log
    ## at the same time.

    tail -f /var/log/mail.log /var/log/apache/error_log


    • tac--Concatenate in Reverse
    反过来显示一个指定的文件或者输入,同cat反过来记忆比价有效。

    What is cat spelled backwards? Well, that's what tac's functionality is all about. It concatenates file order and their contents in reverse. So what's its usefulness? It can be used on any task that requires ordering elements in a last-in, first-out (LIFO) manner. Consider the following command line to list the three most recently established user accounts from the most recent through the least recent.


    # (5) last 3 /etc/passwd records - in reverse
    $ tail -3 /etc/passwd | tac
    curly:x:1003:100:3rd Stooge:/homes/curly:/bin/ksh
    larry:x:1002:100:2nd Stooge:/homes/larry:/bin/ksh
    moe:x:1001:100:1st Stooge:/homes/moe:/bin/ksh


    • nl--Numbered Line Output
    nl简单的一个行过滤器,可以在文件的前面标注上行号,并可以从指定的行号开始,还可以指定分割符号等等,请看下面的描述和例子。
    nl is a simple but useful numbering filter. I displays input with each line numbered in the left margin, in a format dictated by command-line options. nl provides a plethora of options that specify every detail of its numbered output. The following commented examples demonstrate some of of those options:


    # (6) Display the first 4 entries of the password
    # file - numbers to be three columns wide and
    # padded by zeros.
    $ head -4 /etc/passwd | nl -nrz -w3
    001root:x:0:1:Super-User:/:/bin/ksh
    002daemon:x:1:1::/:
    003bin:x:2:2::/usr/bin:
    004sys:x:3:3::/:
    #
    # (7) Prepend ordered line numbers followed by an
    # '=' sign to each line -- start at 101.
    $ nl -s= -v101 Data.txt
    101=1st Line ...
    102=2nd Line ...
    103=3rd Line ...
    104=4th Line ...
    105=5th Line ...
    .......

    • fmt--format
    fmt可以用来给指定的文件限制宽度,使显示起来符合规则,好看些。另外还有些特殊的参数可以使用,参考下面的描述。
    The fmt command is a simple text formatter that focuses on making textual data conform to a maximum line width. It accomplishes this by joining and breaking lines around white space. Imagine that you need to maintain textual content that was generated with a word processor. The exported text may contain lines whose lengths vary from very short to much longer than a standard screen length. If such text is to be maintained in a text editor (like vi), fmt is the command of choice to transform the original text into a more maintainable format. The first example below shows fmt being asked to reformat file contents as text lines no greater than 60 characters long.


    # (8) No more than 60 char lines
    $ fmt -w 60 README.txt > NEW_README.txt
    #
    # (9) Force uniform spacing:
    # 1 space between words, 2 between sentences
    $ echo "Hello World. Hello Universe." |
    fmt -u -w80
    Hello World. Hello Universe.


    • fold--Break Up Input
    fold有点类似于fmt地用法,但是用法更加特殊,参考下面的描述。
    fold is similar to fmt but is used typically to format data that will be used by other programs, rather than to make the text more readable to the human eye. The commented examples below are fairly easy to follow:


    # (10) format text in 3 column width lines
    $ echo oxoxoxoxo | fold -w3
    oxo
    xox
    oxo
    # (11) Parse by triplet-char strings -
    # search for 'xox'
    $ echo oxoxoxoxo | fold -w3 | grep "xox"
    xox
    # (12) One way to iterate through a string of chars
    $ for i in $(echo 12345 | fold -w1)
    > do
    > ### perform some task ...
    > print $i
    > done
    1
    2
    3
    4
    5

    上面的例子似乎不那么有效,更加有效的方法是:

    for i in `seq 5`; do print $i;done

    seq(1) allows to define start, stop, step and more.


    • tr
    tr is a simple pattern translator. Its practical application overlaps a bit with other, more complex tools, such as sed and awk [with larger binary footprints]. tr is quite useful for simple textual replacements, deletions and additions. Its behavīor is dictated by "from" and "to" character sets provided as the first and second argument. The general usage syntax of tr is as follows:


    # (13) tr usage
    tr [options] "set1" ["set2"] <> output

    Note that tr does not accept file arguments; it reads from standard input and writes to standard output. When two character sets are provided, tr operates on the characters contained in "set1" and performs some amount of substitution based on "set2".


    • pr
    相对于tr或者其他命令,pr命令主要用于打印的格式。
    pr shares features with simpler commands like nl and fmt, but its command-line options make it ideal for converting text files into a format that's suitable for printing. pr offers options that allow you to specify page length, column width, margins, headers/footers, double line spacing and more.

    Aside from being the best suited formatter for printing tasks, pr also offers other useful features. These features include allowing you to view multiple files vertically in adjacent columns or columnizing a list in a fixed number of columns.



    • Miscellaneous
    The following two commands are specialized parsers used to pick apart file path pieces.


    Basename/Dirname
    The basename and dirname commands are useful for presenting portions of a given file path. Quite often in scrīpting situations, it's convenient to be able to parse and capture a file name or the containing-directory name portions of a file path. These commands reduce this task to a simple one-line command. (There are other ways to approach this using the Korn shell or sed "magic", but basename and dirname are more portable and straightforward).

    basename is used to strip off the directory, and optionally, the file suffix parts of a file path. Consider the following trivial examples:


    :# (14) Parse out the Java Class name
    $ basename
    /usr/local/src/java/TheClass.java .java
    TheClass
    # (15) Parse out the file name.
    $ basename srcs/C/main.c
    main.c

    dirname is used to display the containing directory path, as much of the path as is provided. Consider the following examples:


    # (16) absolute and relative directory examples
    $ dirname /homes/curly/.profile
    /homes/curly
    $ dirname curly/.profile
    curly
    #
    # (17) From any korn-shell scrīpt, the following
    # line will assign the directory from where
    # the scrīpt was launched
    scrīpt_HOME="$(dirname $(whence $0))"
    #
    # (18)
    # Okay, how about a non-trivial practical example?
    # List all directories (under $PWD that contain a
    # file called 'core'.
    $ for i in $(find $PWD -name core )^
    > do
    > dirname $i
    > done | sort -u
    bin
    rje/gcc
    src/C


    至此,这几个命令就介绍完了,希望能给你的工作带来帮助,如果要更详细地用法,可以参考man命令的帮助。
  • Top的一个bug

    2007-02-01 23:13:55

    今天发现了top命令的一个bug, 监控进程的时候,明明当前有三个进程,可是top的低版本只能取到其中一个进程的资源信息.

    但是用高版本的就可以,操作日志如下:

    [root@MD_100_22 bin]# ./linux_top -b -n 2 -d 2 > old.log

    [root@MD_100_22 bin]# top -b -n 2 -d 2 > new.log

    查看了一下版本信息:

    [root@MD_100_22 bin]# ./linux_top -version

    procps version 2.0.11

    [root@MD_100_22 bin]# top -version

    top: procps version 3.2.5

    usage: top -hv | -bcisS -d delay -n iterations [-u user | -U user] -p pid [,pid ...]

    然后对文件内容进行对比:

    [root@MD_100_22 bin]#

    [root@MD_100_22 bin]# grep "media_director" old.log

    1563 root 15 0 104M 2292 1884 S 9.9 0.1 10:46 0 media_director

    1563 root 15 0 104M 2292 1884 S 9.4 0.1 10:46 0 media_director

    [root@MD_100_22 bin]# grep "media_director" new.log

    1567 root 15 0 1037m 601m 408m S 11.9 29.7 10:48.14 media_director

    1563 root 15 0 104m 2292 1884 S 0.0 0.1 0:00.01 media_director

    1568 root 16 0 104m 2304 1892 S 0.0 0.1 0:00.02 media_director

    1567 root 15 0 1037m 601m 408m S 8.0 29.7 10:48.30 media_director

    1563 root 15 0 104m 2292 1884 S 0.0 0.1 0:00.01 media_director

    1568 root 16 0 104m 2304 1892 S 0.0 0.1 0:00.02 media_director

    [root@MD_100_22 bin]#


    new的结果分析,显然每次有三个media_director的进程,可是老版本的居然只能取到一个,看来真的是bug.

    man top的时候看到报bug可以发邮件

    Michael K. Johnson <johnsonm@redhat.com> is now the maintainer.

    Please send bug reports to procps-list@redhat.com

    发了俩封邮件居然都被reject,,郁闷

    下载最新的top,请到官方网址.

  • vmstat命令中增加时间戳

    2007-02-01 23:12:50

    前阵子,利用vmstat生成的实时性能数据文件,美中不足是没有时间信息,缺少一个时间戳,测试结束后开发在debug问题的时候,往往需要定位性能瓶颈发生的具体时间,因此,需要为vmstat性能文件添加一个时间戳.

    利用强大的awk即可实现这个功能,如下:

    [autoadm@AUTOTEST-ATS ATSLib]$ vmstat 2 5 | awk '{print strftime("%H:%M:%S") $0}'
    17:08:07 procs memory swap io system cpu
    17:08:07 r b w swpd free buff cache si so bi bo in cs us sy id
    17:08:07 1 0 0 8 316280 251760 995400 0 0 23 75 120 108 1 1 98
    17:08:09 0 0 0 8 316280 251760 995400 0 0 0 12 105 70 0 2 98
    17:08:12 0 0 0 8 316280 251760 995400 0 0 0 344 113 75 0 0 100
    17:08:14 0 0 0 8 316072 251760 995400 0 0 0 0 116 167 6 2 92
    17:08:16 0 0 0 8 316072 251760 995400 0 0 0 340 114 76 0 0 100
    [autoadm@AUTOTEST-ATS ATSLib]$

    其中的命令strftime好像在solaris下面不行,不知道是否因为strftime函数的缘故?如果是这样,用系统命令date应该也可以,尝试如下:

    [autoadm@AUTOTEST-ATS ATSLib]$ vmstat 2 5 | awk '{ system("date +%H:%M:%S"); print $0}'
    17:10:25
    procs memory swap io system cpu
    17:10:25
    r b w swpd free buff cache si so bi bo in cs us sy id
    17:10:25
    0 0 0 8 316092 251764 995584 0 0 23 75 120 108 1 1 98
    17:10:27
    0 0 0 8 316092 251764 995584 0 0 0 98 109 77 0 0 100
    17:10:29
    0 0 0 8 315884 251764 995584 0 0 0 0 117 174 5 3 92
    17:10:31
    0 0 0 8 315892 251764 995584 0 0 0 348 137 97 0 0 100
    17:10:33
    0 0 0 8 315888 251764 995584 0 0 0 12 157 200 16 3 80
    [autoadm@AUTOTEST-ATS ATSLib]$

    不过这样也有一个缺陷,就是时间和性能数据分在了两行, 不知有什么好办法可以搞到一行?


    其实不仅仅是vmstat,其他的不带时间戳的命令都可以用这种方法得到时间戳.
  • TCL脚本的历史

    2007-02-01 23:11:32

    如果要做自动化测试,首选语言是Tcl,大家不要一提TCL就想到做电器的那个TCL公司,这个Tcl非那个TCL,这是一种功能强大的脚本语言,全名: Tool Command Language. 自从2003年初次接触自动化,我就跟这个语言结下了不解之缘. 做自动化,有Tcl还远远不够,Tcl的一个强大扩展Expect也是不能不提的, 想想看,如果没有这些个强大的交互式命令语言,怎么能真正把大家从电脑面前解脱出来.

    Expect提供的基本命令很少,spawn, send,expect应该是最最常用的三个了吧.依靠这三个简单的命令,可以完成非常强大的工作,写一个脚本,自动执行,是多少程序员,测试人员梦寐以求的事情啊.

    说Tcl的历史,就一定要看看下面这篇文章:
    Tcl脚本的历史

    Tcl的下载可以到官方地址

    另外,中文方面的资料也很多:
    Tcl的中文网站


    如果要系统学习Tcl,有一本权威的教材,一定要看,吐血推荐:


    上次团购,我们team的人现在可是人手一本啊, ^_^
  • 利用sed处理好看的报表

    2007-02-01 23:10:22

    前阵子做性能测试的时候,系统工程师有个要求,所有的性能测试项目提供一份vmstat的结果作为参考,最好能做出性能的曲线. 做曲线这么艰巨任务就只能由Excel去完成肋,我们所要做的就是把vmstat的结果输出为一个标准的格式文件,步骤如下:

    1. 首先利用vmstat -n 5 10 > vmstat.csv把结果输出到一个文本文件中. (不过solaris好像没有-n :) ,只能自己想办法去掉)
    2. 利用利器sed把其中的第一个空格去掉,命令如下:
    :1,$s/^ //
    我尝试了N次,都没能成功,居然把行首的^给忘记了,自责中...
    3. 利用sed把其中所有的空格替换成, 这样csv格式就可以直接被打开,利用如下的命令:
    :1,$s/ \+//g
    这个命令也有一段来历,刚开始尝试的时候用的不是+而是*,结果每次都把后面的所有内容都替掉了,后来在伟大的Grant同学指点下(一个冰淇淋换来的)用转义后的+解决. 不过为什么*不用转义,而+需要转义一直表示不解,直到看了如下解释:

    [grant@autotest16 test]$ man grep

    ……………………………

    In basic regular expressions the metacharacters ?, +, {, |, (, and )

    lose their special meaning; instead use the backslashed versions \?,

    \+, \{, \|, \(, and \).

    ……………………………


    4. 把替换好的格式文件,传到Windows,然后用Excel打开,并通过简单的操作,得到如下漂亮的图案.

数据统计

  • 访问量: 28169
  • 日志数: 29
  • 图片数: 2
  • 书签数: 1
  • 建立时间: 2006-12-28
  • 更新时间: 2007-05-14

RSS订阅

Open Toolbar