• Chapter 7 Intermediate Shell Tool I

    1.1 Sifting Through Files for a String

    The grep command searches through files looking for the expression you supply:

    1. [maxwell@MaxwellDBA test]$ grep printf *.c
    2. grep: *.c: No such file or directory
    3. [maxwell@MaxwellDBA test]$ grep bash *.sh
    4. check_arg_count.sh:#!/usr/bin/env bash
    5. checkfile.sh:#!/usr/bin/env bash
    6. checkstr.sh:#!/usr/bin/env bash
    7. check_unset_parms.sh:#!/usr/bin/env bash
    8. chmod_all2.sh:#!/usr/bin/env bash
    9. chmod_all.sh:#!/usr/bin/env bash
    10. dbinit.sh:#!/usr/bin/env bash
    11. donors.sh:#!/bin/bash
    12. ext.sh:#!/bin/bash
    13. fun_calc.sh:#!/usr/bin/env bash
    14. func_choice1.sh:#!/bin/bash
    15. func_choose.sh:#!/bin/bash
    16. myscript.sh:#!/bin/bash
    17. readability.sh:#!/bin/bash
    18. rpncalc.sh:#!/usr/bin/env bash
    19. select_dir.sh:#!/bin/bash
    20. simpls.sh:#!/bin/bash
    21. strvsnum.sh:#!/usr/bin/env bash
    22. suffixer.sh:#!/usr/bin/env bash
    23. suffixer.sh:# rename files that end in .bad to be .bash
    24. suffixer.sh:    mv "${FN}" "${FN%bad}bash"
    25. trackmatch.sh:#!/usr/bin/env bash
    26. use_up_option.sh:#!/usr/bin/env bash
    27. [maxwell@MaxwellDBA test]$ 
    1. [maxwell@MaxwellDBA test]$ grep bash /home/maxwell/shelllearning/test/*.sh /home/maxwell/shelllearning/anotherdir/*.sh
    2. /home/maxwell/shelllearning/test/check_arg_count.sh:#!/usr/bin/env bash
    3. /home/maxwell/shelllearning/test/checkfile.sh:#!/usr/bin/env bash
    4. /home/maxwell/shelllearning/test/checkstr.sh:#!/usr/bin/env bash
    5. /home/maxwell/shelllearning/test/check_unset_parms.sh:#!/usr/bin/env bash
    6. /home/maxwell/shelllearning/test/chmod_all2.sh:#!/usr/bin/env bash
    7. /home/maxwell/shelllearning/test/chmod_all.sh:#!/usr/bin/env bash
    8. /home/maxwell/shelllearning/test/dbinit.sh:#!/usr/bin/env bash
    9. /home/maxwell/shelllearning/test/donors.sh:#!/bin/bash
    10. /home/maxwell/shelllearning/test/ext.sh:#!/bin/bash
    11. /home/maxwell/shelllearning/test/fun_calc.sh:#!/usr/bin/env bash
    12. /home/maxwell/shelllearning/test/func_choice1.sh:#!/bin/bash
    13. /home/maxwell/shelllearning/test/func_choose.sh:#!/bin/bash
    14. /home/maxwell/shelllearning/test/myscript.sh:#!/bin/bash
    15. /home/maxwell/shelllearning/test/readability.sh:#!/bin/bash
    16. /home/maxwell/shelllearning/test/rpncalc.sh:#!/usr/bin/env bash
    17. /home/maxwell/shelllearning/test/select_dir.sh:#!/bin/bash
    18. /home/maxwell/shelllearning/test/simpls.sh:#!/bin/bash
    19. /home/maxwell/shelllearning/test/strvsnum.sh:#!/usr/bin/env bash
    20. /home/maxwell/shelllearning/test/suffixer.sh:#!/usr/bin/env bash
    21. /home/maxwell/shelllearning/test/suffixer.sh:# rename files that end in .bad to be .bash
    22. /home/maxwell/shelllearning/test/suffixer.sh:    mv "${FN}" "${FN%bad}bash"
    23. /home/maxwell/shelllearning/test/trackmatch.sh:#!/usr/bin/env bash
    24. /home/maxwell/shelllearning/test/use_up_option.sh:#!/usr/bin/env bash
    25. /home/maxwell/shelllearning/anotherdir/func_choice1.sh:#!/bin/bash
    26. /home/maxwell/shelllearning/anotherdir/func_choose.sh:#!/bin/bash
    27. [maxwell@MaxwellDBA test]$

    If you don't want to see the specific filenames,you may turn this off using the -h switch to grep:

    1. [maxwell@MaxwellDBA test]$ grep -h bash *.sh 
    2. #!/usr/bin/env bash
    3. #!/usr/bin/env bash
    4. #!/usr/bin/env bash
    5. #!/usr/bin/env bash
    6. #!/usr/bin/env bash
    7. #!/usr/bin/env bash
    8. #!/usr/bin/env bash
    9. #!/bin/bash
    10. #!/bin/bash
    11. #!/usr/bin/env bash
    12. #!/bin/bash
    13. #!/bin/bash
    14. #!/bin/bash
    15. #!/bin/bash
    16. #!/usr/bin/env bash
    17. #!/bin/bash
    18. #!/bin/bash
    19. #!/usr/bin/env bash
    20. #!/usr/bin/env bash
    21. # rename files that end in .bad to be .bash
    22.     mv "${FN}" "${FN%bad}bash"
    23. #!/usr/bin/env bash
    24. #!/usr/bin/env bash

    If you don't want to see the actual lines from the file, but only a count of the number of times the expression is found, then use the -c option.

    1. [maxwell@MaxwellDBA test]$ grep -c bash *.sh 
    2. check_arg_count.sh:1
    3. checkfile.sh:1
    4. checkstr.sh:1
    5. check_unset_parms.sh:1
    6. chmod_all2.sh:1
    7. chmod_all.sh:1
    8. dbinit.sh:1
    9. donors.sh:1
    10. ext.sh:1
    11. fun_calc.sh:1
    12. func_choice1.sh:1
    13. func_choose_man1.sh:0
    14. func_choose.sh:1
    15. myscript.sh:1
    16. quoted.sh:0
    17. readability.sh:1
    18. rpncalc.sh:1
    19. select_dir.sh:1
    20. simplest.sh:0
    21. simpls.sh:1
    22. strvsnum.sh:1
    23. suffixer.sh:3
    24. trackmatch.sh:1
    25. tricky.sh:0
    26. trythis.sh:0
    27. use_up_option.sh:1
    28. [maxwell@MaxwellDBA test]$

    1.2 Getting Just the Filename from a Search

    Use the -l option of grep to get just the filenames:

    1. [maxwell@MaxwellDBA test]$ grep -l bash *.sh
    2. check_arg_count.sh
    3. checkfile.sh
    4. checkstr.sh
    5. check_unset_parms.sh
    6. chmod_all2.sh
    7. chmod_all.sh
    8. dbinit.sh
    9. donors.sh
    10. ext.sh
    11. fun_calc.sh
    12. func_choice1.sh
    13. func_choose.sh
    14. myscript.sh
    15. readability.sh
    16. rpncalc.sh
    17. select_dir.sh
    18. simpls.sh
    19. strvsnum.sh
    20. suffixer.sh
    21. trackmatch.sh
    22. use_up_option.sh
    23. [maxwell@MaxwellDBA test]$

    1.3 Getting a Simple True/False from a Search

    Use -q, the "quiet" option for grep. Or,for maximum portability,just throw the output away by redirecting it into /dev/null.Either way, your answer is in the bash return status variable $?

    1. [maxwell@MaxwellDBA test]$ grep -q bash fun_calc.sh
    2. [maxwell@MaxwellDBA test]$ if [ $? -eq 0 ] ; then echo yes ; else echo nope ; fi
    3. yes
    4. [maxwell@MaxwellDBA test]$

    If we list multiple filenames after grep -q, then grep stops searching after the very first occurence of the search string being found.if you really need to read through all the files, then rather than use -q you can do this :

    1. [maxwell@MaxwellDBA test]$ grep bash fun_calc.sh > /dev/null
    2. [maxwell@MaxwellDBA test]$ if [ $? -eq 0 ] ; then echo yes ; else echo nope ; fi
    3. yes
    4. [maxwell@MaxwellDBA test]$

    The redirecting to /dev/null sends the output to a special kind of device, a bit bucket, that just throws everything you give it away.

    1.4 Searching for Text While Ignoring Case

    Use the -i option on grep to ignore case:

    1. [maxwell@MaxwellDBA test]$ grep -i bash *sh
    2. check_arg_count.sh:#!/usr/bin/env bash
    3. checkfile.sh:#!/usr/bin/env bash
    4. checkstr.sh:#!/usr/bin/env bash
    5. check_unset_parms.sh:#!/usr/bin/env bash
    6. chmod_all2.sh:#!/usr/bin/env bash
    7. chmod_all.sh:#!/usr/bin/env bash
    8. dbinit.sh:#!/usr/bin/env bash
    9. donors.sh:#!/bin/bash
    10. ext.sh:#!/bin/bash
    11. fun_calc.sh:#!/usr/bin/env bash
    12. func_choice1.sh:#!/bin/bash
    13. func_choose.sh:#!/bin/bash
    14. myscript.sh:#!/bin/bash
    15. readability.sh:#!/bin/bash
    16. rpncalc.sh:#!/usr/bin/env bash
    17. select_dir.sh:#!/bin/bash
    18. simpls.sh:#!/bin/bash
    19. strvsnum.sh:#!/usr/bin/env bash
    20. suffixer.sh:#!/usr/bin/env bash
    21. suffixer.sh:# rename files that end in .bad to be .bash
    22. suffixer.sh:    mv "${FN}" "${FN%bad}bash"
    23. trackmatch.sh:#!/usr/bin/env bash
    24. trackmatch.sh:  echo Track ${BASH_REMATCH[2]} is ${BASH_REMATCH[3]}
    25. trackmatch.sh:  mv "$CDTRACK" "Track${BASH_REMATCH[2]}"
    26. use_up_option.sh:#!/usr/bin/env bash
    27. [maxwell@MaxwellDBA test]$

    A case-insensitive search finds messages written "BASH","bash","Bash",as well as ones like "BaSH" and "bAsH".This option is particularly useful for finding words anywhere that you might have mixed-case text, including words that might be capitalized at the beginning of a sentence or email addresses.

    1.5 Doing a Search in a Pipeline

    Just pipe your result into grep

    If you also want to have grep search error messages that come from the previous command, be sure to redirect its error output into standard output before the pipe:

    gcc bigbadcode.c 2>&1 | grep -i error

    This command attempts to compile some hypothetical, hairy piece of code. We redirect standard error into standard output ( 2>&1) before we proceed to pipe (|) the output into grep, where it will search case-insensitively (-i) looking for the string error.

    1.6 Paring Down What the Search Finds

    Pipe the results into grep -v with an expression that describes what you don’t want to see.

    1. # you’re not sure if it’s always written as Dec, so to be sure to catch them all you type:
    2.  
    3. [maxwell@MaxwellDBA test]$ grep -i dec logfile
    4. error on Jan 01: not a decimal number
    5. error on Feb 13: base converted to Decimal
    6. warning on Mar 22: using only decimal numbers
    7. error on Dec 16 : the actual message you wanted
    8. error on Jan 01: not a decimal number
    9.  
    10. # A quick and dirty solution in this case is to pipe the first result into a second grep and tell the second grep to ignore any instances of “decimal”:
    11.  
    12. [maxwell@MaxwellDBA test]$ grep -i dec logfile | grep -vi decimal
    13. error on Dec 16 : the actual message you wanted
    14. [maxwell@MaxwellDBA test]$ grep -i dec logfile | grep -vi decimal | grep -vi decimate
    15. error on Dec 16 : the actual message you wanted
    16.  
    17. # a better solution would be to use a more powerful regular expression to match the December date. one that looked for "Dec" followed by a space and two digits:
    18. [maxwell@MaxwellDBA test]$ grep 'Dec [0-9][0-9]' logfile
    19. error on Dec 16 : the actual message you wanted
    20. [maxwell@MaxwellDBA test]$ grep Dec\ [0-9\ ][0-9] logfile
    21. error on Dec 16 : the actual message you wanted
    22. [maxwell@MaxwellDBA test]$

    1.8 Search for an SSN

    1. [maxwell@MaxwellDBA test]$ grep '[0-9]\{3\}-\{0,1\}[0-9]\{2\}-\{0,1\}[0-9]\{4\}' datafile
    2. grep: datafile: No such file or directory
    3. [maxwell@MaxwellDBA test]$ 
    [0-9]\{3\} -\{0,1\} [0-9]\{2\} -\{0,1\} [0-9]\{4\}

    The first grouping says “any digit” then “exactly 3 times.” The next grouping says “a dash” then “0 or 1 time.” The third grouping says “any digit” then “exactly 2 times.” The next grouping says “a dash” then “0 or 1 time.” The last grouping says “any digit” then “exactly 4 times.”

    1.9 Grepping Compressed Files.

    zgrep is simply a grep that understands various compressed and uncompressed files (which types are understood varies from system to system).

    zgrep 'search term' /var/log/messages*

    zcat is simply a cat that understands various compressed and uncompressed files (which types are understood varies from system to system).

    1.10 Keeping Some Output,Discarding the Rest

    The following code prints the first word of every line of input:

    The awk utility reads data from the filename supplied on the command line, or from standard input if no filename is given.

    1. [maxwell@MaxwellDBA test]$ awk '{print $1}' myinput.file
    2. zcat
    3. (which
    4. Words
    5. supplied
    6. [maxwell@MaxwellDBA test]$ awk '{print $1}' < myinput.file
    7. zcat
    8. (which
    9. Words
    10. supplied
    11. [maxwell@MaxwellDBA test]$ 
    12. [maxwell@MaxwellDBA test]$ cat myinput.file | awk '{print $1}'
    13. zcat
    14. (which
    15. Words
    16. supplied
    17. [maxwell@MaxwellDBA test]$ cat myinput.file
    18. zcat is simply a cat that understands various compressed and uncompressed files
    19. (which types are understood varies from system to system).
    20. Words are delineated by whitespace. The awk utility reads data from the filename
    21. supplied on the command line, or from standard input if no filename is given. Therefore, you can redirect the input from a file, like this
    22. [maxwell@MaxwellDBA test]$

    1.11 Keeping Only a Portion of a Line of Output

    Pipe ls into awk,and just pull out the fields that you need.
    awk has a built-in variable called NF that holds the number of fields found on the current line, $NF always refers to the last field.

    Just remember that you don’t use a $ to read the value of an awk variable (unlike bash variables). NF is a valid variable reference by itself. Adding a $ before it changes its meaning from “the number of fields on the current line” to “the last field on the current line.”

    1. [maxwell@MaxwellDBA test]$ ls -l | awk '{print $1, $NF}'
    2. total 156
    3. -rw-rw-r-- amazing.data
    4. -rw-rw-r-- another.file
    5. -rw-rw-r-- a.out
    6. -rwxr-x--- asdfwednnrw2rs2esff.txt
    7. -rw-rw-r-- check_arg_count.sh
    8. -rw-rw-r-- checkfile.sh
    9. -rw-rw-r-- checkstr.sh
    10. -rw-rw-r-- check_unset_parms.sh
    11. -rw-rw-r-- chmod_all2.sh
    12. -rw-rw-r-- chmod_all.sh
    13. -rwxr-x--- cong.txt
    14. -rw-rw-r-- dashes
    15. [maxwell@MaxwellDBA test]$

    1.12 Reversing the Words on Each Line

    1. [maxwell@MaxwellDBA test]$ awk '{for (i=NF; i>0; i--) {printf "%s ", $i;} printf "\n" }'
    2. sdjsf
    3. sdjsf 
    4. fsf
    5. fsf 
    6. ssfs
    7. ssfs 
    8. sfs
    9. sfs 
    10. sfs
    11. sfs 
    12. ^C
    13. [maxwell@MaxwellDBA test]$ awk '{                                                       
    14.                                for (i=NF; i>0; i--) {
    15.                                    printf "%s", $i;
    16.                                }
    17.                                printf "\n"
    18.                            }'
    19. sdafada
    20. sdafada
    21. ddsda
    22. ddsda
    23. fssdn
    24. fssdn
    25. ^C
    26. [maxwell@MaxwellDBA test]$

    1.13 Summing a List of Numbers

    Use awk both to isolate the field to be summed and to do the summing.

    1. [maxwell@MaxwellDBA test]$ ls -l | awk '{sum += $5} END {print sum}'
    2. 9527
    3. [maxwell@MaxwellDBA test]$ ls -l
    4. total 156
    5. -rw-rw-r-- 1 maxwell maxwell   0 Jul 26 10:20  amazing.data
    6. -rw-rw-r-- 1 maxwell maxwell  65 Jul 21 21:10  another.file
    7. -rw-rw-r-- 1 maxwell maxwell  10 Jul 21 15:39  a.out
    8. -rwxr-x--- 1 maxwell maxwell  49 Jul 21 20:43  asdfwednnrw2rs2esff.txt
    9. -rw-rw-r-- 1 maxwell maxwell 487 Jul 25 10:17  check_arg_count.sh
    10. -rw-rw-r-- 1 maxwell maxwell 487 Jul 26 10:19  checkfile.sh
    11. -rw-rw-r-- 1 maxwell maxwell 292 Jul 26 14:35  checkstr.sh
    12. -rw-rw-r-- 1 maxwell maxwell 263 Jul 25 12:23  check_unset_parms.sh
    13. -rw-rw-r-- 1 maxwell maxwell 199 Jul 25 09:36  chmod_all2.sh
    14. -rw-rw-r-- 1 maxwell maxwell 160 Jul 25 07:55  chmod_all.sh
    15. -rwxr-x--- 1 maxwell maxwell  19 Jul 21 15:40  cong.txt
    16. -rw-rw-r-- 1 maxwell maxwell 399 Jul 26 20:23  dashes
    17. -rw-rw-r-- 1 maxwell maxwell 185 Jul 26 20:36  dbinit.sh
    18. -rw-rw-r-- 1 maxwell maxwell  15 Jul 21 15:40  def.conf
    19. -rwxrwxr-x 1 maxwell maxwell 123 Jul 22 12:01  donors.sh
    20. -rw-rw-r-- 1 maxwell maxwell 867 Jul 25 05:43  embedded_documenta

    1.14 Counting String Values

    Use awk's associative arrays for your counting.

    1. [maxwell@MaxwellDBA test]$ ls -lR /usr/local | awk -f asar.awk
    2. ls: cannot open directory '/usr/local/aegis/globalcfg/aegisdb': Permission denied
    3. ls: cannot open directory '/usr/local/aegis/globalcfg/cache': Permission denied
    4. ls: cannot open directory '/usr/local/aegis/globalcfg/quara_hex': Permission denied
    5. ls: cannot open directory '/usr/local/aegis/globalcfg/quarantine': Permission denied
    6. ls: cannot open directory '/usr/local/aegis/globalcfg/vulfilebackup': Permission denied
    7. root owns 5306 files
    8. [maxwell@MaxwellDBA test]$ cat asar.awk
    9. #
    10. # cookbook filename: asar.awk
    11. #
    12. NF > 7 {
    13.         user[$3]++
    14.      }
    15.  END {
    16.          for (i in user)
    17.              {
    18.                 printf "%s owns %d files\n", i, user[i]
    19.              }
    20.      }
    21.  
    22. [maxwell@MaxwellDBA test]$

    1.15 Showing Data As a Quick and Easy Histogram

    Use the associate arrays of awk, as discussed in the previous recipe:

    1. [maxwell@MaxwellDBA test]$ cat hist.awk
    2. #
    3. # cookbook filename: hist.awk
    4. #
    5. function max(arr, big)
    6. {
    7.    big = 0
    8.    for (i in user)
    9.    {
    10.        if (user[i] > big) { big=user[i];}
    11.    }
    12.    return big
    13. }
    14.  
    15. NF > 7 {
    16.         user[$3]++
    17.      }
    18.  END {
    19.         # for scaling
    20.         maxm = max(user);
    21.         for (i in user)
    22.         {
    23.           #printf "%s owns %d files\n",i,user[i]
    24.           scaled = 60 * user[i] / maxm ;
    25.           printf "%-10.10s [%8d]:", i, user[i]
    26.           for (i=0; i
    27.               printf "#";
    28.           }
    29.           printf "\n"
    30.         }
    31.      }
    32. [maxwell@MaxwellDBA test]$ ls -lR /usr/local | awk -f hist.awk
    33. ls: cannot open directory '/usr/local/aegis/globalcfg/aegisdb': Permission denied
    34. ls: cannot open directory '/usr/local/aegis/globalcfg/cache': Permission denied
    35. ls: cannot open directory '/usr/local/aegis/globalcfg/quara_hex': Permission denied
    36. ls: cannot open directory '/usr/local/aegis/globalcfg/quarantine': Permission denied
    37. ls: cannot open directory '/usr/local/aegis/globalcfg/vulfilebackup': Permission denied
    38. root       [    5306]:############################################################
    39. [maxwell@MaxwellDBA test]$

    1.16 Showing a Paragraph of Text After a Found Phrase

    1. [maxwell@MaxwellDBA test]$ awk -f para.awk < myinput.file
    2. [maxwell@MaxwellDBA test]$ cat para.awk
    3. /keyphrase/ { flag=1 }
    4. { if (flag == 1) { print $0 }}
    5. /^$/ { flag=0 }
    6. [maxwell@MaxwellDBA test]$
  • 相关阅读:
    20_数组的常见操作
    【正则】详解python正则表达式之re.group()用法
    shell脚本（三）结构化命令
    fork函数
    hash——模拟散列表（拉链法和开放寻址法）
    【今天是Ajax同学的到来，我们了解一下她吧】
    图解八道经典指针笔试题
    Leetcode 496. 下一个更大元素 I
    测试代码1
    JUC笔记(六) --- 不可变共享模型
  • 原文地址：https://blog.csdn.net/u011868279/article/details/126006643