andy@coma.UUCP (Andreas Lampen) (07/31/90)
Hi folks, several weeks ago, I asked for Your most favourite (sophisticated, cryptic, funny) pipe ! I received a lot of them. Thank you all. Here is a list -------------------------------------------------------------------------- 1) Matt Crawford (matt@oddjob.uchicago.edu) Here's one csh alias I like, although the pipe is not the best part of it. alias ptr "(" echo set q=PTR ";" echo \!\$:e.\!\$:r:e.\!\$:r:r:e.\!\$:r:r:r.in-addr.arpa ")" "|" nslookup Here's a script which is one long, fun pipe. (I wrote it when I was new to unix and didn't know that sh was better than csh for scripts.) #! /bin/csh -f # Script to notify heavy disk users of their sins # Change the constant at the beginning of the awk command as needed. # (/usr/etc/quot -f /dev/rxy0e ; /usr/etc/quot -f /dev/rxy0f ; \ /usr/etc/quot -f /dev/rxy1e ; /usr/etc/quot -f /dev/rxy1f) | \ egrep -v 'root|/dev/' | \ sort -nr | \ awk '$1 > 8000 { printf "mail -s Disk-Usage %s << EOM\n", $3 ;\ printf "Your %d files in your home directory are using %d KB.\\n",$2,$1 ;\ printf "This makes you the number %d disk-hog.\n",NR ;\ printf "EOM\n" ; }' | \ csh -sf 2) Ken Yap (ken@cs.rochester.edu) One of my favourites for renaming all .pas files to .p (or similar) is ls *.pas | sed 's/\(.*)\.pas/mv & \1.p/' | sh 3) Andy Behrens (andyb@coat.com) The next is part of a shell script that runs multiple copies of compress in parallel, for rapidly compressing large directories. (Our host is a multiprocessor, so this really does save time). The case statement probably causes a subshell to be started up, so I should rewrite it so it will run faster. PROGRAM=compress PARALLEL=10 # number of processors P=`expr $N / $PARALLEL + 1` # each copy of compress gets $P files echo "$files" | xargs -n$P | case "$trace" in -v) sed "s/.*/$PROGRAM -v & 2>\&1 | cat \&/"; echo wait ;; *) sed "s/.*/$PROGRAM & 2>\&1 \&/"; echo wait ;; esac | sh | sed 's/-- replace.*//' 4) Frank P. Bresz (..!uunet!ittc!fbresz) #! /bin/csh -f # # Postport Postscript File Conversion. TLM # # 4/10/89 - modified to make lpr use -s option. TLM # if ($#argv == 0) then expand | sed -f /usr/local/bin/postsed | awk -f /usr/local/bin/postawk | lpr -s else cat $argv | expand | sed -f /usr/local/bin/postsed | awk -f /usr/local/bin/postawk | lpr -s endif This is postsed s/\\/\\\\/g s/[(]/\\(/g s/[)]/\\)/g This is postawk BEGIN { cpage = 0 newline = 720 newpage = 0 print "%!\n%%Pages: (atend)\n%%DocumentFonts: Courier" print "%%EndComments\n/docstart save def" print "/Courier findfont 11 scalefont setfont\n%%EndProlog" } { if (newpage == 0) { cpage++ newpage++ print "%%Page: \"" cpage "\" " cpage "\n/pagestart save def" } } { if ( length($0) != 0 ) { { if ( $0 != "\f" ) { printf "(%s) 25. %3d. moveto show\n", $0, newline } else { { if ( newline != 720 ) { newline = 64 } else { newline += 12 } } } } } } { newline -= 12 } { if (newline <= 64) { newline = 720 newpage-- print "pagestart restore\nshowpage" } } END { { if (newline != 720) { print "pagestart restore\nshowpage" } } {print "%%Trailer\ndocstart restore\n%%Pages: " cpage } } 5) John M. Blasik (john@mintaka.mlb.semi.harris.com) rsh other.sun screendump | screenload [We had a lot of fun with that, Andy] 6) David Elliott (dce@smsc.sony.com) There are two cases of pipes that I use a lot. The first is backquotes, and I only mention it to keep you honest. I often do things like vi `grep -l pattern *.[ch]` In general, I use the more typical pipes interactively (you don't want to deal with sh program type pipes, do you?) in an iterative way. For example, a csh session might look like: % grep pattern1 *.[ch] <too much stuff> % !! | grep -v pattern2 grep pattern1 *.[ch] | grep -v pattern2 <still too much> % !! | grep pattern3 ... until I finally get the set of lines I was looking for. This isn't efficient, but it's conceptually the easiest way to do this, as having to use egrep with OR'ed patterns takes a lot more typing. 7) Dick Dunn (rcd@ico.isc.com) A short pipe sequence I use fairly often is whatever-stuff | sort | uniq -c | sort -nr | more which produces a frequency count (in descending order) of lines produced by "whatever-stuff". 8) Brian Rice (rice@dg-rtp.dg.com) Wenn man mehreren Prozessen mit aehnlichen Namen (z.B., "biod") toeten will, kann er benuetzen: ps -e | grep biod | grep -v grep | awk '{print $2}' | xargs kill -9 (Mit Berkeley UNIX setzt man "-agx" an die Stelle von "-ef", und auch "grep -v PID | " vor "xargs" einschaltet.) 9) boyd@necisa.ho.necisa.oz.au It's not one you'd type on the command line, but it's pretty involved. case "$1" in 8???????) ;; "") echo "usage: `basename $0` vaddr" 1>&2 exit 1 ;; *) echo "`basename $0` \"$1\" is not a valid virtual address." 1>&2 exit 1 ;; esac ( echo 8i ( addr="`echo "$1" | tr '[a-f]' '[A-F]'`" echo "*pc* |0`echo 16i8o${addr}p | dc`|.text" nm -vo unix ) | sed -e '/^etext/s/$/.text/' -e '/\.text$/!d' -e 's/|02//' -e 's/|.*//' -e 's/^/[/' -e 's/ */ ]P/' -e 's/$/pc/' ) | dc | sort -n +1 | awk ' BEGIN { addr[1] = 0 addr[2] = 0 pc = -1 symbol = "zero" } $1 == "*pc*" { pc = $2 next } pc != -1 && $2 > pc { addr[2] = $2 exit } { symbol = $1 addr[1] = $2 } END { printf("0x%x 0x%x\t%s+0x%x (0x8%07x)\n", addr[1], addr[2], symbol, pc - addr[1], pc) } ' 10) Paul Davey (pd@ixi.co.uk) alias tarcp " (cd \!:1 ; tar cf - . ) | ( cd \!:2 ; tar xf - )" alias psg 'ps aux | sed -n -e "/sed -n -e /d\\ /\!$/p\\ /TIME COMMAND/p"' set lsfields = `/bin/ls -ld / | wc -w` if ( "$lsfields" == "8" ) then setenv SYS BSD else setenv SYS SYS5 endif unset lsfields 11) Charles Geyer (charlie@milton.u.washington.edu) #! /bin/sh if test $1 = '-b' then shift B='-b' else B='' fi if test -f $1.spell then sed 's/\\[a-zA-Z]*//g' $1.tex | spell $B | sort | comm -23 - $1.spell else sed 's/\\[a-zA-Z]*//g' $1.tex | spell $B | more #! /bin/sh CODA=">/dev/null 2>&1" # This is the magic rhost=$1 shift options=$* host=`hostname`.stat.washington.edu DISP="-display $host":0.0 if test `xhost | grep $rhost | wc -l` -eq 0 then xhost $rhost fi echo "xterm -n $rhost -ls $options $DISP $CODA &" | rsh $rhost sh 12) Martin Weitzel (martin@mwtech.UUCP) Consider the classic example where all files with name *.c should be renamed to *.c.bak. The first approach does it this way: for i in `ls *.c` do mv $i $i.bak done Besides that it is difficult to invert if run-time performance is an issue (renaming *.c.bak to *.c commonly requires execution of an `expr ... : ...` in every cycle of the loop), it has more drawbacks: The list of names generated by ls *.c can not exceed the limit of several KByte (excact value depends, but is sufficiently low to cause problems sometimes) and there are some hassles if you must take the possibility of "no matching files" into account. Now, compare this to ls | sed -n '/\.c$/s=.*=mv & &.bak=p' | sh which has none of the above drawbacks. Doing the reverse is not much more complicated: ls | sed -n 's=\(.*\.c\)\.bak$=mv & \1=p' | sh Furthermore, *very* sophisticated things can be done by tailoring some shell script "on the fly" (as above with "sed", with "awk", .... or even with an "echo" from another shell) and piping this to a shell. As shown above this technique at least will take the burden of looping away from the shell, because the loop is embedded now in the method that generates the data ("ls"ing a directory "cat"ing the contents of a file or whatever). 13) Peter da Silva (peter@ficc.ferranti.com) [Responding to Martin Weitzel (above)] I would write ls *.c | while read i; do mv $i $i.bak; done Or, ls *.c | sed 's/\(.*\).c$/& \1.bak/' | while read i j; do mv $i $j; done This does have a bit of looping overhead, and is a big help when what you want to do is a bit too complex to put into your sed command: find . -name *.c -print | while read file do grep '^something:' $file | while read junk body do process $file -options $body and more stuff done done If this can qualify as a single 'pipeline', then I have some real bottlers. 14) Jan Christiaan van Winkel (jc@atcmp.nl) I once had three big files (just below 1 MB each) that when catenated made a uuencoded compressed TAR file. The problem was that on the system I work on, the ulimit is set to 1 Mb, i.e. I could not create files larger than 1 Mb. Therefore I had to use a pipe to un-tar the file, for example something like cat xaa xab xac | uudecode | compress -d | tar xvf - However, uudecode INSISTS on writing to the file named in the first line. In this case 'tarfil.Z'. This would create a file larger than 1 Mb. (Okay, I could have used the source of uudecode to change that, but I wanted to do it quick 'n' dirty). I used a named pipe with the name tarfil.Z to let uudecode write to. Thus, the commands became: cat xaa xab xac | uudecode & compress -d < tarfil.Z | tar xvf - Mayby not sophisticated, but it sure hepled me! 15) Mitchell Wyle (wyle@inf.ethz.ch) tr -cs A-Za-z '\012' | tr A-Z a-z | sort | uniq -c | sort -rn Try that one in a "high-level" language! 16) Joe Bush (bush@evax.arl.utexas.edu) Here is one I concocted to search for key words in include files: echo -n "key=";set kw=`line`;find /usr/include/. /sys/. -name \*.h -print | xargs hgrep "$kw" {} I keep a file of such one-liners like the one above (file of one line pipe-programs is named $HOME/.syscom) and have the following line in my .cshrc: alias g 'set j=`cat ${home}/.syscom|wc -l`;source -h ${home}/.syscom; history | tail -"$j"' Then when I enter "g" from the keyboard, my csh history mechanism gets primed for easy execution. I find it quite handy... 17) Bjorn Engsig (bengsig@oracle.nl, bengsig@oracle.com) Neither sophisticated, cryptic or funny, but often useful: $ make foo make: don't know how to make `foo' $ make -n all | grep foo | sh -x 18) Thomas Truscott (trt@rti.rti.org) Here is a shell script that has a pipeline in it somewhere. #! /bin/sh # you pick a word, e.g. "hangman". # if antihang guesses "a", the new pattern is ".a...a.". # if antihang then gueses "t", the new pattern is ".a...a." (no change). # antihang fails on words not in /usr/dict/words wordlist=/usr/dict/words myguess=e guesses= while [ "$myguess" ]; do guesses=$guesses$myguess echo -n "I guess $myguess, new pattern: " read pattern srchpat=`echo "$pattern" | sed "s/\./[^$guesses]/g"` myguess=`grep "^$srchpat$" $wordlist | \ sed -e "s/[$guesses]//g" -e 's/\(.\)\(.*\)\1/\1\2/g' -e 's/./&\\ /g' | \ sort | uniq -c | sort -nr | \ sed -n -e 's/^.*\([^ ]\)$/\1/p' | sed 1q` done 19) Heiner Marxen (heiner@specs.uucp) Du sammelst doch Pfeifen :-), also hab' ich mal in meine aliase gesehen. Unten ein Exerpt meines ".login" (fuer csh). Interessant ist vielleicht das alias "wot", es enthaelt eine pipe in `` als Teil einer pipe. # Define macros for symbolic naming of directories # and fast travelling between them. a hier set GO'\!:1'=\`/bin/pwd\` a here set GO'\!:1'='"`dirs | sed '"'"'s/ .*//'"'"'`"' a go echo cd \"\$GO'\!:1'\" \; cd \$GO'\!:1' a gp echo pd \"\$GO'\!:1'\" \; pd \$GO'\!:1' a wo set \| sed '"/^GO/\\!d;s/^GO//"' a Wo wo \| sort +1 a wg wo \| grep a wot wo \| egrep '" (`dirs|sed '"'"'s/ .*//'"'"'`|`pwd`)"' a wo. wo \| egrep '" (`dirs|sed '"'"'s/ .*//'"'"'`|`pwd`)"'\\$ a unhere unset GO'\!:1' a hereset set GO'\!:1'='\!:2' a putgo set \| \ sed "'"'/^GO/\\!d;s/$/"/;s/ / "/;s/^GO/hereset /'"'" \ \>\! ~/.gosave a getgo "if( -e ~/.gosave ) source ~/.gosave" Dann noch ein gar nicht so untypischer Fall fuer Kommandos, die mit "sh" implementiert werden. Das folgende ist Teil eines lokalen scripts um unseren Postscript-Laser zu beschicken: case $# in 0) expand $tab | pr5 -w190 $two -f -l77 $prflags ;; *) for f do t="$two" l=`sed -e 68q "$f" | wc -l | sed -e 's/^ *//' -e 's/ .*//'` case $l in ? | [1-5]?) t="" ;; 6?) case `expr $l \<= 67` in 1) t="" ;; esac esac expand $tab "$f" | pr5 -w190 $t -f -l77 $prflags -h "$f" done ;; esac | lpscript -s7 -r -o1.0 | lpr -Pps 20) Phil Budne (budd@bu-it.bu.edu) For seeing if anything of mine is stuck in the mqueue (I call it "mq"); #!/bin/sh # display my items in the mail queue if [ $# -gt 0 ]; then who=$1 else who=`whoami` fi echo Pending messages on `hostname` from user $who # mailq | \ awk " BEGIN { pr = items = found = 0; } \ /^AA/ { if( \$0 ~ /$who/ || \$0 ~ /$who\@\.\*/ ) { \ pr = 1; found++; } \ else { pr = 0; } items++; } \ { if( pr ) print } \ END { if( found > 0 ) fm = \"(\" found \" displayed)\"; print items \" total \" fm } " A summation of rwho output (I call it "uz"); #!/bin/sh if tty -s <&1; then rwho $* | sed 's/[ ].*//' | uniq | fmt else rwho $* | sed 's/[ ].*//' | uniq fi A count of users as displayed by "uz" (I call it "uc"); rwho $* | sed 's/[ ].*//' | uniq | wc -l For seeing what batched news is queued for a site (I call it "gq") #/bin/sh # NOTE! our /usr/spool/news is in /news SPOOLDIR=/news if [ "$1" -a -f $SPOOLDIR/batch/$1 ]; then sed -e 's@/[0-9].*$@@' \ -e "s@^$SPOOLDIR/@@" \ -e 's@/@.@g' < /usr/spool/news/batch/$1 |\ sort | uniq -c | sort -rn fi Show most frequently executed commands (I call it "lastfreq"); lastcomm | sed 's/ .*//' | sort | uniq -c | sort -rn -- ---- Andreas Lampen, Tech. Univ. Berlin andy@coma.cs.tu-berlin.de
reddy@lion.austin.ibm.com (/150000;Austin) (08/01/90)
How about including useful short pipes (which could make our lives (novices) a little easier (and productive). Satish
tbray@watsol.waterloo.edu (Tim Bray) (08/02/90)
>How about including useful short pipes (which could make our lives (novices) >a little easier (and productive). What have I been doing lately in this big directory? ls -lt | head
turner@ksr.com (James M. Turner) (08/03/90)
>How about including useful short pipes (which could make our lives (novices) >a little easier (and productive). tar cf - . | (cd ../otherdir;tar xfpB -) Copy everything in this directory to "../otherdir", preserving owner and date. -- Name: James M. Turner * Great Moments in Aviation #21: While on a Company: Kendall Square Research * NDB approach into Hanscom, pilot Ted Hertz Email: turner@ksr.com, ksr!turner * accidently tunes WEEI on his NAV instead Phone: (617) 895-9400 * of the LOM, and lands on top of the Pru.
hascall@cs.iastate.edu (John Hascall) (08/03/90)
reddy@lion.austin.ibm.com (/150000;Austin) writes: }How about including useful short pipes (which could make our lives (novices) }a little easier (and productive). Here's a good one (stolen from an Ultix man page): (cd fromdir; tar cf - .) | (cd todir; tar xpf -) And it even has a smilie at the end -) John Hascall / john@iastate.edu / hascall@atanasoff.cs.iastate.edu
trt@rti.rti.org (Thomas Truscott) (08/04/90)
> Here's a good one (stolen from an Ultix man page): > (cd fromdir; tar cf - .) | (cd todir; tar xpf -) I have learned that (cd fromdir && tar cf - .) | (cd todir && tar xpf -) is a safer form of this command. Guess how I learned :-) In general, use "a && b" rather than "a ; b" if you only want to do "b" if "a" succeeds. Some versions of /etc/rc have a line to delete trash from /tmp: (cd /tmp; rm -f *) # Not recommended I wonder what happens if /tmp is inadvertently missing, or mode 0? I guess it just deletes trash in some other directory :-) (rm -f /tmp/*) # Recommended seems wiser, particularly since files in /tmp can be named "-r" or "-i". Tom Truscott
cosell@bbn.com (Bernie Cosell) (08/04/90)
trt@rti.rti.org (Thomas Truscott) writes: } (rm -f /tmp/*) # Recommended }seems wiser, particularly since files in /tmp can be named "-r" or "-i". On the other hand, adding '-r' makes some sense, unless you'd really like to leave a subdirectory of /tmp around forever. Also, I do "rm -f /tmp/.*" [*NO* '-r' on this one, thank you very much! :-)]. /Bernie\
staff@cadlab.sublink.ORG (Alex Martelli) (08/09/90)
tbray@watsol.waterloo.edu (Tim Bray) writes: >>How about including useful short pipes (which could make our lives >>(novices) a little easier (and productive). >What have I been doing lately in this big directory? >ls -lt | head Yup, VERY useful - but, more portably (not everybody has "head"): ls -lt | sed 11q (note how the '1' and 'q' are nicely arranged near each other on your keyboard, just for this purpose...:-) -- Alex Martelli - CAD.LAB s.p.a., v. Stalingrado 45, Bologna, Italia Email: (work:) staff@cadlab.sublink.org, (home:) alex@am.sublink.org Phone: (work:) ++39 (51) 371099, (home:) ++39 (51) 250434; Fax: ++39 (51) 366964 (work only; any time of day or night).