techwiki:bash

Differences

This shows you the differences between two versions of the page.


Previous revision
techwiki:bash [2022/12/03 10:55] (current) – [Bash profile] ying
Line 1: Line 1:
 +This page contains Bash cmd info in general, and specific binary tool command in Mac and Linux.
 +====== Bash profile ======
 +user profile located at : <code bash>~/.bash_profile</code>
 +
 +edit user profile environment variable in the bash profile:
 +  * display: <code bash>echo $PATH</code>
 +  * display with escape <code bash>echo \$PATH</code>
 +  * display raw text: <code bash>echo '$PATH'</code>
 +  * display with variable: <code bash>echo "Path is $PATH"</code>
 +  * display block of text: <code bash>echo << EOF
 +Block of text here
 +Text for $PATH
 +EOF</code>
 +  * display block of raw text: <code bash>echo << "EOF"
 +Block of text here
 +Text Again but no $PATH
 +EOF</code>
 +    * ref: http://en.wikipedia.org/wiki/Here_document
 +  * session based variable set:<code bash>path=/myPath:$PATH</code> 
 +  * permanently change to path in profile file:<code bash>export PATH=/myPath:$PATH</code>
 +
 +Note: by adding your_Path_To_3rd_Party_Binary to the $PATH variable, you can directly call them in terminal.
 +
 +**My linux bash profile code**
 +
 +<code bash>
 +# User specific aliases and functions
 +alias fx='cd ~/App/firefox;./firefox'
 +# refresh fx installation
 +alias fxnew='cd ~/App;rm -r firefox;tar xjvf firefox-91.9.1esr.tar.bz2'
 +</code>
 +====== Bash tips ======
 +  * find path to a command file <code bash>which ls</code>
 +  * escape by "\"
 +  * create a txt file <code bash>echo "hello" > text.txt</code>
 +  * create a empty file or update a file <code bash>touch empty.txt</code>
 +  * append to a txt file <code bash>echo "127.0.0.1\thome" >> /etc/hosts</code>
 +  * create a symbolic link <code bash>ln -s ~/real_dir ~/Desktop/alias_name</code>
 +  * make folder accessible to all <code bash>sudo chmod 777 /your_folder_name</code>
 +  * batch file rename<code bash>
 +for i in *.JPG ; do mv $i ${i/-[0-9.]*.JPG/.JPG} ; done
 +</code>
 +
 +  * Vi text editor <code>
 +vi
 +# cmd mode
 +#  i : enter text edit mode
 +#  dd: delete line
 +#  x : delete a char
 +#  w : write file wq: save & quit
 +#  q : quite q!: quit without save
 +# edit mode
 +# esc: back to cmd mode
 +# ':':enter cmd mode
 +</code>
 +
 +^ shortcut ^^
 +| ctrl + z | suspend | 
 +| ctrl + a | goto cmd line start |
 +| ctrl + e | goto cmd line end |
 +
 +Full bash terminal shortcuts: http://www.shell-tips.com/2006/10/29/working-quickly-with-some-usefull-bash-shortcuts/
 +
 +^ quick list ^^
 +| man | manual, q:quit man; space: next |
 +| ls | ls -la: long list directory; ls -sh: list size human read | 
 +| cp | cp -R: recursive copy directory |
 +| mv | move file |
 +| rm | rm -R: recursive delete directory |
 +| mkdir | make directory | 
 +| source | source a shell script file | 
 +| %%|%% | pipe prev cmd to next cmd |
 +| cmd& | run cmd background |
 +| clear | clear screen |
 +
 +====== bash - computer management: user, admin, network ======
 +<code bash>
 +# check users: w, who, users
 +#
 +#<----------------------------------->
 +# ifconfig
 +wifi(){
 + #os x version
 + ifconfig en1 | grep 'inet ' | cut -d ' ' -f 2
 + # linux version
 + # ifconfig en1 | grep 'inet addr:' | cut -d: -f2 | awk '{ print $1}'
 +}
 +
 +#<---------------------------------->
 +# kill
 +
 +# kill by name
 +alias kbn=ka;
 +ka(){
 +for X in `ps acx | grep -i $1 | awk {'print $1'}`; do
 +  kill $X;
 +done
 +}
 +alias ki='ka itunes'
 +</code>
 +
 +====== bash - file operation ======
 +<code bash>
 +
 +#<---------------------------------->
 +# ls
 +
 +lss(){
 +ls -R | grep ":$" | sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^/ /' -e 's/-/|/'
 +}
 +
 +lsss(){
 +ls -trR
 +}
 +
 +# ls -file tools
 +# ref: http://www.thegeekstuff.com/2009/07/linux-ls-command-examples/
 +alias a='ls'
 +alias aa='ls -AF'
 +alias aaa='ls -lF'
 +alias ll='ls -lF'
 +alias lt='ls -ltF' # by time
 +alias lt.='ls -ltrF' # by time reverse
 +alias lsB='ls -S'
 +alias lsS='ls -Sr'
 +alias lh='ls -lhF'
 +alias l='ls'
 +alias ld='ls -d */'
 +alias 1='ls -1F'
 +# find and remove all
 +# sudo find * -type d -name .svn -exec rm -rf {} \;
 +#
 +#<--------check md5-------------------------->
 +md5(){
 +openssl md5 "$1"
 +# tip: quote $1
 +}
 +
 +# print directory structure
 +
 +graph(){
 +set `pwd`
 +set `basename $1`
 +ls -FR>~/Desktop/$1.txt
 +}
 +
 +# clean ._mac file
 +#find . -name ._* -exec rm '{}' 
 +
 +# advanced sys function
 +
 +
 +uflat(){
 +find . -not -type d -print0 | xargs -0J % mv -f % .
 +find . -type d -depth 1 -print0 | xargs -0 rm -rf
 +}
 +
 +# replace text; usage: urep \[array of file] [find] [replace]
 +urep(){
 +for i in $1; do j=`echo $i | sed 's/$2/$3/g'`; mv "$i" "$j"; done
 +
 +#for i in *.txt; do j=`echo $i | sed 's/2010/2011/g'`; mv "$i" "$j"; done
 +}
 +urep2(){
 +rename 's/$2/$3/' $1
 +}
 +
 +# rename prefix; usage: upre/upost \[array of file] [pre/post]
 +upre(){
 +for i in $1; do mv "$i" $2."$i"; done
 +}
 +upost(){
 +for i in $1; do mv "$i" "$i".$2; done
 +}
 +
 +# switch ext; usage: uext \[array of file] [old ext] [new ext]
 +uext(){
 +for i in $1; do mv "$i" "${i/.$2}".$3; done
 +}
 +uext2(){
 +for i in $1; do mv "$i" "`basename $i .$2`".$3; done
 +}
 +uext3(){
 +rename 's/\.$2/\.$3/' $1
 +}
 +
 +# rename case switch; usage: ucase \[array of file]
 +ucaseUnix(){
 +rename 'y/A-Z/a-z/' $1
 +}
 +
 +# clean space in name; usage: uclean \[array of file]
 +ucleanUnix(){
 +rename 's/ //' $1
 +}
 +
 +# working (test)
 +# add date to pre/post; usage: udate \[array of file] [pre/post] [ext]
 +udate(){
 +if [ $2 ] && [ $3 ]; then
 + if [ $2 == "e" ] || [ $2 == "b" ] || [ $2 == "p" ] || [ $2 == "post" ]; then
 + for i in $1
 + do mv "$i" "${i/.$3}".`date '+%Y-%m-%d'`.$3
 + done
 + fi
 +else
 + for i in $1; do mv "$i" `date '+%Y-%m-%d'`."$i";done
 +fi
 +}
 +
 +</code>
 +
 +Get running script directory
 +<code bash>#!/bin/bash
 +echo "The script you are running has basename `basename $0`, dirname `dirname $0`"
 +echo "The present working directory is `pwd`"
 +</code>
 +
 +  * get file list <code bash>find *.jpg #find all jpg in fold</code>
 +  * rename find list <code bash>for i in `find *v10*.jpg`; do j=`echo $i | sed 's/v10/v010/g'`; mv "$i" "$j"; done 
 +# for each file, generate new name, replace old name with new name</code>
 +====== bash - utility ======
 +
 +===== grep =====
 +  * get lines contains word A <code bash>grep wordA sample.txt</code>
 +  * ignore case <code bash>grep -i wordA sample.txt</code>
 +  * match exact word <code bash>grep -w wordA sample.txt</code>
 +  * inverse selection, lines without word A <code bash>grep -v wordA sample.txt</code>
 +  * count <code bash>grep -c wordA sample.txt</code>
 +  * line number <code bash>grep -n wordA sample.txt</code>
 +  * list file <code bash>grep -l wordA *.txt</code>
 +
 +  * Pattern
 +    * ^wordA: line starts with wordA
 +    * wordA$: line ends with wordA
 +    * ^$: blank line
 +    * [set]: match one from set [0-9][A-Za-z] [abc] [x-z]
 +    * [:class:]: match one from class [:alnum:] [:alpha:] [:blank:] [:digit:] [:lower:] [:upper:] [:space:]
 +    * escape: \
 +    * .: any char
 +    * \< \>: begin/end of word
 +    * wordA|wordB: wordA or wordB
 +    * {n}: how many times appera in line {min,max}
 +    * ?: at most once
 +    * *: none to unlimited
 +    * +: at least once
 +    * ^: inverse range; begin of line
 +
 +ref:http://www.cyberciti.biz/faq/howto-use-grep-command-in-linux-unix/
 +
 +===== sed =====
 +
 +line editor
 +<code bash>sed '/range/s/find/replace/g' <input >output
 +# same with _
 +sed '/range/ s_find_replace_g' <input >output
 +#line range ^start, $end
 +sed '1,100 s/find/replace/'
 +</code>
 +
 +remove all comments in bash script
 +<code bash>sed '/start/,/stop/ s/#.*//'</code>
 +
 +ref: http://www.grymoire.com/Unix/Sed.html#uh-0
 +
 +===== awk =====
 +awk is a words grabber, like grep for lines grabber
 +
 +ref: http://www.grymoire.com/Unix/Awk.html
 +
 +
 +===== Vi =====
 +<code bash>
 +# vi
 +# --- change current line to lower case
 +# :s/.*/\L&/
 +
 +# -- set file ending with DOS or Unix
 +# :set ff=dos or :set ff=unix
 +
 +# -- mass converting dos<->unix
 +# vim +"argdo set ff=<format>" +wqa <files>
 +</code>
 +
 +
 +====== Mac specific binary tool ======
 +An A-Z Index of the Mac OS X command line : http://ss64.com/osx/
 +
 +enable root on mac : http://support.apple.com/kb/HT1528
 +
 +  * osascript <code applescript>osascript theScript.scpt</code>
 +  * open : open cmd opens every thing, like "start" in DOS
 +
 +===== mac specific cmd - disk related operation =====
 +
 +  * mount smb share folder (Windows,Linux,Mac share folder by SMB) <code bash>
 +if [ ! -d /Volumes/me ]; then mkdir /Volumes/me; mount_smbfs //username@PCNAME/shareFolderName /Volumes/me; fi
 +</code>
 +  * Dmg file creation and convertion <code bash>
 +# make an DMG image name $2 from folder called $1
 +dmgF(){
 +echo "CMD: hdiutil create -srcdir \"$1\" $2.dmg"
 +hdiutil create -srcdir "$1" "$2.dmg"
 +}
 +
 +# make an ISO image named as $2 from folder called $1
 +isoF(){
 +hdiutil makehybrid -iso -o "$2.iso" "$1"
 +}
 +
 +# burn DMG file $1
 +burn(){
 +hdiutil burn $1
 +}
 +
 +# create a DMG file with size $1 MB named as $2
 +dmg(){
 +hdiutil create -size $1m $2.dmg -fs HFS+ -volname $2
 +}
 +
 +# convert DMG file of $1 to an ISO image file
 +dmg2iso(){
 +hdiutil convert $1.dmg -format UDTO -o $1.iso
 +}
 +</code>
 +  * other disk related cmds <code bash>
 +################################
 +
 +# force reject disk
 +#drutil tray open
 +#drutil eject
 +
 +#<---------------------------------->
 +vo(){
 +cd /Volumes/"$1";.
 +}
 +
 +#<---------------------------------->
 +# dd function for disk dump
 +#
 +# dd back cd
 +# dd if=/dev/disk1 of=myDVDcopy.img bs=2048
 +# dd if=/dev/disk1 of=dvd.iso
 +
 +# dd backup disk
 +# dd bs=512 if=/dev/rXX# of=/some_dir/foo.dmg conv=noerror,sync
 +# dd bs=512 if=/dev/rdisk2s3 conv=noerror,sync | gzip -9 > foo.dmg.gz
 +
 +# file rescue
 +# dd bs=512 if=/Volumes/HD1/badfile.mpg of=/Volumes/HD2/newrecoveredfile.txt conv=noerror,sync
 +
 +# back and split
 +# dd if=/dev/hda# | gzip -9 | split -b 4096m /dev/sda#/somedir/macosx
 +# cat
 +# cat /dev/sda#/somedir/macosx* | gzip -d | dd of=/dev/hda# 
 +
 +# list disk and device
 +# disktool -l
 +# df -k
 +# diskutil list
 +
 +
 +#<---------------------------------->
 +# disktool
 +eject2 () {
 +/usr/sbin/disktool -e $(df | grep $1 | awk '{print $1}' | sed 's/\/dev\///')
 +}
 +
 +eject(){
 +eject2 /Volumes/$1
 +}
 +
 +#<---------------------------------->
 +# tar
 +#create tar
 +#tar -czvf MyArchive Source_file 
 +#tar --create --gzip --verbose --file=MyArchive Source_file
 +#extract tar
 +#tar -xzvf MyArchive Source_file 
 +#tar --extract --gunzip --verbose --file=MyArchive Source_file
 +
 +#tar xjvf filename.tar.bz2
 +
 +#tar -pvczf BackUpDirectory.tar.gz /path/to/directory
 +#tar --exclude=".*" -pvczf BackUpDirectory.tar.gz /path/to/directory
 +#tar -xvzf filename.tar.gz -C /desired/path
 +tarp(){
 +tar -xvzf $1 -C $2
 +}
 +tard(){
 +tar czvf $1.$(date +%Y%m%d-%H%M%S).tgz $1
 +exit $?
 +}
 +
 +#tar -czvf my_backup.$(date +%Y%m%d-%H%M%S).tgz source_file
 +
 +# unrar x test.part1.rar
 +
 +</code>
 +===== mac specific bash cmd - others=====
 +<code bash>
 +# convert to pdf
 +alias pdf2="/System/Library/Printers/Libraries/convert"
 +pdf(){
 +pdf2 -f "$1" -o "$1.pdf";
 +}
 +#<---------------------------------->
 +# man
 +function wman() {
 +   url="man -w ${1} | sed 's#.*\(${1}.\)\([[:digit:]]\).*\$#http://developer.apple.com/documentation/Darwin/Reference/ManPages/man\2/\1\2.html#'"
 +   open `eval $url`
 +}
 +
 +xman() {
 +   url="http://bashcurescancer.com/man/cmd/"
 +   open $url$1
 +}
 +</code>
 +
 +====== Linux specific binary tool ======
 +
 +  * convert
 +    * convert image sequence into gif<code bash>convert -delay 50 -loop 1 render.*.jpg animation.gif</code>
 +    * convert big image into small tile parts <code bash>convert -crop 64x64 +repage big.tif small_%02d.tif</code>
 +    * convert several images into one big image horizontally or vertically \\ <code bash>convert +append 1.jpg 2.jpg out.jpg # horizontally
 +convert -append 1.jpg 2.jpg out.jpg # vertically
 +</code>
 +  * zip \\ zip a folder into a zip file <code bash>zip -9 -r myZipFile.zip myFileOrFolderName</code>
 +  * xwininfo \\ get window info
 +  * sha1sum \\ get sha1 sum of a file like md5
 +  * 7zip \\ 7z file opener <code bash># install or http://p7zip.sourceforge.net/
 +sudo apt-get install p7zip
 +# extract
 +p7zip e example.7z
 +# more ref: http://www.dotnetperls.com/7-zip-examples
 +</code>
 +
 +====== Other 3rd Party tool ======
 +  * lame - convert aiff to mp3 <code bash>lame --r3mix -q0 --id3v2-only --tt \"[title]\" --ta \"[artist]\"
 +\"123.aiff\" \"123.mp3\"</code>
 +  * ffmpeg - convert image & video formats ([[http://ffmpeg.org/ffmpeg.html|ref]]) <code bash>
 +// convert image seq into avi
 +ffmpeg -f image2 -i seq.%04d.jpg -r 30 -s 640x480 -vcodec mpeg4 test.avi
 +  // -f: format; -i: input, %04d means 0001 like pattern; -r: framerate; -b:v video bitrate; -fs: file size limite; -s resolution; -vcodec: encode format
 +
 +// get info
 +ffmpeg -i video.avi
 +
 +
 +// img seq to video
 +ffmpeg -f image2 -i image.%04d.jpg video.mpg
 +// ref: https://ffmpeg.org/trac/ffmpeg/wiki/UbuntuCompilationGuide
 +ffmpeg -r 8 -i frame.%04d.jpeg -vcodec libx264 -vpre medium -crf 24 -threads 0 -s 640x480 output.mp4
 +
 +// video to img seq
 +ffmpeg -i video.mpg image.%04d.jpg
 +
 +// video to iphone mp4 or psp
 +ffmpeg -i source_video.avi -acodec aac -ab 128kb -vcodec mpeg4 -b 1200kb -mbd 2 -flags +4mv+trell -aic 2 -cmp 2 -subcmp 2 -s 320x180 -title X final_video.mp4
 +
 +ffmpeg -i source_video.avi -b 300 -s 320x240 -vcodec xvid -ab 32 -ar 24000 -acodec aac final_video.mp4
 +
 +// save sound
 +ffmpeg -i source_video.avi -vn -ar 44100 -ac 2 -ab 192 -f mp3 sound.mp3
 +
 +// wav to mp3
 +ffmpeg -i son_origine.avi -vn -ar 44100 -ac 2 -ab 192 -f mp3 son_final.mp3
 +
 +// mpg to avi, avi to mpg
 +ffmpeg -i video_origine.avi video_finale.mpg
 +ffmpeg -i video_origine.mpg video_finale.avi
 +
 +// video to gif
 +ffmpeg -i video_origine.avi gif_anime.gif
 +
 +// add souble
 +ffmpeg -i son.wav -i video_origine.avi video_finale.mpg
 +
 +// video to flv
 +ffmpeg -i video_origine.avi -ab 56 -ar 44100 -b 200 -r 15 -s 320x240 -f flv video_finale.flv
 +
 +// video to dvd
 +ffmpeg -i source_video.avi -target pal-dvd -ps 2000000000 -aspect 16:9 finale_video.mpeg
 +
 +// ogm to mpg
 +ffmpeg -i film_sortie_cinelerra.ogm -s 720x576 -vcodec mpeg2video -acodec mp3 film.mpg
 +</code>