Shell Script

Regex

  • . : any one character
  • * : match any number of previous (include 0)
  • + : match any number of previous
  • $ : end of the line
  • ^ : beginning of the line
  • [^] : any character but the ones in brackets
  • \S : any none white space character
  • \s : any white space character
  • \b : any border (any white space/end of file not included in the returned match)
  • ? : optional
  • [a-z] : any lowercase letter
  • [A-Z] : any uppercase letter
  • [a-zA-Z] : any letter
  • [0-9] : any number
  • \ : scape something

Tips

#!/bin/bash
var="aaa bbb ccc"
echo ${var%% *}
echo ${var##* }
echo ${var% *}
echo ${var#* }

#Results:
#aaa
#ccc
#aaa bbb
#bbb ccc

General

#!/bin/bash
ps axc #all system processes
pidof var #process ID of var
pigrep var #process ID of var
uname -a #system information 
alias var #what is var alias for
tr "[:lower:]" "[:uper:]" #replace all lower case characters with upercase

File Information

#!/bin/bash
du -h #used space or file size
df #free space

Grep

-i : case insensitive

Sed

sed -i 's/{old pattern}/{new pattern}/' data.txt # The first instance of the search string in each line is replaced.

# [ ] : multiple character pattern
sed '/[mM]z/!d' "PATH to a file"` = `grep -i mz "PATH to a file"

sed -i 's/{old pattern}/{new pattern}/g' data.txt # Global replacement all occurrences of the search string in each line.

sed -i 's/\b{old pattern}\b/{new pattern}/g' data.txt # Using the word-boundary expression (\b) to ensures the partial words are not matched

sed -i 's/{old pattern}/{new pattern}/gI' data.txt # Case insensitive

Cut

cut -f 3 data.txt # Cut the third field of each line.

cut -f 2-4 data.txt # Cut the second-through-fourth field.

cut -f 1-2,4-5 data.txt # Cut only the first-through-second and fourth-through-fifth field.

cut -f 3- data.txt # Cut the third field and every field after it.

cut -c 3-12 data.txt # Output only the third-through-twelfth character of every line.

cut -b 3-12 data.txt # Counting bytes instead of characters.

cut -f 1,3 -d ':' /etc/passwd # Specifying a delimiter other than tab

cut -f 1,3 -d ':' /etc/passwd # Display each username and user ID number

cut -f 1,3 -d ':' --output-delimiter=' ' /etc/passwd # delimite the output by a tab

Awk

\t # tab in strings in print and printf

\n # new line in strings in print and printf

%ns # A n characters string in printf

%nd # A n characters integer in printf

%n.mf # A n.m characters float in printf

a += b # a = a + b

Flags

-f # Get path of an awk script file

-F”,” # Set a delimiter

BEGIN {starting actions}
awk '(pattern) {action}' PATH-TO-INPUT-FILE
END {ending actions}
awk -f PATH-TO-CODE-FILE.awk PATH-TO-INPUT-FILE

/regular expression patterns/ # put regular expression patterns between //

awk '{ print $2 "---" $1, $3 }' input.txt # print second column"---"first column third column of input.txt

== vs ~ : exact match vs contain the pattern

! : not

$0 : whole line

awk '{print $0}' "PATH to a file"`= `cat "PATH to a file"`

$1 $2 … : first column second column ..

awk '$1 == "var" {print $2}' "PATH to a file" # Print out the second column of every line with var in their first column.
awk '{$1=""; $2=""; sub(" ", " "); print}' input.txt = 
awk '{print substr($0, index($0, $3))}' input.txt = awk '{$1=$2=""}1' input.txt = cut -d " " -f 3- input.txt # skip first 2 columns

awk 'FNR==NR{a[NR]=$3;next}{$2=a[FNR]}1' input2.txt input1.txt # Replace second column of input1 with third column of input2

awk '/var/ "PATH to a file" = 
grep var "PATH to a file"

Special Variables

NR : number of record (line number)

awk'(NR>=0 && NR<=10){print} NR==11{exit}' "PATH to a file"` = `head "PATH to a file"` = `sed 10q "PATH to a file"

NF : number of field (column number)

FNR : file number of record (line number of input file)

FS : field separator

FPAT : field pattern

FILENAME: file name of input file

ARGC : how many input arguments are passed to the awk code

ARGV : an array of arguments that are passed to the awk code

tolower : lower case all characters

awk 'tolower($0) ~ /mz/' "PATH to a file" = 
grep -i mz "PATH to a file"

Array

array[key] = value

Dimensional Arrays

array[key1][key2] = value

Example 01

ex1.awk:

#!/usr/bin/awk -f
NF == 2 { phones[$1] = $2 }
NF == 3 { phones[$1] = $3 }
END{
        for (p in phones) {
                printf "|%7s|%10d|\n    ", p,phones[p]
        }
}

input.txt

John 987654321
Jack Smith 123456789
ex1.awk input.txt # Or awk -f ex1.awk input.txt

# Result:
# |   Jack| 123456789|
# |   John| 987654321|

Functions

BEGIN { printf "%2.20f\n", atan2(0, -1) } :  3.14159265358979311600

func multiply(a, b) {
    return a * b
}

Heredoc

#!/bin/bash
while read $var1 $var2 ...
do
    echo "var1=$var1 var2=$var2 ..."
done <<EOF
a 1
b 2
c 3
...

#Result:
#var1=a var2=1
#var1=b var2=2
#var1=c var2=3
...

Autorotation With Xrandr

#!/bin/bash
# https://github.com/gevasiliou/PythonTests/blob/master/autorotate.sh original source
# December 2020; tested and works on Medion Akoya E3213 (MD60986)
# This is a bash script to make screen autorotation possible based on device orientation
# This script *should* be added to startup applications for the user.

function Tablet {
	[[ $1 == "1" ]] && action="enable"
	[[ $1 == "0" ]] && action="disable"
	while read i; do
		xinput $action $i;
	done <<< "$(xinput list | grep -e 'AT.*keyboard' -e 'Synaptics' | grep -Po 'id=\K[0-9]+')";
}

# Use multistage grep to find the display connector (VGA-1/eDP-1/etc)
SCREEN=$(xrandr | grep 'connected' | grep -v 'disconnected' | grep -oE '[a-zA-Z]+[\-]+[^ ]')
TOUCHPAD="SYNA3602:00 0911:5288 Touchpad"
TOUCHSCREEN="FTSC1000:00 2808:50C5"
# Launch monitor-sensor - store output in a variable on tmpfs (RAM) to be parsed by the rest script
monitor-sensor >> /dev/shm/sensor.log 2>&1 &

# Parse output of monitor-sensor to get the new orientation whenever the log file is updated
# Possibles are: normal, bottom-up, right-up, left-up. Light data will be ignored
# Use inotifywait to wait for a modification of a file (/dev/shm/sensor.log) "-e modify" waits for modification
while inotifywait -e modify /dev/shm/sensor.log; do
	ORIENTATION=$(tail -n 1 /dev/shm/sensor.log | grep 'orientation' | grep -oE '[^ ]+$')  # Read the last line that was added to the file and get the orientation
	case "$ORIENTATION" in # Set the actions to be taken for each possible orientation
	normal)
			xrandr --output $SCREEN --rotate normal
			xinput set-prop "$TOUCHPAD" "Coordinate Transformation Matrix" 1 0 0 0 1 0 0 0 1
			xinput set-prop "$TOUCHSCREEN" "Coordinate Transformation Matrix" 1 0 0 0 1 0 0 0 1
			zenity --question --text"Disable keyboard?"
			Tablet $? ;;
	bottom-up)
			xrandr --output $SCREEN --rotate inverted
			xinput set-prop "$TOUCHPAD" "Coordinate Transformation Matrix" -1 0 1 0 -1 1 0 0 1
			xinput set-prop "$TOUCHSCREEN" "Coordinate Transformation Matrix" -1 0 1 0 -1 1 0 0 1
			zenity --question --text="Disable keyboard?"
			Tablet $? ;;
	right-up)
			xrandr --output $SCREEN --rotate right
			xinput set-prop "$TOUCHPAD" "Coordinate Transformation Matrix" 0 1 0 -1 0 1 0 0 1
			xinput set-prop "$TOUCHSCREEN" "Coordinate Transformation Matrix" 0 1 0 -1 0 1 0 0 1
			zenity --question --text="Disable keyboard?"
			Tablet $? ;;
	left-up)
			xrandr --output $SCREEN --rotate left
			xinput set-prop "$TOUCHPAD" "Coordinate Transformation Matrix" 0 -1 1 1 0 0 0 0 1
			xinput set-prop "$TOUCHSCREEN" "Coordinate Transformation Matrix" 0 -1 1 1 0 0 0 0 1
			zenity --question --text="Disable keyboard?"
			Tablet $? ;;
	esac
done

Fun

List all packages that explicitly installed on arch

pacman -Qet | awk '{ print $1 }' > names && pacman -Qi $(pacman -Qet | awk '{ print $1 }') | awk -F":" '/Description/ {$2 = "0 ,"$2;print $2}' > descriptions && awk 'FNR==NR{a[NR]=$1;next}{$1=a[FNR]}1' names descriptions > packages.csv && rm names descriptions

pacman -Qqetti | grep -e Name -e Description | awk -F': ' '{if ($1 ~ /^Name/) printf $2;else print", \""$2"\""}'

Small script to generate a random password. Works on Linux with busybox, haven’t tested on GNU.

hexdump -e '"%02x""\n"' /dev/random | tr -d '\n' | dd count=32 bs=1 2>/dev/null; echo

Net usage:

ip -s -h -c link show wlan0 

RX=$(ip -s -h -c link show wlan0 | awk 'FNR==4{print val,$1}')
TX=$(ip -s -h -c link show wlan0 | awk 'FNR==6{print val,$1}')

Paste a command output to a pastebin:

If you are stuck in a terminal and don’t want to use hastebin.com, you can use ix.io or paste.rs to paste from the CLI:

<command to print output> |& curl -F 'f:1=<-' ix.io
<command to print output> |& curl --data-binary @- https://paste.rs

GNU parallel

Executing jobs in parallel

for i in *jpeg; do convert $i $i.png ; done # a serial process
find . -name "*jpeg" | parallel -I% --max-args 1 convert % %.png # parallel

# Or

$ cat jobs2run
bzip2 oldstuff.tar
oggenc music.flac
opusenc ambiance.wav
convert bigfile.tiff small.jpeg
ffmepg -i foo.avi -v:b 12000k foo.mp4
xsltproc --output build/tmp.fo style/dm.xsl src/tmp.xml
bzip2 archive.tar

$ parallel --jobs 6 < jobs2run

# Or

$ ls -1
12_LS_establishing-manor.avi
12_wildsound.flac
14_butler-dialogue-mixed.flac
14_MS_butler.avi
...and so on...

$ ls -1 | parallel --max-args=2 ffmpeg -i {1} -i {2} -vcodec copy -acodec copy {1}.mkv

TTY screenshot

sudo cat "/dev/vcstty | sed 's:/dev/tty::'" | sed "s:.\{tput cols\}:&\n:g" | convert -negate -font Courier label:@- screenshot.png