In this series, we are going to learn Linux Shell Scripting. Shell scripting is a very important skill that Unix/Linux administrators can never go without. Shell Scripting means writing scripts (programs) for the UNIX/Linux Shell.

So, let me tell you what Linux Shell is.

What is Shell?

A shell in a Linux operating system takes input from you in the form of commands, processes it, and then gives an output. It is the interface through which a user works on the programs, commands, and scripts. A shell is accessed by a terminal which runs it.

What are shell scripts?

A Shell Script is a collection of Linux commands to be executed in sequence. The commands are written and stored in an ordinary text file. When the script is to be executed, the shell reads the file contents, interprets and executes them as if they were typed on the shell command prompt.

Why Shell Scripts?

As being set of Linux commands, a shell script can be used to achieve one or more tasks. For example, a script could be written to start a database, shutdown a database, start backup, check whether a specific service is up, and if not, it could start it. So many other tasks can be done using shell scripts. In general, when a specific task is frequently needed, it is wise to write a script that can do it for you. This is much better than having to remember, and type the commands every time the task is needed. Writing shell scripts half-automates the administration tasks, saves the administrator’s time, and minimizes typos effects.

Shell scripts combined with cron jobs:

Crontab job is a Unix/Linux tool that helps system administrators schedule jobs to run at specific times on periodic basis.

Together, crontab jobs and shell scripts can fully-automate many administration and monitoring tasks. For instance, the weekly backup that run every Saturday at midnight, the purging of old log files that takes place on the first of every month, and the system monitoring utility that checks the CPU, memory, file systems, paging space utilizations every 10 minutes, and notifies the admin by email, all considered typical usages for the shell scripts combined with crontab job automation.

What Types of Shell are Available?

• The Bourne Shell, which is the default shell in many UNIX operating systems.
• The Bourne Again Shell (Bash). This is the de facto standard shell for almost all Linux Distributions, like Red Hat, CentOS, Fedora, Ubuntu, etc.
• The Korn Shell (ksh).

In this series, we will use the Bash shell.

Preparing your tools:

Before writing shell scripts (which are collections of Linux commands), we must be first aware of how to use the Linux shell, and how to type and execute commands in and the most important is to have rich knowledge of Linux commands.

Basics of Linux:

Linux Directory Structure Diagram: 

• /bin : All the executable binary programs (file) required during booting, repairing, files required to run into single-user-mode, and other important, basic commands viz., cat, du, df, tar, rpm, wc, history, etc.
• /boot : Holds important files during boot-up process, including Linux Kernel.
• /dev : Contains device files for all the hardware devices on the machine e.g., cdrom, cpu, etc
• /etc : Contains Application’s configuration files, startup, shutdown, start, stop script for every individual program.
• /home : Home directory of the users. Every time a new user is created, a directory in the name of user is created within home directory which contains other directories like Desktop, Downloads, Documents, etc.
• /lib : The Lib directory contains kernel modules and shared library images required to boot the system and run commands in root file system.
• /lost+found : This Directory is installed during installation of Linux, useful for recovering files which may be broken due to unexpected shut-down.
• /media : Temporary mount directory is created for removable devices viz., media/cdrom.
• /mnt : Temporary mount directory for mounting file system.
• /opt : Optional is abbreviated as opt. Contains third party application software. Viz., Java, etc.
• /proc : A virtual and pseudo file-system which contains information about running process with a particularProcess-id aka pid.
• /root : This is the home directory of root user and should never be confused with ‘/‘
• /run : This directory is the only clean solution for early-runtime-dir problem.
• /sbin : Contains binary executable programs, required by System Administrator, for Maintenance. Viz.,iptables, fdisk, ifconfig, swapon, reboot, etc.
• /srv : Service is abbreviated as ‘srv‘. This directory contains server specific and service related files.
• /sys : Modern Linux distributions include a /sys directory as a virtual filesystem, which stores and allows modification of the devices connected to the system.
• /tmp :System’s Temporary Directory, Accessible by users and root. Stores temporary files for user andsystem, till next boot.
• /usr : Contains executable binaries, documentation, source code, libraries for second level program.
• /var : Stands for variable. The contents of this file are expected to grow. This directory contains log, lock,spool, mail and temp files.

 Exploring Important file, their location and their Usability:

Linux is a complex system which requires a more complex and efficient way to start, stop, maintain and reboota system unlike Windows. There is a well-defined configuration files, binaries, man pages, info files, etc. for every process in Linux. 

• /boot/vmlinuz : The Linux Kernel file.
• /dev/hda : Device file for the first IDE HDD (Hard Disk Drive)
• /dev/hdc : Device file for the IDE Cdrom, commonly
• /dev/null : A pseudo device, that don’t exist. Sometime garbage output is redirected to /dev/null, so that it gets lost, forever.
• /etc/bashrc : Contains system defaults and aliases used by bash shell.
• /etc/crontab : A shell scriptto run specified commands on a predefined time Interval.
• /etc/exports : Information of the file system available on network.
• /etc/fstab : Information of Disk Drive and their mount point.
• /etc/group : Information of Security Group.
• /etc/grub.conf : grub bootloader configuration file.
• /etc/init.d : Service startup Script.
• /etc/lilo.conf : lilo bootloader configuration file.
• /etc/hosts : Information of Ip addresses and corresponding host names.
• /etc/hosts.allow : List of hosts allowed to access services on the local machine.
• /etc/host.deny : List of hosts denied to access services on the local machine.
• /etc/inittab : INIT process and their interaction at various run level.
• /etc/issue : Allows to edit the pre-login message.
• /etc/modules.conf : Configuration files for system modules.
• /etc/motd : motdstands for Message Of The Day, The Message users gets upon login.
• /etc/mtab : Currently mounted blocks information.
• /etc/passwd : Contains password of system users in a shadow file, a security implementation.
• /etc/printcap : Printer Information
• /etc/profile : Bash shell defaults
• /etc/profile.d : Application script, executed after login.
• /etc/rc.d : Information about run level specific script.
• /etc/rc.d/init.d : Run Level Initialisation Script.
• /etc/resolv.conf : Domain Name Servers (DNS) being used by System.
• /etc/securetty : Terminal List, where root login is possible.
• /etc/skel : Script that populates new user home directory.
• /etc/termcap : An ASCII file that defines the behaviour of Terminal, console and printers.
• /etc/X11 : Configuration files of X-window System.
• /usr/bin : Normal user executable commands.
• /usr/bin/X11 : Binaries of X windows System.
• /usr/include : Contains include files used by ‘c‘ program.
• /usr/share : Shared directories of man files, info files, etc.
• /usr/lib : Library files which are required during program compilation.
• /usr/sbin : Commands for Super User, for System Administration.
• /proc/cpuinfo : CPU Information
• /proc/filesystems : File-system Information being used currently.
• /proc/interrupts : Information about the current interrupts being utilised currently.
• /proc/ioports : Contains all the Input/Output addresses used by devices on the server.
• /proc/meminfo : Memory Usages Information.
• /proc/modules : Currently using kernel module.
• /proc/mount : Mounted File-system Information.
• /proc/stat : Detailed Statistics of the current System.
• /proc/swaps : Swap File Information.
• /version : Linux Version Information.
• /var/log/lastlog : log of last boot process.
• /var/log/messages : log of messages produced by syslog daemon at boot.
• /var/log/wtmp : list login time and duration of each user on the system currently.

Basic Commands:

cd	Change the current directory. With no arguments “cd” changes to the users home directory.
chmod	chmod<specification><filename> – Effect: Change the file permissions.
	Ex: chmod 751 myfile		Effect: change the file permission to rwx for owner, re for group
	Ex: chmod go=+r myfile		Effect: Add read permission for the owner and the group
	character meanings u-user, g-group, o-other, + add permission, – remove, r-read, w-write,x-exe
	Ex: chmod a +rwxmyfile		Effect: Allow all users to read, write or execute myfile
	Ex: chmod go -r myfile		Effect: Remove read permission from the group and others
	chmod +s myfile – Setuid bit on the file which allows the program to run with user or group privileges of the file.
	chmod {a,u,g,o}{+,-}{r,w,x} (filenames) – The syntax of the chmod command.
chown	chown<owner1><filename> Effect: Change ownership of a file to owner1.
	“chown -R owner:owner /filename”
cp	cp<source><destination> Copy a file from one location to another.
find	Ex: find $Home –name readme Print search for readme starting at home and output full path.
	How to find files quickly using the find command:
	Ex: find ~ -name report3 –print
	“~” = Search starting at the home directory and proceed through all its subdirectories
	“-name report3” = Search for a file named report3
	“-print” = Output the full path to that file
ln	Make links between files.
	“ln -sf /usr/jdk/instances/jdk1.7.0_25/bin/java /usr/bin/”
locate	File locating program that uses the slocate database.
	“locate filename”
ls	List files. Option -a, lists all, see man page “man ls”
	Ex: “ls Docum Projects/Linux” – The contents of the directories Docum and Projects/Linux are listed.
	To list the contents of every subdirectory using the ls command:
	1. Change to your home directory.
	2. Type: ls -R
mkdir	Make a directory.
	“mkdir /name”
mv	Move or rename a file. Syntax: mv <source><destination> Ex: mv filename directoryname/newfilename
pwd	Print or list the working directory with full path (present working directory).
rm	Ex: “rm .*” – Effect: Delete system files (Remove files) –i is interactive option.
rmdir	rmdir<directory> – Remove a directory. The directory must be empty.
stat(1u)	Used to print out inode information on a file.
touch	Change file timestamps to the current time. Make the file if it doesn’t exist.
whatis	Search the whatis database for complete words.
wheris	Locate the binary, source and man page files for a command.
which	Show full path of commands where given commands reside.

 

ed	Editor
emacs	Full screen editor.
gitview	A hexadecimal or ASC file viewer.
head	head linuxdoc.txt – Look at the first 10 lines of linuxdoc.txt.
jed	Editor
joe	Editor
less	q-mandatory to exit, Used to view files.
more	b-back q-quit h-help, Used to view files.
pico	Simple text editor.
tail	tail linuxdoc.txt – Look at the last 10 lines of linuxdoc.txt.
vi	Editor with a command mode and text mode. Starts in command mode.

File compression, backing up and restoring:

ar	Create modify and extract from archives.
bunzip2	Newer file decompression program.
bzcat	Decompress files to stdout.
bzip2	Newer file compression program.
bzip2recover	Recovers data from damaged bzip2 files.
compress	Compress data.
cpio	Can store files on tapes. to/from archives.
dump	Reads the filesystem directly.
gunzip	unzip <file> – unzip a gz file.
gzexe	Compress executable files in place.
gzip	gzip<file> – zip a file to a gz file.
mt	Control magnetic tape drive operation.
tar	Can store files on tapes.
	Usage: tar cvf<destination><files/directories> – Archive copy groups of files
	Ex: tar /dev/fdo temp Effect: Copy temp to drive A:
uncompress	Expand data.
unzip	unzip <file> – unzip a zip file. Files ending in “.gz” or “.zip” are compressed.
zcat	Used to restore compressed files.
zcmp	Compare compressed files.
zdiff	Compare compressed files.
zforce	Force a .gz extension on all gzip files.
zgrep	Search possibly compressed files for a regular expression.
zmore	File filter for crt viewing of compressed text.
znew	Recompress .z files to .gz files.
zip	zip <file> – make a zip file.

Extra control and piping for files and other outputs:

basename	Strip directory and suffix information from filenames.
cat	Ex: cat < filename — Effect: put keyboard input into the file. CTRL-D to exit (end).
cmp	Compare two files.
colrm	Remove columns from a file.
column	Columnate lists.
comm	Ex: comm file1 file2 — Effect compare the contents of file1 and file2 produces 3 columns of output. Lines in the first file, lines in second file, lines in both files.
csplit	Split a file into sections determined by context lines.
cut	Remove sections from each line of files.
diff	Show the differences between files. Ex: diff file1 file2
diff3	Find differences between 3 files.
dirname	Strip the non-directory suffix from a filename.
echo	Display a line of text.
egrep	Similar to grep -E, compatible with UNIX egrep.
expand	Convert tabs to spaces.
expr	Evaluate expressions.
FALSE	Do nothing. Exit with a status indicating failure.
fgrep	Same as grep -F.
fold	Wrap each input line to fit in specified width.
join	Join lines of two files in a common field.
grep	grep pattern filename.
	Ex: grep ” R ” — Effect: Search for R with a space on each side
	Ex: ls –a |grep R — Effect: List all files with an R in them or their info listing.
hexdump	asc, decimal, hex, octal dump.
logname	Print user’s login name.
look	Display lines beginning with a given string.
mkfifo	Create named pipes with the given names.
nl	Write each file to standard output with line numbers added.
od	Dump files in octal and other formats.
patch	Apply a diff file to an original.
paste	Combines from 2 or more files. Ex: paste file1 file 2
printf	Print and format data.
rev	Reverses lines in a file.
script	Make a typescript of a terminal session.
sdiff	Find differences between 2 files and merge interactively.
sed	A stream editor. Used to perform transformations on an input stream.
sleep	Delay for a specified amount ot time.
sort	Sort a file alphabetically.
split	Split a file into pieces.
strings	Print the strings of printable characters in files.
tac	Concatenate and print files in reverse.
tee	Read from standard input and write to standard output and files.
tr	Translate or delete characters.
uniq	Remove duplicate lines from a sorted file.
wc	Count lines, words, characters in a file. Ex: wc filename.

Bash Scripts:

Bash scripts are primarily written as a text file of commands that can be executed. They can end in the .sh extension or no extension at all. They can be executed with the Bash command preceding-it— bashmyscript.sh — or by having the file mode set as executable and placing the path to Bash’s executable at the beginning as such:

Let us understand the steps in creating a Shell Script

• Create a file using a vi editor(or any other editor).  Name  script file with extension .sh
• Start the script with #! /bin/sh
• Write some code.
• Save the script file as sh
• For executing the script type bash sh

“#!” is an operator called shebang which directs the script to the interpreter location. So, if we use“#! /bin/sh” the script gets directed to the bourne-shell

Let’s create a small script for better understanding:

#!/bin/bash

Then you can execute that script by giving the path and file name ./myscript.sh

In Bash, comments are created with the pound symbol. For the sake of this post, I will be writing # Output: inline in cases where it will be more helpful.

value="Hello World!"
echo value      ### Output: value
echo $value     ### Output: Hello World!
echo ${value}  ### Output: Hello World!
echo "value"    ### Output: value
echo "$value"   ### Output: Hello World!
echo "${value}" ### Output: Hello World!

 As you can see above, even though the variable value was passed, it wasn’t interpreted as a variable unless we preceded it with the dollar sign. Also notice that there are no spaces in value=”Hello World!”. This is very important in Bash as many things won’t work if the spaces aren’t as they need to be.

Conditional checks in Bash:

 value=0
 if [ $value == 0 ]; then
  echo "Zero"
else
  echo "Not Zero"
fi
#### Output: Zero

 

The if syntax is a bit strange but not too difficult to learn. Note the space inside of each outer edge square bracket for the if condition. This is important or the script won’t work. In Bash, the semicolon is the equivalent of a new-line for your code. You could also place then on the next line instead of using a semicolon.

For a second condition statement, you can use elif for else if. You will still need to follow the same ; then pattern for that.

Writing Functions in Bash is very simple:

cow() {
echo Moo
}
cow   ###### Output: Moo

 In bash, parameters passed from the console or to a function take a dollar number format, where the number indicates which position of the parameter it is.

cow_eat() {
  echo Cow chews $1
}
 cow_eat grass  ######## Output: Cow chews grass

 Bash loads a .bashrc file from your home directory for all its defaults. You can place as many functions as you want in there and they will be available on the command line at any time as if they’re a program on your system.

What are shell variables?

Variables store data in the form of characters and numbers. Similarly, Shell variables are used to store information and they can by the shell only

For example, the following creates a shell variable and then prints it:

variable ="Hello"
echo $variable
#!/bin/sh
echo "what is your name?"
read name
echo "How do you do, $name?"
read remark
echo "I am $remark too!"

get prepared for more stuffs like this in the next blog………..