Unix Training by Dhanabal
Unix Training by Dhanabal
Dhanabal S.
But, it will definitely help you to be a confident sailor in the ocean of Unix no matter what the circumstance is.
Dont be nervous,please
This training will teach you about
90
commands!!!
350 slides!!!!
But dont be nervous remember the utilities of the commands for this presentation will be with you for the rest of your life to provide you the syntax for using them.
Duration 15 Hours ( 3 Days) Schedule 29th Sept-1st Oct,2003 Timings 11:00 am 1 pm 2:00 pm 5 pm
Purpose
Get introduced to Unix Operating System
Operating System
An operating system is a living, breathing software entity. The soul of the computing machine, it is the nervous system that turns electrons and silicon into a personality. It brings life to the computer - from
Mike Gancarz's "The UNIX Philosophy"
The OS sits between users and hardware providing translation services. It speaks the language of the hardware to perform basic tasks such as the definition of memory or the allocation of disk space using the hardware.
Hardware
In UNIX, the operating system is broken into three pieces: the kernel, the shell, and the built-in utilities. The kernel is responsible for low level hardware communication, the shell provides human users with a user-friendly interface, and the built-in utilities provide basic tools for doing work.
Kernel
Heart of The Unix OS. Collection of C programs directly communicating with hardware Part of Unix system loaded into memory when Unix is booted
Manages:1. System resources 2. Allocates time between user and processes 3. Decides process priorities
Exit
Shell
Human interface point for Unix Program layer provides an environment for the user to enter commands to get desired results. Korn Shell, Bourne Shell, C Shell are various shells used by Unix users
Exit
getty calls program login which scans file /etc/passwd to match username and password
After validation, control passes to session startup program /bin/sh , session startup program
Program /bin/sh reads file /etc/profile and .profile and sets up system wide and user specific environment.
User gets a shell prompt
Exit
Understanding Commands
Controlling Environment
Process
Handling Files
System Administration
Filters
Communication
To get work done by Unix, there is no way but to execute : commands. Commands can be typed in from keyboard or taken from a file
Independent Commands
Do not require any input for execution
$pwd /home/ems2000
Dependent Commands
Require input for execution
$type ls
<arguments> : May be a string or the name of file(s) on which the command will act upon for desired output.
Example : $grep -l ems2000 *.sh The above command from the OS prompt searches for the string ems2000 in all the files with extension : .sh and shows the name of the files containing it.
/dev/pts/tb
First, pwd command is executed which displays the output : /home/ems2000/queue Then , tty command shows the terminal no: /dev/pts/tb Any number of commands can be specified at the OS prompt for execution separated by a ;
tty terminal information script get the whole job uname- machine info bc Unix Calculator
Example
date +%b
date +%c date +%C date +%d date +%e date +%H date +%I
date +%m
date +%M date +%n date +%p
date +%t
date +%u date +%w date +%V date +%x date +%X date +%y date +%Y date +%Z
Tab character
Weekday as 1 digit number[1-7 for Monday-Sunday] Weekday as 1 digit number [ 0-6 for SundaySaturday] Week number of the year Current date as dd/mm/yy Current time in hh:mm:ss Two digit year 4 digit year Time zone name
Examples of Date
$ cat calendar
$who sqian ems2000 $who -Hu NAME sqian ems2000 $who am i ems2000 pts/te Sep 10 01:41 LINE pts/td pts/te TIME IDLE PID COMMENTS Sep 9 18:27 7:32 28935 135.148.207.175 Sep 10 01:41 0:02 29362 129.42.68.104 pts/td pts/te Sep 9 18:27 Sep 10 01:41
$uptime -u 2:17am up 122 days, 11:03, 2 users, load average: 0.41, 0.41, 0.41 $uptime -h sqian pts/td 6:27pm 7:39 1:41am rlogin htstbw00 -ksh ems2000 pts/te $uptime -l 2:18am up 122 days, 11:04, 2 users, load average: 0.40, 0.41, 0.41
Scale:By default, bc performs truncated division.One have to set scale to the number of digits of precision before performing any division. $bc scale=2 10/4 2.50 If answer to division is greater than the value as dictated by the scale variable, then the value dictated by the scale is ignored and the real value is shown.
^d
$
ibase and obase:By default, the input and output are interpreted as decimal values.But, if the demand required input and/or output in different number system(binary, hexadecimal), variables ibase and obase are set. To convert binary input to decimal output:ibase=2 To convert decimal input(default) into binary output:obase=2 For hexadecimal systems, value 16 is used in ibase/obase variable.
Handling Variables:Variables can be used in bc mode and values can be assigned to them.
$bc X=12 ; y=19 Z=x+y Z
31
Length of a Number
Syntax : length ( x ) Example : $bc
length ( 1234.5678 )
8
Trigonometric Functions
To use trigonometric functions of bc, one have to include math library. For that , issue the following command from Os prompt:$ bc l
Various trigonometric functions available with bc are:s(x) c(x) e(x) l(x) sine cosine exponential log
String handling
To find length of a string:expr <string> : .* Example : $ expr Unix : .* 4
To extract a substring from a string:$ expr Subhendu : (\..\) he # Shows 4th to 5th character
To locate first position of a character in a string:$expr Subhendu : [^d]*h 4 # Shows h is at 4th position
factor
Finds out factor of the integer provided
$factor 15
$factor 18
15
3 5
18
2 3 3 $
primes
Shows all prime numbers between integers <lower value> and <upper value>. If upper value is not provided, it is considered to be 2,147,483,647.
$primes 0 10 2 3 5 7 $
units
Converts quantities expressed in various standard scales to equivalents in other scales. Acts interactively as follows:System Prompt You have: You want: User Response inch cm their
The system responds with two factors; one used if multiplying (preceded by *), the other if dividing (preceded by /): * 2.540000e+00 / 3.937008e-01
tput
Controls screen display Options
tput clear tput cup <r> <c> tput bold tput blink tput rev tput cols tput bel
Significance
Clears the screen Moves cursor to row <r> and column <c> Bold display Blink display Reverse display Shows number of columns in the screen Echo bell character
tput lines
tput smso tput rmso
Examples
time
Syntax : time <command>
Times a command.
Command <command> is executed and time prints:Elapsed time during the command Time spent in the system Time spent executing the command
bounce_off.ksh:# Developed by
drawbox.ksh:# Developed by header:# heading:# real user 6.6 0.8 Developer Developer
: SUBHENDU MAJUMDAR
: SUBHENDU MAJUMDAR
sys
0.7
Handling Files
This section will introduce you with
Architecture Types of Files Inode File System pwd-Current Directory cd Change Directory ls List contents of a dir Cat Create,View, append Files VI Visual Editor more display files pg View files cp,mv Copy and rename files rm Remove Files wc Count word,line and characters file know file type chmod Change permissions chown Change owner chgrp Change group touch Change time stamp of a file ln Link a file to other
Handling Filescontd
mkdir Create Directory rmdir Remove Directory cmp compare two files comm Compare two files sdiff find differences between two files dircmp Compare Directories lp Print a file in a printer
Partition
Partition
Directory
Directory
Directory Directory
Directory
Types of Files
Normal files
Can be text/binary files.
Directory files
Contains no external data, but details of files and sub-directories it contains.
Device files
Printers, tapes, floppy drives, CD ROMs, hard disks, terminals all are considered as files
Inode
Inode is a fixed format structure containing the attributes of the files stored in the file system. Every file has one inode, and a list of such inodes is kept in a disk area not directly accessible by user. Each inode is accessed by inode number, specifying the position of the inode in the list.
Some of the important information that inode contains are:Information Mode Link count Significance Permission mask and type of file No. of links associated with the file
User id
Group Id Size Access time Mod time
ID of the owner
ID of the group No. of bytes in the file Time of last access of the file Time of last modification of the file
Inode time
Unix
bin
dev
usr
tmp
etc
usr2
usr3
bin
/tmp contains temporary files /usr/bin contains additional binary unix commands
/bin contains binary executable files /dev contains device related files /usr is the home directory for all users /usr/usr1 is the home dir. for user : usr1
pwd
Shows the directory where the user is currently in
Syntax : pwd
Example
$pwd /home/ems2000
cd
Navigates from the current directory to another directory
Significance
Moves to directory : scripts under current working directory Change to directory program residing in the current directory's parent directory Change to the directory whose absolute pathname is : /usr/fin/subhendu/ manfiles Move to home directory of the user
cd /usr/fin/subhendu/ manfiles
cd cd ~
Commands
cd .. cd ~/subhendu
Significance
Move to the parent directory of the current working directory Move to folder : subhendu under the home directory for the user
cdpath
This is an environmental variable which specifies the list of directories to be searched when an user issues a cd command.
$CDPATH=.:..:$HOME
This means, when a cd command is issued, search for the new directory first in current working directory(.) If not found, move to the parent directory of the current directory and search there. If, still, not found, search for the directory under users home directory
ls
Shows the contents of a directory/existence of specific files with their attributes
Significance
Shows the name of all the files and directories under the current directory, excluding those starting with . Lists all files including those starting with . Multi columnar output Shows all files and recursive listing of all files in subdirectories Long listing showing seven attributes of a file Marks executables with * and directories with /
Commands
ls t ls ut ls r ls ltr
Significance
Sorts files by modification time the file modified most recently comes at the top Sorts files by access time Sorts file in reverse order Shows long listing of files with their attributes, sorted in reverse order by access time(most recently edited file comes last in the list) Shows inode number of a file Shows the name of all files with .ksh at the end of their name Shows the files with name starting with vowels Lists all files starting with d and ending with .sh in their name Lists all files with first letter as d and third letter as l Shows the files with names not starting with vowels
cat
Creates , shows, concatenates,copies files
Commands
cat >file1 cat >>file1 cat file1 cat file1>file2 cat file1 file2 > file3 cat file1 >>file2
Significance
Creates file file1 where a user enters text and presses <Ctrl-D> to end text editing Append lines to existing content of file : file1 and is ended when <Ctrl-D> is pressed Shows the contents of the file: file1 Copies the contents of file : file1 into new or existing file : file2 Concatenates the content of file1 and file2 and places it into new or existing file file3 Appends the contents of file1 after the last line of file2. If file2 does not exist, new file is created
Commands
cat n file1 cat b file1 cat e file1 cat r file1 cat t file1 cat s file1 file2 >file3
Significance
Displays the contents of file : file1 with line number Displays the contents of file : file1 with line number for all lines excepting the blank lines Prints $ at the end of each line Replaces multiple consecutive empty lines with one empty line Prints tab character as ^I and form feed character as ^L Suppresses error and does the job. If file file2 does not exist, the command will copy the contents of file1 into file3
Examples
Examplescontinued
Examplescontinued
VI
First Unix Full screen Editor
First full screen editor, developed by William(Bill) Joy, a graduate student from University of California, Berkley.
Divides Unix users into two camps: Those who hate vi Those who love vi Haters say that it is the worst thing ever happened in the Unix world. Lovers are totally biased towards it and would go any length supporting its cause.
Modes of Operation
Ex mode
Environmental variables
Append mode
Command mode
view
Mode of Operation
Command mode : Default mode when a file is opened using
vi. All the keys pressed by the user are interpreted as user commands
Command Mode
R,R,I,I,c,C,o, O,s,S,a,A
Esc
Enter
Append Mode
Ex Mode
Step 1 : Create a new file by typing the following command from the OS Prompt : vi newfile
vi clears the screen and display a window. The _ on the top line indicates that the cursor is waiting for commands
Step 2 : Press i to enter into Append mode. Add text to the file
Step 3 : Press :. The cursor takes to the ex mode at the command line. Enter wq and press enter.
Append Mode
Inserting Texts Cmd
i I a A
Significance
Appends text from the left of the current cursor position Appends text at the start of the current line. Appends text from the right of the current cursor position Appends text at the end of the current line.
o
O
Significance
Replaces <n> characters from current cursor posn. with inserted text Replaces text from cursor to right Replaces <n> characters from cursor with entered text Replaces <n> lines from the current cursor line with entered text Changes from cursor to beginning of line with the text entered
c$
C
<n>cw Changes <n> words from the current cursor position with text entered <n>cc cG Replaces <n> lines from the current cursor line with entered text Changes from current cursor position to end of the file with entered text.
Command Mode
Saving work in a file and quit
Cmd
ZZ
Significance
Saves the work done in the file and quits editing by vi editor
Deleting texts/lines
Cmd
<n>x <n>dd or <n>D
Significance
Deletes <n> characters from current cursor position Deletes <n> lines counting from current cursor line to below
d$
dG d<n>G df<char> d/<pattern> d?<pattern>
Significance
Yank <n> lines starting from current line onwards into undo buffer Yank <n> words starting from current cursor position onwards into undo buffer Yank from current cursor position to end of the line in undo buffer Yank from current cursor position to end of the file in undo buffer Yank <n> lines starting from current line onwards into buffer named a Paste the contents of undo buffer( as a result of deleting or yanking) after the cursor position
P
ap bd
Paste the contents of undo buffer( as a result of deleting or yanking) before the cursor position
Paste the contents of buffer a after the cursor position Delete text into named buffer b
Cmd
<n>h <n>l <n>b <n>w
Significance
Moves cursor left to nth previous character w.r.t. the current cursor position Moves cursor right to nth next character w.r.t. the current cursor position Moves cursor left to start of nth previous word w.r.t the current cursor position. Punctuation marks are taken into account. Moves cursor right to start of nth next word w.r.t the current cursor position. Punctuation marks are taken into account.
<n>e
f<ch> F<ch> t<ch> T<ch> ;
Moves cursor right to end of nth next word w.r.t the current cursor position. Punctuation marks are taken into account.
Move the character to the next character <ch> on same line Move the character to the prv. character <ch> on same line Move the character to one column before the next character <ch> on same line Move the character to one column after the next character <ch> on same line Repeats search in the same direction along which the prv. Search was made using f/t
Cmd
,
Significance
Repeats search in the opposite direction along which the prv. Search was made using f/t Moves the cursor to specified column <n> Moves to 1st character of the current line Moves to last character of current line Moves to 1st non-space character of the line Moves cursor left to start of nth previous word w.r.t the current cursor position.Punctuation marks are ignored Moves cursor right to start of nth next word w.r.t the current cursor position. Punctuation marks are ignored
<n>E
Moves cursor right to end of nth next word w.r.t the current cursor position. Punctuation marks are ignored
Significance
Move the cursor down to the <n>th next line in the same column Move the cursor up to the <n>th prv line in the same column Moves the cursor to the top line of the screen Moves the cursor to the last line of the screen
M
<n>G + -
Significance
Makes the current line the last line of the screenand redraws the screen Makes the current line the first line of the screenand redraws the screen Makes the current line the middle line of the screenand redraws the screen
Ctrl-l
/pattern/z-
Cmd
<n>^f
Significance
Move forward by <n> screens
<n>^b
<n>^d <n>^u <n>^e
<n>^y
Significance
Searches for specified <pattern> forward. IF end of file is reached, search wraps around.
?pattern
n N /pattern/+<n> /pattern/-<n>
Significance
Joins current line and <n> lines below it together to form a single line
Significance
Undo last change
Significance
Marks position of the file with mark <char>
Moves to portion of the file marked with <char> Toggle to most recently marked location
Significance
Paste the content of <n>th last delete ( n<=9) Till the last change is found
Significance
Sort from current line to line no. <n>
Translates all the characters from current line to line <n> to uppercase
Translates all the characters of current line to uppercase
Significance
Inserts <ch> character <n> number of times in input mode at a stretch
Miscellaneous in command mode Cmd ~ Significance Change the character under cursor from lowercase to uppercase and vice versa
.
<n>. << >>
Significance
Recover the file <filename> as much as possible after system crash and open it
vi R <filename>
vi +<n> <filename> vi + <filename> vi w<n> <filename> vi +/<pattern> <filename> vi x <filename>
Ex Mode
Saving work in a file Cmd :w Significance Save the changes made to the file
:w <filename>
Same as Save As.. in windows. Saves the contents to the specified file <filename> . If it does not exist previously, a new file is created
Save the changes to file <filename>, if the file already exists Append the contents of the opened file after the last line of the file <file1>. File <file1> should exist previously Copies the contents of lines <n1> to <n2> into a new file <newfile> Moves the contents of lines <n1> to <n2> into an existing file <newfile> , overwriting its previous contents
Cmd
:.,.+<n>w <newfile> :.,.+<n>w >> <nextfile> :q :q!
Significance
Appends from current line to <n> number of lines below it into file <newfile>
Appends from current line to <n> number of lines below it after the last line of the file <nextfile> Quits the file editing in vi, provided no unsaved change remains Quits vi neglecting all the unsaved changes made to the file
:wq or :x
Save the unsaved changes in the opened file and quit vi editor
Cmd
:sh
Significance
Temporarily allows the user to come out of the vi file and use the shell. After the job of the user is done and command : exit is triggered from OS prompt, control returns to vi editor again
Significance
Custor moves to line number <n>
Significance
All lines where matching pattern is found Current line Refers from line <n1> to <n2> Last line First to last line From current cursor line to <n> number of lines downwards
Significance
Substitutes 1st occurrence of string ex with vi by showing them and asking for confirmation. When each string will be shown with pause in cursor, press y for substitution Replaces amaze , where available as a full word, with delight. Note, any word like amazed will not be replaced
:g/subhendu/s/majumdar/ mazumder/g
:g/.\ {9\ }9/s/0/*/g :g/^$/d :g!/complete/s/$/To be done/
Replaces every occurrence of string majumdar with mazumder on all lines containing the pattern subhendu
Replaces 0 with * at all lines having 9 after 9th position Delete all blank lines Append the string To be done at the end of all lines not containing the string complete
Example
:g/vi/s/^/editor/ :%s/$/ : see my note/g :g/^.$/d :g/^..o/d :%s/$//g
Significance Append the string editor at the first of all lines containing the string vi Appends the string : see my note at the end of all lines Deletes all lines containing 4 letters Deletes all lines with o as 3rd character Delete the last three character of every line
Reading below the current line in the vi editor Command :r <nextfile> :r! <command> Significance Reads the contents of the file <nextfile> below current line Places the output of the command <command> below the current line
Significance
Stops editing the current file; leaves the current file and starts editing file <nextfile>; provided there are no unsaved changes in the current file Edits file <nextfile> abandoning all the changes done to the current file Loads last saved version of current file Edits next file mentioned in the vi queue Edits first file in the command line Edit starts at line <n> of file <nextfile>
Significance
When the user writes the string <short_string> in input mode, the <long_string> is written
Mapping
Command
:map g :w^M
Significance
Pressing g, one wants to save the file(:w is for saving, and ^M is for pressing Enter key. While writing it in the command line, write ^V^M) When you position your cursor to any character in a line and press z , the line will be broken from that point and two lines will be formed. The control will remain in command mode (^[ represents <Escape> key) Pressing z in command mode saves the file and executes it in one shot
:map z i^M^[
:map z :w^M:!%^M
Miscellaneous Command :! <command> Significance Executes the command <command> remaining in vi editor
:f
^g
To set an environment variable to customize vi, the following syntax needs to be followed::set <env.variable> [= <value>]
Environment variables
autoindent(ai)
Significance
Newly inserted lines of text are indented to the same distance from left margin as the preceding line. Opposite of this option is noautoindent(noai) Automatically saves the unsaved changes in a file before opening the next file with :n or using a shell command with :! <command>. The opposite to this option is noautowrite(noaw) Sounds the bell when error occurs. Opposite is noeb Allows an .exrc file in the current directory to override the .exrc file in users home directory. Opposite is noex Displays special characters in the screen: tabs are shown as ^I, end of line are marked with $. Opposite is nolist System messages allowed when vi is running. Opposite is nomesg Displays line numbers. Opposite is nonu
autowrite(aw)
Environment variables
report=<val> scroll=<val>
Significance
When any operation affects more lines than this settings, message is displayed Number of screen lines to scroll
shiftwidth(sw)=<val>
showmatch(sm) showmode tabstop=<val> ignorecase(ic) wrapmargin(wrm)=<val>
view
Syntax : view <filename> Description :Shows the file <filename> in vi mode. File remains
read-only. No changes done to the file cannot be saved.
more
Syntax : more [ -n <number> ] [ -<number>] [ -d] [-i] [-e] [ -c] [-f] [-s] [ +/pattern] [filename]
Filter for examining continuous text, one screenful at a time. It pauses after each screenful,printing the filename at the bottom of the screen. To display one or more next lines, press <Return> To display another screenful, press <Space>
Significance
Sets the no of lines in the display window to that <number>. Same as above Prompts user with message :- Press space to continue, q to quit, h for help at the end of each screenful of display Performs case independent pattern matching Draw each page by starting at the top of screen, and erase each line before drawing on it Squeeze multiple blank lines from the file , showing only one blank line
+/pattern
Start listing such that the current position isset to two lines above the line matching the regular expression pattern.
Options
f b q /pattern
Significance
Scrolls forward one screen Scrolls backward one screen Quit Searches pattern <pattern> forward
?pattern
v n or N <n>j <n>G G . :n
Moves to line number <n> Goes to last line of the file Repeats prv. command Moves to next file specified in the command line
Options
:p
i<space>
Significance
Moves to previous file specified in the command line
Scrolls forward by i number of screens
i<return>
<i>^d/ <I> <i>^u /<i>u <n>k/ <n>^y <n>z <n>f / <n>^f <n>b / <n>^b ^g
Options
h
!<command>
Significance
Display a descriptive list of all the more commands
Invoke a shell with a command <command>
:e <newfile>
<I>:n r / ^l . m<char>
Leaves the current file and starts viewing a new file <newfile>
Examines the <I>th next file specified in the command line Refreshes the screen Repeats the prv command Marks the current position with letter <char>
<char>
pg
Displays the content of a file one page at a time
Significance
Clear the screen at the end of each page of display and start the display at the top of the screen Continues to the next file after the end of one file, if more than one files are specified at the command line Truncate lines longer than the width of the screen display Display the <string> at the pg command prompt.Default is : . If string is %d the pageno is displayed atr the prompt Highlights all messages and prompts issued by the pg command Start the display from line no <n> of the file specified
Flag
-<n> +/pattern/
Significance
Sets the size of the display screen to <n> number of lines Search for pattern <pattern> in the file and start the display at that line
Keystrokes in pg mode
Option
-<n> +<n> l
Significance
Go backward by <n> number of pages Go forward by <n> number of pages Go forward by one line
<n>l
+<n>l -<n>l d
-d
^l
Go backward by screen
Redraws the screen
Option
$
Significance
Move to last page of the file
<n>/<pattern>
Searches forward for the pattern <pattern> in the file from beginning of the next page. If a number <n> is specified, pg searches for the specified occurrence number <n> of the <pattern>
Starts showing the <n>th previous file.
<n>p
n
q
cp
Copies one file to another file/one directory to another directory/files into directories
-r
Recursive copying
Examples
Example
cp file1 file2
Significance
Copies the contents of file1 into new or existing file file2
mv
Syntax :mv [-flag] [ old files/directories] [ new files/directories]
Moves/renames:A file to new or existing file One/more files to existing directory One or more directories to a new or existing directory. If the access permission of the destination directory or existing destination file forbids writing, mv command asks for overwriting the file Flag
-f
-i
Significance
Performs move operation without prompting for permission
Interactive moving
rm
Syntax :rm [-f|-i] [ -r|-R] [files|directories]
Removes the entries for one/more files from a directory.
Significance
Forceful deletion Interactive deletion Recursively delete the entire contents of the cirectory before removing the directory itself
Examples
Example mv *.txt ../testdir rm *.temp rm r backupdir Significance Moves all the files with .txt extension under the directory testdir residing under the parent directory of the current directory Remove all files with extension : .tmp under the current directory Remove the directory backupdir with all its contents
wc
Syntax :wc [-c|-l|-w] <filename(s)>
Counts the number of words/bytes/characters/lines in a file Flag
-w -l -c
Significance
Counts the total no. of words Counts the total no. of lines Counts the total no. of characters
Examples
$ wc c file1 32 file1 $ wc l file1 2 file1 $ wc w file1 8 file1
file
Syntax : file [-f ffile] [-h] file ...
Determines the file type of a file or list of files.
Performs a series of tests on each file in an attempt to classify it. If file appears to be an ASCII file, file examines the first 512bytes and tries to guess its language.
File /etc/magic is ued to identify files that have some sort of magic number, that is, any file containing a numeric or string constant that indicates its type. Commentary at the beginning of /etc/magic explains the format.
chmod
Syntax : chmod [file permission] <files|directories>
A file can have three type of permissions:-
A file can be accessed by:User : Person creating the file. He grants all the authorizations to the file Group : Group user for the file
Others: All other users not belonging to the group or are not the creator.
Significance
Read permission
2
1 6(=4+2) 7(=4+2+1)
Write permission
Execute permission Read and write permission Read,write and execute permission
5(=4+1)
Example of granting numeric authorization Syntax: chmod <val1><val2><val3> <filename(s)|directory name(s)> val1 is for users val2 is for group val3 is for others Any permission on a directory percolate down to the files and subdirectories under it. Example
chmod 744 file1 chmod 776 chmod 777 file1
Significance
Grant all permissions to : User, and read permission to group and others Grant all permission to user and group, read and write permission to others Grant all permission to all
Significance
Read permission Write permission Execute permission
Value
u
g o a
Significance
User creating the file
Other users in the same group of the creator Any other users All(creator, other users in the same group, and other users)
Significance
Assign read,write and execute permission to user, but only read permission to group and others Revoke execute permission from others Assign execute permission to everybody Traverse the directory subtree under directory /home/ems2000 making all regular files readable by user and group only, revoke read permission from others and grant execute permission to all
Chown chgrp
and
chown changes owner of the file and chgrp changes group of each file specified. To change user or groyp, one must own the file and have the CHOWN privilege granted to him by the system administrator
Options
-h
-R
Significance
Change owner/group of a symbolic link
Recursively change the owner and group of all the files and sub-directories under the directory named
Example
chown ems2000 auto.profile chgrp ems2000 auto.login chown R ems2000: users shell_scripts
Significance
User ems2000 becomes the owner of the file auto.profile User ems2000 becomes the group user to access the file auto.login The command searches the directory : shell_scripts and changes each file in that directory to owner : ems2000 and group : users
touch
Updates access time/modification time of file(s)
Flag
-a
-m -r ref-file -t <time>
Significance
Change the access time of the file to the time specified/ if no time is specified, use the current time
Change the modification time of the file to the time specified/ if no time is specified, use the current time. Use the corresponding time of file ref-file to change the modification/access time of the file Use the specified time <time> instead of current time. The time format is : <YYYY><MM><DD><hh><mm>.<ss>
Example
touch a wot.ksh
Significance
Changes the access time of wot.ksh with the current date and time (Changes can be perceived if the command : ls ut is
fired in the directory containing the file : the file wot.ksh will come at the top of the list)
touch m wot.ksh
Changes the modification time of wot.ksh with the current date and time (Changes can be perceived if the command : ls lt is
fired in the directory containing the file : the file wot.ksh will come at the top of the list)
Changes the modification time of file : new.del to Dec 12,2010 time : 23:00 Seeing the attributes of the file new.del will reveal the information:-rw-rw-r-- 1 ems2000 dba 10 Dec 12 2010 new.del
touch -m -r new.del new1 Change the modification time of file : new1 and make it same as the modification time of the file : new.del
ln
Description Link files and directories
Command
ln [-f|-i] file1 newfile
ln [-f|-i] file1 file2 testdir ln [-f|-i] dir1 dir2 testdir
Significance
Links file : file1 to a new or existing file : file1
Creates link for files new1 and new2 under the directory : testdir with two new files/existing files new1 and new2 Creates link for directories dir1 and dir2 under the directory : testdir with two new /existing directories : dir1 and dir2
Flag Significance
-f
-i
mkdir
Creates directories
-m <mode>
Example
Mkdir m755 testdir mkdir p testdir/subhendu
Significance
Directory : testdir is created with permission = 755 If testdir is not created, it is first created. Then, a directory : subhendu is created under the directory testdir
rmdir
Removes the directory entry for each empty directory referred
Significance
Forcefully remove a directory, even though it is not empty
Interactive removal of empty directory Path removal.If, after, removing a directory with more than one pathname component, the parent directory of that directory is empty, rmdir removes the parent directory also. This continues till rmdir encounters non-empty parent directory
cmp
Syntax : cmp [-l|-s] <file1> <file2>
Significance
Print the byte number(decimal) and differing byte(octal) for each difference Prints nothing; return exit codes only
Return codes
0 1
Significance
Files identical Files not identical
Inaccessible/missing arguments
comm
Syntax : comm [-[123]] <file1> <file2>
Significance
Produces a three columnar output:1st column displays lines only in file: filenew 2nd column displays lines only in file:fileold 3rd column displays lines common to both files Suppress display of 1st column
sdiff
Syntax : sdiff [-l|-s] <file1> <file2> Description Side by side file difference program.
$cat file1
Suvendu Subhasish Dibyendu Diptoman Akash
$cat file2
Subhendu Subhasish Arka Dibyendu diptoman
Flag
-l
Significance
Only print on the left hand side where columns are identical
-s
dircmp
Syntax : dircmp [ -d|-s|-wn] <directory1> <directory2> Description Examines two directories and generates various
tabulated info about the contents of the directories.
Flag
-d
Significance
Also compares the contents of the files with same name in both directories and output a list telling what must be done to bring them into agreement Suppresses messages about identical files
-s
dir1 dir2
./nsf
Aug 13 00:42 2003 Comparison of dir1 dir2 Page 1 directory different same same . ./abc ./def ./samefile1
lp
Syntax : lp -d<printer> [-m] [-n<copies>] [-t<title>] <file(s) Description Prints one/more files to a specified printer
Flag
-d<printer> -m -n<copies>
Significance
Specify the printer <printer> where the print request is to be directed Notify the requesting user at successful completion of the print request by mail Specifies number of copies to print
-t<title>
lpstat
Syntax : lpstat [ -p<printer>] [-t] [-u<username>] [-v
<printername>]
Significance
Shows all printing jobs queued at printer <printer> Displays detailed status information about the print from all users Displays the status of print requests triggered by user <username> Displays a list for the specified printername
cancel
Description
Used to cancel a specific print request or cancel all queued requests to a specific printer queue. Any ordinary user can cancel only those jobs triggered by his own userid
Example:-
$ cancel 734
$ cancel lipi
pr
Description Prepares a file for printing by adding suitable
headers, footers and formatted texts. Example
pr file1
Significance
Adds 5 lines of margin at the top and 5 lines at the bottom of file : file1. Header shows date and time of last modification of the file along with filename and pageno Changes default page size from 66 to 72 Show file file1 from 10th page Prints a file in three columns Suppresses page headers and footers for file : file1 Form feed is used for a new page instead of a sequence of line feed characters
pr w 80 file1
Example
pr P file1 pr o5 file1 pr d file1 pr h List of files file1
Significance
Pauses after each page is displayed in the terminal Indent each line by 5 columns Generate output with double specing Print List of files instead of the filename as header on each page.
Compressing files
Compress
Compress <filename(s)>
Reduces the size of the named files using adaptive Lempel-zev coding. If reduction is possible, each file is replaced by a new file with same name suffixed sith .Z. Original ownership, modes, access and modification times are preserved.To uncompress a file, issue the command : uncompress <filename>.Z . To view a compressed file, issue : zcat <filename>.Z
Gzip
gzip <filename(s)>
Reduces the size of the named files . If reduction is possible, each file is replaced by a new file with same name suffixed sith .gz. Original ownership, modes, access and modification times are preserved. To uncompress a file, issue the command : gunzip <filename>.gz . To view a compressed file, issue : gzcat <filename>.Z
Filters
Filters
By end of this section , you will be able know
head First few lines of a file tail Last few lines of a file cut View columns/fields paste join lines of files split Break long lines fmt Formats texts fold Folds long lines Sort Sorts file contents tr Translates file contents nl Shows line numbers Spell Catch spelling mistakes find Find files grep Search Pattern in a file Sed Display specific lines awk Reporting Tool
head
Gives first few lines of a file
Significance
Shows first 5 lines of file : newfile Same as above
head -l -n 5 newfile
head -c n 14 newfile
Same as above
Shows first 14 characters of newfile
tail
Gives last few lines/characters of a file
Syntax :
Flag
-n <no>
tail [-f] [-c number] [file] tail [-f] [-n number] [file]
Significance
Shows last <no> lines of the file if <no> is specified with no sign or ve sign. If specified with a + sign, it shows from <no>th line onwards till end of the file Shows last <no> bytes of the file if <no> is specified with no sign or ve sign. If specified with a + sign, it shows from <no>th byte onwards till end of the file Works in constant pilot mode . Used for variable sized files those are growing constantly.
-c <no>
-f
Example
tail 5 newfile tail n 5 newfile tail +5 newfile tail n +5 newfile tail -c 5 newfile tail c +5 newfile tail fn 3 newfile
Significance
Shows last 5 lines of newfile Same as above Shows from 5th line to end of the file Same as above Shows last 5 bytes of a file Shows from 5th byte till end of the file Constantly displays the last three lines of file : newfile and leave the tail in follow mode. The $ prompt does not return even after the work is over. One have to abort the process to exit to the shell.
cut
Syntax : cut [-c list] [file]
cut [-f list] [-d <char>] [file]
Extracts selected fields/characters on each line of a file
Example
cut c 6-22,24-32 file1 cut c 22 newfile cut c 6, 22,24- file1
Significance
Extracts texts from column 6 to 22 and 24 to 32 and displays in the terminal in two columns Extracts text from column 1 to 22 from the file : newfile and displays it in the terminal Extracts texts from column 1 to 6, column 22 and from column 24 to end of the line and display it in the terminal
Extracts field contents of 2nd and 3rd fields and display in the terminal. Use | as a field separator while extracting fields
Using | as field separator, it extracts field contents of 1st to 5th fields and display in the terminal.
paste
Syntax :
paste file1 file2 ... paste -d list file1 file2 ... paste -s [-d list] file1 file2 ... Merges same lines of several files or subsequent lines of same file
Example
paste file1 file2
Significance
Display in the screen the join of line 1 of file1 with line 1 of file 2 with a space between them, line 2 of file1 with line 2 of file2 with a space delimiter and so on.
Same as above with | as delimiter between the fields of both files on each line.
Joins all the lines of file1 into a single line and show it in the terminal
Significance
Displays in the screen a single line which is :Join of all the lines in file1 + Join of all the lines in file2 4th into another line and so on.
paste s -d t\n\ file1 Combines 1st and 2nd lines into a single line, 3rd and
split
Syntax : split [-<no. of lines to be put in each file>] [<initials>]
Splits up a file into equiline smaller lines.
Large files are sometime difficult to edit with an editor. The split command breaks up a larger file into several equi line smaller files, each containing a default of 100 lines.
It creates a group of files xaa,xabtill xaz and then again from xba,xbb.. till xbz. Total 26x 26 = 676 files can be created in this way.
Example:
File: newfile consists of 100 lines. $ split 20 newfile 5 files nfa,nfb,nfc,nfd and nfe will be prepared each containing 20 lines from newfile
fmt
Syntax : fmt [-s] [-w <width>] [file]
Simple text formatter that fills and join lines /split lines to produce output lines upto the number of characters specified in the w option(default is 72). It also counts the spaces between words in a lineand also considers a space between joining of two lines. The s option split lines only. It does not join short lines to form longer ones.
$ >cat file1 Today, we have a meeting. It will start at 6 pm. Bye $ >fmt -w10 file1 Today, we have a meeting. It will start at 6 pm. Bye
fold
Syntax :
fold [-b|-s] [-w <width>] <file.> Folds long lines
Flag Significance
-b -s Width in bytes for counting Break the line on the last blank character found before the specified number of column position specified in the w<width> option(default : 80)
$ cat file1 Today, we have a meeting. It will start at 6 pm. Bye $ fmt -s -w15 file1 Today, we have a meeting. It will start at 6 pm. Bye
sort
Sorts the contents of a file. Starts with 1st character on each line and proceeds to the next character only when the prv. Character in the two lines are identical.
Can also work on fields. Default field separator is the space.This can also be changed.
Example
sort file1 Sort t | +2 file1 Sort t | r +1 file1 Sort c file1
Significance
Shows the sorted contents of file1 as per character sorting Sort the contents of file1 based on 3rd field Reverse sort on 2nd field Checks whether the file : file1 is already sorted or not. Value of system variable $? Is 0 if it is sorted
sort t | +2 c file1
Example
Sort o sort.lst +3 file1 Sort t | +3 4 file1
Significance
Sort the contents of file : file1 based on 4th field and store the output in a file : sort.lst Start sorting of file1 after 3rd field(+3) and stop sorting after 4th field(-4). Basically sort file : file1 on 4th field Sorts the contents of the file : file1 and removes duplicate entries Start sort after 5th(+7.5) column of 8th field (+7.5) and stop sort on 6th column(11.6) of 12th field(11.6) Sort numerically file : file1. Used when the file contains numeric entries.
tr
Syntax :
tr [-c] [-d] <expr1> <expr2> < <file>
By default, it translates each character in <expr1> found in the file to its mapped character in <expr2>
Example
tr ABC abc < filenew tr [a-z] [A-Z] <filenew tr d | < fileold tr s A <fileold tr cd |/ < fileold
Significance
Translates the uppercase letters A,B and C in file : filenew to lowercase Translates all the lowercase letters in file : filenew to uppercase Deletes all the | characters from file : fileold Compress multiple consecutive presence of character A to one A Delete all characters except | and /.
nl
Syntax :
nl [-w < number>] [ -s <delimiter_char>] <filename.> Shows line number at the left of each line in the file. Reserves six characters for the number
Example
-w <number> -s <delimiter_char>
Significance
Specifies the width option for line number display to <number> characters Used to specify the character used for delimiting between line no and line content
spell
Syntax :
spell [-a|-b] <filename(s)>
Lists all spellings in the file (s) the program recognizes as mistakes.
Example
-a -b
Significance
Use American spelling system Use British spelling system
find
Syntax :
find <path_list> <mode> <sel-criteria> <action>
Significance
Selects file with name <filename> . Pattern matching is allowed Selects file owned by user specified by <username> Selects type of files specified by <f> Selects file if owned by group <grpname>
-size +<x>[<c>]
-atime +<x>
Selects file if the size exceeds <x> blocks(characters if <c> is also specified)
Selects file if accessed before <x> days
-amin +<x>
-atime -<x> -amin -<x> -mtime -<x> -mmin -<x> -newer <flname>
Action
-perm <permission_mode>
-exec <command> {} \; -ok <command> {} ; -print
Significance
Finds the files with permission mode specified
Executes command <command> after finding every file on the search Executes command <command> after user confirmation Shows selected files in display
grep
Syntax :
grep [-c|-n|-v|-l|-I|-e|-h|-x ] <pattern> <filename(s)> Searches a file for a pattern
Flag
-c
Significance
Counts number of occurrences. Output shows only the file name and number of times the searched pattern found within it
-n
-v -l -i -e -h -x
Displays the line number containing the pattern, along with the lines
Displays all lines excepting those containing the pattern Displays only file names containing the pattern Ignores case while searching for the pattern Extended search. More than one pattern is to be searched Suppresses printing filename when printing multiple files where the pattern is found Matches are recognized only when the entire line of the file searched matches the fixed string
Example
grep while wot.ksh
Significance
Searches for the string while in the file : wot.ksh and if found, displays the lines containing the pattern Searches for the string while in all files with extension .ksh and shows the file name and the line containing the pattern Searches for the string List of names in all .txt files. Case-independent searching of the string while takes place in all the .ksh files and the the name of the files where matched is displayed only Searches for pattern Subhendu and Suvendu in the file : name.lst
egrep
Description Extended grep Expression
<ch>+
<ch>? <expr1>|<expr2> (<exp1>|<exp2>|<exp3>
Significance
Matches one/more presence of character <ch>
Matches zero / presence of character <ch> Matches <expr1> or <expr2> Matches expression <exp1><exp3> or <exp2><exp3>
Expression
egrep '(Mano|Mono)roma' name.lst egrep 'Aka+sh' name.lst egrep f srchlist name.lst
Significance
Matches string Monoroma or Manoroma Matches Akash , Akaash,Akaash File : srchlist contains all the strings to be searched in file : name.lst
sed
Syntax :
sed <options> <address,action> <filename(s)>
Stream text editor, used for:Displaying specific lines from a file by line no/pattern matching Inserting or changing texts in a file Deleting lines from a file
Addressing by lines
Done by specifying line number , or a pair of them separated by comma to specify the lower and upper boundaries of selection. Single word specifying action are:-
p
q $p
Print
Quit Last line
Examples
sed 3q name.lst sed n 3,5p name.lst sed n 10,$p name.lst
Significance
Show first three lines of file : name.lst and quit Show 3rd to 5th line of file : name.lst Show from line no. 10 to end of file
sed n e 1,2p e 7,9p e $p Show 1st to 2nd line, 7th to 9th line and last line name.lst of the file : name.lst
Examples
$ sed 'a\ > subhendu > ' del>del1 $ sed i\ > subhendu > ' del>del1 $ sed $a\ > subhendu > ' del>del1
Significance
Inserts the line subhendu after every line in the file : del and save the display in file : del1 Inserts the line subhendu before every line in the file : del and save the display in file : del1 Inserts the line subhendu at the end of file : del and saves under file : del1
Context addressing
Example
sed n /getopts/p emp.ksh sed n /getopts/, /charstring/p emp.ksh sed n /[cC]hatterjee/p emp.lst sed -n '/Aka*sh/p' name.lst sed n /^A.a.s/p emp.lst
Significance
Searches the string getopts in file : emp.ksh and displays it Searches for the string getopts and charstring in file : emp.ksh Searches for the string Chatterjee or chatterjee in file : emp.lst Searches for any string with the pattern specified Searches for any line containing A as 1st character, a as 3rd character and s as 5th character Shows all but the blank lines from file emp.lst Shows all lines with 4 characters only Shows lines longer than 100 characters Shows lines containing 9 after 9 characters in a line
sed n /^$/!p emp.lst sed /^$/d emp.lst sed n /^.$/p emp.lst sed -n '/. {101, }/p' del sed -n '/. {9 }9/p' del
Examples
sed n /if/w iflist /while/w wlist emp.ksh sed n f instr.lst emp.ksh
Significance
Searches file emp.ksh for pattern : if and writes matching lines into file : iflist Searches file emp.ksh for pattern : while and writes matching lines to file : wlist Searches file : emp.ksh for the strings specified in the file instr.lst
Substitution Examples
sed s/exhausted/tired/ list1 sed s/exhausted/tired/p list1 sed 1,5s/basic/prelim/ list1
Significance
Shows the content of the file : list1 with the string exhausted replaced by tired at all occurences Shows from 1st to 5th line of file : list1 with the string basic replaced by prelim at all occurences
awk
Reporting tool for Unix.
Syntax :
AWK is a pattern matching and processing language . It can search file(s) searching for a pattern, and when found, performs specified action.
Works best with ascii files, preferably not to use with binary files.
0011 | Tamal Sen Sharma 0010 | Ratna Sengupta 0015 | Raghab Dasgupta 0025 | Rajib Banerjee 0012 | Damini Sen 0007| Akash Nag 0009| Anindya Das 0019| Sougata Das
| Programmer| Avaya | 12000.00 | Accountant | Avaya | 12000.00 | Programmer| Avaya | 12000.00 | Programmer| Avaya | 12000.00 | Operator | Nestle| 10000.00 | Salesman | Nestle| 7800.00 | Manager | Nestle| 30000.00 | Manager | Avaya | 50000.00
| | | | | | | |
Significance
Number of command line arguments Conversion format for numbers Whitespace separated string for the width of input fields. Provides a facility for fixed-length fields instead of using field separators. Name of current input file Current record number
FILENAME NR
FS
IGNORECASE
NF
FNR
Significance
Output field separator Output record separator(default is newline) Input record separator( default is new line) Name of current input file Current record number Input field separator
IGNORECASE
ENVIRON ERRNO
$ cat name2.lst
0001 | Sourav Dutta 0002 | Ratna Sengupta $ cat name.awk { FS = "|" ; OFS = " ** " ; ORS = "\n-------------------------------------------\n"; print ARGC , ENVIRON["TERM"], CONVFMT , FILENAME ; print NR , $2 , $3 , "Total fields" , NF ; } | Project M | Avaya | 25000.00|
| Programmer| GFS
------------------------------------------2 ** Sourav Dutta ** Project M ** Total fields ** 6 ------------------------------------------3 ** vt100 ** %.6g ** name2.lst ------------------------------------------3 ** Ratna Sengupta ** Programmer ** Total fields ** 4 -------------------------------------------
Significance
Newline Tab Starts match at the beginning
$
. [ABC] [A-Ca-c] [^ABC] Desk|Chair [ABC][DEF]
Significance
Matches zero/more occurences of A,B or C Matches one/more occurences of A,B or C Matches to an empty string or more of A,B or C
( )
Significance
Is equal to Less than Less than or equal to
>
>= != ~ !~
Greater than
Greater than or equal to Not equal to Matched by regular expression Not matched by regular expression
Significance
Logical AND Logical OR Logical NOT
( )
Significance
Shows 3rd,4th and 5th record
.........................
.........................
END { < show subtotals> }
0006 | Sumit kumar Basu | Programmer| Avaya | 12000.00| 0011 | Tamal Sen Sharma | Programmer| Avaya | 12000.00| 0015 | Raghab Dasgupta | Programmer| Avaya | 12000.00| 0025 | Rajib Banerjee | Programmer| Avaya | 12000.00|
$ awk -F"|" '$3==" Programmer" { print $2,$4,$5 }' name.lst Raghab Das Avaya 12000.00 Sumit kumar Basu Avaya 12000.00 Tamal Sen Sharma Avaya 12000.00 Raghab Dasgupta Avaya 12000.00 Rajib Banerjee Avaya 12000.00 (Shows 2nd,4th and 5th field of all records where 3rd field contains the word Programmer)
$ awk -F"|" '$3~ /Manager/ && $4~ /Nestle/ ' name.lst 0009 | Anindya Das | Manager | Nestle| 30000.00| (Shows the records where 3rd field contains the string Manager and 4th field contains the string Nestle.
$ awk -F"|" '$3!~ /Programmer|Accountant/' name.lst 0001 | Subhendu Majumdar | Team Lead | Avaya | 25000.00| 0012 | Damini Sen | Operator | Nestle | 10000.00| 0007 | Akash Nag | Salesman | Nestle| 7800.00 |
| Manager | Manager
(Shows all records where 3rd fiels contains strings other than Programmer and Accountant) $ awk -F"|" '$5>=30000 { printf "%20s t t %d n" , $2, $5 }' name.lst Anindya Das 30000 Sougata Das 50000
$ awk -F"|" '$3~ /Manager/ { > kount = kount + 1 > print kount,$2 } ' name.lst 1 Anindya Das 2 Sougata Das
$ awk -F"|" '$5>=30000 { printf "%20s t t %d n" , $2, $5 }' name.lst Anindya Das 30000 Sougata Das 50000
$ cat salavg.awk
BEGIN { printf "\n\t\t Salary Report of Managers \n\n" } $3~ /Manager/ { kount++ tot = tot + $5 print kount,$2,$5 } END { print "\n\nTotal managers found is :",kount print "Average salary is :",tot/kount }
30000.00 50000.00
String Operators
(A) Concatenating Strings x = abcdef y = ghi z=xy # z = abcdefghi # x = abcdef
Built-in String Functions Functions gsub( <reg>,<string>,<target>) Significance Substitutes string <string> in string <target> every time the regular expression <reg> is matched Returns the position of the string <search> in string <string> Returns the length of the string <string> Returns the position in string <string> that matches the expression <reg> Splits string <string> into array elements of <store> based on delimiter <delim> Substitutes string <string> in <target> the first time the regular expression <reg> is matched.
sub(<reg>,<string>,<target>)
Built-in String Functions Functions substr(<string>,<pos>,<len>) Significance Extracts the portion of the string <string> starting from position <pos> of length <len> Translates the string <string> rto lower case Translates the string <string> rto upper case
tolower(<string>) toupper(<string>)
Use of Character Functions in awk $ cat awkf.awk $5 >= 30000 { print "Length of the string is : " , length ( $2 ) print "First three characters are : " , substr( $2,1,3 ) print "String in lowercase is : " , tolower( $2 ) print "String in uppercase is : " , toupper( $2 ) nm = substr( $2,2,4 ) print " 2nd to 5th characters are :", nm }
Use of Character Functions in awkContd $ awk -F"|" -f awkf.awk name.lst Length of the string is : 19 First three characters are : An String in lowercase is : anindya das String in uppercase is : ANINDYA DAS 2nd to 5th characters are : Anin Length of the string is : 19 First three characters are : So String in lowercase is : sougata das String in uppercase is : SOUGATA DAS 2nd to 5th characters are : Soug
Use of Character Functions in awkUse of Split function $ cat split.awk BEGIN { FS="|" printf "\n Details of name of the Employees : " print "\n\t Name \t\t\t\t First name \t\t Last name" print "\n\t............................................................" } $5 > 15000 { split( $2 , a , " " ) print "\n\t", $2, "\t\t", a[1] , "\t\t" , a[2] }
Use of Character Functions in awkUse of Split function $ awk -f split.awk name.lst Details of name of the Employees : Name First name Last name ........................................................... Subhendu Majumdar Anindya Das Sougata Das Subhendu Anindya Sougata Majumdar Das Das
Special String Constants Expression \\ \a \b \f \n \r \t \v \ Significance Backslash Alert or bell character Backspace Formfeed Newline Carriage return Tab character Vertical tab Double quote
Built-in Numerical Functions Expression cos(x) int(x) exp(x) log(x) rand( ) sin(x) Significance Cosine of x in radians Integer value of x Returns e raised to power of x Natural log of x Returns random number between 0 and 1 Returns the sine of x in radians
sqrt(x)
systime()
Arithmetic Functions Expression x^y x**y x%y x+y x-y x*y x/y x++ ++x y-Significance Raises x to the power of y Raises x to the power of y Calculates the reminder of division of x by y Adds y to x Subtracts y from x Multiplies y with x Divides x by y Increments x by 1 and then uses it Uses x and then increments it by 1 Decrements y by 1 and then uses it
--y
Arithmetic Functions Expression x+=y x-=y x*=y x/=y x++=y Significance x = x+y x = x-y x = x*y x = x/y x=x+1+y
Arrays Arrays are normally used to handle more than one related piece of data. One accesses the individual elements within an array by enclosing the subscript within double brackets. In AWK, one does not have to declare a variable to be an array, and does not have to define the maximum no. of elements. When one uses an element for the first time, it is created. In awk arrays, subscript is a string ; viz., tot_sales[Cal] = 10.15
One can use this in conditional flow:If Cal in tot_sales Delete tot_sales[Cal]
Multidimensional Arrays One can use two subscripts to form a multi-dimensional array. Example : tot_sales[India,Calcutta] = 100 And can use this in conditional statements also:If (India,Calcutta) in tot_sales
Examples of Arrays BEGIN { FS = "|" print "\n\t Name \t\t Basic \t\t da \t\t hra \n" print "---------------------------------------------------------------" } $3~ /Manager/ { da = $5 * 0.25; hra = $5 * 0.3; print $2 ,"\t" , $5 , "\t" , da , "\t\t" , hra ; tot[1] += $5; tot[2] += da ; tot[3] += hra ; tot[4] += $5 + da + hra ; } END { print "\n-------------------------------------------------------------" print "Total" , "\t\t\t" , tot[1] , "\t\t" , tot[2] , "\t\t" , tot[3] ; print "---------------------------------------------------------------" print "\n\n Grand Total : ", tot[4] ; }
Examples of Arrayscontd
$ awk f salcalc.awk name.lst Name Basic da
hra
--------------------------------------------------------------Anindya Das 30000.00 7500 9000 Sougata Das 50000.00 12500 15000 ------------------------------------------------------------Total 80000 20000 24000 ---------------------------------------------------------------
Multidimensional Arrays-Examples
Multidimensional Arrays-Examplescontd $ cat pop.awk BEGIN { system(" tput clear " ); print "\n\nPopulation in different parts of the cities in India\n\n" FS = "|" } { print $2 , "\t" , "\t" , $3 , "\t" , $4 ; tot_pop[$1,$2]+=$4; } END { print "--------------------------------------------------------" print "Population for Calcutta : " ,tot_pop["India","Calcutta"]; print "Population for Chennai" , tot_pop["India","Chennai"]; }
Multidimensional Arrays-Examplescontd
$ awk f pop.awk pop.lst Population in different parts of the cities in India Chennai West 150000 Calcutta North 234500 Calcutta East 134500 Chennai East 100000 Calcutta South 234508 Calcutta West 100500 -------------------------------------------------------Population for Calcutta : 704008 Population for Chennai 250000
Conditional Flow { if .. else if .. else if .. else } $ cat sal.awk 4 ~/Nestle/ { if ( $5 > 10000 ) print $2 , $5 , "Taxable"; else print $2 , $5 , "Non taxable"; }
$ awk -F"|" -f sal.awk name.lst Damini Sen 10000.00 Non taxable Akash Nag 7800.00 Non taxable Anindya Das 30000.00 Taxable
$ cat sal.awk $4 ~/Nestle/ { $5 > 10000 ? taxyn = "Taxable" : taxyn = "Non taxable" ; print $2 , $5 , taxyn ; } $ awk -F"|" -f sal.awk name.lst Damini Sen Akash Nag Anindya Das 10000.00 Non taxable 7800.00 Non taxable 30000.00 Taxable
Loops
Loops can be of three types : Do While For Used to perform some repeatitive jobs.
Example of Do Loop $ cat doloop.awk BEGIN { linesep = "-"; linedraw = "-"; FS = "|"; } { res = $5 / 1000 ; do { linedraw = linedraw"-" ; res--; } while ( res >= 1 ); print $2 , linedraw ; linedraw = linesep ; }
Example of Do Loopcontd $ awk f doloop.awk name.lst Subhendu Majumdar -------------------------Raghab Das ------------Sumit kumar Basu ------------Tamal Sen Sharma ------------Ratna Sengupta Raghab Dasgupta Rajib Banerjee Damini Sen Akash Nag Anindya Das Sougata Das --------------------------------------------------------------------------------------------------------------------------------------
Example of While Loop $ cat while.loop BEGIN { linesep = "-"; linedraw = "-"; FS = "|"; } { res = $5 / 1000 ;
Example of While Loopcontd $ awk f while.loop name.lst Subhendu Majumdar -------------------------Raghab Das ------------Sumit kumar Basu ------------Tamal Sen Sharma ------------Ratna Sengupta Raghab Dasgupta Rajib Banerjee Damini Sen Akash Nag Anindya Das Sougata Das --------------------------------------------------------------------------------------------------------------------------------------
For Loop
For ( <var> = <value> , var <operator> <value_last> , <var> [++|--] <statements> For ( <subscript> in <array> ) <statement>
Example of For Loop $ cat for.loop BEGIN { linesep = "-"; linedraw = "-"; FS = "|";
}
{ res = $5 / 1000 ; for ( res = res ; res >= 1 ; res -- ) { linedraw = linedraw linesep ; res--; } print $2 , linedraw ; linedraw = linesep ; }
Example of For Loopcontd $ awk f for.loop name.lst Subhendu Majumdar -------------------------Raghab Das ------------Sumit kumar Basu ------------Tamal Sen Sharma ------------Ratna Sengupta Raghab Dasgupta Rajib Banerjee Damini Sen Akash Nag Anindya Das Sougata Das --------------------------------------------------------------------------------------------------------------------------------------
Loop Breaking Statements Statement Break Continue Significance Exits the loop Leaves the current record and continues with the next record in the loop from beginning of the loop processing statements.
Format
%c %d %i %e %f %s %g
Meaning
Ascii character Integer Integer Floating point number using scientific notation Floating point number(eg., 10.43) String of characters Awk chooses %e or %f display format(whichever is shorter) suppressing non-significant zeroes.
Output to another file Statement Printf( Hello world\n) > datafile Printf( Hello world\n) >> datafile Significance Creates a file called datafile with the output Appends the output at the end of existing file datafile or creates a new file.
A file should be closed after it receives output from an awk program. Syntax : close(filename)
Functions
Block of code, accesses usually in multiple places in the code. When awk reaches the end of the function, it implicitly returns the control to the calling routine. To make an explicit return to the main program, one can explicitly use the return statement. Syntax : function fname(parameter list) { < function_code> }
Example of Functions $ cat subcalc.awk BEGIN { FS = "|" show_heading( "\n\t Name \t\t Basic \t\t da \t\t hra \n" ) ; show_heading( "-------------------------------------------------"); } $3~ /Manager/ { da = calc_comp(0.25,$5) ; hra = calc_comp(0.3,$5) ; print $2 ,"\t" , $5 , "\t" , da , "\t\t" , hra ; } function show_heading(heading) { print heading ; } function calc_comp(rate,val) { val = val * rate ; return val ; }
Example of Functionscontd $ awk f salcalc.awk name.lst Name Basic da hra --------------------------------------------------------------Anindya Das 30000.00 7500 9000 Sougata Das 50000.00 12500 15000
Use of inputs from user in awk program BEGIN { FS="|" printf "%20s%3d\n\n", "Cut-off basic pay : ",cobp } $5 > cobp { kount++ print $2 , $3 , $5 , "\n" } END { print "\n\n The End\n\n" }
Use of inputs from user in awk programContd $ awk -v cobp=15000 -f tkinp.awk name.lst Cut-off basic pay : 15000 Subhendu Majumdar Team Lead 25000.00 Anindya Das Sougata Das The End Manager Manager 30000.00 50000.00
Controlling Environment
The Unix environment is controlled by a number of pre-defined environment variables. They are usually defined in the file .profile for the user or are defined by user as and when required.
Pre-defined variable
ERRNO
Significance
Non-zero exit code of last command that failed. Value changes only when a command fails
LINENO
Meaningful only within a shell script. Its value is the line no. of the line in the script currently being executed
Value is always the full pathname of the directory where the control resided before coming into current working directory Current working directory Integer number of seconds since one invoked the Korn shell Value of the process id of the parent process of $$ Absolute directory path under which the programs for the commands reside.
OLDPWD
Pre-defined variable
EDITOR CDPATH
Significance
Vi or emacs List of colon separated directory name, which is followed by the system when the user issues a cd command Pathname of the shell script containing commands to be executed when the korn shell is invoked Filename of the korn shell history file Integer number containing maximum number of commands to be retained in history files Pathname of home directory for an user
Pre-defined variable
IFS
Significance
Zero/more characters to be treated by the shell as de-limiters when parsing a command line into words or using the read command Display width used by Korn shell edit mode- vi or emacs Integer number representing the number of lines displayed by the terminal. Pathname of a file to be monitored by the shell for a change in its date of last modification. If change is noted, the shell issues the message : You have mail at the next oppurtunity
MAILCHECK
PS1
No. of seconds after which the shell should check for a change in the MAIL file
Primary shell prompt string.Prompt where commands are issued by the user
Pre-defined variable
PS2
Significance
Secondary prompt screen. Shown when a command pawns more than one line.
PS4
SHELL TERM TMOUT
Debug prompt screen , shown when a shell program is executed using v option.
Pathname of the shell the user is using Symbolic alpha numeric string that identifies the type of your terminal Seconds after which automatic logout occurs.
stty
Controls terminal output and sets terminal characteristics Options
stty a stty echoe stty echo
Significance
Displays all current settings Enables backspacing remove character from the display Keyboard entry is not echoed
stty echo
stty intr ^Z stty eof \ ^a stty erase ^H stty quit ^d
Options
stty susp ^Z
stty stop ^S stty start ^Q stty rows 20 stty rows 20 column 80 stty iuclc stty olcuc stty size
Significance
Enables suspending a foreground process when <ctrl-Z> is pressed
Enables halting the current session by pressing ^S Enables starting the current halted session by pressing ^Q Set 20 rows in display Sets 20 rows and 80 columns in display Maps uppercase alphabets to lowercase Maps lowercase alphabets to uppercase Gives the current screen size in terms of rows and columns
stty eol ^J
stty sane
Process
Running Jobs
Scheduling Jobs
What is a process?
Process is a program that has its own address space. Every command fired in Unix has a process associated with it. Is born when a program starts execution, and remains alive as long as the program is active.After execution is complete, it dies. Has a name, usually the name of the program being executed.
Sh Process
A process is immediately set up by the Kernel when a user logs into Unix system.This os usually a Unix command(sh/ksh/bash etc). Any command executed is actually input to the shell process. Can be known from the value of shell variable $$. This process for the shell remains alive till the user logs out.
ps process status
Displays the attributes of a process Command
ps
Significance
Selects the process associated with the current terminal.Shows output in four columns: PID, TTY(terminal info), TIME(total cpu time used by the process), CMD(command gererating the process) Displays process ancestry.Produces 8 column output:- UID(userid),PID,PPID(Parent PID), C( amt of CPU time consumed by a process),STIME(time the process started),TIME,CMD
ps f
Command
ps u ems2000 ps a ps x ps e
Significance
Displays processed of user : ems2000 Displays all processes of all users; excepting the system processes Shows command line in extended format Displays system processes
Internal and External Commands External program : Commands like : cat, ls,
grep, sed etc are external programs.Shell creates a process for each of these commands. spawning to another shell, which then becomes the parent of the commands featured in the script. which are executed directly by the current shell. No additional processes are generated. Example : cd , echo etc
Shell Scripts : Shell executes shell scripts by Internal Commands : These are commands
Unix is a multi-tasking system , allowing to perform more than one job at a time.
One job can be executed in foreground at a time, but many can be executed at background. & operator at the end of the command line interprets the job to be done at background. Shell returns the PID of the background job for tracking.
Korn or bash shell users can use the job control facility of the shell to manipulate jobs.One can put a job in the background, bring it back to foreground, suspend it, run it later, or even kill it.
bg
You are running a job in the foreground. It is taking a lot of time. You want to do some other jobs. Press ^Z. The job running in the foreground will be suspended. Enter the command : bg at the OS prompt and push it to be executed in the background. Unix will return you a PID for the process pushed to background.
jobs
Shows the jobs running in background. Command : jobs l prints PID of each job besides job number. This command is used by the user to push a job from background to foreground.
fg
Used to bring a background job to foreground and execute it. fg [%<job_no>] brings the job from background to foreground and executes it. For example , issuing of the command : fg %2 executes job number 2 (viewed by jobs command) from background to foreground and executes it. Execution of fg command alone from the OS prompt resumes latest created background process to be executed inforeground.
kill
Example
kill 2905 kill 9 2905 kill s SIGKILL 2905 kill 15 2906 kill s SIGTERM 2906 kill 24 2900 kill 25 2900 kill 26 2900 kill s SIGCONT 2900 kill &!
Significance
Kills the process with Pid : 2905 Surely kills a process with PID 2905 Terminates the process with PID 2906 Pause the process with PID 2900 Start the paused process with PID 2900 Kill the latest background job
nice
Processes in the Unix are usually executed with equal priority. But, high priority jobs must be executed at the earliest. Unix offers : nice command to change the priority of execution of jobs. Ordinary users can only reduce priority; super user can do both. Nice value ranges from 0 to 39 ; commands run with a value of 20 in both. Example : nice n 15 prog1.sh & The above command reduces the pririty value to 35 The nice and priority values of the processes can be displayed with ps l command.
batch
Schedules job for later execution, when the system load permits it. Prevents too many high load jobs to run at the same time. Syntax for running a program : batch < prog1.sh
Any job scheduled in this way also goes to the at queue and can be removed using : at r , provided it is fired before the job has been executed.
at
Schedules a program for execution at a specified time.
Flag
-l -m -r <joblist>
Significance
Displays the list of jobs scheduled by the user Mail a report of successful execution of the job Remove the jobs specified in the joblist from queue
Examples
at 2300 my_job.sh at 11:00 pm my_job.sh
Significance
Executes program my_job.sh at 11:00 pm tonight Schedules execution of njb.sh after 6 hours from now
at 11:00 P my_job.sh
at 2300 today my-job.sh at now + 6 hours njb.sh
at 6:30 pm next week njb.sh Schedules execution of program : njb.sh at 6:30 pm next week at now + 1 year njb.sh Schedules execution of program : njb.sh after 1 year from now Schedules execution of program : njb.sh at 3:08 pm tomorrow.
Examples
at 9 am Mon njb.sh at 9 am tomorrow at now + 5 minutes < njb.sh at f njb.sh now + 5 minutes
Significance
Schedules execution of njb.sh at 9 am coming Monday. Schedules execution of njb.sh at 9 am tomorrow. Schedules execution of njb.sh after 5 minutes from now.
at 08:15 Jan 12 < njb.sh Schedule execution of njb.sh at 08:15 am on January 24.
cron
Runs jobs periodically. Mostly dormant, but wakes every minute and checks for a file in /usr/spool/cron/crontabs directory for programs to be executed at that instant. A user may be permitted to place a crontab file after his login name in this directory. This file contains a list of commands, along with a schedule for execution.
3rd field(1 to 31) controls the day of the month. * means all.
4th field(1-12) indicates the month
User can modify the file containing his cron commands as follows:Create a file : cron.txt in the format described earlier. Issue the following command from the Os prompt: crontab cron.txt. There will be now a file ems2000( for eg., the name of the user) in /usr/spool/cron/crontabs with the contents of cron.txt. One can see the contents of crontab file issuing the command : crontab -l
System Administration
System Administration
shutdown Shutdown Unix server su Go as you like wall Send message to everybody umask Set Default Permission ulimit Set maxm. Space for user du Show disk usage df Free bytes available
shutdown
Executed from the root user to shut down the system.
Commands
shutdown g2 shutdown y g0 shutdown y g0 i6 shutdown 17:30 shutdown r now
Significance
Powers down system after 2 minutes Immediate shutdown Shutdown and reboot Shutdown at 5:30 pm Shutdown and reboot
shutdown m
su
Takes to super user mode.
Commands
su
Significance
Takes to super user mode. One can have all the accesses same as the root user. Root password is required after executing the command One logs in as the user : mat. Asks for the password.
su mat
wall
Addresses all the users simultaneously. $ wall . ^D All the text written will be reflected in the terminals of all the users. $ wall < mesg.txt Contents of the file mesg.txt are sent as the wall message.
umask
Used by System Administrator to set the default permissions to assign to each file to be created by the user. User can modify this default settings. To give permission 751 as a default to newly created files, umask value will be (777 751) = 026. This value is assigned as a command : umask 026
ulimit
Faulty programs or processes can eat up disk space.So, a restriction is to be imposed on the mazimum size of a file that an user is permitted to create.
Ordinary user can reduce the default value . Super user can increase / decrease it.
Examples
ulimit a ulimit Ha ulimit t <size> ulimit f <size>
Significance
Shows soft limits Shows the hard limits Set the CPU time in seconds Set the maxm. Size in blocks
ulimit d <size> Sets the maxm. size of data blocks in KB ulimit m <size> Sets the maxm. Size of memory in KB.
du
Shows disk usage amount of space taken by a group of files in units of 512 bytes or KB. Descends all sub-directories from the directory in which the command is fired.
Flag Significance
-a
-k -s -r
Print entries for each file encountered in the directory hierarchies in additional to the normal output
Give the block count in terms of 1024 bytes Give the grand total of disk usage for each of the directories Print messages about directories that cannot be read, files cannot be accessed etc.
df
Displays the number of free 512 byte blocks and free inodes available for file systems by examining the count kept in the superblocks.
Flag Significance
-b -e -f -i -k Reports only the number of KB free Report the number of files free Reports only the actual count of the blocks in the free list Reports the total number of inodes, number of free inodes, number of used inodes,% of inodes in use Reports the allocation in KB
finger
By default, the command finger lists for each user in the system: Login name Full given name Terminal write status( if write permission is denied) Idle time Login time Users home directory and login shell Project on which the user is working Any plan put by user in the file .plan under home directory Office location and telephone number Last time the user received the mail, the last time user read the mail.
Flag Significance
-b -f -h -i -l -p -R Suppresses information on home directory and shell Suppresses the header info Suppresses the project information Suppresses the info on idle time Suppresses long output Suppresses plan Print users hostname
-s
Significance
Who file
Last login file Users name, office etc Users plan Users project info Mail directory
crypt
Used to encode/decode files using personal password
Options
crypt < uncrypted_file > crypted_file crypt passwd < file1 > file2 crypt passwd < file2 > file1 vi x file2
Significance
Crypts file uncryped_file to crypted_file. Asks for a password. Crypts file : file1 to : file2 based on the password : passwd Decrypts file : file2 to : file1 using the password : passwd Helps to view an encrypted file : file2. Asks for a password
Communication
Communication
mailx Send and receive mails
mailx
Used to send/receive mails Syntax for sending mails:-
mailx -e
mailx [-HL] [-u user] mailx -f [-HL] [filename]
Case : 2
$ mailx [email protected]
If no subject is entered at the command line, a prompt asks for the subject.
Case : 3
$ echo Hello mike|mailx s Heading1 [email protected] The job performed above will send a mail to : [email protected] with subject of the mail as : Heading1 and body of the mail as : Hello mike.
Case : 4
$ cat msgfile| mailx s Heading1 [email protected] The job performed above will send a mail to : [email protected] with subject of the mail as : Heading1 and body of the mail with the texts written in the file : msgfile.
Case 5- You want to send a mail to some user and cc it to some other users.
$ mailx [email protected] Subject : Test mail number 2 this is a test mail for communication ~h To: [email protected] Subject: testing Cc: [email protected] Bcc: [email protected] (continue) testing mail--please ignore ~. EOT .
Flag
-e
Significance
Test for presence of mail. mailx prints nothing and exits with a successful return code if there is mail to read.
-H/-L
-u user
$ mailx
mailx Revision: 1.179.214.2 date: 98/12/01 01:29:55 Type ? for help. "/var/mail/ems2000": 2 messages 2 unread >U 1 [email protected] Mon Aug 18 04:50 33/1199 RE: 3rd mail
U 2 ems2000 schedule
?_
It shows a list of unread mails with a ? prompt below, waiting for commands from the user.
Flag
<n> type quit(q) next reply <n> mail <user>
Significance
Shows message number <n> , the number typed at the first column for every message.
Shows all the messages in one shot, one by one Quit mail utility
Shows next message Reply to message number <n> Opens mail utility to write mail to user <user>
write
Type text into other users terminal for information
Case : 1
$ write ems2000 Hello, how are you ^d
The command flashes the message : Hello, how are you to all persons terminal who have logged in as user : ems2000.
Case : 2
$ write ems2000 pts/tc Hello, how are you ^d
The command flashes the message : Hello, how are you to that persons terminal who is working in terminal pts/tc and logged in as user : ems2000.
Case : 3
$ write ems2000 < fileshow.txt
The command flashes the message written in the file : fileshow.txt to all persons terminal who have logged in as user : ems2000. User must be logged in and mesg should be set to y( mesg=y) for successful write communication.
ftp
Syntax : ftp [-i] [-v] [-n] <remote_host>
Copies file over a network connection between local host to remote host. ftp runs on client server.
Flag Significance
-i
-n
-v
Significance
Executes command <command> in the local server Copy file <lf> in the local server to the end of file : <rf> in the remote server. If no such file exists in remote server, a new file is created. Sets the file transfer type to ASCII Sets the file transfer type to binary Close the connection to remote host, if it is open, and exit Remote file names with name containing all letters in uppercase are translated to lowercase when the file is transported from remote server to local. Set the working directory in the remote server to <remdir> Set the current directory of the remote server to the parent of current working directory.
cd <remdir> cdup
Command
chmod <mode> <file> close mdelete <remote_files> mdir <rem_files> <lcl_file>
Significance
Changes the mode of the file : <file> to mode <mode> Terminate the connection to the remote server Deletes the files in the remote server Writes a list of remote files to local file <lcl_file>
mget <rem_files>
get <rem_file> [<lcl_file>] mkdir <dirname> mls <remfiles> <lclfile> modtime <rem_file>
Command
dir <rem_dir> [<lcl_file>] disconnect beep lcd [local_dir]
Significance
Shows a listing of remote directory <rem_dir> to terminal/ into local file <lcl_file> . Close the connection to remote host Beeps when command is executed Specifies the current working directory of the local server to directory : <local_dir> . If unspecified, users home directory is considered for navigation. get file if remote file <file_name> is newer than local file <file_name> Toggle forcing interactive prompting on multiple commands Toggle store unique for local files. IF turned on, then while receiving files from remote host, if a remote file already exists with a name equal to a local file, a .1 is appended to the name. For another repeatition, a .2 is appended till 99 such cases are found.
Command
size <remote_file> quit reget <remfile> [<lclfile>]
Significance
Shows the size of the remote file bye Acts like get. If local file <lclfile> exists and is smaller than the remote file <remfile>, local file is assumed to be partially transferred from remote server.Then, the transfer is continued from the apparent point of failure. This command is very useful to transfer very large files over the netwoprk that tend to drop connections. Shows the current status of ftp- informations are shown on remote server name and all the settings(prompt,type,bell,case etc) Toggle store unique for remote files. If turned on, then while transferring files to remote host, if a remote file already exists with a name equal to a local file, a .1 is appended to the name of the file transferred from local to remote server.. For another repeatition, a .2 is appended till 99 such cases are found.
status
sunique
Command
system umask [new mask]
Significance
Shows the type of operating system running in remote server Set the default umask on the remote server to new value. IF nothing is specified, the current umask value of the remote server is displayed. If only help is typed from ftp prompt, it shows a list of all commands that can be used from ftp prompt. If a command is mentioned for help, it shows one line help explaining its significance. toggle metacharacter expansion of local file names. If put on, one can use * , ? etc characters to specify portion / all of the file name.
help [<command>]
glob
rlogin
Syntax : rlogin <rhost> [-l <username>]
Logs into remote host <rhost> as user <username> with/without any password. If the users are same on both remote and local server, then option <username> is not required. Prompting for password depends on the entries in file : /etc/hosts.equiv or .rhosts under remote users home directory of the remote server
rcp
Copies file(s)/directory(ies) from one server to another Copy Single File
rcp [-p] <srcfile1> <rem_user> @<remserver>:<pathname>/<dest_file>
Shell Programming
Or
$ sh < prog1.sh where prog1.sh is the name of the program. It may not have execute permission at this point of time.
. <progname>
Whenever a shell script is executed, it runs in a sub-shell under the current shell. To make the script run in the current shell, one have to trigger the program as : . <progname>. The program need not have execute permission before executing in this fashion.
read p_name
Echo Hello $p_name
Hello Suman
One can declare it at the top of the program /anywhere at the time of usage.
Example g_val=20 g_name=two words Significance Declares a variable : g_val and assigns a value 20 to it. Declares a variable : g_name and assigns a value two words to it.Place the value to be assigned within quotes if contains more than one words
Declares a variable : g_cnt and assigns a value 20 to it. Makes the value in the variable : g_cnt readonly. Further change in the value is not allowed.
unset g_cnt
Parameters $!
$$
$_
$echo $1 $2
$ Happy birthday
$echo $1 $3 $2 $6
$ Fri 19 Aug 2003
$echo $1 $2 $3 $4 $5 $6 $7 $8 $9
$ I know this is not a valid issue to $ shift; echo $1 $2 $3 $4 $5 $6 $7 $8 $9
Performing Calculations
Expr, echo let and (( )) comes handy cnt_val=`expr $cnt_val + 1` cnt_val=`echo $cnt_val + 1|bc` let cnt_val=cnt_val + max_limit ((z=x+y+2)) cnt=`expr $a \* 2` a=`expr $rt / 2` per=`expr \( $m1 + $m2 \) / 5`
echo
Displays strings , values in terminal or stores them in files.
Examples echo Hello dear Significance String : Hello dear shown in terminal
echo hello `hostname` String : Hello sb1dtl02 shown in terminal a=256; echo $a The value of variable a is displayed in terminal
Creates a file called : del or overwrites an existing file , the content of the file being : happy
Appends the contents o the variable $string at the end of the file : del
\b
\t \r \c
Backspace
Tab Carriage return Positioning the cursor after the statement
tput
Syntax : tput [<options>] Formats display
Options
clear cup r c bold blink rev smul rmul bel
Significance
Clears the screen Moves cursor to row r and column c Bold display Blinking display Reverse display Starts mode underline Ends mode underline Bells the terminal
lines
ed el
Decision making If
Making decisions based on conditions is the most popular practice in every programming language. Shell provides decision building using If statements.
If <conditions> then <actions> fi If <conditions> then If <conditions> then <actions> elif <conditions> then <actions>
<actions>
else <actions> fi
else
<actions> fi
Test
Used to compare numerical values,strings,find properties of different files etc.
Numericl test operators Significance -gt Greater than
-lt
-eq(=) -ne(!=) -ge -le
Lesser than
Equal to Not equal to Greater than or equal to Lesser than or equal to
Significance True if string1 is same as string2 True if string1 is not equal to string2
-n string
-z string
while [ -z "$userid" ] do tput cup 5 5 echo "Enter userid:\c" read userid done
if test $userid = super then tput cup 11 5 echo No such userid allowed exit fi
while [ -z "$pswd" ] do tput cup 7 5 echo "Enter password:\c" stty -echo read pswd stty echo done
while [ -z "$npswd" ] do tput cup 9 5 echo "Confirm password:\c" stty -echo read npswd stty echo done if test $pswd = $npswd then tput cup 11 5 echo o.k else echo wrong entry.... fi
And,Or
Operators -a -o Significance and or echo "Enter city and state:\c"
E x a m p l e
read city state if test $city = cal -a $state = wb then echo You stay in Calcutta and in West Bengal elif test $city != cal -a $state = wb then echo You stay in West Bengal, not in Calcutta elif test $state != wb -a \( $city = bom -o $city = del \) then echo you stay in other metros fi
Case
Used to build logic depending on different values of same object. Syntax
case <variable> in <value1>) <actions> ;; <value2>) <actions> ;; *) <actions> esac
echo "Enter an animal:\c" read animal case "$animal" in [Tt]iger|[Ll]ion) echo He is the king;; Crow|Sparrow|parrot) echo They are birds;; [0-9]*) echo Name cannot start with numbers;; ?) echo You entered a single alphabet only;; ??) echo You entered two alphabets only;; [hH]ip*m*) echo Is it hippo?;; *) echo Correct entry , but could not be identified esac
Loop processing
Three common loops used in shell scripts are :1. While 2. Until 3. For
While
Syntax
while <condition is true> do <actions> done while [ -z "$userid" ] do tput cup 5 5 echo "Enter userid:\c" read userid done
Until
Syntax
until <control does not return true> do <actions> done
For
Syntax
for <variable> in <value1> <value2> <value3> .
do
<actions> done
done
n=20 while [ $n -le 100 ] do echo $n ((n=n+1)) if test $n -eq 30 then break fi done
Sleep
Syntax: sleep < number of seconds> Idly waits for number of seconds specified. Mainly used to display an event clearly to user.
Basename
Extracts the base filename from an absolute pathname Example:$ basename /home/ems2000/shell_scripts/ppc1.sh ppc1.sh When used with second argument, it strips off the second argument from the first
E x a m p l e
echo "Enter city code:\c" read city_code grep "^$city_code" << stcity 01 Calcutta 02 Bombay stcity if [ $? -ne 0 ] ; then echo invalid code fi
echo First name:\c" read fname echo Middle name:\c" read mname echo Last name:\c" read lname echo your full name is $fname $mname $lname
$ prog8.sh<<end > soma > sen > gupta > end First name:Middle name:Last name:your full name is soma sen gupta
read city
set --`grep "^$city" <<tillend 01 Calcutta India WestBengal
echo "\n\tState
else echo Invalid code fi
: $4"
Command grouping
Group Significance () Executes the commands placed inside ( and ) separated by ; in a sub-shell under the current shell. They have the same PID Same as above; but uses current shell only. The closing curly brace must be on a separate line. To have both the braces on the same line, terminate the last command with a ; $pwd /home/ems2000 $ { cd scripts;pwd; } /home/ems2000/scripts $pwd /home/ems2000/scripts
{ }
Conditional Execution
Group cmd1 && cmd2 cmd1 || cmd2 cmd1 || cmd2 && cmd3 Significance Command cmd2 is executed only when command cmd1 succeeds Command cmd2 is executed only when command cmd1 fails Metacharacter coming first gets higher priority
$grep ems2000 prog.sh && echo String found String found $ grep ems99876 prog.sh || echo String not found String not found
String Handling
Syntax $ expr <string> : .* $ expr <string> : \(..\) $expr <string> : [^d]*<char> Significance Finds the length of the string : <string> Extracts the fourth and fifth character from the string Locates position of character <char> in the string <string> Significance Finds the length of the string : Training Extracts the fourth and fifth character from the string Locates position of character l within string : Anamoly
Options Significance
= If the variable <var> is null, it is assigned the string <stg>. echo Enter filename:\c read flname grep $pattern ${flname:=emp.lst} IF flname is null, flname=emp.lst
Evaluates the parameter if the variable is assigned and nonnull, otherwise it echoes the string following it. The shell is also terminated. This option is useful in terminating a script if the user fails to respond properly to shell directives. echo Enter pattern:\c read pattern ${pattern:? No pattern}
Merging Streams
Usually, the inputs required for a program are typed in by user in terminal, all the outputs in terms of messages and all the error messages ( shell genarated) are shown in the terminal .
But, one can make a program take inputs from a file, store outputs(messages) and / or error messages in a file, instead of displaying them in a terminal. Symbol Significance
0 1 2 Standard input Standard output Standard error
Significance All the output messages and prompts will be taken into file : sclist. However, the inputs to the program will be received from terminal. $ cat outlist Enter pattern: Pattern is Patternfromterminal $
$ cat prog2.sh echo "Enter pattern:\c" read pattern echo \nPattern is $pattern"
Significance All the inputs prompted by program prog1.sh are taken from file : namelist
$ cat prog1.sh echo "Enter first name:\c" read fname echo \nEnter last name:\c" read lname echo \nWelcome $fname $lname"
$ prog1.sh <namelist Enter first name: Enter last name: Welcome Suman Das
Significance All the inputs prompted by program prog1.sh are taken from file : namelist and all the output messages are stored in file : sclist $ prog1.sh <namelist >sclist $ cat sclist Enter first name: Enter last name: Welcome Suman Das
$ cat prog1.sh echo "Enter first name:\c" read fname echo \nEnter last name:\c" read lname echo \nWelcome $fname $lname"
Example $ 1>&2
$ cat prog2.sh
ans=y while [ "$ans" = "y" -o "$ans" = "Y" ] do echo "\n Enter city:\c" 1>&2 read city echo "$city" echo "\n More(y/n):\c" 1>&2 read ans done
$cat outlist
calcutta bombay
Example $ 2>&1
$ cat prog3.sh echo "Searching all the .kkl files" ls -ltr *.kkl $ prog3.sh Searching all the .kkl files *.kkl not found $prog3.sh > out.2 *.kkl not found
$cat out.2 Searching all the .kkl files *.kkl not found
$ prog3.sh>out.2 2>&1 $
Using subroutines
Using subroutines is one of the most popular practice in any programming language. Codes in a subroutine does a piece of job. It can take inputs from an user, do some jobs and can return some value to the calling program.
Subroutines are used to: Modularize program sections. Perform repeatitive jobs.
Do Calculations.
Syntax:<subroutine_name> { <actions> } function <function_name> { <actions> } Inputs to a subroutine are identified within the subroutine as : $1,$2,$3 etc. A subroutine is called from a program as :-
$ function show_name { grep "$1" *.ksh if [ $? -eq 0 ] ; then return 1 else return 2 Fi }
echo "Enter pattern:\c" read pattern show_name "$pattern" if [ $? -eq 1 ] ; then echo "Pattern found..." else echo "Not found" fi