Set up password less authentication in ssh

To setup SSH passwordless authentication(User Equivalence) Between 2 Servers Linux, we can follow the below steps.

Lets assume we have only 2 servers : ServerA, ServerB and we need to configure password-less login between both the servers.

We need to login to ServerA and identify the .ssh directory exists under the home directory

In case it doesn't exist we can create it.

cd /root

mkdir .ssh

chmod 700 .ssh

On Linux for root user it is normally /root/.ssh

$ cd /root/.ssh

$ ssh-keygen -t rsa

This will create 2 files id_rsa and id_rsa.pub. One is a private key file and other is the public key file.

$ cat id_rsa.pub >> auth_keys_a

Now copy this file to the ServerB using scp utility

$ scp auth_keys_a root@ServerB:/root/.ssh

Now on ServerB identify the .ssh directory which should ideally be on the same location /root/.ssh

$ cd /root/.ssh

$ ssh-keygen - rsa

This will again create the public and private keyfile

$ cat id_rsa.pub >> auth_keys_b

Now scope this file back to ServerA

$ scp auth_keys_b root@ServerA:/root/.ssh

Now we can try to ssh between the nodes

From ServerA

ssh root@serverB

From ServerB

ssh root@serverA

Removing particular Host name form SSH known_host list

To remove a specific host from the SSH known_hosts list. we can use the below command:

ssh-keygen -R hostname

Instead of hostname, we can directly use the IP also.

 If we want to remove non-standard ssh port,  then we can use this format

ssh-keygen -R [ssh.xxx.com]:[port no]




Stop NetworkManager to overwrite /etc/resolv.conf

Network manager overwrite the /etc/resolv.conf file. To stop overwriting this file. We can follwo the below steps

# systemctl disable NetworkManager.service
# systemctl stop NetworkManager.service

Or If we don't want to stop or disable the network manager service, in that case. We will have to tell NetwoekManager not to modify the DNS settings:

 /etc/NetworkManager/NetworkManager.conf
 [main]
 dns=none

SSH Server related basic command

Install

To install the openssh-server, you need to install the openssh-server package:

sudo dnf install -y openssh-server;

Start

To start the sshd daemon (openssh-server) in the current session:

sudo systemctl start sshd.service;

Stop

To stop the active (if any) sshd daemon in the current session:

sudo systemctl stop sshd.service;

Enable

To configure the sshd daemon to start automatically at boot time:

sudo systemctl enable sshd.service;

You will get an output similar to this:

ln -s '/usr/lib/systemd/system/sshd.service' '/etc/systemd/system/multi-user.target.wants/sshd.service'

Disable

To configure the sshd daemon to stop automatic initialization at boot time:

sudo systemctl disable sshd.service;

Linux command terminal output and redirecting to file

Finding all files in a directory containing a specific text in Linux

grep -rnw '/path/to/target/location/' -e 'search_string'

-r or -R is recursive,

-n is line number, and

-w stands for match the whole word.

-l (lower-case L) can be added to just give the file name of matching files.

Along with these, --exclude, --include, --exclude-dir flags could be used for efficient searching:

This will only search through those files which have .log or .out extensions:

grep --include=\*.{log,out} -rnw '/path/to/target/location/' -e "search_string"

This will exclude searching all the files ending with .txt extension:

grep --exclude=*.txt -rnw '/path/to/target/location/' -e "search_string"

For directories it's possible to exclude a particular directory(ies) through --exclude-dir parameter. For example, this will exclude the dirs dir1/, dir2/ and all of them matching *.dst/:

grep --exclude-dir={dir1,dir2,*.dst} -rnw '/path/to/target/location/' -e "search_string"

This works very well for me, to achieve almost the same purpose like yours.

Displaying lines numbers in Cat command

The syntax is:
cat -n fileName

OR
cat --number paul.txt

or  we cab use the more command/less command as filter when text can not be fitted on the screen:
cat --number foo.c | more
cat --number foo.c | less

The -b / --number-nonblank option number all nonempty output lines, starting with one and the syntax is:
cat -b fileName

OR
cat --number--nonblank filename

Finally, we can suppress or remove repeated empty output lines with the -s / --squeeze-blank option:
cat -s -n fileName
cat -s -n /etc/hosts




we can use the cat or nl command to display line numbers:

cat -n hello.c
nl hello.c



A note about sed
In the case just to print 7th and 9th line we can use the sed command:



sed -n -e 7p -e 9p /etc/resolv.conf

Multiple lines comment and uncommnet in VIM

To comment out blocks in vim:

• press Esc (to leave editing or other mode)
• hit ctrl+v (visual block mode)
• use the up/down arrow keys to select lines you want (it won't highlight everything - it's OK!)
• Shift+i (capital I)
• insert the text you want, i.e. %
• press EscEsc

To uncomment:

Put your cursor on the first # character, press CtrlV (or CtrlQ for gVim), and go down until the last commented line and press x, that will delete all the # characters vertically.

Searching text in the file contents inside a directory in linux

Sometimes, We need to find out specific string of text in the files' Contents of a directory of linux system. Its to look for text within the file, no tin the file name. 


In such case we can user the below command:


grep -r -l "specific test" /directory/path/full/*criteria_if_required_in_file_name*log

How to kill a process on a port on ubuntu

Sometimes, when we strying to up an service and get message like: the port *** is used by another service. And we are notable to start our app to use that port. In that case, we are in need to free the port by stopping the running process. For this purpose a first we need to find out the process which is using that port.

By using the below query we can get the process id using this port:

sudo lsof -t -i:8080 [For this case 8080 is our reference port number]

to get the process in details, we can use:
ps -ef | grep


And then we can kill the process by :
kill -9

We can use the below command at once to kill the process:
sudo kill $(sudo lsof -t -i:8080)

Sheel Script for Deleting files older than some days

find is the common tool for this kind of task :

find ./target_dir ! -name 'stayhere.txt' -mtime +5 -type f -delete


EXPLANATIONS

./target_dir your directory (replace with your own)
! -name File to exclude
-mtime +5 older than 5 days
-type f only files
-delete no surprise. Remove it to test your find filter before executing the whole command
And take care that ./target_dir exists to avoid bad surprises !

Installation of Oracle Database 11g on Linux with ASM

Let’s assume that you have already installed Linux on your server. And disks are already bounded to the server.

1. Creating OS groups and users.

#Creating groups for Grid Infrastructure

groupadd asmadmin
groupadd asmdba
groupadd asmoper

#Creating groups for Oracle Software

groupadd oinstall
groupadd dba
groupadd oper

#Creating user for Grid Infrastructure

useradd -g oinstall -G dba,asmadmin,asmdba,asmoper -d /home/grid grid

#Creating user for Oracle Software

useradd -g oinstall -G dba,oper,asmdba -d /home/oracle oracle

#Setting password for users

passwd grid
passwd oracle

2. Creating necessary directories

mkdir -p /u01/app/grid
mkdir -p /u01/app/11.2.0/grid
mkdir -p /u01/app/oracle
chown -R grid:oinstall /u01
chown oracle:oinstall /u01/app/oracle
chmod -R 775 /u01

3. Creating .bash_profile-s

#For Oracle user

su – oracle
vi .bash_profile

if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi

ORACLE_SID=orcl; export ORACLE_SID

ORACLE_UNQNAME=orcl; export ORACLE_UNQNAME

JAVA_HOME=/usr/local/java; export JAVA_HOME

ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE

ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1
export ORACLE_HOME

ORACLE_TERM=xterm; export ORACLE_TERM

NLS_DATE_FORMAT=”DD-MON-YYYY HH24:MI:SS”
export NLS_DATE_FORMAT

TNS_ADMIN=$ORACLE_HOME/network/admin; export TNS_ADMIN

ORA_NLS11=$ORACLE_HOME/nls/data; export ORA_NLS11

PATH=.:${JAVA_HOME}/bin:${PATH}:$HOME/bin:$ORACLE_HOME/bin
PATH=${PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin
PATH=${PATH}:/u01/app/common/oracle/bin
export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$ORACLE_HOME/oracm/lib
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib
export LD_LIBRARY_PATH

CLASSPATH=$ORACLE_HOME/JRE
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/jlib
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/rdbms/jlib
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/network/jlib
export CLASSPATH

THREADS_FLAG=native; export THREADS_FLAG

export TEMP=/tmp
export TMPDIR=/tmp

umask 022

#For Grid user

su – grid
vi .bash_profile

if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi

ORACLE_SID=+ASM; export ORACLE_SID

JAVA_HOME=/usr/local/java; export JAVA_HOME

ORACLE_BASE=/u01/app/grid; export ORACLE_BASE

ORACLE_HOME=/u01/app/11.2.0/grid; export ORACLE_HOME

ORACLE_TERM=xterm; export ORACLE_TERM

NLS_DATE_FORMAT=”DD-MON-YYYY HH24:MI:SS”; export NLS_DATE_FORMAT

TNS_ADMIN=$ORACLE_HOME/network/admin; export TNS_ADMIN

ORA_NLS11=$ORACLE_HOME/nls/data; export ORA_NLS11

PATH=.:${JAVA_HOME}/bin:${PATH}:$HOME/bin:$ORACLE_HOME/bin
PATH=${PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin
PATH=${PATH}:/u01/app/common/oracle/bin
export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$ORACLE_HOME/oracm/lib
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib
export LD_LIBRARY_PATH

CLASSPATH=$ORACLE_HOME/JRE
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/jlib
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/rdbms/jlib
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/network/jlib
export CLASSPATH

THREADS_FLAG=native; export THREADS_FLAG

export TEMP=/tmp
export TMPDIR=/tmp

umask 022

4. Setting resource limits

Edit the following files:

# /etc/security/limits.conf

[root@orcl ~]# cat >> /etc/security/limits.conf <grid soft nproc 2047
grid hard nproc 16384
grid soft nofile 1024
grid hard nofile 65536

oracle soft nproc 2047
oracle  hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
EOF

# /etc/pam.d/login

[root@orcl ~]# cat >> /etc/pam.d/login <session required pam_limits.so
EOF

# /etc/profile

[root@orcl ~]# cat >> /etc/profile <if [ \$USER = “oracle” ] || [ \$USER = “grid” ]; then
if [ \$SHELL = “/bin/ksh” ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
umask 022
fi
EOF

# /etc/csh.login

[root@orcl ~]# cat >> /etc/csh.login <if ( \$USER == “oracle” || \$USER == “grid” )
then
limit maxproc 16384
limit descriptors 65536
endif
EOF

# /etc/hosts

[root@orcl ~]#  vi /etc/hosts

127.0.0.1 localhost.localdomain localhost
192.168.34.150 orcl

5. Installing ASMlib and creating disk groups

Download link: http://www.oracle.com/technetwork/topics/linux/asmlib/index-101839.html

# To know your kernel version

uname -r
2.6.18-194.el5xen

# Download the following files

oracleasm-support-2.1.7-1.el5.x86_64.rpm
oracleasm-2.6.18-308.el5-2.0.5-1.el5.x86_64.rpm

oracleasmlib-2.0.4-1.el5.x86_64.rpm

# Installing

rpm -Uvh oracleasm-support-2.1.7-1.el5.x86_64.rpm
rpm -Uvh oracleasm-2.6.18-308.el5-2.0.5-1.el5.x86_64.rpm
rpm -Uvh oracleasmlib-2.0.4-1.el5.x86_64.rpm

# Configure ASM

/usr/sbin/oracleasm configure -i

Default user to own the driver interface []: grid
Default group to own the driver interface []: asmadmin
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done

# Load ASM kernel module

/usr/sbin/oracleasm init

##Partition available disks for ASM

#list available disks and partitions

fdisk -l

#partitioning

fdisk /dev/sdb

Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-52216, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-52216, default 52216):
Using default value 52216

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

fdisk /dev/sdc

Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-52216, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-52216, default 52216): +100M

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

# Restart the server

init 6

# Creating ASM disks

/usr/sbin/oracleasm createdisk VOL1 /dev/sdb1
/usr/sbin/oracleasm createdisk CRSVOL /dev/sdc1

# List disks

/usr/sbin/oracleasm listdisks
VOL1
CRSVOL

# Testing disk discovery, that is used by DBCA

oracleasm-discover

Using ASMLib from /opt/oracle/extapi/64/asm/orcl/1/libasm.so
[ASM Library – Generic Linux, version 2.0.4 (KABI_V2)]
Discovered disk: ORCL:CRSVOL [208782 blocks (106896384 bytes), maxio 512]
Discovered disk: ORCL:VOL1 [838849977 blocks (429491188224 bytes), maxio 512]

6. Installing Oracle Grid Infrastructure

Download link: http://download.oracle.com/otn/linux/oracle11g/R2/linux.x64_11gR2_grid.zip

Login as a grid user.

# Unzip and install

mkdir  /home/grid/myinstall
unzip /tmp/linux.x64_11gR2_grid.zip  -d  /home/grid/myinstall

cd /home/grid/myinstall
chmod -R 777 *
./runInstaller

Choose the following options:

6.1. Install and Configure Grid Infrastructure for a Standalone Server

6.2.  English

6.3. English

6.4.
Disk Group Name: CRS
Redundancy: External
Add Disks->Candidate Disks, choose just  ORCL:CRSVOL

6.5. Speciify password(s).

6.6.
ASM Database Administrator(OSDBA) Group : asmdba
ASM Instance Administration Operator(OSOPER) Group: asmoper
ASM Instance Administrator(OSASM) Group: asmadmin

6.7.
Oracle Base: /u01/app/gridSoftware Location: /u01/app/11.2.0/grid

6.8. On the pop-up windows press Yes.

6.9. Inventory Directory: /u01/app/oraInventory

6.10. On the prerequisite checks page, there will be failed checks. To solve:

6.10.1 Press Fix & Check Again
6.10.2 Login as root user and run the following:

[root@orcl ~]# /tmp/CVU_11.2.0.1.0_grid/runfixup.sh

6.10.3 Press OK on the Execute Fixup window

Left just libaio* and unixODBC* packages that should be installed. To solve:

6.10.4 Mount Oracle Enterprise Linux installation disk and go to the Server folder, where rpms are located.

[root@orcl Server]# rpm -Uvh libaio-devel-0.3.106-5.i386.rpm
warning: libaio-devel-0.3.106-5.i386.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing…                ########################################### [100%]
1:libaio-devel           ########################################### [100%]
[root@orcl Server]# rpm -Uvh libaio-devel-0.3.106-5.x86_64.rpm
warning: libaio-devel-0.3.106-5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing…                ########################################### [100%]
1:libaio-devel           ########################################### [100%]
[root@orcl Server]# rpm -Uvh unixODBC-2.2.11-7.1.i386.rpm
warning: unixODBC-2.2.11-7.1.i386.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing…                ########################################### [100%]
1:unixODBC               ########################################### [100%]
[root@orcl Server]# rpm -Uvh unixODBC-2.2.11-7.1.x86_64.rpm
warning: unixODBC-2.2.11-7.1.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing…                ########################################### [100%]
1:unixODBC               ########################################### [100%]
[root@orcl Server]# rpm -Uvh unixODBC-devel-2.2.11-7.1.i386.rpm
warning: unixODBC-devel-2.2.11-7.1.i386.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing…                ########################################### [100%]
1:unixODBC-devel         ########################################### [100%]
[root@orcl Server]# rpm -Uvh unixODBC-devel-2.2.11-7.1.x86_64.rpm
warning: unixODBC-devel-2.2.11-7.1.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing…                ########################################### [100%]
1:unixODBC-devel         ########################################### [100%]

6.10. 5   Press Check Again.(There should not be any error)

6.11.

[root@orcl ~]# /u01/app/oraInventory/orainstRoot.sh
[root@orcl ~]# /u01/app/11.2.0/grid/root.sh

Press enter…

7. Creating ASM disk groups for database

Connect as a grid user.

[grid@orcl ~]$ asmca

7.1 Press Create button.

7.2
Disk Group Name: DATARedundancy: External(None)Select Member Disks->Show Eligible, choose ORCL:VOL1

8. Installing Database, create instance on ASM

Connect as an oracle user.

Download Link: http://www.oracle.com/technetwork/database/enterprise-edition/downloads/index.html?ssSourceSiteId=ocomen

Unzip downloaded file to /home/oracle/myinstall.

[oracle@orcl linux.x64_11gR2_database]$ ./runInstaller

8.1 Uncheck “I with to receive security updates via My Oracle Support”

8.2 On the pop-up window choose yes.

8.3 Install database software only

8.4 Single instance database installation

8.5 English

8.6 Enterprise Edition (4.29GB)

8.7
Oracle Base: /u01/app/oracleSoftware Location:  /u01/app/oracle/product/11.2.0/db_1

8.8
Database Administrator(OSDBA) Group: dba
Database Operator(OSOPER) Group: oper

8.9 I have some failed checks that are fixable so I press Fix & Check Again button, login as a root user and run:

[root@orcl ~]# /tmp/CVU_11.2.0.1.0_oracle/runfixup.sh

Click OK on the Execute Fixup window.

8.10

[root@orcl ~]# /u01/app/oracle/product/11.2.0/db_1/root.sh
Press Enter…
y
y
y

9. Run DBCA as an oracle user.

[oracle@orcl ~]$ dbca

and follow the steps (choose ASM instead of File System option)

That’s all.

Source

Useful VI commands

===================================================================================
Regular Expressions
===================================================================================
. (dot)    Any single character except newline
*            zero or more occurrences of any character
[...]        Any single character specified in the set
[^...]       Any single character not specified in the set
^            Beginning of the line
$            End of line
\<           Beginning of word
\>           End of word
=======================================================================================
1. Add something at the beginning of each line 

:%s/^/@

2. Add something at the end of each line 

:%s/$/.sql

3. Delete blank lines

:g/^$/d
:g/^ *$/d

4.remove all leading white space(s) from all lines
:1,$ s/^\s*//g

5.Remove the ^M characters at the end of all lines (Dos to Unix Conversion)

:%s/^V^M//g  ====The ^v is a CONTROL-V character and ^m is a CONTROL-M. When you type this, it will look like this: 
:%s/^M//g:%s/\r//g : Delete DOS returns ^M " 
Is your Text File jumbled onto one line? use following 
:%s/\r/\r/g : Turn DOS returns ^M into real returns

6. Duplicate every Line

:g/^/t.

7. Delete Duplicate Lines

:%s/^\(.*\)\n\1$/\1/        : delete duplicate lines
:%s/^\(.*\)\(\n\1\)\+$/\1/  : delete multiple duplicate lines [N]

8. Delete blank lines

:v/\S/d                     : Delete empty/blank lines
:g/^\s*$/d                  : delete all blank lines

9. Read lines from external command
:r!ls -ltr

10. Being able to determine the line number of the current line or the total number of 
lines in the file

:.= returns line number of current line at bottom of screen
:=  returns the total number of lines at bottom of screen
^g  provides the current line number, along with the total number of lines,in the file at the 
 bottom of the screen

Find and kill a process in one line using bash

For operation and other pusposes, we need to kill process in linux system. For this purpose, at first we find the process id then we kill the process with the process id. i.e

[agentrpt@localhost ~]$ ps -ef | grep 'jar'
agentrpt 13131     1  0 16:49 ?        00:00:00 java -jar ABS_ALERT_MIDDLEWARE.jar
agentrpt 13278     1  0 16:54 ?        00:00:00 java -jar SMSMailClient.jar
agentrpt 13402 13367  0 16:58 pts/0    00:00:00 grep jar

For killing process of java -jar SMSMailClient.jar, We need to executge command as:
kill 13278

We can do it in a simple way. For the using the script regulary we can create a script with the below command.

In bash, you should be able to do:

kill $(ps aux | grep '[SMS]MailClient.jar' | awk '{print $2}')

Details on its workings are as follows:

The ps gives you the list of all the processes.
The grep filters that based on your search string, [SMS] is a trick to stop you picking up the actual grep process itself.
The awk just gives you the second field of each line, which is the PID.

Remove all files/directories except for one file

find . ! -name 'filetoexist.txt' -type f -exec rm -f {} +

will remove all files except filetoexist.txt. To remove directories, change -type f to -type d and add -r option to rm.

To exclude list of files: ! \( -name one_file -o -name two_file \)

In bash, to use rm !(file.txt), we will have to enable extglob:

$ shopt -s extglob
$ rm !(file.txt)

Note that extglob only works in bash and Korn shell family. And using rm !(file.txt) can cause an Argument list too long error.

In zsh, you can use ^ to negate pattern with extendedglob enabled:

$ setopt extendedglob
$ rm ^file.txt

or using the same syntax with ksh and bash with options ksh_glob and no_bare_glob_qual enabled.

Modifying file using Vim and awk

In some case, we have to change a big file with the same type of text. We can use vim editor for doing this type of task. Suppose, we have a files as below:

Now for a need, we need to generate delete script from this file contents as below:

For doing so, 

1. we have opened the file using vim editor. vim loglistws.txt

2.Then we have used the below commnads to add double quate around the words : 

   :%s/EJB_Str/"EJB_Str/g

   

Then save the file using :w

   :%s/\.log/\.log"/g

3. Now save the file using :wq!

4. Now we have made a new file form the existing file as:

    cat loglistws.txt | awk '{print "rm ",$9,$10,$11,$12}' > myscriptws.sh

To run a command repeatedly in Linux

For many cases, we need to execute the command repeatedly in the shell. We can do it by cron. But in that cat its not feasible to use cron if the interval time is less than 1 min or more. Suppose, we are inclined to get the output like the top command for a specific command. We can do that in the following ways as:

watch Command
# watch free -m [By Default 2 Sec Interval]
# watch -n 10 script.sh [For the interval of 10 Seconds]
Sleep Command:
a) inside For Loop
# for i in {1..10}; do echo -n "This is a test in loop $i "; date ; sleep
b) Inside while Loop
# while true; do echo -n "This is a test of while loop";date ; sleep 5; done