Prakash Sawarkar: Kernel 3.8 Released, how to Compile in Redhat, CenOS and Fedora..

Kernel 3.8 Released, how to Compile in Redhat, CentOS and Fedora.

Friday, 20 November 2009

LVM (Logical Volume Manager) Concept on Linux RHEL / CentOS

 LVM is a method of allocating harddrive space into logical  volumes that can be easily resized instead of partitions.
 With LVM the harddrive (or) set of harddrives are allocated to  one or more physical volumes.
 The physical volumes are combined into volume  groups
 Each volume group is divided into logical volumes which are  assigned mountpoints such as /home and filesystem types such  as   ext3.
To configure LVM

1)Create three LVM partitions
2)Convert them as physical volumes
3)Create volume groups from physical volumes
4)Create logical volumes from volume groups and assign mountpoints

IMPLEMENTATION:
# fdisk /dev/sda
n
+500M
t
<partition number>
8e
w
partprobe
To convert LVM partitions as physical volumes
# pvcreate /dev/sda<partition numbers>
ex: pvcreate /dev/sda{9,10,11}

To view physical volumes
# pvdisplay

To create volume group
# vgcreate <vg name> <partitions>
ex: vgcreate maaza /dev/sda{9,10,11}

To view volume groups
# vgdisplay

To create a logical volume
# lvcreate -L <+size> <vg name> -n <LV name>
ex: lvcreate -L +500M /dev/maaza -n fanta

To view logical volumes
# lvdisplay

To format logical volumes
# mke2fs -j /dev/maaza/fanta

Create a mountpoint and mount logical volume on it
# mkdir /cyber
# mount /dev/maaza/fanta /cyber
# cd /cyber

To extend size of logical volume
# umount <mountpoint>
# lvcreate -L <size> <lv name>
ex:lvcreate -L +500M /dev/maaza/fanta

To make filesystem for extended size
# resize2fs <logical volume>
ex: resize2fs /dev/maaza/fanta
# mount /dev/maaza/fanta /cyber

To resize a logical volume
note:whenever we are reducing an LVM we have to take backup
# mkdir /a
# cp -rf /cyber/* /a

# lvreduce -L <-size> <LVM>
ex: lvreduce -L -100M /dev/maaza/fanta

To format LVM
# mkfs.ext3 <logical volume>
ex:mkfs.ext3 /dev/maaza/fanta

# mount /dev/maaza/fanta /cyber
# cp -rf /a/* /cyber

To remove an LVM
# umount <mountpoint>
# lvremove <logical volume>
ex:lvremove /dev/maaza/fanta

To extend volume group
1)create another LVM partition
2)convert into physical volume

# vgextend <vg name> <partition name>
ex:vgextend /dev/maaza /dev/sda12

To reduce volume group
# vgreduce <vgname> <partition name>
ex:vgreduce /dev/maaza /dev/sda12

To remove volume group
# vgremove <vg name>
ex:vgremove /dev/maaza

To delete physical volumes
# pvremove <partitions>

ex:pvremove /dev/sda{9,10,11,12}

Monday, 28 September 2009

DNS Configuration in CentOS /Redhat

DNS is a service which is used to resolve host to IP address and zone records & configuration files.

MASTER DNS

There are two types of zonerecords
1) Forwad lookup zone
2) Reverse lookup zone

 By defaults, computer connects to another computer with the help of IP address


Forward lookup zone :  It converts names into IP addresses

Reverse lookup zone: It converts IP addresses to names
Steps:
# yum install bind-* caching-nameserver*-y
# service named start
# chkconfig named on
# cd /var/named/chroot/etc
# ls
# cp named-caching-nameserver named.conf
# vi named.conf
  delete ipv6 lines (line nos. 16 &22)

(line no15) Listen on port 53 {127.0.0.1; 192.168.0.254;}
(here 192.168.0.254 is server ip
(line no 21) allow-query  192.168.0.0/24 (clients range)
(line no 30) match-clients {local host; 192.168.0.0/24; } (here 192.168.0.0/24 is clients ip range & subnet mask)
(line no 31) match-destinations    {localhost; 192.168.0.0/24;}
save&quit

#  vi /etc/rfc1912.zones
 copy ten lines from 21 to 31 and paste under 31
 change as follows
 zone "redhat.com" IN { 
  type master;
  file "redhat.for"
   allow-update { none; };
};
zone "0.168.192.IN-addr-arpa IN {
   type master;
   file "redhat.rev"
    allow-update { none; };
};

(here redhat.com is domain name and 0.168.192. is redhat.coms network range redhat.for is forward look up zone & redhat.rev is reverse lookup zone)
save & quit
#  chgrp named named.conf
# chgrp named named.rfc1912.zones
# cd /var/named/chroot/var/named
# cp localhost.zone redhat.for
# cp named.local redhat.rev
#  vi redhat.for

change as follows
$TTL    86400
@               IN SOA  redhat.com.       root.redhat.com. (
                                        42              ; serial (d. adams)
                                        3H              ; refresh
                                        15M             ; retry
                                        1W              ; expiry
                                        1D )            ; minimum

                IN NS           server1.redhat.com.
server1.redhat.com.             IN A            192.168.0.254
www254.redhat.com.           IN CNAME        server1.redhat.com.
station1.redhat.com.           IN A            192.168.0.1
www1.redhat.com.              IN CNAME        station1.redhat.com.
station2.redhat.com.          IN A            192.168.0.2
www2.redhat.com.             IN CNAME        station2.redhat.com.
xxx2.redhat.com.              IN CNAME        station2.redhat.com.
yyy2.redhat.com.              IN CNAME        station2.redhat.com.
station3.redhat.com.        IN A            192.168.0.3
www3.redhat.com.           IN CNAME        station3.redhat.com.
station4.redhat.com.        IN A            192.168.0.4
www4.redhat.com.           IN CNAME        station4.redhat.com.
station5.redhat.com.        IN A            192.168.0.5
www5.redhat.com.           IN CNAME        station5.redhat.com.
station6.redhat.com.            IN A            192.168.0.6
www6.redhat.com.                IN CNAME        station6.redhat.com.



(zone: zone is a storage database which contains all zone records

forward lookup zone: used for resolving hostname to ipaddress & it maintains host to ip mapping information
reverse lookup zone: used for resolving ip address to hostname & it maintains ip to hostname mapping information

Types of records: 
SOA : sort of authority the first record in any zone  it indicates who is authority for this domain
NS :nameserver it identifies the dns server for each zone
A record : resolves hostname to ip address
CNAME record : resolves an alias name to a hostname
PTR record : resolves an ipaddress to a hostname
MX record : resolves mail server ip (used by mail server)
TTL :time to live)

save & quit

# vi redhat.rev
(change as follows)
$TTL    86400
@       IN      SOA redhat.com.    root.redhat.com.  (
                                      1997022700 ; Serial
                                      28800      ; Refresh
                                      14400      ; Retry
                                      3600000    ; Expire
                                     86400 )    ; Minimum
            IN        NS      server1.redhat.com.
254      IN       PTR     server1.redhat.com.
1         IN       PTR     station1.redhat.com.
2        IN       PTR      station2.redhat.com.
3         IN      PTR     station3.redhat.com.
4         IN      PTR     station4.redhat.com.
5         IN      PTR     station5.redhat.com.
6         IN      PTR   station6.redhat.com.

save & quit

#  chgrp named redhat.for
#  chgrp named redhat.rev
#  service named restart

 to check:
# dig server1.redhat.com
# dig -x 192.168.0.1
 (if answer is 1 server is ready if answer is 0 server has some error)

to check error in configuration file
# named-checkconf redhat.com /var/named/chroot/etc/named.conf

to check errors in zone record 
# named-checkzone redhat.com /var/named/chroot/var/named/redhat.for
# named-checkzone redhat.com /var/named/chroot/var/named/redhat.rev

 SLAVE DNS

 Master DNS Server
It is the Master Copy of all the Zone Information.
It is Read/Write copy.

Slave DNS Server
It is Slave Backup of Master zone. It is Read Only

        if any error may occur to your dns server at the time the entir network will stop.sometimes it may cause huge damage.for that one we are createing slave dns for faulttolerance and load balancing.

we need another system which contains server o/s

Steps:

# yum install bind-* caching-nameserver -y
# service named start
# chkconfig named on
# cd /var/named/chroot/etc/named-rfc1912.zone

(change as follows)
copy 10 lines from 21 to 31 paste under 31
zone "redhat.com"
  type slave;
  file "redhat.for"
  masters {192.168.0.254:};

zone "0.168.192. IN-ADDR-arpa" IN {
  type slave ;
  file " redhat.rev"
  masters {192.168.0.254;};

save& quit

#service named restart
go to client 
# vi /etc/resolv.conf

nameserver 192.168.0.254
nameserver 192.168.0.1 (slave dns ip)  

FORWARDERS

If you have trusted relationship with another company  those comapny users can enter into our network & our company users can enter into their network by using this forwarders
Steps:
in master dns server 
#  vi /var/named/chroot/etc/named.conf

add aline 
 forwarders {192.168.10.254:};
 forward only ;
};
(here 192.168.10.254 is trusted companies dns)
save & quit

# service named restart

Sunday, 5 July 2009

Reinstall GRUB Bootloader on md0

After a power outage, an RHEL 4 server was stuck with nothing on the screen but "GRUB" at the top left. Rebooting had the same result. My bootloader was toast.

I put in the RHEL 4 install disc, and at the boot prompt selected the rescue option.

boot: linux rescue

Once the rescue image booted, scanned the system and mounted read/write, I attempted to reinstall grub:

# chroot /mnt/sysimage/
# grub-install /dev/sda
/dev/md0 does not have any corresponding BIOS drive
# grub-install --recheck /dev/sda
/dev/md0 does not have any corresponding BIOS drive
#
An error. The md0 mirror, which is composed of sda and sdb, was complicating things. I though maybe I could fudge the device map by adding an entry to /boot/grub/device.map:

(fd0)   /dev/fd0
(hd0)   /dev/sda
(hd1)   /dev/sdb
(hd2)   /dev/md0

# grub-install /dev/sda
The file /boot/grub/stage1 not read correctly.
Fixed. I then also installed the bootloader on sdb (hd1), just in case.
That was no good either. Finally, I had to run grub manually.
# grub
grub> root (hd0,0)        
root (hd0,0)
 Filesystem type is ext2fs, partition type 0xfd
grub> setup (hd0)
setup (hd0)
 Checking if "/boot/grub/stage1" exists... no
 Checking if "/grub/stage1" exists... yes
 Checking if "/grub/stage2" exists... yes
 Checking if "/grub/e2fs_stage1_5" exists... yes
 Running "embed /grub/e2fs_stage1_5 (hd0)"...  16 sectors are embedded.
succeeded
 Running "install /grub/stage1 (hd0) (hd0)1+16 p (hd0,0)/grub/stage2 /grub/grub.conf"... succeeded
Done.
grub> quit
# reboot

Fixed. I then also installed the bootloader on sdb (hd1), just in case.

Monday, 29 June 2009

Setting up a new LAN CARD (Ethernet) network device

If you added a new network device, but upon reboot, CentOS did not represent the card in the /etc/sysconfig/network-scripts/ifcfg-eth*, you may have to manually make aware the device to the OS. 

Step 1: Make sure that the OS has detected the new card. From the example below, it detected my 2nd card. If you are unable to detect, you may want to reinstall or check the driver.

#  lspci |grep -i ethernet
   0b:00.0 Ethernet controller: VMware VMXNET3 Ethernet Controller (rev 01)
   14:00.0 Ethernet controller: VMware VMXNET3 Ethernet Controller (rev 01)

Step 2: Launch the system-config-network and fill up the necessary information.
# system-config-network
You will see that the system-config-network create an entry at /etc/sysconfig/network-scripts/ifcfg-eth1.
Restart the network
# service network restart

Sunday, 31 May 2009

How DNS Works (Domain Name System)

In any network, the hosts primarily communicate between each other through IP addresses. For example, if my computer is doing a google search, my computer is actually communicating with the IP address of one of the web servers of google.com. However, even if the computer is efficient with numbers, humans on the other hand work better with names. For this reason, the TCP/IP protocol includes the Domain Name System (DNS)  to link between IPs and computer names i.e. hostnames. The DNS is a distributed database of computers that is responsible for resolving hostnames against IP addresses and vice-versa.

Any DNS query involves two parts.
The Resolver: The resolver forms up or initiates the query. The resolver itself does not run as a program. /etc/resolv.conf is an example of a resolver.
Name Server: The Name Server is the service running in the server that responds to the DNS query generated by the resolver i.e. answers to the question of the resolver.

Fully Qualified Domain Name

The fully qulified domain name is the full name of any server. Just like any human needs a full name in the real world, every server on the Internet also need a full name to work. The structure of a FQDN- host.domain.tld. For example, in www.qwe.net “www” is the hostname of the web server, “qwe” is the name of the domain and “net” is the top level domain (TLD). Other examples of TLD are .com, .in,.net, .org, .gov,.oc.in, .mil and so on.

The Root-Servers

As mentioned earlier, the DNS works as a distributed database. If a DNS server does not know the answer to a query, it forwards the query to another server upper in the hierarchy. The query keeps going upwards until it reaches the root. There are 13 root servers responsible for all DNS in the world. The root servers are named as a.root-servers.net to m.root-servers.net. These root servers continuously keep communicating with each other and update each other about what they know. Here is an interesting article about the root DNS server numbers: http://blog.icann.org/2007/11/there-are-not-13-root-servers/

Authoritative Name Servers

The authoritative name servers are servers that are responsible for a domain. For example, if we host the DNS for a domain qwe.net, then the domain requires at least 2 authoritative DNS servers i.e. ns1.qwe.net & ns2.qwe.net.

These two authoritative DNS servers are responsible for any DNS query about the qwe.net domain and should be able to answer any query regarding this domain. The root servers store only the records for the authoritative name servers for different domains.


How DNS Works

I found this step-by-step image in the web, and thought it's worth sharing. Here is the link to the full article- http://www.communityguy.ca/resources/cira-2009-elections-and-an-overview-of-how-dns-works/






















How DNS works (Reference)

And now for the step-by-step analysis.

The client initiates a query to find techsmb.ca. The client sends the query to the DNS server of the ISP. (The DNS Server IP in the client computer is set as the IP address of the DNS Server of the ISP)
The DNS Server of the ISP first checks it's own cache to check whether it already knows the answer. But as the answer is not present, it generates another query. As the TLD of techsmb.ca is .ca, so the DNS server queries CIRA to find who is responsible for techsmb.ca.

The CIRA responds to the ISP by answering the query.


Once the ISP DNS Server knows the authoritative name servers, it contacts the authoritative name servers to find out the IP address for www.techsmb.ca. i.e. the IP address of host www in the domain techsmb.ca.

techsmb.ca responds to the ISP DNS Server by answering the query and providing the IP address of the web server i.e. www
The ISP DNS Server stores the answer in it's cache for future use and answers to the client by sending the IP address of the www server.
The client may store the answer to the DNS query in it's own cache for future use. Then the client communicates directly with the www server of domain techsmb.ca using the IP address.
The www server responds by sending the index.html page.

Sunday, 19 April 2009

Reducing.. Kernel Page cache, Inode and dentry caches.

Kernel 2.6 never provide a mechanism to have the kernel drop the page cache and/or inode and dentry caches on command, which can help free up a lot of memory.

Writing to this will cause the kernel to drop clean caches, dentries and inodes from memory, causing that memory to become free.

To free pagecache:
#  echo 1 > /proc/sys/vm/drop_caches

To free dentries and inodes:
#  echo 2 > /proc/sys/vm/drop_caches

To free pagecache, dentries and inodes:
#  echo 3 > /proc/sys/vm/drop_caches

As this is a non-destructive operation, and dirty objects are not free-able, the user should run "sync" first in order to make sure all cached objects are freed.

Wednesday, 1 April 2009

How to speed up SSH login?

Whenever I login to my Centos Server using SSH, it takes about 6 seconds for me to get the prompt for password, however when I login to my web hosting server it takes about 1 second. What can I do to speed this up?

If you run a lot of terminal tabs or scripts that all need to make OpenSSH connections to the same server, you can speed them all up with multiplexing: making the first one act as the master and letting the others share its TCP connection to the server.

If you don't already have a config file in the .ssh directory in your home directory, create it with permissions 600: readable and writeable only by you.
Speed up SSH 

Step 1 -Add below to /etc/ssh/sshd_config

The number one reason I've seen for this is a configuration option in SSHD UseDNS this option (enabled by default) causes the server to perform DNS resolution on the incoming requests. A time consuming operation. I've seen logins go from one minute plus waiting for password 


#  vi /etc/ssh/sshd_config

UseDNS no
Step 2- Setting up ssh client with compression 

#  vi /etc/ssh/ssh_config

Host *
Ciphers arcfour,blowfish-cbc
Compression yes
AddressFamily inet
ControlMaster auto
ControlPath ~/.ssh/socket-%r@%h:%p

Save and exit !
Step 3- Restart the SSH daemon with service sshd restart

You should see an improvement next time you connect.

Wednesday, 25 March 2009

Linux EXT2/EXT3 Superblock Recovery

Each file system has one superblock, which contains information about file system such as file system type, size, status and information about other metadata structures. If it lost, you would be in trouble so Linux maintains multiple redundant copies of the superblock in every file system.

I read the article “Surviving a Linux Filesystem Failures” on nixCraft, and did the test that destroyed and also recovered the ext2 and ext3 filesystem superblock on RHEL/Centos. The following is the steps.

EXT2 filesystem

In my machine, there is an ext2 filesystem /dev/sda6 which its mount point is /fs2.

Firstly, I should know the location of superblocks and the size of block.
[root@localhost ~]#   dumpe2fs /dev/sda6 | grep –i superblock
dumpe2fs 1.40.8 (23-Mar-2009)
  Primary superblock at 1, Group descriptors at 2-2
  Backup superblock at 8193, Group descriptors at 8194-8194
  Backup superblock at 24577, Group descriptors at 24578-24578
  Backup superblock at 40961, Group descriptors at 40962-40962
  Backup superblock at 57345, Group descriptors at 57346-57346
  Backup superblock at 73729, Group descriptors at 73730-73730
[root@localhost ~]#   dumpe2fs /dev/sda6 | grep –i ‘block size’
dumpe2fs 1.40.8 (23-Mar-2009)
Block size:                               1024 

From the above, I knew there are 5 backup superblocks and the block size is 1024 bytes. The location of primary superblock is at 1. The locations of these 5 backup superblocks are at 8193,24577,40961,57345 and 73729 respectively. If you want to know the filesystem’s block size, there are another two ways that you can use tune2fs and ext3grep commands. For example,
[root@localhost ~]#   tune2fs –l dev/sda6 | grep –i ‘block size’
[root@localhost ~]#   ext3grep /dev/sda6 | grep –i ‘block size’ 

Then, I destroyed the superblock as below.
[root@localhost ~]#   dd if=/dev/zero count=1 bs=1024 seek=1 of=/dev/sda6 

I checked the effect, and found I still could change to /fs2 directory. I could still list its contents. But the system gave me error messages when I tried to create a file or copy files.
[root@localhost ~]#   cd /fs2
[root@localhost ~]#   ll
total 19
drwx—— 2 root root 12288 2009-03-22 08:57 lost+found
-rw-r—r—1 root root   135 2009-03-22 18:15 qaz
 … … 

[root@localhost fs2]#  cp qaz wsx
cp: overwrite  `wsx’? y
cp: writing `wsx’: No space left on device
 [root@localhost fs2]#  cp qaz edc
cp: cannot create regular file `edc’: Input/output error 

When I run the dumpe2fs command, it told me the superblock had been destroyed.
[root@localhost fs2]#  dumpe2fs /dev/sda6
dumpe2fs 1.40.8 (23-Mar-2009)
dumpe2fs: Bad magic number in super-block while trying to open /dev/sda6
Couldn’t find valid filesystem superblock. 

I recovered the superblock.
[root@localhost ~]#   dd if=/dev/sda6 count=1 bs=1024 skip=8193 seek=1 of=/dev/sda5

After that, I checked if I can create and copy files. I found they worked. And the command dumpe2fs could display normally. 

EXT3 filesystem

In my machine, there is another ext3 filesystem /dev/sda5 which its mount point is /fs3.

Check the location of superblocks and the size of block.
[root@localhost ~]#   dumpe2fs /dev/sda5 | grep –i superblock
dumpe2fs 1.40.8 (23-Mar-2009)
  Primary superblock at 0, Group descriptors at 1-1
  Backup superblock at 32768, Group descriptors at 32769-32769
  Backup superblock at 98304, Group descriptors at 98305-98305
  Backup superblock at 163840, Group descriptors at 163841-163841
  Backup superblock at 229376, Group descriptors at 229377-22937
[root@localhost ~]#  dumpe2fs /dev/sda5 | grep –i ‘block size’
dumpe2fs 1.40.8 (23-Mar-2009)
Block size:                               4096 

From the above, I knew there are 4 backup superblocks and the block size is 4096 bytes. The location of primary superblock is at 0. The locations of these 4 backup superblocks are at 32768, 98304, 163840, and 229376.

Destroy the superblock.
[root@localhost ~]#   dd if=/dev/zero count=1 bs=4096 seek=0 of=/dev/sda5 

I found I still can change to /fs3 directory. But when I listed its contents, there is nothing. I used dumpe2fs command to check, it told me the superblock has problem. Then I tried to recover the superblock like the above approach, however, it did not work. I restarted the system. The system displayed it unable to resolve the filesystem and could not boot up. I entered the maintenance mode and tried to recover using the method as below. It did not work.
(Repair filesystem) 1 #   e2fsck –f –b 32768 /dev/sda5
e2fsck 1.40.8 (23-Mar-2009)
e2fsck: Device or resource busy while trying to open /dev/sda5
Filesystem mounted or opened exclusively by another program? 

Then I used another method, after I reboot the system it worded.
(Repair filesystem) 2 #   e2fsck –f  /dev/sda5
(Repair filesystem) 3 #   reboot 

From the test, I realized that the information of the filesystem’s superblock is very important. We’d better backup it to the root because we can still read it through maintenance mode even if the system can not boot up normally. Don’t forget run the following command.
#  dumpe2fs /dev/sda5 > /dumpe2fs-sda5 

Thursday, 19 February 2009

How To Convert VMWare Image (.vmdk) to VirtualBox Image (.vdi) on Linux

QEMU is free software written by Fabrice Bellard that implements a fast processor emulator, allowing a user to run one operating system within another one. It is similar to projects such as Bochs and VMware Workstation
First we need to install QEMU
# yum install qemu
Now using qemu, first we need to convert a .vmdk (VMware image) to a .bin format, which can then be converted to a Virtualbox native .vdi format.
# qemu-img convert /path/to/original.vmdk converted.bin
VirtualBox is a free, powerful and versatile virtualization program which is available for Linux, Mac, and Windows hosts, and can virtualize many different Operating Systems.
VirtualBox was originally developed by Innotek, but was purchased by Sun and renamed Sun xVM VirtualBox.
Now using VBoxManage utility that comes with Virtualbox we can easily convert the .bin file that we have generated using qemu to a native .vdi format:
#VBoxManage convertdd converted.bin converted.vdi