Monitoring n/w traffic using speedometer

Speedometer Monitor network traffic or speed/progress of a file transfer.

How long it will take for 100 mb transfer to finish
How quickly is another transfer going

 Speedometer can print the RX and TX rates on a per-interface basis.

# sudo apt-get install speedometer

# speedometer -rx eth0 -tx eth0

BitMeter OS for bandwidth and data transfer monitor

BitMeter OS keeps track of how much you use your Internet/network connection.

You can monitor your connection usage either via a web browser, or by using the command line tools. The Web Interface displays various graphs and charts that show how your Internet / network connection has been used over time.

Download the package and install it

 bitmeteros_0.7.5-i386.deb
 bitmeteros_0.7.5-amd64.deb

# dpkg -i  bitmeteros_0.7.5-i386.deb

After sucessful installation you can access the BitMeter OS web interface using  web-browser:  http://localhost:2605

The following commands will stop/start/restart the OS BitMeter Web Interface in Linux

sudo /etc/init.d/bitmeterweb stop
sudo /etc/init.d/bitmeterweb start
sudo /etc/init.d/bitmeterweb restart

The below screen shoots showing different options available in  BitMeter

vnStat

vnstat is an excellent and simple tool to monitor your server or interface bandwidth and the results can be displayed both on the server console and on the web interface with vnStat phpFrontend.

 rpm available at : http://packages.sw.be/vnstat/

# rpm -ivh vnstat-1.7-1.el5.rf.i386.rpm

After you install, just edit the /etc/vnstat.conf config file and setup the following lines with according to your needs.

# default interface
  Interface “eth0″

If you want to store it in a separate database or directory just edit the following lines and change it to your new path/directory.

# location of the database directory
    DatabaseDir “/var/lib/vnstat”

Running vnstat as a deamon

 Go to http://humdi.net/vnstat/init.d/

Download the corerct os version of your server and copy it inside the /etc/init.d/. Then enable the daemon by

  # chmod 755 /etc/init.d/vnstat
 # chkconfig vnstat on
 # service vnstat start

Installing the vnStat php Frontend

Download the vnstat php frontend tool directly from http://www.sqweek.com/sqweek/files/vnstat_php_frontend-1.5.1.tar.gz. Copy or move the folder to your document-root directory (say for example /var/www/html)

Edit the following lines in the config.php to setup the language and the interface list and title to finish the setup

$language = ‘en’;
$iface_list = array(‘eth1′, ‘sixxs’);
$iface_title['eth1'] = ‘Internal’;
$iface_title['sixxs'] = ‘SixXS IPv6′; 

access the vnstat url now to display the stats. It is advisable to setup a password protect directory to avoid unauthorized access. 

Access the gui of vnstat as below

http://192.168.1.67/vnstat

We can get all these information from console itself

Statistics by day:

# vnstat -d

By hour

# vnstat -h

By month

# vnstat -m

Live monitoring of the network interface

# vnstat -l

show top10
# vnstat -t

update database
# vnstat -u

calculate traffic
# vnstat -tr

In the output  Rx and Tx were abbreviation's for Receive and Transmit. Data is crossing or traveling over a cable or through a signal in the air.

 

Vim commandline Tricks

ctrl+u  -------> undo in vim editor

:w filename -------> save as filename without exiting from vim

vim ctechz +25 -----> open file and go to line 54, any : command can be run using + on command line

vim -O ctechz1 ctechz2  -----> open ctechz1 and ctechz2 side by side

:ls    -----> list buffers

:bd  ----->  delete buffer and any associated windows

Ctrl+g    ----->  Show file info including your position in the file

vi filename   ----> Opening a file / Creating text

Edit modes: These keys enter editing modes and type in the text of your document. 

i     Insert before current cursor position
I     Insert at beginning of current line
a     Insert (append) after current cursor position
A     Append to end of line
r     Replace 1 character
R     Replace mode
<ESC> Terminate insertion or overwrite mode

Deletion of text:

x       Delete single character
dd     Delete current line and put in buffer
ndd   Delete n lines (n is a number) and put them in buffer
J        Attaches the next line to the end of the current line (deletes carriage
          return).

Undo:

u     Undo last command

Cut and Paste:

yy     Yank current line into buffer
nyy   Yank n lines into buffer
p       Put the contents of the buffer after the current line
P       Put the contents of the buffer before the current line

Cursor Positioning:

^d     Page down
^u     Page up
:n      Position cursor at line n
:$      Position cursor at end of file
^g     Display current line number
h,j,k,l Left,Down,Up, and Right respectivly. Your arrow keys should also work if
                                                your keyboard mappings are anywhere near sane.

String Substitution:

:n1,n2:s/string1/string2/[g]       Substitute string2 for string1 on lines
                                                 n1 to n2. If g is included (meaning global), 
                                                 all instances of string1 on each line
                                                 are substituted. If g is not included,
                                                  only the first instance per matching line is
                                                 substituted.

   ^  matches start of line
    .  matches any single character
    $  matches end of line

These and other "special characters" (like the forward slash) can be "escaped" with \ i.e to match the string "/usr/STRIM100/SOFT" say "\/usr\/STRIM100\/SOFT".

Saving and Quitting and other "ex" commands:

These commands are all prefixed by pressing colon (:) and then entered in the lower left corner of the window. They are called "ex" commands because they are commands of the ex text editor - the precursor line editor to the screen editor  vi. You cannot enter an "ex" command when you are in an edit mode (typing text onto the screen)
Press <ESC> to exit from an editing mode.

:w                      Write the current file.
:w new.file         Write the file to the name 'new.file'.
:w! existing.file  Overwrite an existing file with the file currently being edited.
:wq                     Write the file and quit.
:q                        Quit.
:q!                       Quit with no changes.

:e filename          Open the file 'filename' for editing.

:set number         Turns on line numbering
:set nonumber      Turns off line numbering

HA setup using HeartBeat and Squid

A computer cluster is a group of linked computers, working together closely so that in many respects they form a single computer.
 

Clusters are usually deployed to improve performance and/or availability over that of a single computer, while typically being much more cost-effective than single computers of comparable speed or availability.

Cluster terminology

Node : It’s one of the system/computer which participates with other systems to form a Cluster.

Heartbeat : This a pulse kind of single which is send from all the nodes at regular intervals using a UDP packet so that each system will come to know the status of availability of other node.

It’s a kind of door knocking activity like pinging a system, So that each node which are participating in Cluster will come to know the status of other nodes availability in the Cluster.

Floating IP or Virtual IP : This is the IP assigned to the Cluster through which user can access the services. So when ever clients request a service they will be arrived to this IP, and client will not know what are the back-end/actual ip addresses of the nodes. This virtual IP is used to nullify the effect of nodes going down.

Master node : This is the node most of the time where services are run in a High availability Cluster.

Slave node  : This is the node which is used in High availability Cluster when master node is down. It will take over the role of servicing the users, when it will not receive heartbeat pulse from master. And automatically gives back the control when the master server is up and running. This slave comes to know about the status of master through heartbeat pulse/signals.  

Types of Clusters 

Cluster types can be divided in to two main types

1. High availability : These types of Clusters are configured where there should be no downtime. If one node in the cluster goes down second node will take care of serving users without interrupted service with availability of five nines i.e. 99.999%.

2. Load balancing :  These types of Clusters are configured where there are high loads from users. Advantages of load balancing are that users will not get any delays in their request because load on a single system is shared by two or more nodes in the Cluster.

HeartBeat Configuration files Details

Three main configuration files :

/etc/ha.d/authkeys

/etc/ha.d/ha.cf

/etc/ha.d/haresources

Some other configuration files/folders to know :

/etc/ha.d/resource.d.  Files in this directory are very important  which contains scripts to start/stop/restart a service run by this Heartbeat cluster.

Before configuration of Heartbeat Cluster these below points to be noted.

Note1 : The contents of ha.cf file are same in all the nodes in a cluster,except ucast and bcast derivatives.

Note2 : The contents of authkeys and haresources files are exact replica on all the nodes in a cluster.

Note3 : A cluster is used to provided a service with high availability/high performance, that service may be a web server, reverse proxy or a Database.

Test scenario setup

1. The cluster configuration which I am going to show is a two node cluster with failover capability for a Squid reverse proxy..

2. For Squid reverse proxy configuration please click here.

3. Node details are as follows.

Node1 

IpAddress(eth0):10.77.225.21
Subnetmask(eth0):255.0.0.0
Default Gateway(eth0):10.0.0.1
IpAddress(eth1):192.168.0.1(To send heartbeat signals to other nodes)
Sub net mask (eth1):255.255.255.0
Default Gateway (eth1):None(don’t specify any thing, leave blank for this interface default gateway).

Node2 

IpAddress(eth0):10.77.225.22
Subnetmask(eth0):255.0.0.0
Default Gateway (eth0):10.0.0.1
IpAddress(eth1):192.168.0.2(To send heartbeat signals to other nodes)
Sub net mask (eth1):255.255.255.0
Default Gateway(eth1):None(don’t specify any thing, leave blank for this interface default gateway).

 tyle=”font-family: verdana;”>4. Floating Ip address:10.77.225.20

Lets start configuration of Heartbeat cluster. And make a note that ever step in this Heartbeat cluster configuration is divided in two parts 

1.(configurations on node1)
2.(configurations on node2)

For better understanding purpose

Step1 : Install the following packages in the same order which is shown. If you did not find the packages online you can download it from our site, click here to download the packages.

Step1(a) : Install the following packages on node1
#rpm -ivh heartbeat-2.1.2-2.i386.rpm
#rpm -ivh heartbeat-ldirectord-2.1.2-2.i386.rpm
#rpm -ivh heartbeat-pils-2.1.2-2.i386.rpm
#rpm -ivh heartbeat-stonith-2.1.2-2.i386.rpm

Step1(b) : Install the following packages on node2
#rpm -ivh heartbeat-2.1.2-2.i386.rpm
#rpm -ivh heartbeat-ldirectord-2.1.2-2.i386.rpm
#rpm -ivh heartbeat-pils-2.1.2-2.i386.rpm
#rpm -ivh heartbeat-stonith-2.1.2-2.i386.rpm

Step2 : By default the main configuration files (ha.cf, haresources and authkeys) are not present in /etc/ha.d/ folder we have to copy these three files from /usr/share/doc/heartbeat-2.1.2 to /etc/ha.d/

Step2(a) : Copy main configuration files from /usr/share/doc/heartbeat-2.1.2 to /etc/ha.d/ on node 1
#cp /usr/share/doc/heartbeat-2.1.2/ha.cf /etc/ha.d/
#cp /usr/share/doc/heartbeat-2.1.2/haresources /etc/ha.d/
#cp /usr/share/doc/heartbeat-2.1.2/authkeys /etc/ha.d/

Step2(b) : Copy main configuration files from /usr/share/doc/heartbeat-2.1.2 to /etc/ha.d/ on node 2
#cp /usr/share/doc/heartbeat-2.1.2/ha.cf /etc/ha.d/
#cp /usr/share/doc/heartbeat-2.1.2/haresources /etc/ha.d/
#cp /usr/share/doc/heartbeat-2.1.2/authkeys /etc/ha.d/

Step3 : Edit ha.cf file
# vi /etc/ha.d/ha.cf

Step3(a) : Edit ha.cf file as follows on node1
debugfile /var/log/ha-debug
logfile /var/log/ha-log
logfacility local0
keepalive 2
deadtime 25
warntime 10
initdead 50
udpport 694
bcast eth1
ucast eth1 192.168.0.1
auto_failback on
node rp1.linuxnix.com
node rp2.linuxnix.com

Step3(b) : Edit ha.cf file as follows on node2
debugfile /var/log/ha-debug
logfile /var/log/ha-log
logfacility local0
keepalive 2
deadtime 25
warntime 10
initdead 50
udpport 694
bcast eth1
ucast eth1 192.168.0.2
auto_failback on
node rp1.linuxnix.com
node rp2.linuxnix.com

Let me explain each entry in detail:

Debugfile : This is the file where debug info with good details for your heartbeat cluster will be stored, which is very much useful to do any kind of troubleshooting. 

Logfile : This is the file where general logging of heartbeat cluster takes place.

Logfacility : This directive is used to specify where to log your heartbeat logs(if its local that indicates store logs locally or if it’s a syslog then store it on remote server and none to disable logging). And there are so many other options, please explore yourself.

Keepalive : This directive is used to set the time interval between heartbeat packets and the nodes to check the availability of other nodes. 

Deadtime : A node is said to be dead if the other node didn’t receive any update form it.

Warntime : Time in seconds before issuing a “late heartbeat” warning in the logs.

Initdead : With some configurations, the network takes some time to start working after a reboot. This is a separate “deadtime” to handle that case. It should be at least twice the normal deadtime.

Udpport : This is the port used by heartbeat to send heartbeat packet/signals to other nodes to check availability(here in this example I used default port:694).

Bcast : Used to specify on which device/interface to broadcast the heartbeat packets.

Ucast : Used to specify on which device/interface to uni-cast the heartbeat packets.

auto_failback : This option determines whether a resource will automatically fail back to its “primary” node, or remain on whatever node is serving it until that node fails, or an administrator intervenes. In my example I have given as on that indicate if the failed node come back online, control will be given to this node automatically. Let me put it in this way. I have two nodes node1 and node2. My node one machine is a high end one and node is for serving temporary purpose when node 1 goes down. Suppose node1 goes down, node2 will take the control and serve the service, and it will check periodically for node1 starts once it find that node 1 is up, the control is given to node1.

Node : This is used to specify the participated nodes in the cluster. In my cluster only two nodes are participating (rp1 and rp2) so just specify that entries. If in your implementation more nodes are participating please specify all the nodes.

Step4 : Edit haresources file
# vi /etc/ha.d/haresources

Step4(a) : Just specify below entry in last line of this file on node1
rp1.linuxnix.com 10.77.225.20 squid

Step4(b) : Just specify below entry in last line of this file on node1
rp1.linuxnix.com 10.77.225.20 squid

Explanation of each entry :
rp1.linuxnix.com is the main node in the cluster
10.77.225.20 is the floating ip address of this cluster.

Squid : This is the service offered by the cluster. And make a note that this is the script file located in /etc/ha.d/ resource.d/.

Note : By default squid script file will not be there in that folder, I created it according to my squid configuration.

What actually this script file contains

This is just a start/stop/restart script for the particular service. So that heartbeat cluster will take care of the starting/stopping/restarting of the service(here its squid). Here is what squid script file contains.

Step5 : Edit authkeys file, the authkeys configuration file contains information for Heartbeat to use when authenticating cluster members. It cannot be readable or writeable by anyone other than root. so change the permissions of the file to 600 on both the nodes.

Two lines are required in the authkeys file:
A line which says which key to use in signing outgoing packets.
One or more lines defining how incoming packets might be being signed.

Step5 (a) : Edit authkeys file on node1
#vi /etc/ha.d/authkeys
auth 2
#1 crc
2 sha1 HI!
#3 md5 Hello!
Now save and exit the file

Step5 (b) : Edit authkeys file on node2
#vi /etc/ha.d/authkeys
auth 2
#1 crc
2 sha1 HI!
#3 md5 Hello!
Now save and exit the file

Step6 : Edit /etc/hosts file to give entries of host-names for the nodes

Step6 (a) : Edit /etc/hosts file on node1 as below

10.77.225.21 rp1.linuxnix.com rp1
10.77.225.22 rp2.linuxnix.com rp2

Step6 (b) : Edit /etc/hosts file on node2 as below

10.77.225.21 rp1.linuxnix.com rp1
10.77.225.22 rp2.linuxnix.com rp2

Step7 : Start Heartbeat cluster

Step7 (a) : Start heartbeat cluster on node1
#service heartbeat start

Step7 (b) : Start heartbeat cluster on node2
#service heartbeat start

Checking your Heartbeat cluster:
If your heartbeat cluster is running fine a Virtual Ethernet Interface is created on node1 and 10.77.225.20 Clipped output of my first node.
# ifconfig

Try accessing your browser whether Squid is working fine or not.

DNS Records

A     ## address record,Returns a 32-bit IPv4 address, most commonly used to
            map hostnames to an IP address of the host.
                            eric.ctechz.com. IN A 32.36.7.6
       
 (address) Maps a host name to an IP address. When a computer has multiple adapter cards or IP addresses, or both, it should have multiple address records.

AAAA   ## IPv6 address record, Returns a 128-bit IPv6 address, most commonly used to map hostnames to an IP address of the host.

CNAME  ## Canonical name record, Alias of one name to another: the DNS
                      lookup will continue by retrying the lookup with the new name.
                     CNAME records simply allow a machine to be known by more than
                     one hostname. There must always be an A record for the machine
                     before aliases can be added. The host name of a machine that is
                      stated in an A record is called the canonical.

                      www.ctechz. IN CNAME eric.ctechz.com.

 (canonical name) Sets an alias for a host name. For example, using this record, zeta.microsoft.com can have an alias as www.microsoft.com.

MX     ## mail exchange record, Maps a domain name to a list of message
                   transfer agents for that domain.

(mail exchange) Specifies a mail exchange server for the domain, which allows mail to be delivered to the correct mail servers in the domain.

NS    ## name server record, Delegates a DNS zone to use the given authoritative name servers.
                 ctechz.com. IN NS ravan.ctechz.com.

(name server) Specifies a name server for the domain, which allows DNS lookups within various zones. Each primary and secondary name server should be declared through this record.

PTR   ## pointer record, Pointer to a canonical name. Unlike a CNAME, DNS processing does NOT proceed, just the name is returned. The
most common use is for implementing reverse DNS lookups, but other uses
include such things as DNS-SD.

 (pointer) Creates a pointer that maps an IP address to a host name for reverse lookups.

SOA   ## start of [a zone of] authority record, Specifies authoritative
  information about a DNS zone, including the primary name server, the email of the domain administrator, the domain serial number, and
 several timers relating to refreshing the zone.

(start of authority) Declares the host that's the most authoritative for the zone and, as such, is the best source of DNS information for the zone. Each zone file must have an SOA record (which is created automatically when you add a zone).

    ctechz.com. IN SOA dom.ctechz.com.
        hostmaster.ctechz.com. (
           1996111901 ; Serial
           10800 ; Refresh
           3600 ; Retry
           3600000 ; Expire
           86400 ) ; Minimum

TXT   ## Text record, Originally for arbitrary human-readable text in a DNS
           record. 

The @ symbol in your DNS record refers to the record for your domain name without any www or sub-domain name.

The result of this record is that visitors can connect to your domain name at http://your-domain.com. You may also notice the @ symbol in the CNAME section:

ftp        @
www    @

This will create aliases to the @ A Record, which will point www.your-domain.com and ftp.your-domain.com to the same IP address.

The @ symbol may also be used in an MX record. For example:
@        mail        1
This indicates that the primary MX record for the email sent to @your-domain.com points to the A-record called "mail".

DNS Resolution Process

Let's check how a Domain Name Service works when we entering a name into a client like a browser or a mail client.

1. When a user type a host name (www.ctechz.co.in) in a browser the application then try to find the IP address associated with that domain name. The process associated with it is either called a reverse lookup or a forward lookup.

Checking of IP address associated with a domain name is
known as forward lookup and also the checking of domain name associated with an IP address is called reverse lookup.

There are 13 root name servers on the internet which provides the necessary name server details.

Each country has a name server and each organization has a name server too. Each NS only has information about machines in its own domain as well as information about other name servers. The root NS only has information on the ip address of the name servers of .com, .edu etc (top level domains).

* .in NS only has information on the IP address of the name servers of .org.in, .ac.in, .co.in etc

* .co.in NS only has information on the name servers of all indian companies or which hosted in india.

* .ctechz.co.in NS only has information on the machines at ctechz systems like www.ctechz.ci.in etc...

The Name Resolution Process

Here we can take an example with domain www.ctechz.co.in, the following take place to resolve this name into an IP address. This procedure is called hostname resolution and the algorithm performs this operation is called the resolver.

2.  The application checks local database on the local machine first. If it can get an answer directly from them it proceeds no further.

3. otherwise request will sent to NS to find the IP address associated with www.ctechz.ci.in.

4. NS determines whether that IP has been recently looked up or not. If it is there, no need to ask further. Since the result would be stored in a local cache.

5. NS checks whether the domain is local, ie, if a computer that has a direct information about. In  this case this would only be true if the NS were www.ctechz.co.in's very own NS.

6. NS strips out the TLD (top level domain) .in and it queries a root NS, asking what NS is responsible for .in. It will return an answer say a NS of IP 127.168.2.33. Depend on the answer NS will query authoritative server for IP address.

7. NS strips out the next highest domain .co.in and it queries to 127.168.2.33 asking what NS is responsible for .co.in, it will return an answer say a NS of IP 192.168.55.67.

8. NS strips out next highest domain .ctechz.co.in and it queries 192.168.55.67 asking what NS is responsible for ctechz.co.in, it will return an answer say a NS of IP 196.28.120.5

9. NS queries 196.28.120.5 asking for IP address of  www.ctechz.co.in and the answer will be 160.120.170.3

10. NS returns result to the application.

11. NS stores each of these result in local cache with an expiration date, to avoid having to look up a second time.

Configuring local Machine

Some configuration file in local machines are the following,

  /etc/host.conf

 /etc/hosts

 /etc/resolv.conf

1. Application checks /etc/host.conf which has the line order hosts,bind
 specify it should first check the local database file /etc/hosts and then query the NS specified in /etc/resolv.conf(bind)

The file /etc/hosts contain plain list of IP address and names. If an application can get an answer directly from /etc/hosts it proceeds no further.

2. The application checks in the file /etc/resolv.conf for a line 

nameserver <nameserver>

3. The application sends to the NS a query with the hostname [ checks local db first ]  then proceeds with the hierarchical queries.