Tuesday, December 29, 2015

Varnish HTTP accelerator to make dynamic websites faster

Varnish is a HTTP proxy cache or an HTTP accelerator designed to cache http requests to make dynamic websites a lot faster. Varnish is used by high-profile, high-traffic websites including online newspaper sites

The principal configuration mechanism is Varnish Configuration Language (VCL)

############# Installation for centos/redhat/fedora ####################

yum install varnish

*** Listen port should be 80 and Assing our Allocate Memory of 512 MB for caching

vi /etc/sysconfig/varnish



vi /etc/varnish/default.vcl

Modify the default backend default port 80 to port 81 instead, so the backend section should look like the following

backend default {
 .host = "";
 .port = "81";

Edit the apache httpd.conf to configure apache to listen on port 81 instead

vi /etc/httpd/conf/httpd.conf
Listen 81

service httpd restart

chkconfig varnish on
service varnish start

Test Varnish

yum install curl

curl -I http://localhost


Browse a page from this server  by firefox with firebug ....  (Install firefox and also firebug addon)

Try visiting your site and refresh the page multiple times to make sure it was cached, then using firebug go to the Net tab you should see under the fetched resources that it was served by varnish

visit site and refresh by F5 several times..->click firebug -> click "net" tab -> click cache  header

Monday, December 28, 2015

Linux IDS / IPS firewall by Suricata

Suricata is an open source high performance modern Network Intrusion Detection, Prevention and Security Monitoring System for Unix/Linux, FreeBSD and Windows based systems.
It was developed and owned by a non-profit foundation the OISF (Open Information Security Foundation)

IDS / IPS: Suricata is a rule-based Intrusion Detection and Prevention engine that make use of externally developed rules sets to monitor network traffic, as well as able to handle multiple gigabyte traffic and gives email alerts to the System/Network administrators.

Multi-threading: Suricata provides speed and importance in network traffic determination. The engine is developed to apply the increased processing power offered by modern multi-core hardware chip sets. The engine not only provides keywords for TCP, UDP, ICMP and IP, but also has an built-in support for HTTP, FTP, TLS and SMB. A system administrator can able to create its own rule to detect a match within an HTTP stream. This is going to become different Malware detection and control.

Install Prerequisites::
# yum install epel-release
# yum -y install libpcap libpcap-devel libnet libnet-devel pcre pcre-devel gcc gcc-c++ automake autoconf libtool make libyaml \
libyaml-devel zlib zlib-devel libcap-ng libcap-ng-devel magic magic-devel file file-devel

IPS for 64 bit systems:   we to need “libnfnetlink” and “libnetfilter_queue” packages

 rpm -Uvh

 rpm -Uvh

 rpm -Uvh

 rpm -Uvh

Download latest Suricata source files and Install:

cd  /home/tayab/
tar -xvzf suricata-1.4.4.tar.gz
cd suricata-1.4.4

./configure && make && make install-conf
 ./configure && make && make install-rules
 ./configure && make && make install-full

IPS Installation for 64 bit:
By default, works as an IDS. If you want to add IDS support, install some needed packages as follows.

yum  -y install libnetfilter-queue-dev libnetfilter-queue1 libnfnetlink-dev libnfnetlink0

Suricata Basic Setup
 mkdir /var/log/suricata
 mkdir /etc/suricata

 cd /home/tayab/suricata-1.4.4
 cp classification.config /etc/suricata
 cp reference.config /etc/suricata
 cp suricata.yaml /etc/suricata

### start the “Suricata Engine” first time and specify the interface device name of your preference. Instead of eth0,

suricata -c /etc/suricata/suricata.yaml -i eth0

##After several minutes later, check the engine is correctly working and receives and inspects traffic

cd /usr/local/var/log/suricata/
ls -l

total 300
drwxr-xr-x 3 root root   4096 Dec 28 16:13 .
drwxr-xr-x 3 root root   4096 Dec 28 16:04 ..
-rw-r--r-- 1 root root 145436 Dec 28 16:15 fast.log
drwxr-xr-x 2 root root   4096 Dec 28 16:04 files
-rw-r--r-- 1 root root  15287 Dec 28 16:15 http.log
-rw-r--r-- 1 root root  44844 Dec 28 16:15 stats.log
-rw-r--r-- 1 root root  85800 Dec 28 16:15 unified2.alert.1451297619

## Watch “stats.log” file and make sure the displayed information is up-dated in real time.

 tail -f stats.log


#### Snorby, the web interface frontend for managing IDS alerts
Ruby 1.9.3, at least version 1.9.2 is needed to support Snorby.
wkhtmltopdf, for export to pdf.

bunzip2 wkhtmltopdf-0.10.0_rc2-static-amd64.tar.bz2
tar xvf wkhtmltopdf-0.10.0_rc2-static-amd64.tar
ls -lh
cp wkhtmltopdf-amd64 /usr/bin/wkhtmltopdf

gem install thor i18n bundler tzinfo bui
yum install git
git clone /var/www/html/snorby

 mv snorby-master/ snorby
 cd snorby
 bundle install

#Configuring database.yml. Make a copy of the example database.yml
cd /var/www/html/snorby/config/ && cp database.yml.example database.yml

mysql -u root -p
create user 'snorbyuser'@'localhost' IDENTIFIED BY 'PASSWORD123';
grant all privileges on snorby.* to 'snorbyuser'@'localhost' with grant option;
flush privileges;

vi /etc/mysql/my.cnf
Comment the bind-address line. Then restart mysqld service.

 vi  /var/www/html/snorby/config/database.yml
 Update user: snorbyuser and Password: PASSWORD123

cd /var/www/html/snorby/config/
 cp snorby_config.yml.example snorby_config.yml

## vi /etc/httpd/conf/httpd.conf

<VirtualHost 203.190.X.X:80>
        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/html/snorby/public

        <Directory "/var/www/html/snorby/public">
                AllowOverride all
                Order deny,allow
                Allow from all
                Options -MultiViews


## cd /var/www/html/snorby

Must have to Update ruby 1.8 to 1.9.3
*** If not updated... then
yum update
 yum install gcc-c++ patch readline readline-devel zlib zlib-devel
 yum install libyaml-devel libffi-devel openssl-devel make
 yum install bzip2 autoconf automake libtool bison iconv-devel
curl -L | bash -s stable
tar -zxvf 1.26.11.tar.gz
cd rvm-1.26.11/
 source /etc/profile.d/
rvm install 1.9.3
rvm use 1.9.3 --default
#Check Current Ruby Version
ruby --version

cd /var/www/html/snorby
gem install i18n -v '0.7.0'
bundle update activesupport railties rails
bundle pack && sudo bundle install --path vender/cache

MONIT to monitor Linux server

Monit is a free open source and very useful tool that automatically monitors and manages server processfiles,directorieschecksumspermissionsfilesystems and services like ApacheNginxMySQLFTPSSHSendmailand so on in a UNIX/Linux based systems and provides an excellent and helpful monitoring functionality to system administrators.The monit has user friendly web interface where you can directly view the system status and setup up processes using native HTTP(S) web server or via the command line interface.

Installing Moint for CentOS / Redhat / Fedora
yum install epel*
yum install monit
/etc/init.d/monit start
chkconfig monit on

# We can see logs from   tail -f /var/log/monit

## Config file

 vi /etc/monit.conf

** Can change those default settings

set httpd port 2812 and
    use address localhost  # only accept connection from localhost
    allow localhost        # allow localhost to connect to the server and
    allow admin:monit      # require user 'admin' with password 'monit'

----------- to

set httpd port 2812 and
    use address 203.190.X.X # only accept connection from server 203.190.X.X
    allow 203.190.X.0/21        # allow IP-Network to connect to the server and
    allow admin:tayab321      # require user 'admin' with password 'tayab321'

## Also can see in web format....

username: admin
password: tayab321

Adding other services to Monit
# Adding HTTP service to Monitor
check process httpd with pidfile /var/run/httpd/
group apache
start program = "/etc/init.d/httpd start"
stop program = "/etc/init.d/httpd stop"
if failed host 203.190.X.X port 80
protocol http then restart
if 5 restarts within 5 cycles then timeout

# Adding MySQL service to Monitor
check process mysqld with pidfile /var/run/mysqld/
group mysql
start program = "/etc/init.d/mysqld start"
stop program = "/etc/init.d/mysqld stop"
if failed host port 3306 then restart
if 5 restarts within 5 cycles then timeout

# Adding MAIL POSTFIX service to Monitor

 check process postfix with pidfile /var/spool/postfix/pid/
    start program = "/etc/init.d/postfix start"
    stop program  = "/etc/init.d/postfix stop"
    if cpu > 60% for 2 cycles then alert
    if cpu > 80% for 5 cycles then restart
    if totalmem > 200.0 MB for 5 cycles then restart
    if children > 250 then restart
    if loadavg(5min) greater than 10 for 8 cycles then stop
    if failed host 203.190.x.x port 25 type tcp protocol smtp
       with timeout 15 seconds
      then alert
    if 3 restarts within 5 cycles then timeout

# Adding SSH service to Monitor
check process sshd with pidfile /var/run/
start program "/etc/init.d/sshd start"
stop program "/etc/init.d/sshd stop"
if failed host 203.190.X.X port 7707 protocol ssh then restart
if 5 restarts within 5 cycles then timeout

## To check monit syntax error in config file ::   monit -t

####### Enjoy this

Thursday, December 24, 2015

Denial-of-service (DoS) attack protection by MikroTik

 denial-of-service (DoS) attack is an attempt to make a machine or network resource unavailable to its intended users, such as to temporarily or indefinitely interrupt or suspend services of a host connected to the Internet.

Way to protect by MikroTik is : 

Limit incoming connections
Address with too much connections can be added to address list for blocking.

/ip firewall filter add chain=input protocol=tcp connection-limit=LIMIT,32 action=add-src-to-address-list  address-list=blocked-addr address-list-timeout=1d 

/ip firewall filter add chain=input protocol=tcp src-address-list=blocked-addr connection-limit=3,32 action=tarpit 

where LIMIT is max. number of connection per IP. LIMIT should be 100 or higher as many services use multiple connection (HTTP, Torrent, other P2P programs).
Action tarpit. Instead of simply droping attackers packets(action=drop) router can capture and hold connections and with enough powerful router is can block the attacker.

NAT with MikroTik ( One to One Mapping)

Using Multiple Public IP to NAT private IP addresses

If we use private ip in our server and want to access that server from internet then we need to set 1 to 1 netmap bellow are the configuration of netmap, you just need to change the public IP, Private IP and WAN interface name as per your configuration.

say we have 4 public ip address to
and we want to netmap private IP address to

/ip address add address= interface=wan
/ip address add address= interface=wan
/ip address add address= interface=wan
/ip address add address= interface=wan

/ip firewall nat add chain=dstnat dst-address= action=dst-nat to-addresses=
/ip firewall nat add chain=dstnat dst-address= action=dst-nat to-addresses=
/ip firewall nat add chain=dstnat dst-address= action=dst-nat to-addresses=
/ip firewall nat add chain=dstnat dst-address= action=dst-nat to-addresses=

/ip firewall nat add chain=srcnat src-address= action=src-nat to-addresses=
/ip firewall nat add chain=srcnat src-address= action=src-nat to-addresses=
/ip firewall nat add chain=srcnat src-address= action=src-nat to-addresses=
/ip firewall nat add chain=srcnat src-address= action=src-nat to-addresses=

MikroTik Backup Auto/Schedule Send to EMAIL

Goto:  Tools -> Email 
smtp server :
port: 25
from :

Here we will create schedule for auto backup sent to email address every 1 day and this will start work at every day 8:01PM(20:01:00)

Goto: System ->scheduler

Name :  mail full backup
start date : 20:01:00
Intervale : 1d 00:00:00 

/system backup save name=emailback
/tool e-mail send file=emailback.backup to="" body="Backup of Mikrotik" subject="$[/system identity get name] $[/system clock get time] $[/system clock get date] Backup"

Mail SPAM detect by MikroTik

We need to create a Firewall Filter Rule

/ip firewall filter

add chain=forward protocol=tcp dst-port=25 src-address-list=suspectedspambot \
    action=drop comment="Drop traffic from those on the suspect list"

add chain=forward protocol=tcp dst-port=25 \
    connection-limit=10,32 \
    action=add-src-to-address-list \
    address-list=suspectedspambot \
    address-list-timeout=2d \
    comment="More than 10 simultaneous connections looks spammer"

We use alternated colors for readability. The operation of this approach is quite simple. The first rule (in blue) simply drops any SMTP connection attempts from anyone who is found in the address list called “suspectedspambot”. The second rule (in red) is the one that does the work of actually detecting spammers. What this rule does is watch for SMTP connections and, if the count of connections from a single IP (/32) goes above 10, then the source address of that packet is added to an address list called “suspectedspambot”. On the next connection attempt, the packet will be dropped. The only problem with this approach is that it assumes that there are NO mail servers that MAY be sending more than 10 emails at a time legitimately. If this is the case, you can simply create another address list called “smtpservers” then add a rule as follows ABOVE the rule above (in blue):

add chain=forward protocol=tcp dst-port=25 \

       src-address-list=smtpservers action=accept \

       comment="Allow known smtp servers to send email"

This would allow your known mail servers to send email without fear of being “caught” and tagged as a spam source. One further comment on these rules. This set of rules does not take into account smtp traffic that is going TO your mail server. I will leave that fix as an exercise for the reader. If one of your customers is “tagged” as a suspected spambot, you will find their IP address in the address list and can begin troubleshooting from there.

Saturday, December 19, 2015

Juniper Real-Time Performance Monitoring (RPM)

tayab@MX#  edit services rpm

[edit services]
rpm {
  probe server-network-monitor {
   test icmp-test {
     probe-type icmp-ping-timestamp;
     target address 203.190.x.x;
     probe-count 15;
     probe-interval 1;
     test-interval 600;

set services rpm probe icmp-test test icmp probe-type icmp-ping-timestamp
set services rpm probe icmp-test test icmp target address
set services rpm probe icmp-test test icmp probe-count 15
set services rpm probe icmp-test test icmp probe-interval 1
set services rpm probe icmp-test test icmp test-interval 600
set services rpm probe icmp-test test http probe-type http-get
set services rpm probe icmp-test test http target url
set services rpm probe icmp-test test http probe-count 15
set services rpm probe icmp-test test http probe-interval 1

set services rpm probe icmp-test test http test-interval 600

** To see results

tayab@MX#   run show services rpm probe-results

Configure the target address as the loopback address of the remote device (in this case, the loopback address on any-router).
You must also configure the number of probes in each test, the length of time between probes, and the length of time between tests. Juniper recommends configuring between 10 and 20 probes at one-second intervals. This particular test is going to run every ten minutes

Real-Time Performance Monitoring (RPM) enables you to configure active probes to track and monitor traffic. Probes collect packets per destination and per application, including PING Internet Control Message Protocol (ICMP) packets, User Datagram Protocol and Transmission Control Protocol (UDP/TCP) packets with user-configured ports, user-configured Differentiated Services code point (DSCP) type-of-service (ToS) packets, and Hypertext Transfer Protocol (HTTP) packets. 

Probe configuration and probe results are supported by the command-line interface (CLI) and SNMP.

The following probe types are supported with DSCP marking:

ICMP echo
ICMP timestamp
HTTP get (not available for BGP RPM services)
UDP echo
TCP connection
UDP timestamp
With probes, you can monitor the following:

Minimum round-trip time
Maximum round-trip time
Average round-trip time
Standard deviation of the round-trip time
Jitter of the round-trip time—The difference between the minimum and maximum round-trip time
One-way measurements for ICMP timestamp probes include the following:

Minimum, maximum, standard deviation, and jitter measurements for egress and ingress times
Number of probes sent
Number of probe responses received
Percentage of lost probes

virtualmin domain edit problem


Failed to modify server : Missing or invalid mail username prefix

Monday, December 7, 2015

Juniper Outputs in Web view

tayab@MX> show bgp summary | display xml
tayab@DOL-MX>  show route protocol bgp | display xml

XML Output of BGP Summary
<rpc-reply xmlns:junos="">
    <bgp-information xmlns="">
        <bgp-rib junos:style="brief">
            <bgp-rib-state>BGP restart is complete</bgp-rib-state>
        <bgp-peer junos:style="terse" heading="Peer                     AS      InPkt     Ou
            <elapsed-time junos:seconds="1643733">2w5d0h</elapsed-time>
            <peer-state junos:format="1/43/1/0             0/0/0/0">Established</peer-state>
        <bgp-peer junos:style="terse">
            <elapsed-time junos:seconds="357732">4d 3:22:12</elapsed-time>
            <peer-state junos:format="0/1/1/0              0/0/0/0">Established</peer-state>
        <bgp-peer junos:style="terse">
            <elapsed-time junos:seconds="357731">4d 3:22:11</elapsed-time>
            <peer-state junos:format="2/2/2/0              0/0/0/0">Established</peer-state>
        <bgp-peer junos:style="terse">
            <elapsed-time junos:seconds="677003">1w0d20h</elapsed-time>
            <peer-state junos:format="1875/1875/1875/0     0/0/0/0">Established</peer-state>

** Copy this XML to paste in any web server
You can also convert this XML file to HTML file  online

*****  call  this  route.xml file to an HTML file this way....

<title>BGP Route OUTPUT</title>
<link rel="stylesheet" href="">
<script src=""></script>
<script src=""></script>

// Loading the XML file
$xml = new SimpleXMLElement("route.xml", 0, true);

<div class="container">
<div class="page-header">
<h3><a href="testcode1.php">Test-ISP</a> <small>ISP Description</small></h3>

<table class="table table-hover">
<th>AS Path</th>


$startPage = $_GET['page'];
$perPage = 500;
$currentRecord = 0;
foreach($xml->{'route-information'}->{'route-table'}->{'rt'} as $record){
$currentRecord += 1;
if($currentRecord > ($startPage * $perPage) && $currentRecord < ($startPage * $perPage + $perPage)){

$route = str_replace("\n", "", $record->{'rt-destination'});
$aspath = str_replace("\n", "", $record->{'rt-entry'}->{'as-path'});
$protocolname = str_replace("\n", "", $record->{'rt-entry'}->{'protocol-name'});
$validation = str_replace("\n", "", $record->{'rt-entry'}->{'validation-state'});
echo "<tbody>";
echo "<tr>";
echo "<td class=\"small\">{$route}</td>";
echo "<td class=\"small\">{$aspath}</td>";
echo "<td class=\"small\">{$protocolname}</td>";
if ($validation=="invalid")
{echo "<td class=\"small\"><span class=\"label label-danger\">$validation</span></td>";}
elseif ($validation=="valid")
{echo "<td class=\"small\"><span class=\"label label-success\">$validation</span></td>";}
{echo "<td class=\"small\"><span class=\"label label-default\">$validation</span></td>";}
echo "</tr>";
echo "</tbody>";
}//end of if loop
}//endo of foreach loop

for ($i = 1; $i <= ($currentRecord / $perPage); $i++) {
echo("<a href='testcode1.php?page=".$i."'>\t".$i."</a>");


Thursday, December 3, 2015

Juniper System Management

show host ns lookup
show version displays software running on the box
show system software displays installed packages
show system uptime uptime of the router
show system processes Show the process table
show system statistics Show protocol statistics
show system connections lists only active IP sockets on RE
show system users Show users currently logged into the system
show system storage displays the amount of amount of free disk space on file system
root% df -k show system storage from Shell
show system boot-messages displays contents of boot log  (boot-up messages)
show system virtual-memory displays current memory state
request system reboot Restarts the system
request system halt Stops the router and prepares it to be shut down
request system reboot Reboot the system
request system snapshot Save config in /altconfig and software in /altroot before upgrade
request support information show tech
request system software add <pkgname> performs bsd pgkadd; always use jbundle (4->5 jinstall)