Ubuntu : Centralized Log Server & PHPSyslogViewer

ติดตั้ง Ubuntu 8.04 Server
1.Boot จากแผ่นติดตั้ง เลือกภาษาที่จะใช้ในการติดตั้ง : English
2.เลือกการติดตั้ง : Install Ubuntu Server
3.เลือกภาษาสำหรับระบบ : English
4.เลือกประเทศ : other -> Thailand
5.Detect keyboard layout : No -> Thailand -> Thailand
6.เลือกปุ่มเปลี่ยนภาษา : Alt+Shift
7.ตั้งชื่อ Server : Go Back (เพื่อกลับไปตั้งค่า IP)
8.ตั้งค่า Network : Configure network manually
8.1 IP Address : 192.168.0.251
8.2 Netmask : 255.255.255.0
8.3 Gateway : 192.168.0.3
8.4 Name server : 192.168.0.254
9.ตั้งชื่อ Server : Server1
10.Domain : sci.com
11.จัดการ Partition : Guided - use entire disk -> Yes
12.ตั้งชื่อ user : System Administrator -> sa
13.ตั้งรหัสผ่านและยืนยัน : *** -> ***
14.ตั้งค่า proxy :
15.เลือก Software ที่จะติดตั้ง : OpenSSH server
16.Restart (หลังจากนี้สามารถ ssh มาจากเครื่องอื่นได้)
17.Login ด้วย User ที่สร้าง
18.กรณีไม่ได้ตั้งค่า IP ในขั้นตอนที่ 8 ตั้งค่าได้โดยใช้คำสั่ง
sudo nano /etc/network/interfaces
แก้ไขให้เป็นดังนี้

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
address 192.168.0.251
netmask 255.255.255.0
network 192.168.0.0
broadcast 192.168.0.255
gateway 192.168.0.3
# dns-* options are implemented by the resolvconf package, if installed
dns-nameservers 192.168.0.254
dns-search sci.com

19.Update and Upgrade
sudo apt-get update
sudo apt-get upgrade
reboot

ติดตั้ง Time Server
sudo apt-get install ntp
sudo nano /etc/ntp.conf

แก้ไขให้เป็นดังนี้

# /etc/ntp.conf, configuration for ntpd; see ntp.conf(5) for help
driftfile /var/lib/ntp/ntp.drift

# Enable this if you want statistics to be logged.
#statsdir /var/log/ntpstats/

#statistics loopstats peerstats clockstats
#filegen loopstats file loopstats type day enable
#filegen peerstats file peerstats type day enable
#filegen clockstats file clockstats type day enable

# You do need to talk to an NTP server or two (or three).
server 203.185.69.60 dynamic
server time.navy.mi.th dynamic
server time.nist.gov dynamic
server 127.127.1.0 # local clock
fudge 127.127.1.0 stratum 10
broadcastdelay 0.008
keys /etc/ntp/keys

# Access control configuration; see /usr/share/doc/ntp-doc/html/accopt.html for
# details. The web page
# might also be helpful.
#
# Note that "restrict" applies to both servers and clients, so a configuration
# that might be intended to block requests from certain clients could also end
# up blocking replies from your own upstream servers.

# By default, exchange time with everybody, but don't allow configuration.
#restrict -4 default kod notrap nomodify nopeer noquery
restrict default kod notrap nomodify nopeer noquery

# Local users may interrogate the ntp server more closely.
restrict 127.0.0.1
#restrict ::1

# Clients from this (example!) subnet have unlimited access, but only if
# cryptographically authenticated.
restrict 192.168.0.0 mask 255.255.255.0 nomodify notrap

# If you want to provide time to your local subnet, change the next line.
# (Again, the address is an example only.)
#broadcast 192.168.123.255

# If you want to listen to time broadcasts on your local subnet, de-comment the
# next lines. Please do this only if you trust everybody on the network!
#disable auth
#broadcastclient

restart ntp service
sudo /etc/init.d/ntp restart

ตั้งค่า linux เครื่องอื่นให้มารับเวลาจากเครื่องนี้

# /etc/ntp.conf, configuration for ntpd; see ntp.conf(5) for help
driftfile /var/lib/ntp/ntp.drift

# Enable this if you want statistics to be logged.
#statsdir /var/log/ntpstats/

#statistics loopstats peerstats clockstats
#filegen loopstats file loopstats type day enable
#filegen peerstats file peerstats type day enable
#filegen clockstats file clockstats type day enable

# You do need to talk to an NTP server or two (or three). (192.168.0.251 is Log Server)
server 192.168.0.251

# Access control configuration; see /usr/share/doc/ntp-doc/html/accopt.html for
# details. The web page
# might also be helpful.
#
# Note that "restrict" applies to both servers and clients, so a configuration
# that might be intended to block requests from certain clients could also end
# up blocking replies from your own upstream servers.

# By default, exchange time with everybody, but don't allow configuration.
#restrict -4 default kod notrap nomodify nopeer noquery
#restrict -6 default kod notrap nomodify nopeer noquery
restrict default ignore

# Local users may interrogate the ntp server more closely.
restrict 127.0.0.1
#restrict ::1

# Clients from this (example!) subnet have unlimited access, but only if
# cryptographically authenticated. (192.168.0.251 is Log Server)
restrict 192.168.0.251 mask 255.255.255.255 nomodify notrap noquery

# If you want to provide time to your local subnet, change the next line.
# (Again, the address is an example only.)
#broadcast 192.168.123.255

# If you want to listen to time broadcasts on your local subnet, de-comment the
# next lines. Please do this only if you trust everybody on the network!
#disable auth
#broadcastclient


ตั้งค่า Windows ให้มารับเวลาจากเครื่องนี้ โดยใช้ regedit.exe (save เป็นไฟล์นามสกุล .reg แล้ว double click)

Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Config]
"AnnounceFlags"=dword:00000005
"MaxNegPhaseCorrection"=dword:00000e10
"MaxPosPhaseCorrection"=dword:00000e10

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Parameters]
"NtpServer"="192.168.0.251,0x1"
"Type"="NTP"

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\TimeProviders\NtpClient]
"SpecialPollInterval"=dword:00000384

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\TimeProviders\NtpServer]
"Enabled"=dword:00000001


ติดตั้ง Syslog-NG และ PHP SYSLOGVIEWER
1.ติดตั้ง apache
sudo apt-get install apache2

2.ดาวน์โหลดและเตรียมติดตั้ง PHP SYSLOGVIEWER

wget http://jaist.dl.sourceforge.net/sourceforge/phpsyslogviewer/phpsyslogviewer-7.2.1.tar.bz2
tar xjvf phpsyslogviewer-7.2.1.tar.bz2
cd phpsyslogviewer-7.2.1

3.ติดตั้ง mysql-server
sudo apt-get install mysql-server

4.ติดตั้ง phpmyadmin ไว้ช่วยจัดการฐานข้อมูล
sudo apt-get install phpmyadmin

5.สร้างฐานข้อมูล

mysql -u root -p
mysql> create database syslogng;
mysql> exit;
mysql -u root -p syslogng < install/phpsyslogviewer.sql

6.ติดตั้ง php-cli
sudo apt-get install php5-cli

7.สร้างรายชื่อผู้ใช้
php install/newuser.sql.php
php install/newuser.sql.php | mysql -u root -p syslogng

8.สร้างหน้า web

cp -R htdocs /var/www/phpsyslogviewer
nano /var/www/phpsyslogviewer/config.php
chown root:www-data /var/www/phpsyslogviewer/config.php
chmod 440 /var/www/phpsyslogviewer/config.php

9.ลองเข้าดูได้ที่ http://192.168.0.251/phpsyslogviewer
10.เพิ่มความเร็วในการเก็บ log เข้า MySQL ด้วย speedupd

wget http://jaist.dl.sourceforge.net/sourceforge/phpsyslogviewer/speedupd-7.3.2.tar.bz2
tar xjvf speedupd-7.3.2.tar.bz2
cd speedup-7.3.2/
apt-get install debhelper cmake libdaemon-dev libconfuse-dev fakeroot
apt-get install build-essential libmysqlclient15-dev
dpkg-buildpackage -rfakeroot
cd ..
dpkg -i speedupd_7.3.0_i386.deb
nano /etc/speedupd.conf
update-rc.d speedupd defaults
/etc/init.d/speedupd start


ติดตั้ง Syslog-NG
1.ติดตั้ง
sudo apt-get install syslog-ng

2.ตั้งค่า
nano /etc/syslog-ng/syslog-ng.conf
ป้อนข้อมูลดังนี้

options {
recv_time_zone (+07:00);
send_time_zone (+07:00);
sync (0);
time_reopen (100);
log_fifo_size (1000);
long_hostnames (off);
use_dns (no);
use_fqdn (no);
create_dirs (yes);
chain_hostnames(yes);
keep_hostname (yes);
};

source s_sys {
file ("/proc/kmsg" log_prefix("kernel: "));
unix-stream ("/dev/log");
internal();
#udp(ip(0.0.0.0) port(514));
#tcp(ip(0.0.0.0) port(514) keep-alive(yes));
};

destination d_mysql {
pipe("/var/log/mysql.pipe"
template("INSERT INTO logs (host, facility, priority, level, tag, datetime, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n")
template-escape(yes));
};

filter f_filter1 { facility (kern); };
filter f_filter2 { level(info..emerg) and not facility(mail,authpriv,cron); };
filter f_filter3 { facility(authpriv); };
filter f_filter4 { facility(mail); };
filter f_filter5 { level(emerg); };
filter f_filter6 { facility(uucp) or (facility(news) and level(crit..emerg)); };
filter f_filter7 { facility(local7); };
filter f_filter8 { facility(cron); };

log { source(s_sys); filter(f_filter1); destination(d_mysql); };
log { source(s_sys); filter(f_filter2); destination(d_mysql); };
log { source(s_sys); filter(f_filter3); destination(d_mysql); };
log { source(s_sys); filter(f_filter4); destination(d_mysql); };
log { source(s_sys); filter(f_filter5); destination(d_mysql); };
log { source(s_sys); filter(f_filter6); destination(d_mysql); };
log { source(s_sys); filter(f_filter7); destination(d_mysql); };
log { source(s_sys); filter(f_filter8); destination(d_mysql); };
#####################################################################

# Source from remote client
source s_client {
tcp(ip(0.0.0.0) port(514) keep-alive(yes) max-connections(300));
udp(ip(0.0.0.0) port(514));
};
log {source(s_client); destination(d_mysql); };

3.สร้างคำสั่งสำหรับเขียนลง mysql
nano /usr/local/bin/syslog2mysql.sh
ป้อนคำสั่งดังนี้

#!/bin/bash
if [ ! -e /var/log/mysql.pipe ]
then
mkfifo /var/log/mysql.pipe
fi
while [ -e /var/log/mysql.pipe ]
do
mysql -u root --password=*** syslogng < /var/log/mysql.pipe > /dev/null
done

ทำให้คำสั่ง execute ได้

chmod +x /usr/local/bin/syslog2mysql.sh


4.ทำให้ script run ทุกครั้งที่ boot
nano /etc/init.d/syslog2mysql
ป้อนคำสั่งดังนี้

#!/bin/sh
# run syslog2mysql.sh at boot

case "$1" in
'start')
sh /usr/local/bin/syslog2mysql.sh &
;;
'stop')
;;
*)
echo "Usage: $0 { start | stop }"
;;
esac
exit 0

ทำให้ execute ได้แล้วเพิ่ม startup link ใน RC

sudo chmod +x /etc/init.d/syslog2mysql
sudo update-rc.d syslog2mysql defaults


run script แล้วก็ restart syslog-ng

/etc/init.d/syslog2mysql start
/etc/init.d/syslog-ng restart


5.ติดตั้ง Syslog-NG ที่ Linux Server เครื่องอื่นและตั้งค่าให้ส่ง log มาเก็บที่เครื่องนี้

options {
sync (0);
time_reopen (10);
log_fifo_size (1000);
long_hostnames (off);
use_dns (no);
use_fqdn (no);
create_dirs (yes);
keep_hostname (yes);
};

source s_sys {
file ("/proc/kmsg" log_prefix("kernel: "));
unix-stream ("/dev/log");
internal();
#udp(ip(0.0.0.0) port(514));
#tcp(ip(0.0.0.0) port(5149) keep-alive(yes));
};

destination logserver { tcp("192.168.0.251" port(514)); };

destination d_cons { file("/dev/console"); };
destination d_mesg { file("/var/log/messages"); };
destination d_auth { file("/var/log/secure"); };
destination d_mail { file("/var/log/maillog" sync(10)); };
destination d_spol { file("/var/log/spooler"); };
destination d_boot { file("/var/log/boot.log"); };
destination d_cron { file("/var/log/cron"); };
destination d_kern { file("/var/log/kern"); };
destination d_mlal { usertty("*"); };

filter f_filter1 { facility(kern); };
filter f_filter2 { level(info..emerg) and not (facility(mail) or facility(authpriv) or facility(cron)); };
filter f_filter3 { facility(authpriv); };
filter f_filter4 { facility(mail); };
filter f_filter5 { level(emerg); };
filter f_filter6 { facility(uucp) or (facility(news) and level(crit..emerg)); };
filter f_filter7 { facility(local7); };
filter f_filter8 { facility(cron); };
# Remove the 'squid' log entries from 'user' log facility
filter f_remove { not program("squid"); };

log { source(s_sys); filter(f_filter1); destination(d_cons); };
log { source(s_sys); filter(f_filter1); destination(d_kern); };
log { source(s_sys); filter(f_filter2); filter(f_remove); destination(d_mesg); };
log { source(s_sys); filter(f_filter3); destination(d_auth); };
log { source(s_sys); filter(f_filter4); destination(d_mail); };
log { source(s_sys); filter(f_filter5); destination(d_mlal); };
log { source(s_sys); filter(f_filter6); destination(d_spol); };
log { source(s_sys); filter(f_filter7); destination(d_boot); };
log { source(s_sys); filter(f_filter8); destination(d_cron); };

filter f_squid { program("squid") and facility(user); };

destination d_squid {
file("/var/log/$HOST/$YEAR/$MONTH/squid.$YEAR-$MONTH-$DAY"
owner(root) group(adm) perm(665)
create_dirs(yes) dir_perm(0775));
};
log { source(s_sys); filter(f_squid); destination(d_squid); };

log { source(s_sys); destination(logserver); };


6.ส่ง Log จาก Windows Server มาเครื่องนี้
6.1 Download Lasso (Windows Event Collector) จาก http://open.loglogic.com
6.2 ตั้งค่า hostlist.ini
localhost,*6
6.3 ตั้งค่า lasso.ini

SkipInitDLLScan,0
LogAppliance,192.168.0.251
RepositoryPath,C:\Program Files\Lasso\LassoRepository\
SpoolPath,C:\Program Files\Lasso\LassoRepository\Spool\
EventPollInterval,10
SpoolFileSize,1.0
WatermarkWriteInterval,100
MaxTraceFileSize,20
MaxNumWorkerThreads,4
DllLoadInterval,3600
HighWaterMarks,ON
#DefaultLassoShare,LassoShare=C:\LassoTemp
CheckHostListInterval,3600
NewHostSkipHistorical,0
EnableShareDlls,1
CheckRemHostAvail,0
EnableAdminSharesIfDisabled,0
DebugLevel,0
LogLevel,1
DebugHostFileSize,20
AccessReport,0

6.4 start service Lasso Windows Event Collector