So I'm back out to work after a nice vacation, so I should have seen this coming, right?
When attempting to connect to my VNC Server from my client, a nice error message popped up.
Unable to connect to server.
I managed to ssh into the server, but the VNC Server refused to restart:
[myuser@xxxxxx tmp]# service vncserver restart
Shutting down VNC server: 3:myuser [FAILED]
Starting VNC server: 3:myuser
Warning: xxx.xxx.xxx:3 is taken because of /tmp/.X3-lock
Remove this file if there is no X server xxx.xxx.xxx.xxx:3
A VNC server is already running as :3 [FAILED]
Long story short, I had to delete the lockfile and socket in order to be able restart the VNC Server.
0. Login as root/sudo
1. Remove lock file and socket
rm -rf /tmp/.X3-lock
rm -rf /tmp/.X11-unix/X3
2. Restart vnc service
service vncserver restart
Thursday, January 2, 2014
Tuesday, October 29, 2013
Standalone server reporting with Sar and KSar
Zabbix is a great tool for monitoring and reporting for multiple servers and that installation is covered here. In this instance, Sar and KSar will be used as a standalone data collection tool, as it falls outside the physical and networking reach of my Zabbix server.
Sar is a neat little tool that is part of the sysstat package, more information can be found on the author's website. In my case, we will be using to collect data on CPU, Memory, Swap, Network and all the other metrics that can make or break a Linux based service.
kSar is a separate java based tool that generates some lovely graphs using the collected sar data, because a picture paints a thousand words, or in this case, a graph summarizes a crapload of data.
This tutorial covers the installation of both, as well as a practical usage scenario.
0. Got root/ sudo?
1. Install the packagesyum install sysstat java
2. Set the sysstat cron to run
nano /etc/cron.d/sysstat
Ensure the following lines are active / not commented out. The first line specifies how often the tool should take a snapshot, the second is when the daily summary is processed.
*/10 * * * * root /usr/lib/sa/sa1 1 1
53 23 * * * root /usr/lib/sa/sa2 –A
3. Configure sar to keep a month's worth of data
By default, sar keeps 7 days worth of data. Since we need data monthly, the configuration needs updating.
nano /etc/sysconfig/sysstat
Update line, HISTORY = 7 to now read:
HISTORY = 31
4. Download kSar
Located at http://sourceforge.net/projects/ksar
Create a folder to store kSar and Monthly text files
mkdir /mon
Extract kSar into /mon
Change permissions to make kSar executable
chmod +x /mon/kSar
5. View Daily Server Data (default)
Prep report:
LC_ALL=C sar -A > /mon/sardata.txt
See the graphs
cd /mon/kSarx.x.x/
./run.sh
Click "Data" Menu option -> Load from text file
Select /mon/sardata.txt
6. View Monthly Server Data (see below for actual script)
Prep report:
cd /mon/
./sarprep_monthly.sh
(Ensure script is executable before running!)
See the graphs
cd /mon/kSarx.x.x/
./run.sh
Click "Data" Menu option -> Load from text file
Select /mon/sarmonthly_July.txt (use appropriate month name)
7. All done!
sarprep_monthly.sh
#cleanup old monthly file
rm -rf /mon/sarmonthly_$(date +"%B").txt
#loop through 31 possible days, merge all files into one
for i in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31; do
LC_ALL=C sar -A -f /var/log/sa/sa$i >> /mon/sarmonthly_$(date + "%B").txt
done
That's it! Til next time...
-noveck
Sar is a neat little tool that is part of the sysstat package, more information can be found on the author's website. In my case, we will be using to collect data on CPU, Memory, Swap, Network and all the other metrics that can make or break a Linux based service.
kSar is a separate java based tool that generates some lovely graphs using the collected sar data, because a picture paints a thousand words, or in this case, a graph summarizes a crapload of data.
This tutorial covers the installation of both, as well as a practical usage scenario.
0. Got root/ sudo?
1. Install the packagesyum install sysstat java
2. Set the sysstat cron to run
nano /etc/cron.d/sysstat
Ensure the following lines are active / not commented out. The first line specifies how often the tool should take a snapshot, the second is when the daily summary is processed.
*/10 * * * * root /usr/lib/sa/sa1 1 1
53 23 * * * root /usr/lib/sa/sa2 –A
3. Configure sar to keep a month's worth of data
By default, sar keeps 7 days worth of data. Since we need data monthly, the configuration needs updating.
nano /etc/sysconfig/sysstat
Update line, HISTORY = 7 to now read:
HISTORY = 31
4. Download kSar
Located at http://sourceforge.net/projects/ksar
Create a folder to store kSar and Monthly text files
mkdir /mon
Extract kSar into /mon
Change permissions to make kSar executable
chmod +x /mon/kSar
5. View Daily Server Data (default)
Prep report:
LC_ALL=C sar -A > /mon/sardata.txt
See the graphs
cd /mon/kSarx.x.x/
./run.sh
Click "Data" Menu option -> Load from text file
Select /mon/sardata.txt
6. View Monthly Server Data (see below for actual script)
Prep report:
cd /mon/
./sarprep_monthly.sh
(Ensure script is executable before running!)
See the graphs
cd /mon/kSarx.x.x/
./run.sh
Click "Data" Menu option -> Load from text file
Select /mon/sarmonthly_July.txt (use appropriate month name)
7. All done!
sarprep_monthly.sh
#cleanup old monthly file
rm -rf /mon/sarmonthly_$(date +"%B").txt
#loop through 31 possible days, merge all files into one
for i in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31; do
LC_ALL=C sar -A -f /var/log/sa/sa$i >> /mon/sarmonthly_$(date + "%B").txt
done
That's it! Til next time...
-noveck
Thursday, September 26, 2013
Upgrading / Migrating a MySQL 5.0 Database to MySQL 5.5 [InnoDB]
This post is an ultra, no, make that uber paranoid method of upgrading/migrating a relatively large (20+ GB on file) InnoDB database from MySQL version 5.0 to MySQL 5.5. Some might consider it overkill, but as it relates to a database of this size and maturity, I'd prefer not to take any unnecessary risks.
This assumes that the old server is running mysql 5.0 on CentOS 5.x and that the MySQL 5.5 is installed on a new server running CentOS 6, using the remi repositories. This is covered here.
Phase 1 - Prepare Data on the Old Server
1. Execute Database Check to ensure tables are clean
From terminal:
mysqlcheck –c mydbname –u root –p
<enter password when prompted>
2. Re-index tables before the dump
From mysql: (single line!)
select concat(‘ALTER TABLE`’, table_schema,’`.`’, table_name,’` Engine=InnoDB;’) from information_schema.tables where table_schema =‘mydbname’ into outfile ’/tmp/InnoBatch.sql’;
From shell:
mysql -u root –p --verbose < /tmp/InnoBatch.sql
3. Export the database as a dump file
From shell:
mysqldump -u root –p –e –c --verbose --default-character-set=utf8 --skip-set-charset --max-allowed-packet = 100M --single-transaction --databases mydbname –r /root/Desktop/mydbdump.sql
4. Copy to new DB server
scp –r /root/Desktop/mydbdump.sql root@new.db.srv.ip:/root/Desktop/
Phase 2 - Import to New Server
1. Create empty database shell for import
From mysql:
create database mdbname character set utf8 collate utf8_unicode_ci\
2. Issue Grant permissions to new DB (I hope you have this documented, else you might need to dump/restore the mysql.user table to new DB)
3. Import SQL file.(but first set a really high session value for max_allowed_packet to handle the large data import)
set global max_allowed_packet = 1000000000;
source /root/Desktop/mydbdump.sql
4. Check mysql for transaction warnings
from mysql:
show warnings\G
5. Run upgrade script
From shell:
mysql_upgrade –u root –p --force
6. Rebuild InnoDB tables, which would force the InnoDB tables to upgrade
(source: http://www.mysqlperformanceblog.com/2010/05/14/mysql_upgrade-and-innodb-tables/ )
From mysql: (single line!)
select concat(‘ALTER TABLE`’, table_schema,’`.`’, table_name,’` Engine=InnoDB;’) from information_schema.tables where table_schema =‘mydbname’ into outfile ’/tmp/InnoBatch.sql’;
From shell:
mysql -u root –p --verbose < /tmp/InnoBatch.sql
7. Execute Database Check to ensure newly imported/upgraded tables are clean
From shell:
mysqlcheck –c mydbname –u root –p
Phase 3 - Compare old and new database
Checking data consistency to ensure all the data was transferred via an accurate record count.
http://dev.mysql.com/doc/refman/5.0/en/innodb-restrictions.html
1. On Old db server, generate query to perform record count on each table
from mysql: (single line!)
select concat(‘ SELECT “’,table_name, ‘” as table_name, count(*) as exact_row_count from ‘,table_schema, ‘.’ table_name, ‘ UNION’) from information_schema.tables where table_schema =‘mydbname’ into outfile ’/tmp/TableAnalysisQuery.sql’;
From shell:
nano /tmp/TableAnalysisQuery.sql
remove the LAST Union from the end of last line in the file.
2. Run the query to get table row count for all tables
From shell:
mysql –u root –p < /tmp/TableAnalysisQuery.sql > /root/Desktop/TableAnalysisResults-$(hostname).txt
3. On New db server, generate query to perform record count on each table
from mysql: (single line!)
select concat(‘ SELECT “’,table_name, ‘” as table_name, count(*) as exact_row_count from ‘,table_schema, ‘.’ table_name, ‘ UNION’) from information_schema.tables where table_schema =‘mydbname’ into outfile ’/tmp/TableAnalysisQuery.sql’;
From shell:
nano /tmp/TableAnalysisQuery.sql
remove the LAST Union from the end of last line in the file.
4. Run the query to get table row count for all tables
From shell:
mysql –u root –p < /tmp/TableAnalysisQuery.sql > /root/Desktop/TableAnalysisResults-$(hostname).txt
5. Copy both text files to a third machine for comparison
On OLD db server, from shell:
scp –r /root/Desktop/TableAnalysisResults-myolddb.mydomain.com.txt root@third.machine.ip:/root/Desktop
On NEW db server, from shell:
scp –r /root/Desktop/TableAnalysisResults-mynewdb.mydomain.com.txt root@third.machine.ip:/root/Desktop
ON third server
from shell:
diff –a /root/Desktop/TableAnalysisResults-myolddb.mydomain.com.txt /root/Desktop/TableAnalysisResults-mynewdb.mydomain.com.txt
No output from the previous command means that the data is consistent (as it relates to number of rows on each table) on both servers and the new database can be made active/ brought in production
<EOF>
That's it!
-noveck
This assumes that the old server is running mysql 5.0 on CentOS 5.x and that the MySQL 5.5 is installed on a new server running CentOS 6, using the remi repositories. This is covered here.
Phase 1 - Prepare Data on the Old Server
1. Execute Database Check to ensure tables are clean
From terminal:
mysqlcheck –c mydbname –u root –p
<enter password when prompted>
2. Re-index tables before the dump
From mysql: (single line!)
select concat(‘ALTER TABLE`’, table_schema,’`.`’, table_name,’` Engine=InnoDB;’) from information_schema.tables where table_schema =‘mydbname’ into outfile ’/tmp/InnoBatch.sql’;
From shell:
mysql -u root –p --verbose < /tmp/InnoBatch.sql
3. Export the database as a dump file
From shell:
mysqldump -u root –p –e –c --verbose --default-character-set=utf8 --skip-set-charset --max-allowed-packet = 100M --single-transaction --databases mydbname –r /root/Desktop/mydbdump.sql
4. Copy to new DB server
scp –r /root/Desktop/mydbdump.sql root@new.db.srv.ip:/root/Desktop/
Phase 2 - Import to New Server
1. Create empty database shell for import
From mysql:
create database mdbname character set utf8 collate utf8_unicode_ci\
2. Issue Grant permissions to new DB (I hope you have this documented, else you might need to dump/restore the mysql.user table to new DB)
3. Import SQL file.(but first set a really high session value for max_allowed_packet to handle the large data import)
set global max_allowed_packet = 1000000000;
source /root/Desktop/mydbdump.sql
4. Check mysql for transaction warnings
from mysql:
show warnings\G
5. Run upgrade script
From shell:
mysql_upgrade –u root –p --force
6. Rebuild InnoDB tables, which would force the InnoDB tables to upgrade
(source: http://www.mysqlperformanceblog.com/2010/05/14/mysql_upgrade-and-innodb-tables/ )
From mysql: (single line!)
select concat(‘ALTER TABLE`’, table_schema,’`.`’, table_name,’` Engine=InnoDB;’) from information_schema.tables where table_schema =‘mydbname’ into outfile ’/tmp/InnoBatch.sql’;
From shell:
mysql -u root –p --verbose < /tmp/InnoBatch.sql
7. Execute Database Check to ensure newly imported/upgraded tables are clean
From shell:
mysqlcheck –c mydbname –u root –p
Phase 3 - Compare old and new database
Checking data consistency to ensure all the data was transferred via an accurate record count.
http://dev.mysql.com/doc/refman/5.0/en/innodb-restrictions.html
1. On Old db server, generate query to perform record count on each table
from mysql: (single line!)
select concat(‘ SELECT “’,table_name, ‘” as table_name, count(*) as exact_row_count from ‘,table_schema, ‘.’ table_name, ‘ UNION’) from information_schema.tables where table_schema =‘mydbname’ into outfile ’/tmp/TableAnalysisQuery.sql’;
From shell:
nano /tmp/TableAnalysisQuery.sql
remove the LAST Union from the end of last line in the file.
2. Run the query to get table row count for all tables
From shell:
mysql –u root –p < /tmp/TableAnalysisQuery.sql > /root/Desktop/TableAnalysisResults-$(hostname).txt
3. On New db server, generate query to perform record count on each table
from mysql: (single line!)
select concat(‘ SELECT “’,table_name, ‘” as table_name, count(*) as exact_row_count from ‘,table_schema, ‘.’ table_name, ‘ UNION’) from information_schema.tables where table_schema =‘mydbname’ into outfile ’/tmp/TableAnalysisQuery.sql’;
From shell:
nano /tmp/TableAnalysisQuery.sql
remove the LAST Union from the end of last line in the file.
4. Run the query to get table row count for all tables
From shell:
mysql –u root –p < /tmp/TableAnalysisQuery.sql > /root/Desktop/TableAnalysisResults-$(hostname).txt
5. Copy both text files to a third machine for comparison
On OLD db server, from shell:
scp –r /root/Desktop/TableAnalysisResults-myolddb.mydomain.com.txt root@third.machine.ip:/root/Desktop
On NEW db server, from shell:
scp –r /root/Desktop/TableAnalysisResults-mynewdb.mydomain.com.txt root@third.machine.ip:/root/Desktop
ON third server
from shell:
diff –a /root/Desktop/TableAnalysisResults-myolddb.mydomain.com.txt /root/Desktop/TableAnalysisResults-mynewdb.mydomain.com.txt
No output from the previous command means that the data is consistent (as it relates to number of rows on each table) on both servers and the new database can be made active/ brought in production
<EOF>
That's it!
-noveck
Monday, August 19, 2013
Installing MySQL 5.5 on CentOS 6.x
This post covers the installation of MySQL 5.5 on CentOS 6 (64bit)
By default, CentOS 6 ships with MySQL 5.1, but to take all the advantages of the more recent versions, it is generally recommended to try to use 5.5 or 5.6 if possible, especially on a new server.
0. Login as root/ su
1. Open Terminal Interface
2. Go to Temporary Folder
cd /tmp
3. Get and Install EPEL Repo (this example uses 64bit, get the 32bit rpm if needed! )
wget http://fedora.mirror.nexicom.net/epel/6/x86_64/epel-release-6-8.noarch.rpm
rpm –Uvh epel-release-6-8.noarch.rpm
4. Get and Install REMI Repo
wget http://rpms.famillecollet.com/enterprise/remi-release-6.rpm
rpm –Uvh remi-release-6.rpm
5. Check MySQL version to be installed
yum --enablerepo=remi list mysql mysql-server
**If the following error is observed during this step, see below for resolution
ERROR: Cannot retrieve metalink for repository: epel. Please verify its path and try again.
FIX: Edit epel.repo and change all https references in “mirrorlist” sections to http
cd /etc/yum.repos.d/
nano epel.repo
Find: mirrorlist=https://mirrors.fedorapro……
Change to: mirrorlist=http://mirrors.fedorapro…..
6. Install mysql 5.5
yum install --enablerepo=remi mysql mysql-server
7. Start MySQL and configure to start on boot
service mysqld start
chkconfig mysqld on
8. Run mysql upgrade script
mysql_upgrade -u root –p
9. Change Mysql default Password
/usr/bin/mysqladmin -u root password 'yourpasswordhere'
10. Check to ensure that the mysql is at the desired version
mysql –version
11. Set proper permissions on /tmpchown –R root:root /tmp
chmod –R 1777 /tmp
12. Secure MySQL
Optional but recommended for production servers
See link for details: http://dev.mysql.com/doc/refman/5.5/en/mysql-secure-installation.html
/usr/bin/mysql_secure_installation
13. Restart mysql service
service mysqld restart
That's it!
-noveck
By default, CentOS 6 ships with MySQL 5.1, but to take all the advantages of the more recent versions, it is generally recommended to try to use 5.5 or 5.6 if possible, especially on a new server.
0. Login as root/ su
1. Open Terminal Interface
2. Go to Temporary Folder
cd /tmp
3. Get and Install EPEL Repo (this example uses 64bit, get the 32bit rpm if needed! )
wget http://fedora.mirror.nexicom.net/epel/6/x86_64/epel-release-6-8.noarch.rpm
rpm –Uvh epel-release-6-8.noarch.rpm
4. Get and Install REMI Repo
wget http://rpms.famillecollet.com/enterprise/remi-release-6.rpm
rpm –Uvh remi-release-6.rpm
5. Check MySQL version to be installed
yum --enablerepo=remi list mysql mysql-server
**If the following error is observed during this step, see below for resolution
ERROR: Cannot retrieve metalink for repository: epel. Please verify its path and try again.
FIX: Edit epel.repo and change all https references in “mirrorlist” sections to http
cd /etc/yum.repos.d/
nano epel.repo
Find: mirrorlist=https://mirrors.fedorapro……
Change to: mirrorlist=http://mirrors.fedorapro…..
6. Install mysql 5.5
yum install --enablerepo=remi mysql mysql-server
7. Start MySQL and configure to start on boot
service mysqld start
chkconfig mysqld on
8. Run mysql upgrade script
mysql_upgrade -u root –p
9. Change Mysql default Password
/usr/bin/mysqladmin -u root password 'yourpasswordhere'
10. Check to ensure that the mysql is at the desired version
mysql –version
11. Set proper permissions on /tmpchown –R root:root /tmp
chmod –R 1777 /tmp
12. Secure MySQL
Optional but recommended for production servers
See link for details: http://dev.mysql.com/doc/refman/5.5/en/mysql-secure-installation.html
/usr/bin/mysql_secure_installation
13. Restart mysql service
service mysqld restart
That's it!
-noveck
Wednesday, July 17, 2013
CentOS 5 top menu vanished after update
Well this is a tickler. After I performed an update on a CentOS system, the top menu pulled a Houdini on me. The panel was totally empty and non-responsive, save for the time.
This is a quick and easy fix, so don't panic. Just force the gnome panel to reload.
0. Login as root or sudo.
You should find the Terminal via right-click on the desktop. Thankfully.
1. Reload the gnome panel (forcefully)
killall gnome-panel
2. That's it!
The menu items should be return. If that didn't work, you might need further help.
-noveck
This is a quick and easy fix, so don't panic. Just force the gnome panel to reload.
0. Login as root or sudo.
You should find the Terminal via right-click on the desktop. Thankfully.
1. Reload the gnome panel (forcefully)
killall gnome-panel
2. That's it!
The menu items should be return. If that didn't work, you might need further help.
-noveck
Tuesday, June 11, 2013
Reset MySQL root password on CentOS 5.x
I had one of those oh-crap moments and forgot the mysql root password in one of my development/test machines.
This is a reblog of someone else's post in case it ever gets deleted, I must say it saved my bacon (or at the very lease a couple hours of hair pulling and reinstall)
Credit to: http://gettechgo.wordpress.com/2012/05/10/how-to-reset-mysql-root-password-linux-o-s/
0. Login as root/su
1. Stop the MySQL service
service mysqld stop
2. Start MySQL Safe mode with skip grant tables option
mysqld_safe --skip-grant-tables &
(press ctrl+z to exit, if required)
3. Start the MySQL service
service mysqld start
4. Log into the MySQL server without any password
mysql -u root -p mysql
5. Reset the password for ‘root’ user
UPDATE user SET password=PASSWORD(‘new-password’) where user=’root’;
6. Flush privileges
flush privileges;
7. Restart the MySQL service
service mysqld restart
8. Log-in with the new password
mysql -u root -p
<enter new password when prompted>
Cheers,
Noveck
This is a reblog of someone else's post in case it ever gets deleted, I must say it saved my bacon (or at the very lease a couple hours of hair pulling and reinstall)
Credit to: http://gettechgo.wordpress.com/2012/05/10/how-to-reset-mysql-root-password-linux-o-s/
0. Login as root/su
1. Stop the MySQL service
service mysqld stop
2. Start MySQL Safe mode with skip grant tables option
mysqld_safe --skip-grant-tables &
(press ctrl+z to exit, if required)
3. Start the MySQL service
service mysqld start
4. Log into the MySQL server without any password
mysql -u root -p mysql
5. Reset the password for ‘root’ user
UPDATE user SET password=PASSWORD(‘new-password’) where user=’root’;
6. Flush privileges
flush privileges;
7. Restart the MySQL service
service mysqld restart
8. Log-in with the new password
mysql -u root -p
<enter new password when prompted>
Cheers,
Noveck
Monday, May 13, 2013
Removing a repository from CentOS
Let's chalk this one up to the Monday Morning Blues.
I'm running through a CentOS 6.x test server and needed to install the EPEL repos. For some reason I managed to install the RHEL5 rpm instead of the RHEL6. (D'Oh!)
This is a quick fix on removing the repo completely to install the proper one.
0. Got root? (or sudo)
1. Remove Repo
From terminal
rpm -qa | grep epel
Expected output (in my case)
epel-release-5-4
yum remove epel-release-5-4
yum clean all
That's it!
-noveck
I'm running through a CentOS 6.x test server and needed to install the EPEL repos. For some reason I managed to install the RHEL5 rpm instead of the RHEL6. (D'Oh!)
This is a quick fix on removing the repo completely to install the proper one.
0. Got root? (or sudo)
1. Remove Repo
From terminal
rpm -qa | grep epel
Expected output (in my case)
epel-release-5-4
yum remove epel-release-5-4
yum clean all
That's it!
-noveck
Subscribe to:
Posts (Atom)