Migrating a Virtual Machine from VMWare Player to ESXi

This post covers the migration of a virtual machine from a VMWare Player environment, which is located on my workstation (Windows 7 64-bit), to a VMWare ESXi server in my datacenter.

Both are the free versions, so they don't have all the fancy bells and whistles as the awesome licensed versions.

It took a bit of doing, but it's relatively straightforward.


1. Upgrade VMWare player (if necessary)
I realized that I was using version 3.0, and was a 'few' versions behind. Newer versions contain the OVFTool, which we will use to create the ESXi ready OVF.
Visit here: http://www.vmware.com/products/player


2. Upgrade and Reboot your pitiful workstation.

3. Read the VMWare OVFTool guide. Or not.

4. Login to the Virtual Machine in VMWare Player and uninstall the VMWare Tools. Reboot VM.

5. When VM boots, ignore any driver errors. Do a clean shutdown.

6. Export the OVF from the VM
Open Command Prompt as Administrator
cd C:\Program Files\VMWare\VMWare OVF Tool

Make sure the tool works, run the help
ovftool -h

You should be presented with a range of flag options.

Export the OVF
ovftool "C:\Path to VMname.vmx" "C:\Path to place OVF"
If the output does not look like this, it might be a good idea to check the documentation.

7. Import the OVF Into VMWare ESXi
Login to your ESXi Host
Click File -> Deploy OVF Template
Browse to "C:\Path to place OVF" as specified in step 6.
Use defaults, or configure accordingly.
When completed boot VM in ESXi, enable the network adapter and update the IP/Hostname or any other relevant configuration.

That's it!

Issues with Percona Xtrabackup on CentOS 5/MySQL 5.0

I've got a couple servers that really can't move from CentOS5 / MySQL 5.0 just yet, and while attempting to restore a MySQL InnoDB database backup using percona's innobackupex/xtrabackup, I got the following error (hashtags added, 'cause I can't remember the exact preamble.):


Can't exec "xtrabackup_51": No such file or directory at /usr/local/bin/innobackupex-############

#### fatal error: no 'mysqld' group in MySQL options
 
Absolutely freaking beautiful.

 Turns out that it's caused due to the automatic yum update to  version 2.1.x of the xtrabackup application, which uses the MySQL 5.1+ plugins and fortunately, it's a very quick fix, even though I nearly pulled out what little hair I have to figure it out. Thankfully percona includes the last version with their repo, so it's just to specify the version and it's gonna work again.


0. Got root/sudo

1. Remove existing xtrabackup package.  
yum remove xtrabackup

2. Check that the older version of the package is available from the repo 
yum list | grep percona


3. Specify/Install version 2.0 explicitly (choose based on your OS version!)
yum install percona-xtrabackup-20.x86_64


It's as simple as that! No need to pull out your hair just yet or try to build from source, which is it's own piece of work thanks to cryptic documentation.

To avoid this from happening again, I'd suggest disabling the percona repo when doing a yum update for the entire server, or manually patching the version by calling yum update percona-xtrabackup-20.

Cheerio
-noveck

SSL Secured Apache Webserver

Here's a quick way to run an SSL Secured Webserver. Ideally, a trusted Certificate Authority should be used, but as a proof of concept, we'll be generating our own self-signed certificate.

This assumes a fully functional Apache Webserver running on CentOS Linux.

0. Login as root/sudo into the terminal

1. Install prerequisites
yum install mod_ssl openssl

2. Generate Certificate / Private Key
(or use instructions from trusted CA with a purchased certificate) 
openssl genrsa -out ca.key 1024

3. Generate Certificate Signing Request (CSR)
openssl req -new -key ca.key -out ca.csr

4. Generate Self Signed Key
openssl -x509 -req -days 365 -in ca.csr -signkey ca.key -out ca.crt

5. Copy files to appropriate locations
cp ca.crt /etc/pki/tls/certs
cp ca.key /etc/pki/tls/private/ca.key
cp ca.csr /etc/pki/tls/private/ca.csr

6. For SELinux
restorecon -Rvf /etc/pki

7. Update the Apache SSL config file
vi +/SSLCertificateFile /etc/httpd/conf.d/ssl.conf

Edit the two entries in the file

SSLCertificateFile /etc/pki/tls/certs/ca.crt
SSLCertificateKeyFile /etc/pki/tls/private/ca.key

8. Restart Apache
service httpd restart


9. Configure the firewall to accept incoming SSL requests
iptables -A INPUT -p tcp --dport 443 -j ACCEPT
service iptables save
iptables -L –v


10. Test
From a web browser hit https://servername.com and the page should be displayed.


Finito!
Now get some coffee.:)

-noveck

Troubleshooting VNC Connection error

So I'm back out to work after a nice vacation, so I should have seen this coming, right?

When attempting to connect to my VNC Server from my client, a nice error message popped up.
Unable to connect to server.
I managed to ssh into the server, but the VNC Server refused to restart:

[myuser@xxxxxx tmp]# service vncserver restart
Shutting down VNC server: 3:myuser                         [FAILED]
Starting VNC server: 3:myuser
Warning: xxx.xxx.xxx:3 is taken because of /tmp/.X3-lock
Remove this file if there is no X server xxx.xxx.xxx.xxx:3
A VNC server is already running as :3                         [FAILED]







Long story short, I had to delete the lockfile and socket in order to be able restart the VNC Server.

0. Login as root/sudo
1. Remove lock file and socket
rm -rf /tmp/.X3-lock
rm -rf /tmp/.X11-unix/X3

2. Restart vnc service
service vncserver restart

Standalone server reporting with Sar and KSar

Zabbix is a great tool for monitoring and reporting for multiple servers and that installation is covered here. In this instance, Sar and KSar will be used as a standalone data collection tool, as it falls outside the physical and networking reach of my Zabbix server.


Sar is a neat little tool that is part of the sysstat package, more information can be found on the author's website. In my case, we will be using to collect data on CPU, Memory, Swap, Network and all the other metrics that can make or break a Linux based service.

kSar is a separate java based tool that generates some lovely graphs using the collected sar data, because a picture paints a thousand words, or in this case, a graph summarizes a crapload of data.

This tutorial covers the installation of both, as well as a practical usage scenario.

0. Got root/ sudo?



1. Install the packagesyum install sysstat java



2. Set the sysstat cron to run
nano /etc/cron.d/sysstat
Ensure the following lines are active / not commented out. The first line specifies how often the tool should take a snapshot, the second is when the daily summary is processed.


*/10 * * * * root /usr/lib/sa/sa1 1 1
53 23 * * * root /usr/lib/sa/sa2 –A

3. Configure sar to keep a month's worth of data
By default, sar keeps 7 days worth of data. Since we need data monthly, the configuration needs updating.

nano /etc/sysconfig/sysstat

Update line, HISTORY = 7 to now read:
HISTORY = 31

4. Download kSar
Located at http://sourceforge.net/projects/ksar
Create a folder to store kSar and Monthly text files
mkdir /mon
Extract kSar into /mon
Change permissions to make kSar executable
chmod +x /mon/kSar


5. View Daily Server Data (default)
Prep report:
LC_ALL=C sar -A > /mon/sardata.txt

See the graphs
cd /mon/kSarx.x.x/
./run.sh

Click "Data" Menu option -> Load from text file
Select /mon/sardata.txt

6. View Monthly Server Data (see below for actual script)

Prep report:
cd /mon/
./sarprep_monthly.sh
(Ensure script is executable before running!)

See the graphs
cd /mon/kSarx.x.x/
./run.sh

Click "Data" Menu option -> Load from text file
Select /mon/sarmonthly_July.txt (use appropriate month name)

7. All done!

sarprep_monthly.sh
#cleanup old monthly file
rm -rf /mon/sarmonthly_$(date +"%B").txt
#loop through 31 possible days, merge all files into one
for i in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31; do
LC_ALL=C sar -A -f /var/log/sa/sa$i >> /mon/sarmonthly_$(date + "%B").txt
done


That's it! Til next time...
-noveck

Upgrading / Migrating a MySQL 5.0 Database to MySQL 5.5 [InnoDB]

This post is an ultra, no, make that uber paranoid method of upgrading/migrating a relatively large (20+ GB on file) InnoDB database from MySQL version 5.0 to MySQL 5.5. Some might consider it overkill, but as it relates to a database of this size and maturity, I'd prefer not to take any unnecessary risks.

This assumes that the old server is running mysql 5.0 on CentOS 5.x and that the MySQL 5.5 is installed on a new server running CentOS 6, using the remi repositories. This is covered here.

Phase 1 - Prepare Data on the Old Server

1. Execute Database Check to ensure tables are clean
From terminal:
mysqlcheck –c mydbname –u root –p
<enter password when prompted>


2. Re-index tables before the dump
From mysql: (single line!)
select concat(‘ALTER TABLE`’, table_schema,’`.`’, table_name,’` Engine=InnoDB;’) from information_schema.tables where table_schema =‘mydbname’ into outfile ’/tmp/InnoBatch.sql’;

From shell:
mysql -u root –p --verbose < /tmp/InnoBatch.sql

3. Export the database as a dump file
 From shell:
mysqldump  -u root –p –e –c --verbose --default-character-set=utf8 --skip-set-charset --max-allowed-packet = 100M --single-transaction --databases mydbname –r /root/Desktop/mydbdump.sql

4. Copy to new DB server
scp –r /root/Desktop/mydbdump.sql root@new.db.srv.ip:/root/Desktop/


Phase 2 - Import to New Server

1. Create empty database shell for import
From mysql:
create database mdbname character set utf8 collate utf8_unicode_ci\

2. Issue Grant permissions to new DB (I hope you have this documented, else you might need to dump/restore the mysql.user table to new DB)

3. Import SQL file.(but first set a really high session value for max_allowed_packet to handle the large data import)
 set global max_allowed_packet = 1000000000;
source /root/Desktop/mydbdump.sql

4. Check mysql for transaction warnings
from mysql:
show warnings\G

5. Run upgrade script
From shell:
mysql_upgrade –u root –p --force

6. Rebuild InnoDB tables, which would force the InnoDB tables to upgrade
(source: http://www.mysqlperformanceblog.com/2010/05/14/mysql_upgrade-and-innodb-tables/ )

From mysql: (single line!)
select concat(‘ALTER TABLE`’, table_schema,’`.`’, table_name,’` Engine=InnoDB;’) from information_schema.tables where table_schema =‘mydbname’ into outfile ’/tmp/InnoBatch.sql’;
From shell:
mysql -u root –p --verbose < /tmp/InnoBatch.sql

7. Execute Database Check to ensure newly imported/upgraded tables are clean
From shell:
mysqlcheck –c mydbname –u root –p

Phase 3 - Compare old and new database
Checking data consistency to ensure all the data was transferred via an accurate record count.
http://dev.mysql.com/doc/refman/5.0/en/innodb-restrictions.html

1. On Old db server, generate query to perform record count on each table
from mysql: (single line!)
select concat(‘ SELECT “’,table_name, ‘” as table_name, count(*) as exact_row_count from ‘,table_schema, ‘.’ table_name, ‘ UNION’) from information_schema.tables where table_schema =‘mydbname’ into outfile ’/tmp/TableAnalysisQuery.sql’;

From shell:
nano /tmp/TableAnalysisQuery.sql
remove the LAST Union from the end of last line in the file.

2. Run the query to get table row count for all tables
From shell:
mysql –u root –p < /tmp/TableAnalysisQuery.sql > /root/Desktop/TableAnalysisResults-$(hostname).txt


3. On New db server, generate query to perform record count on each table
from mysql: (single line!)
select concat(‘ SELECT “’,table_name, ‘” as table_name, count(*) as exact_row_count from ‘,table_schema, ‘.’ table_name, ‘ UNION’) from information_schema.tables where table_schema =‘mydbname’ into outfile ’/tmp/TableAnalysisQuery.sql’;

From shell:
nano /tmp/TableAnalysisQuery.sql
remove the LAST Union from the end of last line in the file.

4. Run the query to get table row count for all tables
From shell:
mysql –u root –p < /tmp/TableAnalysisQuery.sql > /root/Desktop/TableAnalysisResults-$(hostname).txt

5. Copy both text files to a third machine for comparison

On OLD db server, from shell:
scp –r /root/Desktop/TableAnalysisResults-myolddb.mydomain.com.txt root@third.machine.ip:/root/Desktop

On NEW db server, from shell:
scp –r /root/Desktop/TableAnalysisResults-mynewdb.mydomain.com.txt root@third.machine.ip:/root/Desktop

ON third server
from shell:
diff –a /root/Desktop/TableAnalysisResults-myolddb.mydomain.com.txt /root/Desktop/TableAnalysisResults-mynewdb.mydomain.com.txt
No output from the previous command means that the data is consistent (as it relates to number of rows on each table) on both servers and the new database can be made active/ brought in production

<EOF>

That's it!
-noveck

Installing MySQL 5.5 on CentOS 6.x

This post covers the installation of MySQL 5.5 on CentOS 6 (64bit)

By default, CentOS 6 ships with MySQL 5.1, but to take all the advantages of the more recent versions, it is generally recommended to try to use 5.5 or 5.6 if possible, especially on a new server.

0. Login as root/ su

1. Open Terminal Interface

2. Go to Temporary Folder
cd /tmp

3. Get and Install EPEL Repo  (this example uses 64bit, get the 32bit rpm if needed! )

wget http://fedora.mirror.nexicom.net/epel/6/x86_64/epel-release-6-8.noarch.rpm
rpm –Uvh epel-release-6-8.noarch.rpm

4. Get and Install REMI Repo
wget http://rpms.famillecollet.com/enterprise/remi-release-6.rpm
rpm –Uvh remi-release-6.rpm

5. Check MySQL version to be installed
yum --enablerepo=remi list mysql mysql-server

**If the following error is observed during this step, see below for resolution

ERROR: Cannot retrieve metalink for repository: epel. Please verify its path and try again.

FIX: Edit epel.repo and change all https references in “mirrorlist” sections to http
cd /etc/yum.repos.d/
nano epel.repo
Find: mirrorlist=https://mirrors.fedorapro……
Change to: mirrorlist=http://mirrors.fedorapro…..

6. Install mysql 5.5
yum install --enablerepo=remi mysql mysql-server

7. Start MySQL and configure to start on boot
service mysqld start
chkconfig mysqld on

8. Run mysql upgrade script
mysql_upgrade -u root –p
 
9. Change Mysql default Password
/usr/bin/mysqladmin -u root password 'yourpasswordhere'

10. Check to ensure that the mysql is at the desired version
mysql –version

11. Set proper permissions on /tmpchown –R root:root /tmp
chmod –R 1777 /tmp

12. Secure MySQL
Optional but recommended for production servers
See link for details: http://dev.mysql.com/doc/refman/5.5/en/mysql-secure-installation.html

/usr/bin/mysql_secure_installation

13. Restart mysql service
service mysqld restart

That's it!

-noveck