Monday, November 5, 2012

Batch convert MyISAM tables to InnoDB from Terminal


A couple months ago I posted on a modified php script to convert tables MySQL tables from MyISAM to InnoDB. It worked all well and good for my test environment, but I realized that it may not be the best solution for a dedicated database server where php is not installed.

In this case, this post assumes you have both su access and mysql root access in order to batch generate the queries to convert the storage engine. I strongly recommend isolating the database from the front end at this point, it means shutting your application down until the operation has been complete.

Please TEST before running in your production environment!

Credit to this post on RackerHacker for the kickstart query generation.

This post does not cover the advantages/disadvantages of either storage engine, that is covered at MySQL's site and at the Oracle Blog.

The benefits of Moodle on InnoDB is covered here.

Some prerequisites to get out of the way.

a. Ensure that InnoDB is enabled on the server
mysql -u root -p *****
SHOW ENGINES;

Expected output
...
InnoDB | YES | Supports transactions...
b. Disable the writeback cache (as per MySQL's recommendation)
hdparm -W0 /dev/sdx (where sdx contains the /var/lib/mysql or your specific mysql location )


c. Tune the my.cnf according to MySQL's recommendation
Later on, the Day32's Tuning Primer and the Mysql Tuner Script can be used to fine tune the other server parameters.

Prereq's aside, let's push forward onto the crux of the matter.

0. Login as root on your database server.

1. Take a backup before proceeding any further.

2. Double check that you have a backup. Please.

3. Run a check on the database to avoid a GIGO issue.
mysqlcheck -c -o -r -f -h localhost -u mydbusername -p mydbname
<enter your db password>

4. Login as MySQL root and generate the batch queries, send to output file (note the single quotes plus backticks)
select concat(‘ALTER TABLE `’,table_schema,’`.`’,table_name,’` ENGINE=InnoDB;’)
from information_schema.tables
Where table_schema=’mydbname’
and ENGINE=’MyISAM’
into outfile ‘/tmp/InnoBatchConvert.sql’

5. Logout from MySQL root and back to shell
 At the shell:
mysql -u root -p --verbose < /tmp/InnoBatchConvert.sql
<enter password>

6. After the script has completed, check all the tables to ensure the storage engine is InnoDB
select table_name, engine
from information_schema.tables
where table_schema = 'mydatabasename';

7. Check the database again.
mysqlcheck -c -h localhost -u mydbusername -p mydbname
<enter your db password>

That's it! All you need to do now is ensure you have a decent backup strategy in place and the InnoDB backup script is covered here. I'll follow up soon with a detailed restore procedure as well as perhaps my own version of my innodb backup scripts with rotation built in.

-noveck

Tuesday, October 2, 2012

Send email from command line through an MS Exchange Relay (Alt title: Installing SSMTP on CentOS 5.x)

The situation required me to use an existing MS Exchange Server as a mail relay to send an email from a custom notification shell script I'm working on.

I tried configuring Postfix, Sendmail, Mailx, Nail, all to no avail. This took quite some time in troubleshooting and Google was not particularly helpful.

Eventually I stumbled across this post, which mentioned using the EPEL repo's to install SSMTP. Simple, but effective.

Therefore, this entry focuses on how to setup your server to relay mail, using the SSMTP package (because it works)


0. Login as root or su.

1. Check to ensure that the permissions are correct to relay mail.
telnet my.emailserver.ip.address 25
EHLO
MAIL FROM: myvalidsenderaddress@foobar.org
RCPT TO: myvalidrecipient@foobar.org
DATA
This is a test message.
. <Hit Enter>


quit

You should have received an email if the permissions are correct.

2. Install the EPEL repo

cd /tmp
wget http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-4.noarch.rpm

 Please use this alternative if the original does not work. - http://download.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm

rpm -Uvh epel-release-5-4.noarch.rpm

3. Remove sendmail
yum remove sendmail

4. Install SSMTP
yum install ssmtp --enablerepo=EPEL

5. Configuration
Backup original conf file
cp /etc/ssmtp/ssmtp.conf /etc/ssmtp/ssmtp.conf.bkp

Edit the conf file
nano /etc/ssmtp/ssmtp.conf 

Edit the following lines:
mailhub=my.emailserver.ip.address:25
Hostname=ValidDomainHostName.foobar.org

*Depending on your configuration, you may or may not need TLS, check the SSMTP.conf man for details.

6. Restart service
service ssmtp restart

7. Test the application
echo "This is the body of my email" | ssmtp myemail@foobar.org

That's it!

This worked for me where all the others mysteriously failed. Now I'm off to finish my script. I'll post another update soon!

-noveck

Monday, September 17, 2012

Installation of VNC Server on CentOS 5.x

 I honestly thought I documented this already, but I searched the blog and was unable to locate it.

This covers installation and quick configuration of VNC on a server, in the event that Desktop access is needed on a remote server. In some cases, the CLI is not enough, and I've found this very useful.

On to the entry:

1. Login as root (or su)

2. Install required packages:
yum  -y install vnc-server

3. Secure the VNC server:
vncpasswd
Set mypassword
cd ~/.vnc
ls

the passwd folder should be listed

4. Configure the server:
nano /etc/sysconfig/vncservers
add to end of file:
VNCSERVERS="3:myusername"
VNCSERVERARGS [3] = "-geometry 1024x768"

5. Start the VNC service and configure service to autostart on boot:
service vncserver start
chkconfig vncserver on


6. Further configuration:
cd ~/.vnc
nano xstartup

add under the line "# Add the foll..."
(while true ; do xterm ; done) &
remove comments(#) from
#unset SESSION_MANAGER
#exec /etc/x11/xinit/xinitrc
7. Restart the vnc service
Service vncserver restart

8. Configure firewall to allow VNC server traffic
system-config-securitylevel-tui
add 5903:tcp to exceptions list

9. Open VNC client from remote peer and connect to VNC Server:
server: 192.168.x.x:3
username myusername
password: mypassword (as set in Step 3)


finito
-noveck

Sunday, September 9, 2012

LinuxCon2012!

Just got back from Linux Foundation's LinuxCon2012 (Aug 29-31) in San Diego.

Some great sessions and the slides were made public. It was a powerful reminder of the immense possibilities of open source and who are some of the companies running Linux in their enterprise.

See Linux Foundation's site for some of the brilliant presentations. One of my personal favourites was the fact that SpaceX's shuttle is running on linux.

 Lots of great ideas in hand, stay tuned for some fresh projects!

-noveck

Monday, August 6, 2012

Identifying and Repairing MySQL (MYISAM) Table fragmentation

I had to perform some server maintenance recently, so I decided to re-run the handy mysql tuner tool, as mentioned in my Database Tuning entry.

Lo and behold, a couple tables were identified as fragmented, even though I have a weekly optimization script that checks and repairs any fragmentation.

Just in case there was some sort of anomaly that would bite me in the butt, I decided to run a check to identify the fragmented tables and manually repair those specifically. I found this post by Sean Hull on the DatabaseJournal very helpful!

To quote the author:
"MySQL tables, including MyISAM and InnoDB, two of the most common types, experience fragmentation as data is inserted and deleted randomly. Fragmentation can leave large holes in your table, blocks which must be read when scanning the table. Optimizing your table can therefore make full table scans and range scans more efficient."


The process:

0. Login as root on the Mysql server
mysql -u root -p ************

1. Identify fragmented tablesTake note of the tables, or perhaps select into an outfile if you so desire. This may come in handy if you want to script or automate the identification and repair of the fragmented tables.

select table_schema, table_name, data_free, engine
from information_schema.tables where table_schema
not in ('information_schema', 'mysql') and data_free > 0;


2. Repair the table fragmentation
This could be scripted, but was manually executed as I only had about 4 fragmented tables.
optimize table tablename1;

3. Rerun Step 1 to ensure the fragmentation has been resolved.
 
-noveck

Tuesday, July 31, 2012

Mounting an ISO to an installed VM on VMWare ESXi


I had a little maintenance item to do which involved mounting an ISO to a working VM. Even when the ISO was attached from the VM settings, it somehow was not booting from the device, apparently ESXi sets the first boot device to the HDD by default once a OS is loaded and it will never boot from the CD/ISO. In this case I was using Linux gparted, but for the Windows guys it can be a Windows Server Rescue or something.
This is how I got it to work:
1. Set boot delay to 3000 milliseconds.
                 -Right Click VM -> Edit settings
                 - Options Tab, edit Boot options -> Power on Boot Delay parameter (I used 3000ms or 3 seconds)

2. Mount the ISO from the Datastore / Client DVD
                 - Right Click VM -> Edit settings
                 - Options Tab, CD/DVD Drive x, browse to uploaded ISO or Client Device.

3. Start the VM, at the boot menu hit ESC

4. Select CD/DVD boot option

5. Use your ISO media / Application.

6. When completed reset the Boot Delay parameter to 0.

-noveck

Monday, July 30, 2012

Installing gparted on CentOS 5.x

Okay, so a situation arose where I need to use gparted to do a quick resize on a production server, and this package is not part of the base repository.

It is in the EPEL repo, so that will need to be installed first (if not already done).

Anyhow, it's really simple, so read on...

Assumptions: Base installation of CentOS 5.x

0. Login as root or su

1. Install additional repos

Navigate to temporary directory

cd /temp
download EPEL repo

wget
http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-4.noarch.rpm

Note: The above URL is sporadic. Please use this alternative if the original does not work. - http://download.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm

Install the repo
rpm -Uvh epel-release-5-4.noarch.rpm

2. Install the gparted package and dependencies.

yum install gparted --enablerepo=epel disablerepo=rpmforge

*note the rpmforge repo must be disabled before installing (where applicable)

3. Run the application from terminal
gparted

More information can be found here: http://gparted.sourceforge.net/


-noveck