Monday, November 7, 2011

Moodle Apache/DB Connection Errors!

The Moodle's config.php includes a section to email connection errors to an email address

$CFG->emailconnectionerrorsto = ''

I've noticed some random errors popping up in my inbox, at the same time my stats script is configured to run.
Something along the lines of:

WARNING: Database connection error:

I found this especially curious, especially my Zabbix Server was not reporting any errors. So I dug a little deeper, and ran a couple of the Mysql Reporting tools I use.

Turns out I was maxing out my max_connections config.
My old config was 385 and I had originally increased this to 500 with the deployment of the LB/HA Moodle Environment (earlier post)

I had to do some more tweaking of the mysql configuration file. /etc/my.cnf

The default wait_timeout is very long: 14400 sec.
In my original deployment in May this year, I reduced this to 3600 sec.
With these errors randomly cropping up during the statsrun, I had to tweak it yet again.

I reduced the key_buffer_size to increase the max_connections to 550. This was necessary to ensure that the server does not over-allocate RAM. I also reduced the wait_timeout to 1800 sec, which is fine considering the hardware my server uses. (2 Quad core proc, 32GB RAM)

A quick save and restart of the mysql service was all it took.
96+ hours and I'm still actively monitoring, no more connection errors and we're down to about a max of 300 connections at the peak.