Using Grep to search for a string inside a file

Simple story here really.

I had a borked config somewhere inside my /etc/ folder that was throwing some application errors. It's a development machine with poor documentation so I had to try to figure out where the typo happened.

Grep is a simple, but incredibly powerful command that took care of this quite easily. The flags I used  provided the filenames that contain the matching string as well as the line number where the string is matched. Helpful for fixing or tweaking some borked config or code.

Usage is quite simple:


grep -inr "badconfigurationstring" /searchdirectory

the flags are as follows;
-i = ignore case sensitivity
-n = print line number of matching string
-r = recursively read all files under search directory

more info at the grep man page: http://linux.die.net/man/1/grep

--end.

TurnitinTwo Issues with Moodle 3 on CentOS 6

I recently stumbled upon a Turnitin bug on my Moodle 3 environment where it was simply failing to connect with Turnitin through the configuration interface.


The message was fairly cryptic and the logs even more so.

The error message was simply: "Could not connect to Turnitin, Double check your API URL setting". This was fine, so I looked at the API log, which indicated a "Curl error: Proxy CONNECT aborted", alongside an error 502.

When Turnitin was contacted, they gave this response, which did not work.

If you encounter connectivity issues while using the Turnitin Moodle Direct V2 integration (error: Turnitin API Base URL incorrect or unavailable, or error: Double check your API URL setting) this could be related to a CA certificate being unavailable to cURL. Viewing the Turnitin Apilog files will identify if this is the case.

The Moodle Direct plugin uses the server operating system's implementation of cURL. If cURL has an out of date (or no) CA certificates, the interaction with Turnitin will fail due to cURL performing peer SSL certificate verification and not being able to verify the Turnitin SSL certificate. Until cURL 7.18.0 some CA certificates were provided, but after 7.18.0 no CA certificates have been provided at all. Because of this, the Moodle server administrator would need to ensure that an up to date CA certificate bundle is used.

For Debian and RedHat based distributions:
CA certificates are distributed in the ca-certificates package. Gentoo servers provide them via the app-misc/ca-certificates ebuild. It's also a good idea to make sure that the OpenSSL libraries (libssl) and cURL libraries (libcurl) are up to date on your server.

You will also need to place a file with the Bundle of CA Root Certificates (downloadable from http://curl.haxx.se/ca/cacert.pem) on your webserver and make a curl.cainfo reference to this file in your php.ini.

For Windows based servers:

1. You need to be running PHP 5.3.7 or later.
2. Download https://raw.github.com/bagder/curl/master/lib/mk-ca-bundle.vbs from the Curl repository on GitHub.
3. Open a Command Prompt as Administrator and go to the directory in which you downloaded mk-ca-bundle.vbs .
4. Run mk-ca-bundle.vbs . Accept the default file name and do not include the text information for each certificate.
5. After running this you will end up with a file ca-bundle.crt.
6. Copy that to a known location, e.g. {path}/ca-bundle.crt.
7. Add curl.cainfo={path}/ca-bundle.crt to php.ini. See PHP Runtime Configuration for more details
[PHP]
;;;;;;;;;;;;;;;;;;;
; CURL Settings ;
;;;;;;;;;;;;;;;;;;;
curl.cainfo={path}/ca-bundle.crt
8. Restart the IIS web site

We were eventually able to resolve the issue by a combination of factors:

1. In addition to having an http_proxy environment variable in the operating system, I also needed to explicitly set an https_proxy. This is dependent on if your server currently uses an http_proxy environment variable. Do not make any changes if your server can access the web directly.

vi /etc/bashrc
add linesexport http_proxy=’http://yourproxyip:port/’ export https_proxy=’http://yourproxyip:port/’ save and exit shell

2. The CA-Certificates bundle from curl.hexx.se did not work and resulted in a bunch of errors related to the SSL CA Cert (Message: Problem with the SSL CA cert (path? access rights?)) , so I re-installed the ca-certificates bundle from CentOS repositories.

yum reinstall ca-certificates openssl

3. I then used the “update-ca-trust” package to update the certificate store.

update-ca-trust

4. I removed the Moodle Proxy configuration from the Moodle application interface.

    Dashboard / ► Site administration / ► Server / ► HTTP (Server Proxy section)

Please note that these steps fixed the issue with my particular environment, if you are faced with similar issues I'd suggest starting with steps 2 and 3. If you are having proxy connect issues beforehand, you may try step 1 first.

Good luck and happy moodling!

-Noveck

Merging multiple files using the cat command

Nothing major today, this is a straightforward case that I was surprised would work with the Linux cat command.

I had a folder with several mp3's that were part of one long mixtape but were split into dozens of files, when they should have really been one big file.

I could have used a tool to fix this, but it can be accomplished natively using a Linux or Mac terminal.

First navigate to the folder where the files are located. If they are in numerical order, that's short and easy.

 cat *.mp3 > ../mylongsong.mp3

This combines all the files into one and dumps in one directory above to avoid an endless loop, depending on the shell you're using.


If the files have arbitrary names, but you can figure out the order, you can use a slightly longer command:

cat 1.mp3 2.mp3 a.mp3 myfav.mp3 wow.mp3 > ../mylongsong2.mp3

The caveat is that all the files need to be encoded the same way and be of the same filetype. It will not merge mp3 with wma, it will not merge a 320kbps mp3 with a 128kbps version.

Installing MySQL 5.6 on CentOS 6

MySQL on CentOS6 is currently limited to version 5.1, which sucks. A lot of modern web applications can benefit of running the most recent version of MySQL (up to 5.6 at the writing of this post), and as a result we will be installing MySQL 5.6 on CentOS 6.x, patched to the latest sub-version.

There are several ways of accomplishing this, but I think it's a good idea to take the software directly from the source.

It's also a pretty good idea to have a backup mechanism, and sadly the free version does not play well with hot backups, which are performed while the database is running in full read/write made.

Percona has a wonderful set of tools, and offer a free hot backup solution for MySQL that works awesomely. I'll provide the instructions to install the tool, you can feel free to peruse their docs to implement your own backup strategy :)

More on Percona Xtrabackup here: http://www.percona.com/doc/percona-xtrabackup/2.2/
More on MySQL Community Server here: http://dev.mysql.com/downloads/mysql/


0. Got root/sudo

1. Get and Install the RPM from MySQL/Oracle

cd /tmp
wget  http://dev.mysql.com/get/mysql-community-release-el6-5.noarch.rpm
rpm -Uvh mysql-community-release-el6-5.noarch.rpm

2. Install MySQL and start / enable service

yum install mysql-community-server 
service mysqld start && chkconfig mysqld on

3. Secure MySQL server

./usr/bin/mysql_secure_installation

4.  Install Percona with repos

rpm -Uvh http://www.percona.com/downloads/XtraBackup/XtraBackup-2.2.9/binary/redhat/6/x86_64/percona-xtrabackup-2.2.9-5067.el6.x86_64.rpm

 Verify successful install and check version

xtrabackup --version

--EOF

Pluggable Authentication Modules (PAM) - some basic tricks on CentOS 6

I've been playing around with PAM on a couple distros recently, and I thought I'd share some quick tips and tricks in setting up a secure CentOS 6 Linux multi-user environment. Whilst these are not bulletproof password policies, they are a step beyond the default distribution configuration and are not too complex that the users would be bugging you, the friendly neighbourhood sysadmin.

As usual, any feedback is appreciated, so drop me a line: noveck@woblag.com. Once it gets past the spam filters, I'll try my best to respond asap.

1. Use PAM to disable the use of null passwords in user Accounts.

vi /etc/pam.d/system-auth

Find line 

password sufficient pam_unix.so md5 shadow nullok try_first_pass use_authtok

Remove/delete nullok option, so the line now reads:
password sufficient pam_unix.so md5 shadow try_first_pass use_authtok

save and close file

2. Use PAM to prevent re-using/recycling passwords .

This example prevents the use of the last 3 passwords.

vi /etc/pam.d/system-auth
find line
password sufficient pam_unix.so md5 shadow nullok try_first_pass use_authtok

Add to end of line
password sufficient pam_unix.so md5 shadow nullok try_first_pass use_authtok remember=3

save and close file

3. Set password minimum length

This example sets the minimum password length to 8 characters.

vi /etc/pam.d/system-auth

find line
password sufficient pam_unix.so md5 shadow nullok try_first_pass use_authtok

Add new line BEFORE
passwd password requisite pam_cracklib.so minlen=8
save and close file

4. Configure server to deny access with multiple incorrect login attempts

This example temporarily denies access after 5 attempts. The temporary lockout time can also be configured for a certain time, which will be set to 1 hour (3600 seconds) in this example.

vi /etc/pam.d/system-auth

Add the following line to end of file
auth required pam_tally.so onerr=fail deny=5 unlock_time=3600

save and close file

--END

Fedora 20 as a (viable!) Windows Desktop Replacement

So I've been bound to windows desktops for so long, that it's been my second nature for workstations. My de facto preference for most server environments are Linux, so I've been between both worlds for quite some time.

My Windows7 machine suffered an OS failure a couple months ago, after almost 4 years of constant everyday use. I installed Windows 8 and was relatively happy with it until it just decided to screw over my WIFI. Nobody messes with my WIFI.

I finally decided to chuck the 15 year Redmond tether and go for broke. I'm gonna run Fedora 20 as a desktop environment, and I'm never looking back. Or so I hope.

Anyhow, FC20 is nice, but to make the environment a bit more aesthetic and a bit more intuitive for use by longtime Windows users I've compiled a "Sanity list". These make everyday use a bit less complicated and quite easy to forget that you're running Linux, traditionally seen as a big, bad, complex beast.

I'll explain why in line, followed by the instructions to accomplish on Fedora 20. This assumes a base Fedora 20 installation has been completed, with a default Gnome desktop.

If you ever decide to take the plunge, this can probably help you break the windows habit ;)

Why Fedora?

Tweak List as of 06-OCTOBER-2014

- install cinnamon, a pretty good looking desktop environment.
This can be selected from the settings icon on the default login screen and it will be the future default afterwards.

sudo yum groupinstall cinnamon

- install infinality, some font eyecandy.
There are a number of styles, I prefer the OSX style on my 4 year old Dell Latitude display

sudo rpm -Uvh http://www.infinality.net/fedora/linux/infinality-repo-1.0-1.noarch.rpm
sudo yum -y install freetype-infinality fontconfig-infinality
cd /etc/fonts/infinality/
./infctl.sh setstyle osx 

(personal pref - more options are in README documentation)

- install MSTTCoreFonts
You have to admit that MS ships with some solid fonts. If you want to avoid formatting issues with opening MS Office docs with LibreOffice, this is a pretty good idea.

sudo yum install cabextract
sudo yum install http://sourceforge.net/projects/mscorefonts2/files/rpms/msttcore-fonts-installer-2.6-1.noarch.rpm

- install vlc
This is another personal preference; VLC is a solid media player and works great on Windows as well as Linux

su -c 'yum localinstall --nogpgcheck http://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm http://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm'
sudo yum install vlc

- install flash player
As much as I didn't want to, there are still several sites that I use that are flash based. A necessary evil.

rpm -ivh http://linuxdownload.adobe.com/adobe-release/adobe-release-i386-1.0-1.noarch.rpmrpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-adobe-linuxyum check-updateyum install flash-plugin nspluginwrapper alsa-plugins-pulseaudio libcurl

- install google chrome
Chrome is a great addition to Fedora, compliments the use of Firefox, which I personally find to be a bit bloated now.

cat << EOF > /etc/yum.repos.d/google-chrome.repo
[google-chrome]
name=google-chrome - \$basearch
baseurl=http://dl.google.com/linux/chrome/rpm/stable/\$basearch
enabled=1
gpgcheck=1
gpgkey=https://dl-ssl.google.com/linux/linux_signing_key.pub
EOF
yum install google-chrome-stable

- install skype
Skype, almost a universal tool for free webcalling. I installed it because I use it a lot for international calling to the US for free.

wget http://download.skype.com/linux/skype-4.2.0.11-fedora.i586.rpmsudo yum install skype-4.2.0.11-fedora.i586.rpm

- install dropbox
I've been using dropbox on all my boxes, it's a great way to keep some personal docs on the cloud and have multiple versions for backup.

cat << EOF > /etc/yum.repos.d/dropbox.repo
[Dropbox]
name=Dropbox Respository
baseurl=http://linux.dropbox.com/fedora/19/
gpgkey=http://linux.dropbox.com/fedora/rpm-public-key.asc
EOF
sudo yum install nautilus-dropbox

- install google talk
I consider this an optional item. You may or may not actually need this.

wget http://dl.google.com/linux/direct/google-talkplugin_current_i386.rpm
yum install google-talkplugin_current_i386.rpm --nogpgcheck

- install teamviewer
TeamViewer is an excellent desktop sharing/control tool that comes in handy to access my laptop remotely if needed.

wget http://download.teamviewer.com/download/teamviewer_linux.rpm

Additional tweaks:

- tweak boot timeout behaviour
Shorten the time to countdown the OS selection

vi /etc/default/grub
GRUB_TIMEOUT=0
GRUB_HIDDEN_TIMEOUT_QUIET=true
grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg

- logind tweaks
These control some suspend/hibernate options when the lid is closed. You can decide if you need these.

vi /etc/systemd/logind.conf
HandleSuspendKey=ignore
HandleHibernateKey=ignore
HandleLidSwitch=ignore
HandlePowerKey=poweroff

- eyecandy that can be installed through the extensions menu
(Settings -> Extensions)
Opacify


That's it for now. I'll add to this post and drop a datestamp with any new additions!

Image courtesy http://www.fedoraproject.org

Fedora 20 - Missing WIFI driver/support on clean install

I've started a little experiment on porting a crashed windows 7 Laptop (Dell Inspiron 1525) over to Linux. Why Linux at this point? The crash was largely due to the infamous Windows Update Issue. For me that was the straw that broke the camel's back.

The experiment part of this update comes from the fact that the user has never really used Linux before, so this is really a true test of the user-friendliness of the OS, once initially set up. My weapon of choice was Fedora 20, largely because I'm entrenched in the RHEL distribution tree for almost everything that I do, so it's naturally easier for me to troubleshoot quickly.

The installation and initial configuration went pretty smoothly, except for the fact that out of the box, WIFI appeared nonexistent. This was not a big problem, as it worked on a Kali Linux Live USB, so I knew that the hardware was fine. From the old install I knew that the device was Broadcom based, so I had a launch point.

Credit to http://wireless.kernel.org/en/users/Drivers/b43 for thoroughly documenting the resolution of this issue.

First check the device for support:
0. As root:
lspci -vnn -d 14e4:

It spat out some device information, including the model, in my case the Broadcom Corporation BCM4312, which was supported by the b43 driver. I compared the output to the corresponding CHIP ID segment on the table
Taken from: http://wireless.kernel.org/en/users/Drivers/b43

PCI-ID	Supported?	Chip ID	Modes	PHY version	Alternative
14e4:4315	yes	BCM4312	b/g	LP (r1)	wl

The Fedora specific instructions are as follows:
1. Install the firmware extractor tool

su -

yum install b43-fwcutter wget

2. Install the driver

Define the Firmware path (Fedora specific)

export FIRMWARE_INSTALL_DIR="/lib/firmware"
Download and extract the Broadcom driver

wget http://mirror2.openwrt.org/sources/broadcom-wl-5.10.56.27.3_mipsel.tar.bz2

tar xjf broadcom-wl-5.10.56.27.3_mipsel.tar.bz2

Install the driver

sudo b43-fwcutter -w "$FIRMWARE_INSTALL_DIR" broadcom-wl-5.10.56.27.3/driver/wl_apsta/wl_prebuilt.o

3. Reboot

These steps were sufficient for me, as soon as I rebooted the WIFI was available to connect. If these steps do not solve your issue, you may need to blacklist older drivers. Take a read here: http://wireless.kernel.org/en/users/Drivers/b43

Maybe I'll break the chains of Microsoft bondage on my own laptop - which oddly enough, works swimmingly on a fresh Windows 8.1 installation. We'll see.

-noveck