Friday, September 30, 2016

Installing Docker on CentOS 7 (behind corporate proxy)

As part of my research into Percona's new Open Source offering, their Percona Monitoring and Management platform, I realized that a core component is provided via a Docker container. Now I've previously played around with docker on a small scale, but this needed to be done on an actual server environment on the corporate network. There was one little item that caused a brief moment of grief with the proxy, but I eventually sorted it out.

Docker Engine installation on CentOS7

0. Login as root
1. Update machine
yum update
2. Add the Docker yum repo

tee /etc/yum.repos.d/docker.repo <<-'EOF'[dockerrepo]name=Docker Repositorybaseurl=https://yum.dockerproject.org/repo/main/centos/7/enabled=1gpgcheck=1gpgkey=https://yum.dockerproject.org/gpgEOF

3. Install the Docker Engine Package
yum install docker-engine

4. Start the Daemon
systemctl start docker

5. Set to run at boot
systemctl enable docker

6. Verify Operation with simple test
docker run hello-world

**Note: if you're behind a proxy, you may notice an error 
... dial tcp xx.xx.xx.xx:53: getsockopt: connection refused

You may need to do the following

a. create a systemd drop-in directory for the docker service
mkdir /etc/systemd/system/docker.service.d

b. create a proxy configuration file
...in the directory just created, in my case I needed both an HTTP and an HTTPS proxy to get it to work
vi /etc/systemd/system/docker.service.d/http-proxy.conf

Add the following (one line)
[Service]
Environment="HTTP_PROXY=http://your.proxy.ip.addr:port/" "HTTPS_PROXY=http://your.proxy.ip.addr:port/"

Save and exit

c. flush changes
systemctl daemon-reload

d. Verify that the configuration has been loaded:
systemctl show --property=Environment docker

e. Restart Docker:
systemctl restart docker
Verify Operation with simple test (works, yay!)
docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
c04b14da8d14: Pull complete
Digest: sha256:0256e8a36e2070f7bf2d0b0763dbabdd67798512411de4cdcf9431a1feb60fd9
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker Hub account:
 https://hub.docker.com

For more examples and ideas, visit:
 https://docs.docker.com/engine/userguide/

Wednesday, August 17, 2016

BootCamp: OSX (El Capitan) and Windows 10

The only reason I'm posting this is because I was stuck for a few hours and the internet had tons of conflicting fixes for this that led me nowhere. Still stuck with GPT errors and a whole lot of general bollocks. For some reason this refused to work on the USB port on the left of my macbook and for reasons known only to Job's & Gates' lovechild,  worked on the one located on the right. Sorcery.

This is what worked for me, I can't guarantee that it will work for you and I'm by no means a Mac expert - so use at your own risk. Caveat emptor. No screenshots because of reasons.

What you need:

1. MacBook or other Mac Computer that is listed here: https://support.apple.com/en-us/HT204990
2. Windows 10 64-bit ISO.
3. A fresh USB stick, 8G or more would be fine.
4. At least 100GB free on your Mac for a 64GB Windows install.


What's next?
0. Update your mac if needed.
1. Run bootcamp and select the two options
- Create a Windows 7 or later setup disk
 - Download the latest windows support software from Apple
1.1. Click Continue, grab a beer. or three. This takes a while.

2. Exit bootcamp. Because of reasons.

3. Fire up bootcamp and this time select the option to install (only).

4. Drag a partition size that suits you, I went with 64GB - then click Continue

5. Magic happens then the computer would probably reboot right into OSX. Manually reboot and this time hit the Option Key, Feel free to bang away at it. (Disclaimer: Do not bang away at it.)

6. When presented with boot options, please ignore the Windows option. Select UEFI boot.

7. Oh looky, run through the windows install until you get to the Disk screen. At this point choose your desired windows partition created in step 4 and then click format. The next button magically appears and you should now be able to proceed with installation.

8. Magic happens again, then a couple reboots later and Win 10 is done. Grab another beer. or three.

9. The next time you bring up the dual boot screen, Only OSX and Windows would be presented and the UEFI option is gone.

10. Happy dualbooting. Hell you'd be happy too after six beers.


 




Tuesday, July 12, 2016

Using Grep to search for a string inside a file

Simple story here really.

I had a borked config somewhere inside my /etc/ folder that was throwing some application errors. It's a development machine with poor documentation so I had to try to figure out where the typo happened.

Grep is a simple, but incredibly powerful command that took care of this quite easily. The flags I used  provided the filenames that contain the matching string as well as the line number where the string is matched. Helpful for fixing or tweaking some borked config or code.

Usage is quite simple:


grep -inr "badconfigurationstring" /searchdirectory

the flags are as follows;
-i = ignore case sensitivity
-n = print line number of matching string
-r = recursively read all files under search directory

more info at the grep man page: http://linux.die.net/man/1/grep

--end.






Monday, February 15, 2016

TurnitinTwo Issues with Moodle 3 on CentOS 6

I recently stumbled upon a Turnitin bug on my Moodle 3 environment where it was simply failing to connect with Turnitin through the configuration interface.


The message was fairly cryptic and the logs even more so.

The error message was simply: "Could not connect to Turnitin, Double check your API URL setting". This was fine, so I looked at the API log, which indicated a "Curl error: Proxy CONNECT aborted", alongside an error 502.


When Turnitin was contacted, they gave this response, which did not work.
If you encounter connectivity issues while using the Turnitin Moodle Direct V2 integration (error: Turnitin API Base URL incorrect or unavailable, or error: Double check your API URL setting) this could be related to a CA certificate being unavailable to cURL. Viewing the Turnitin Apilog files will identify if this is the case.

The Moodle Direct plugin uses the server operating system's implementation of cURL. If cURL has an out of date (or no) CA certificates, the interaction with Turnitin will fail due to cURL performing peer SSL certificate verification and not being able to verify the Turnitin SSL certificate. Until cURL 7.18.0 some CA certificates were provided, but after 7.18.0 no CA certificates have been provided at all. Because of this, the Moodle server administrator would need to ensure that an up to date CA certificate bundle is used.

For Debian and RedHat based distributions:
CA certificates are distributed in the ca-certificates package. Gentoo servers provide them via the app-misc/ca-certificates ebuild. It's also a good idea to make sure that the OpenSSL libraries (libssl) and cURL libraries (libcurl) are up to date on your server.

You will also need to place a file with the Bundle of CA Root Certificates (downloadable from
http://curl.haxx.se/ca/cacert.pem) on your webserver and make a curl.cainfo reference to this file in your php.ini.

For Windows based servers:

1. You need to be running PHP 5.3.7 or later.
2. Download
https://raw.github.com/bagder/curl/master/lib/mk-ca-bundle.vbs
from the Curl repository on GitHub.
3. Open a Command Prompt as Administrator and go to the directory in which you downloaded mk-ca-bundle.vbs .
4. Run mk-ca-bundle.vbs . Accept the default file name and do not include the text information for each certificate.
5. After running this you will end up with a file ca-bundle.crt.
6. Copy that to a known location, e.g. {path}/ca-bundle.crt.
7. Add curl.cainfo={path}/ca-bundle.crt to php.ini. See PHP Runtime Configuration for more details
[PHP]
;;;;;;;;;;;;;;;;;;;
; CURL Settings ;
;;;;;;;;;;;;;;;;;;;
curl.cainfo={path}/ca-bundle.crt
8. Restart the IIS web site
We were eventually able to resolve the issue by a combination of factors:

1. In addition to having an http_proxy environment variable in the operating system, I also needed to explicitly set an https_proxy. This is dependent on if your server currently uses an http_proxy environment variable. Do not make any changes if your server can access the web directly.


vi /etc/bashrc
add linesexport http_proxy=’http://yourproxyip:port/’ export https_proxy=’http://yourproxyip:port/’ save and exit shell

2. The CA-Certificates bundle from curl.hexx.se did not work and resulted in a bunch of errors related to the SSL CA Cert (Message: Problem with the SSL CA cert (path? access rights?)) , so I re-installed the ca-certificates bundle from CentOS repositories.
yum reinstall ca-certificates openssl
3. I then used the “update-ca-trust” package to update the certificate store.
update-ca-trust
4. I removed the Moodle Proxy configuration from the Moodle application interface.
    Dashboard / ► Site administration / ► Server / ► HTTP (Server Proxy section)


Please note that these steps fixed the issue with my particular environment, if you are faced with similar issues I'd suggest starting with steps 2 and 3. If you are having proxy connect issues beforehand, you may try step 1 first.

Good luck and happy moodling!

-Noveck

Thursday, April 30, 2015

Merging multiple files using the cat command


Nothing major today, this is a straightforward case that I was surprised would work with the Linux cat command.

I had a folder with several mp3's that were part of one long mixtape but were split into dozens of files, when they should have really been one big file.

I could have used a tool to fix this, but it can be accomplished natively using a Linux or Mac terminal.

First navigate to the folder where the files are located. If they are in numerical order, that's short and easy.

 cat *.mp3 > ../mylongsong.mp3

This combines all the files into one and dumps in one directory above to avoid an endless loop, depending on the shell you're using.


If the files have arbitrary names, but you can figure out the order, you can use a slightly longer command:

cat 1.mp3 2.mp3 a.mp3 myfav.mp3 wow.mp3 > ../mylongsong2.mp3

The caveat is that all the files need to be encoded the same way and be of the same filetype. It will not merge mp3 with wma, it will not merge a 320kbps mp3 with a 128kbps version.



Thursday, February 19, 2015

Installing MySQL 5.6 on CentOS 6

MySQL on CentOS6 is currently limited to version 5.1, which sucks. A lot of modern web applications can benefit of running the most recent version of MySQL (up to 5.6 at the writing of this post), and as a result we will be installing MySQL 5.6 on CentOS 6.x, patched to the latest sub-version.

There are several ways of accomplishing this, but I think it's a good idea to take the software directly from the source.

It's also a pretty good idea to have a backup mechanism, and sadly the free version does not play well with hot backups, which are performed while the database is running in full read/write made.

Percona has a wonderful set of tools, and offer a free hot backup solution for MySQL that works awesomely. I'll provide the instructions to install the tool, you can feel free to peruse their docs to implement your own backup strategy :)

More on Percona Xtrabackup here: http://www.percona.com/doc/percona-xtrabackup/2.2/

More on MySQL Community Server here: http://dev.mysql.com/downloads/mysql/


0. Got root/sudo

1. Get and Install the RPM from MySQL/Oracle
cd /tmp
wget  http://dev.mysql.com/get/mysql-community-release-el6-5.noarch.rpm
rpm -Uvh mysql-community-release-el6-5.noarch.rpm

2. Install MySQL and start / enable service
yum install mysql-community-server 
service mysqld start && chkconfig mysqld on

3. Secure MySQL server

./usr/bin/mysql_secure_installation
4.  Install Percona with repos
rpm -Uvh http://www.percona.com/downloads/XtraBackup/XtraBackup-2.2.9/binary/redhat/6/x86_64/percona-xtrabackup-2.2.9-5067.el6.x86_64.rpm
 Verify successful install and check version
xtrabackup --version



xtrabackup-screenshot

--EOF

Thursday, January 8, 2015

Pluggable Authentication Modules (PAM) - some basic tricks on CentOS 6

I've been playing around with PAM on a couple distros recently, and I thought I'd share some quick tips and tricks in setting up a secure CentOS 6 Linux multi-user environment. Whilst these are not bulletproof password policies, they are a step beyond the default distribution configuration and are not too complex that the users would be bugging you, the friendly neighbourhood sysadmin.

As usual, any feedback is appreciated, so drop me a line: noveck@woblag.com. Once it gets past the spam filters, I'll try my best to respond asap.

1. Use PAM to disable the use of null passwords in user Accounts.

vi /etc/pam.d/system-auth

Find line 
password sufficient pam_unix.so md5 shadow nullok try_first_pass use_authtok

Remove/delete nullok option, so the line now reads:
password sufficient pam_unix.so md5 shadow try_first_pass use_authtok

save and close file


2. Use PAM to prevent re-using/recycling passwords .

This example prevents the use of the last 3 passwords.

vi /etc/pam.d/system-auth
find line
password sufficient pam_unix.so md5 shadow nullok try_first_pass use_authtok

Add to end of line
password sufficient pam_unix.so md5 shadow nullok try_first_pass use_authtok remember=3

save and close file

3. Set password minimum length

This example sets the minimum password length to 8 characters.

vi /etc/pam.d/system-auth

find line
password sufficient pam_unix.so md5 shadow nullok try_first_pass use_authtok

Add new line BEFORE
passwd password requisite pam_cracklib.so minlen=8
save and close file

4. Configure server to deny access with multiple incorrect login attempts

This example temporarily denies access after 5 attempts. The temporary lockout time can also be configured for a certain time, which will be set to 1 hour (3600 seconds) in this example.

vi /etc/pam.d/system-auth

Add the following line to end of file
auth required pam_tally.so onerr=fail deny=5 unlock_time=3600

save and close file

--END